At EnquireLabs, we use Postgres as our core database and for anyone who’s spent years in excel… it’s a dream. After a few searches, I realized there was no central resource/how-to for e-commerce analysis in Postgres (short for PostgreSQL), especially for Shopify. The majority of direct-to-consumer brands don’t have data engineers in-house, so as a brand owner or digital marketer, getting a database up and running could feel like a monumental task. But don’t fret: setting up a database with Shopify is quite trivial. This post will walk you through all the necessary steps to get you up and running.
Let’s say you have a report building process in Excel that takes 30 minutes every time you want to update it. The drill usually involves exporting your Shopify orders data, running a few vlookups and pivot tables, and possibly some manual entry. We’ve all done it. Now imagine, instead of exporting your Shopify order data every time you want to run a simple report, that order data is automatically updated to your Excel spreadsheet, or in our case a database. No more filtering and slicing & dicing. The data is already cleaned and ready to use, so you can spend those 30 minutes of your day on something smarter -- like analysis, or lunch
As a digital marketer, you may or not be overly familiar with databases, as you’re used to using Excel or Google Sheets for most, if not all, of your analysis. So what is a database? The easiest way to conceptualize a database is to compare it to what you already know: an Excel spreadsheet. Within a database you have tables, which are similar in theory to the columns and rows of Excel. I like to envision a database as a self-updating Excel file. It’s like a little employee hiding inside a computer, handing you an updated file every time you ask for it.
1. Creating Your Database
Good news. You don’t need a developer to create a Postgres database instance. With Heroku’s database-as-a-service, creating a Postgres database only takes a few minutes, even for non-technical people.
First, you’ll need to create a Heroku account if you don’t already have one (you can select any language in the on-boarding flow). After you’ve created an account, you’ll be presented with the “Create New App” screen–simply follow the prompts to create an app. Once your app is created click “Resources”, and in the “Add-Ons” search for “Postgres”. Heroku’s free plan unfortunately doesn’t offer enough rows to fully extract your Shopify instance, so you’ll need to select Hobby Basic ($9/month).
After you’ve provisioned your database, select Heroku Postgres in your add-ons list to open your dashboard. Select “Settings” and click “View Credentials”. You’ll need these in the steps below.
2. Extracting Data from Shopify to Your Postgres Database
We’ll use an ETL service to pull your order data from your Shopify account to your newly created Postgres database. ETL stands for extract, transform, & load; three database functions that when combined into one solution extract data from one database and push it to another. There are numerous ETL options to choose from, but we prefer StitchData. Their free account should provide enough data replication for most Shopify stores under $10mm in annual revenue.
After you’ve created your StitchData account, click “Destination” and enter your new Postgres Database credentials (you can leave the encryption type to None). Next, add your Shopify account, by simply clicking “Add Integration” and selecting Shopify and installing the StitchData app.
After you’ve completed these steps, StitchData will begin extracting your Shopify data and loading it into your Postgres DB. This process should take 30 minutes to an hour.
3. Querying Your Shopify Data In Postico
While you’re waiting for StitchData to load your data, you can install Postico and insert your database credentials. Postico is a SQL client that will allow you to query your data from your Mac (if you’re on Windows, you can use TablePlus). To add your database to Postico, simply input your Heroku database credentials after clicking File → New Favorite.
Now for some fun! Once StitchData is done syncing and your database credentials are entered in Postico, you can start querying your data. Take some time to explore the Shopify data structure by querying “SELECT *” from the various tables that are now in your newly created database.
4. Visualizing Your Data
To visualize your data, I recommend using Redash. Redash allows you to both query your database and build visualizations with your data. To add your data, simply create an account and add your Postgres database as a source. If you’d like, you can also write queries directly in Redash and avoid downloading a local client.
To note, we previously recommended using Heroku DataClips feature to create visualizations in Google Sheets. Unfortunately, due to how Heroku authenticates dataclips and caches data, this is no longer a viable solution.
Now that you’re all set up, have at it! We’ll be updating this public GitHub repo regularly with some queries you can use to get you started.