One of our strengths at Metabase has been getting you to a pretty simple chart on your own without having to know (or use) SQL.
But simplicity can only take you so far. Every time we’ve added new features to our trusty query builder, it’s been challenging to do so without either making those features somewhat convoluted or making the entire thing more confusing in the process. We’ve definitely felt the strain and we’re sure some of you have too.
So, in the spirit of trying to retain what’s worked but also set up Metabase to be all it can be going forward, we went into our workshop to try and solve this problem. We’re happy to say we’ve emerged with some new stuff for you all to try.
Two for the price of one
We realized one size fits all just isn’t going to cut it so there are now two ways for you to work with your data depending on your mood or needs (three if you count SQL which is of course sticking around)
A more powerful editor for joining data (finally), multiple aggregations and filter steps and more. All still possible without having to know or use SQL. Pop in here when you’ve got some work to do and really need to craft the right dataset and turn it into the complex answers you’re looking for.
A simpler and lighter experience for summarizing tables, filtering and creating charts. This is where you’ll land by default and where you can drill through and filter like you’re used to.
With this set of changes we’ll be able to more easily add powerful features for folks that need them without complicating things for people who just want to quickly explore.
Here’s where you come in
Given these changes are pretty massive and nobody likes their cheese to be moved without a bit of a heads up, we’re trying a longer preview period for this release. For those of you that are able, we’d love for you to give these new tools a shot and let us know what you like or dislike so both before and after these experiences ship we can keep making them better for you. As such you’ll probably see multiple preview releases and things may move around a little before the final release.
So what do I do if I have opinions or find bugs? Great question.
For general feedback, we’d love for you to chime in on this thread in our discussion forum. We’ll be over there listening to your thoughts on the new features.
We’re really excited about what these changes unlock for the future of Metabase and can’t wait to get it in your hands.
We’ll see you back here for the official 0.33 release post with the full set of new features soon.
Since we launched back in 2015, Metabase has been one of the easiest analytics and BI systems to host yourself. We’ve worked hard to make it easy to install, upgrade and maintain.
Offering up Metabase as an open source application you host yourself has allowed us to be used by companies who are especially security conscious or in highly regulated industries. You can audit our code, build your own binaries, and firewall us to your heart’s content.
When we started the project we hoped that in the next few years, the ecosystem of cloud providers would make it trivial to offer a docker image, offer upgrades, and easily attach a persistent datastore for users’ accounts, dashboards, etc (what we call our “application database” in the docs). We figured that the world was moving this way, and it made no sense to offer a hosted version of Metabase. If the world was going to make it trivial to offer a Container As A Service (with managed updates, troubleshooting, etc.), why get in the way of history? We also underestimated how widely Metabase would spread past the core group of ops-savvy data world.
Since then, we’ve both received a lot of demand (several emails a week or more) asking about a hosted version of Metabase, and the container ecosystem has yet to produce an easy to use Container As A Service offering. So…
We’re going to offer a hosted version of Metabase. We’ll be rolling out a beta this summer, and plan to enter General Availability later this fall.
The beta is aimed at companies that are already using Metabase and are happy with it, but don’t want to deal with the care and feeding of YetAnotherSystem. If you’re interested, please fill out the survey. If you’re a good fit for the program, we’ll get in touch and let you know the details.
Questions you should be asking:
(AKA Frequently Asked Questions)
Will you do what every other Open Source project does and make it a pain to self-host so you can drive up hosting revenue?
No. We still believe that, in an ideal world, self-hosting your analytics infrastructure makes it easier to deal with security, compliance (whether PCI, HIPPA or GDPR), and that your BI workloads should live near your data. That said, the world is imperfect and that is not always possible or the best choice for a given company.
Will this be free?
No. Our Ops engineers have to eat, and servers cost money so a hosted version of Metabase will also cost money.
Will you be offering the Enterprise Edition features on a hosted basis?
Yes, we’ll be offering a number of feature tiers, including the Enterprise Edition features.
Those of you who have been with us for a while have known that a lot has changed in Metabase over the years. We’ve added collections, new dashboard features, embedding, permissions, and a whole lot more. But one specific piece of Metabase, the query builder, has remained largely unchanged. So finally, after years of faithful service, we’re retiring it and replacing it with something we think is a whole lot better. Somethings actually, and we’d love to tell you about them.
A tale of two modes
Borrowing from a certain phrase, we’ve aimed to keep the simple things simple, and make harder things possible without having to resort to SQL when you’re working with your data. That’s why instead of trying to cram every possible tool into one space, we’ve split things into two different modes. To start, we’ve sanded things down and created a streamlined and simple place to view tables and visualizations, and quickly filter or summarize what you’re looking at.
When you need a bit more power and a touch more space, we’ve added a whole new experience to let you join data (yes, finally) and do multiple summary and filter steps when the question calls for it. You can preview each step to see how it affects your result, and then visualize it when you’re ready.
You can easily switch back and forth between these two modes when you need to, and start a new question using your preferred experience.
Easier browsing
You’ll also notice you can browse through your data from anywhere now from the nav bar. Click through your databases and tables to find what you need and you’ll see the data right away. Then, you can easily summarize or filter it using that mode we mentioned up above. If you’re a seasoned SQL pro and have been wishing you could get to writing faster, there’s also a one-click button to get you going from anywhere in Metabase.
Enhanced search
Search now also includes tables, metrics, and segments, so if you know what you’re looking for you can find it in a jiffy.
Bug fixes and enhancements
We’ve closed nearly 40 bugs in this release. Check out the full list of closed bugs.
Also, our amazing community of contributors have helped us to add Dutch as a supported language. Just set your browser’s preferred language to switch Metabase over. If you’d like to help translate Metabase into your own language of choice, head on over to our POEditor project.
How to get the latest release
For instructions on how to upgrade, head on over to our Operations Guide. Always make sure to make a backup of your Metabase app database before you upgrade!
If you haven’t set Metabase up before, here’s the best place to get started.
Tell us what you think!
We’d love to hear what you think of these new changes. Head over to our discussion board to give us your thoughts!
Since we started offering Metabase in different languages over a year ago, translated by the open source community, we have been blown away by the amount of involvement we have seen. Our translation project on POEditor currently has more than 500 contributors(!), and there are 18 languages besides English that have been at least 90% translated. So, before we say anything else, we want to make sure we say “thank you.”
One thing we’ve struggled with along the way, though, has been making our policies about language inclusion and upkeep consistent and clear. Sometimes languages that had been added to Metabase would start to drift away somewhat considerably from the 100% translation mark but wouldn’t be removed, while others would get quite close to 100% but not get included.
Additionally, one of Metabase’s primary tenets is that the product should always provide the absolute best user experience possible, and the core team began to discuss whether it’s truly a good experience to continue to allow users to experience translations that are not complete, resulting in a mishmash of their language and English. Should Metabase really have a “make things look bad” option? Making things worse, the parts of the product that often remain untranslated tend to be the new, marquee features in Metabase, increasing the likelihood of users encountering these less-than-ideal experiences.
So after much discussion and thought, we’re clarifying our policy for what it takes for a translation to be added to Metabase and to remain available in Metabase in subsequent releases:
Any new translation must reach 100% to be added to Metabase, and can be added in the next major or minor version.
All existing translations in Metabase must stay at 100% to continue being included in the next major version of Metabase.
We understand that this is a high bar, so we are holding ourselves to the following new process:
Before each major release, any additions or changes to text in the product will be completed 10 calendar days before the release ships, at which point we will notify all translators of the deadline to complete translations.
Our intent with these changes is to make it crystal clear what the bar is for a language to be included, and to give Metabase users the best possible experience. We hope that these new changes underscore our commitment to both clarity and great UX, while continuing to help make Metabase accessible to as many people around the world as possible.
To give the community sufficient advance notice, these policy changes will not go into effect for the upcoming version 0.34, scheduled for early December. Instead, they will go into effect starting with version 0.35.
If you’d like to help translate Metabase to your language, or to help maintain a translation, you can join our POEditor project.
Thanks once again for all you do to help make Metabase what it is.
We decided to sneak in a stocking stuffer here towards the end of the year. As always you can read the upgrade guide if you’re ready to get going, but for a bit more on the changes this time around read on below.
Showing values on data points
Now when you’re looking at a bar chart we’ll automatically show you the values above the points to give you a quicker read of what’s going on. You can also turn this new setting on for single series line charts. When the setting is on Metabase will fit as many labels as it can, or you can take things into your own hands and make it show values for every point if you wish. As this was one of our most upvoted issues, we’re sure this will make a few of you happy.
Mongo variables
We’ve added support for native query variables in MongoDB! Now you can set up custom widgets and filter to your heart’s content. Read more about it in the docs.
Timezone fixes
We rolled up our sleeves and reworked how Metabase handles timezones throughout the product, and in the process we managed to close out a lot of related issues. We hope you’ll find Metabase is now more reliable when it comes to displaying dates and time in your results.
SQL editor updates
We spent some time improving the quality of life for those of you who frequently use the SQL editor. Highlights include preventing the results of your query from fading out when editing the query, and making it easier to resize the editor. We’ve also given the variables panel a new coat of paint.
Community highlights
Here are a few community contributions we wanted to highlight this time around:
Dashboard cards now scroll the browser window when dragging to move them (thanks @maxzheng!)
Updated and optimized world and US states map files (thanks @flamber!)
And as always we’d like to give a big “thank you” to our translation community for working to make Metabase available in different languages.
Bug fixes and miscellany
We took care of a bunch of bugs in this release, so be sure to check out the full list of fixes and other items.
We hope you enjoy, and we’ll see you again in the new year — we have some big things planned for 2020.
Hi, everyone. We hope that all of you are staying healthy and safe.
We wanted to let you know that we’ve got another big release for you with a bunch of new features and enhancements that should help make it even easier to ask your questions in Metabase.
As always you can read the upgrade guide if you’re ready to get going, but here’s a bit more about what you’ll find in 0.35.
Filter expressions, string extracts, and more
At long last, we’ve added the ability to create filter expressions in the custom question builder, which allow you to include ORs in your filters. You can now write expressions like, ([Subtotal] > 100 AND [Tax] < 10) OR [Product.Category] = "Gizmo"
We’ve also added a whole host of new spreadsheet-like functions that can be used when creating filter expressions, custom columns, or custom aggregations. So you can now use things like case statements, string concatenation, or sumif and countif functions. Check out the documentation to see all the new functions you can use.
Reference saved questions in your SQL queries
You can now include the results of another saved question in your SQL queries using syntax like with Example as {{ #123 }} or select * from {{ #123}}, where #123 is the ID of the saved question. (Note that you can only include questions based on the same database as the one you’ve currently selected.)
Performance improvements
We’ve reworked the way Metabase deals with the saved questions “database” that allows you to nest saved questions inside of new ones. The result is a much faster experience on many pages throughout Metabase.
We also improved the way we fetch results from your data warehouse. Instead of buffering the results in memory, we now stream the results directly from the database to the client. Metabase driver authors should take a look at what’s new in 0.35 for a full list of changes.
Last but not least, we’ve improved some of the API endpoints to limit the amount of metadata being fetched in Metabase environments with a large number of database tables or questions.
Bug fixes and miscellany
We took care of a bunch of bugs in this release, so be sure to check out the full list of fixes and other enhancements.
We hope you enjoy all the updates, and we’ll see you again soon.
In version 0.35 we added a bunch of new functions to the custom column, custom filter, and custom aggregation expression formula editors. These let you do string manipulation on columns, construct filters that use ORs, calculate the share of a portion of your data relative to the whole, and much more. If you want to dig into all of the new functions that are available, check out the documentation.
Another powerful but perhaps not obvious thing these new functions let you do is create time series comparisons. For example, if you wanted to compare your revenue per month or users per day in 2019 vs. 2018, overlaid on top of each other, the new SumIf and CountIf aggregation functions let you. Here’s how.
Step 1: start a new custom question
All of these new functions are found in the custom question or “notebook” editor. To get there, we’ll click on the big “New question” button in the main navigation, then “Custom question.” Then we’ll pick the table that has the data that we have a question about. We’ll be using the sample “Orders” table that’s part of the sample dataset that comes with Metabase.
Step 2: add your first metric
In our example, we’re going to sum the Subtotal column in our Orders table to get our total revenue, and then we’ll break that number out by month. But the one different thing we’re going to do is create a separate metric for each year that we want to see, using the SumIf function. You could just as easily use the CountIf function, though, if you’re interested in counting the rows in your table over time instead of summing a column.
So to add our first metric, in the “Summarize” area of the notebook we’ll click the button that says “Pick the metric you want to see,” and scroll down to the “Custom Expression” option.
In the box that pops up, we’ll write the following:
What this is saying in effect is, “sum up the Subtotal column for all rows where the Created At column is between January 1, 2019, and December 31, 2019.” Let’s look at the specific parts of this expression individually:
The first part of the SumIf function is where we enter the column we want to sum, wrapped in square braces: [Subtotal].
The second part of the SumIf function is where we write the condition to specify which rows should be summed up. What we’re doing here is using the between function to specify that we only want to add up the Subtotal for orders that were placed within 2019. The reason we’re doing this here instead of in a filter is so that each additional metric we add can specify a different time range. In the between function, [Created At] is the time column in our table that we’re using to specify the time range. The first date in quotes is the start date of our range, and the second is the end date.
In the between function, we have to write the dates in this format: YYYY-MM-DD. I.e., four characters for the year, then two for the month, then two for the day, all separated by dashes. Note that you always need to use two characters for the month and the day: you have to write “2019-01-01” not “2019-1-1”.
Then we’ll click Done to add our metric.
Step 3: repeat step 2 for each additional year you want to compare
To add additional years to our comparison, we’ll click the plus (+) button and add an additional custom expression for each year. The easy way to do this is to copy the first expression we added in step 2, then paste that into each new expression and modify the year part of the between function.
Step 4: break out by month-of-year
Next we need to click “Pick a column to group by” to choose our breakout column. In most situations we’d pick a time column and choose the “by month” granularity option. But in this case, in order to overlay the series on top of each other, we’re going to choose the “month of year” option. (If you were to choose “Month” instead of “Month of year,” you’d see the series next to each other, one after the other.)
To do that, we’ll hover over the Created At date column and click “by month” to show the rest of the granularity options, then pick “month of year.”
Step 5: visualize the result!
To see our results, we just click on the “Visualize” button at the bottom of the notebook. Metabase will automatically show our results as a bar chart, with a different series for each year, broken out by each month of the year.
If you’d rather see this as a line chart, you can click the Visualization button in the bottom left of the screen to see your options.
Taking this further
You’re not limited to doing year-over-year comparisons with this pattern. You can compare any metric over an arbitrary period of time to another, just by changing the dates within the between function, and the granularity of the date grouping. For example, we could count the rows in our users table during March of 2020 and compare that to users in March of 2019 by writing two expressions like this:
We’d then break these two metrics out by Created At with the granularity set to “Day of Month” in order to overlay the two series on top of each other. Here’s the result, visualized as an area chart:
We hope that this opens up all sorts of new possibilities for you and your team, especially for folks who are maybe comfortable with Excel but not so comfortable with SQL. If you’d like to show us what you’re doing with version 0.35, go ahead and say hi to us on Twitter— we’d love to see what you’re up to.
Building effective dashboards often involves providing context for folks to give them insight into how or why a calculation exists. While precise titles, descriptions, and labeled axes can go a long way toward clarifying your visualizations, Metabase also allows you to add flexible text boxes to your dashboards. This post will show you how to take full advantage of this feature using Markdown syntax.
Adding text to dashboards
Select a dashboard you’d like to add text to, and click on the pencil icon to edit the dashboard. Click on the Aa button in the dashboard toolbar. A text box will appear, which you can move around just as you would a question box. To resize the text box, click and drag on its bottom right corner.
Writing in the text box
To interact with a text box, you’ll need to be in edit mode in a dashboard. Click on the pencil icon in the dashboard toolbar in the upper right of the dashboard.
Text boxes in Metabase have two modes.
Edit text (paper and pencil icon).
Rendered view (eye icon).
The text editor in Metabase employs a lightweight markup language called Markdown. If you’ve never used Markdown before, it takes some getting used to, but you’ll soon learn to appreciate its simplicity. Markdown can make writers feel like coders and coders feel at home.
How Markdown works
You write a line of Markdown:
# Heading
And Metabase converts your plain text into formatted HTML code.
<h1>Heading<H1>
Which renders like this:
Heading
You can access six levels of headings just by adding more #s.
You can use Markdown syntax to add links, images, gifs, lists, codeblocks, blockquotes, and more. Here’s a text box with heading, paragraph, blockquote, and code block:
You can do everything this post does and more using text boxes in Metabase. The big deal with Markdown is that you don’t have to write tedious HTML, and Markdown is human-readable even when it’s not rendered. Its minimalist feature set will keep you focused on the content, and provide a standardized look across your dashboards.
Here is an example dashboard with a question and text box:
Here is the Markdown code used in the text box above:
# Analysis
Although **Gadgets** outsold **Gizmos** in 2019, we only introduced **Gizmos** and **Doohickeys** in September of 2019. Additionally, both **Gadgets** and **Widgets** were heavily discounted during our spring, summer, and holiday sales.
We expect sales to continue to grow in the **Gizmo** and **Doohickey** product lines.
# SQL query
SELECT "PRODUCTS__via__PRODUCT_ID"."CATEGORY" AS "CATEGORY",
sum("PUBLIC"."ORDERS"."QUANTITY") AS "sum"
FROM "PUBLIC"."ORDERS"
LEFT JOIN "PUBLIC"."PRODUCTS""PRODUCTS__via__PRODUCT_ID" ON
"PUBLIC"."ORDERS"."PRODUCT_ID" = "PRODUCTS__via__PRODUCT_ID"."ID"
WHERE ("PUBLIC"."ORDERS"."CREATED_AT">= timestamp with time zone '2019-01-01 00:00:00.000Z'
AND "PUBLIC"."ORDERS"."CREATED_AT"< timestamp with time zone '2020-01-01 00:00:00.000Z')
GROUP BY "PRODUCTS__via__PRODUCT_ID"."CATEGORY"
ORDER BY "sum" ASC, "PRODUCTS__via__PRODUCT_ID"."CATEGORY" ASC
# Contact
If you have questions, reach out to us on the [\#product](https://fakemessageservice.com/product) channel.
The hypothetical analyst provided some context, code, and contact info: but you can include whatever context will help readers of your report.
Note: in the example above, the analyst pasted the raw SQL query generated by the notebook editor for reference. You can view the SQL composed by the notebook editor by clicking on the console icon to “View the SQL” when in editing mode.
One last pro tip for gif aficionados
The image syntax,
also works for gifs. Because there are far more important use cases for dashboard text boxes:
In fact, you don’t even need to include data in your dashboards; you could just go full gif:
In mathematics, expressions are collections of symbols that together express a value. If you’ve ever worked with spreadsheet software before, expressions are formulas, like =SUM(A1, B1).
Custom expressions in Metabase’s notebook editor are powerful tools that can cover the vast majority of analytics use cases without the need to take SQL out of the toolbox. In fact, there are big advantages to using Metabase’s notebook editor that you don’t get when you use SQL:
Drill-through. Drill-through allows people to filter, break out, zoom in, or x-rays the data on click. Metabase Enterprise adds even more functionality by allowing you to customize the drill-through experience so that you can determine what happens when people click through your charts. Drill-through is not available for questions composed in the SQL editor.
Extensibility. Using the notebook editor to build queries allows those with whom you share your questions to learn from and build on your questions without needing to know any SQL.
And you can always switch to SQL at any point during your development by converting an existing notebook editor question to a native SQL question.
Using custom expressions
There are three places in the notebook editor where we can use custom expressions:
Custom columns use functions to compute values (like + to add numbers) or manipulate text (like lower to change text to lowercase).
Custom filters use functions like contains that evaluate to either true or false.
Custom summaries use functions like count or sum to aggregate records.
Custom columns
We can add a custom column to our data using an expression to compute a new column.
Let’s see an expression in action. Here is the Orders Table from the Sample Data included with Metabase.
Let’s say we want to know the discount percentage applied to orders based on the pre-tax subtotal.
For example, if we gave a $1 discount on a $10 order, we’d want to see a column show that we had discounted that order by 10%.
Unfortunately, a quick scan of the columns in the preview tells us that the database does not store that computation (i.e. there is no “discount percentage” column). We only have the subtotal of the order, and the discount total.
Thanks to math, however, we can use the discount total and order subtotal to compute the percentage. Here’s where expressions come into play: we can use an expression to compute the discount percentage for each row, and store that computed value in a new column.
Let’s walk through how to create a custom column.
When in the notebook editor, we select Add custom column in the Data section.
To calculate the discount percentage, we’ll need the original total (i.e. the subtotal + the discount), and then we’ll need to divide the discount by the original total to get the discount percentage.
In expressions, we reference columns using brackets. For example, we can refer to the Discount column in the Orders table as [Discount]. If we need to reference a column from another table linked by a foreign key, we can use a . between the table and the column, as in [Table.Column] (alternatively you can select [Table -> Column] from the dropdown menu that appears when you type an open bracket ([). For example, we could enter [Product.Category] which will resolve to: [Product → Category].
For now, we’re just interested in columns in the Orders table, so there’s no need to reference another table. Here’s the expression (or formula), we’ll use to compute our custom discount percentage column:
= [Discount] / ([Discount] + [Subtotal])
Enter that expression in the field formula, then give the new column a name: “Discount Percentage”.
Click done, then click Visualize to see your new column.
Since the value in our new Discount percentage column concerns discounts, let’s move the column next to the Discount column. You can move columns around on tables by clicking on the column header and dragging the column to your target location, like so:
Since we’re computing a percentage, let’s fix the formatting so it’s easier to read. Click on the Discount percentage, and select Formatting.
Metabase will slide out a sidebar with formatting options.
Let’s change the Style to Percent, and bump up the number of decimal places to 2. And since the title Discount Percentage takes up a lot of space, let’s rename the column to Discount %.
There’s an option to add a mini bar chart as well. This bar chart won’t show the percentage with respect to 100%; instead the mini bar chart will show us the discount percentage relative to the discount percentage given to other orders. Let’s leave the mini bar chart off for now.
Here’s the finished question with the added Discount % column:
Custom filters
Metabase comes with a lot of filtering options out of the box, but you can design more sophisticated filters using custom filter expressions. These are particularly useful for creating filters that use OR statements, and that’s what we’ll be covering here.
Normally in the notebook editor, when we add multiple filters to our question, Metabase implicitly combines the filters with an AND operator. For example, if we add a filter for products that are Enormous and a filter for products that are Aerodynamic, our question will only return products that are both Enormous AND Aerodynamic, which (enormous, aerodynamic products) do not exist in Metabase’s Sample Dataset.
To filter for products that are either Enormous OR Aerodynamic, we’ll select Custom Expression from the Filter dropdown, and use the contains function to check if the product has either Enormous or Aerodynamic somewhere in the title.
contains(string1, string2)
contains checks to see if string1 contains string2 within it. So string1 is the string to check (the haystack), and string2 is the text to look for (the needle). And since we want to look for either Enormous or Aerodynamic products, we can write two contains expressions with an OR operator in between:
= contains([Title], "Enormous") OR contains([Title], "Aerodynamic")
The resulting data set will contain products that are either Enormous or Aerodynamic:
Note that custom filter expressions must always resolve to either true or false. In the case of the contains function used above, the expression evaluates as true if the title has either
Enormous or Aerodynamic in it, otherwise the expression evaluates as false.
You can, however, nest expressions that do not resolve to true or false within statements, like:
because the outermost function (contains) resolves to either true or false. Whereas you couldn’t use concat([First Name], [Last Name]) as a filter, as it would resolve to a string of text (though you could use concat to create a custom column like Full Name).
Custom summaries
Custom expressions unlock many different ways to aggregate our data. Let’s consider the Share function, which returns the percent of rows in the data that match the condition, as a decimal. For example, say we want to know the total percentage of paper products in our product line, i.e. what share of our product line is composed of paper products?
To start, we’ll select the Products table from the Sample Dataset. Next, we’ll click on Summarize in the notebook editor, and select Custom Expressions. Then, we’ll select Share from the dropdown menu, which will prompt us for a condition. In this case, we want to know which products have “Paper” in their title, so we’ll use the contains function to search through Title.
= Share(contains([Title], "Paper"))
Then we name our expression (e.g., Percentage of Paper Products) and click Done. Click Visualize, and Metabase will compute the share of paper products.
To change the formatting, select Settings in the bottom-left to bring up the settings sidebar, and change the Number options->Style to Percent.
Putting it all together
Let’s create a fairly complex (contrived) question using expressions. Say we’ve been tasked to find the average net inflow for wool and cotton products by month in 2019, with net inflow being the selling price minus the cost we paid for the product. In other words: for each wool and cotton product unit sold, how much money on average did we make (or lose) per unit each month in 2019?
To the get these fascinating numbers, we’ll need to use expressions to:
Compute the selling price per unit (custom column)
Filter results to only include wool or cotton products (custom filter), and limit those results to 2019.
Compute the average net inflow (custom summary), and group by month.
Let’s go:
We create a custom column, named Unit price. To compute the Unit price, we’ll use an expression to divide the subtotal by the number of units sold (Quantity):
= [Subtotal] / [Quantity]
Next, we’ll use a custom filter expression to filter for orders of Wool and Cotton products (i.e. for products that contain “Wool” or “Cotton” somewhere in their Product.Title).
We’ll also filter for orders between 01/01/2019 and 12/31/2019.
We’ll use a custom expression to create a custom summary. Let’s assume the standard retail markup of 50% (the keystone markup). So if the Product.Price is $2, we’ll assume the product cost us $1 to acquire per unit. Given this assumption, we can simply define net inflow per unit sold to be the Unit price minus half of the Product.Price. Then we’ll summarize that data by taking the average of those numbers for each order.
= Average([Unit Price] - [Product → Price] / 2)
Lastly, we’ll group those orders by Orders.Created_At by month.
Here’s our notebook:
And lo, our chart:
Plus, since we composed this question using the notebook editor, we can drill through our data:
Metabase version 0.36 marks another big release from the Metabase team, with help as always from our contributors and the community.
If you’re ready to get going, you can start with our upgrade guide, or read on to learn what’s new.
New features and improvements
SQL snippets in the native query builder
Highlight SQL code and click to save as snippet.
You can now save bits of SQL and refer to them in your SQL editor queries via a tag, like {{ snippet: My Great Snippet }}. Updating the definition of a snippet will update every query that uses that snippet. Just click the icon with the three lines in the native query editor to open up the SQL snippets sidebar.
Find and edit snippets in the Snippets sidebar, and insert them into your query.
Any user with SQL editor permissions for at least one database can use, create, and edit snippets.
Instead of changing the language in the UI for each user based on their browser’s language settings, Metabase now lets admins pick the language for the instance on the Localization tab of the Settings page in the Admin Panel. Users can also change their personal language preferences from their Account Settings page.
Better value labels for line, area, bar, and combo charts
You can now turn on value labels for charts with multiple series, and selectively turn labels on or off for individual series.
Toggle value labels for individual series.
A reorganized Data Model section
We’ve moved things around a bit in the Data Model section: there are now tabs at the top to see all of the Segments and Metrics you’ve defined across your whole instance, so you no longer need to drill down through your tables to find a Segment or Metric.
Data, Segments, and Metrics tabs in the Data Model section.
Reorderable table columns in the Data Model
You can now change the default order for table columns, and choose between database order, alphabetical, custom order, or the old “smart” ordering. You can also click and drag on columns to reorder them manually.
Drag to reorder columns.
Improvements and bugs fixes
See the release notes for a list of all the improvements we made (and all the bugs we squashed).
Thanks to all who submitted issues, pull requests, and suggestions: you are a big part of why Metabase continues to get better and better.
Stay healthy and safe, everyone, and enjoy the new release!
SQL Snippets are one of our most anticipated features among our power users. If you’re just tuning in, here’s why: you can now save SQL code as a snippet, and then you or other SQL authors can refer to that snippet in different SQL queries. If you ever need to update that code, you can edit the snippet, and those changes will propagate to all questions that use that snippet. Any user with SQL editor permissions for at least one database can use, create, and edit snippets.
Highlight SQL code and save it as a snippet.
Snippets are a simple but powerful feature, so let’s unpack them. We’ll also cover an upcoming Enterprise feature: Snippet Folders. These folders and their permissions will help teams keep their snippets organized, and they’ll be shipping with our upcoming 1.36 release of Metabase Enterprise Edition.
First, a simple example
Take this short join statement:
orders AS o
LEFT JOIN products AS p
ON o.product_id = p.id
You can highlight that code, name the snippet “Orders and products”, add a helpful description, and summon that snippet in any SQL query using the snippet: tag, like so:
SELECT *
FROM {{snippet: Orders and products}}
That’s all it takes to create and use a snippet. Now let’s dig in to why they’re so useful.
Why use SQL Snippets?
There are three main use cases for SQL snippets:
Standardization. How does your organization define a popular product? Is it by number of units sold? Or by reviews with an average rating greater than 4? You can define those qualifications for a popular product and codify them in a SQL snippet, {{snippet: popular products}}, and have that code populate in every question that uses that snippet. If down the line this definition needs to change, simply update the snippet’s SQL, and the change will propagate to all questions that use that snippet. Similar to how segments (a named filter or set of filters) and metrics (a named computation) can standardize analytics in your organization, SQL snippets offer another way to ensure correctness and consistency across teams.
Sloth (a.k.a. efficiency). Find yourself copying and pasting SQL code often? Don’t want to bother remembering which foreign keys map to which tables? Write that complicated join once, save it as a snippet, and summon the snippet as needed.
Education. Snippets can level up folks who are new to SQL (or even experienced analysts) by exposing them to your organization’s “canonical SQL,” or to more efficient or more complex queries. Reading, copying, and building upon quality code is one of the best ways to develop skills. It can also save your organization time: people can copy a snippet’s code, modify it to obtain different results, and save that code as a new snippet for others to use.
SQL snippets in action
Snippet sidebar
Metabase’s SQL editor has a sidebar for viewing, creating, and editing snippets, which you can access by mousing over to the right side of the editor and clicking on the snippet icon (three uneven horizontal lines).
Access the Snippets sidebar by clicking on the snippet icon to the right of the SQL editor.
Once you create snippets, you can insert them in the editor using the snippet tag: {{snippet: Orders and products}}, or by selecting them from the snippet sidebar:
Previewing and inserting a snippet from the Snippets sidebar.
Clicking on a snippet’s name in the sidebar will insert it at your cursor’s current location in the query.
Editing snippets
Updating the SQL code of a snippet will update every query that uses that snippet, so make sure to test your code before saving it to avoid breaking people’s questions. Of course, if you make a mistake and cause chaos, you can simply correct the code, save the snippet again, and restore order to the universe.
Note that if you use aliases in your snippet (e.g., orders AS o), you’ll need to use those aliases outside of the snippet to refer to that data in your query.
Archiving snippets
You can archive a snippet via the Snippet Edit modal.
Snippets are indestructible; you cannot delete them. You can, however, archive (and unarchive) snippets. Archiving a snippet will 1) keep your Snippets sidebar tidy, as archived snippets do not populate the sidebar, and 2) prevent the archived snippet from appearing in the typeahead results in the SQL editor. I.e., the archived snippet won’t show up as an autocomplete option when you type {{snippet: in the SQL editor.
Archiving a snippet does not affect questions that use that snippet, so you can safely archive a snippet without breaking anyone’s questions.
Snippet folders in the Enterprise Edition
The upcoming Enterprise Edition of Metabase, 1.36, will include snippet folders and permissions to help keep large numbers of snippets organized.
Snippet folders and permissions
Similar to a file system, admins can add snippets to folders, and place folders within folders.
Saving a snippet to a folder.
Additionally, admins can add permissions to those folders, which control who can view or edit snippets in those folders:
Changing a folder's permissions.
Permissions are granted to groups for each folder, with three permission levels:
No access. Cannot view snippets in the sidebar and autocomplete menus. People without access can, however, run queries that include these snippets.
Archiving and unarchiving snippets has no affect on permissions, though you’ll need edit access to the folder to archive and unarchive its snippets.
Use folders to keep snippets organized and standardized among and between teams, as well as to keep sensitive snippets visible only to the appropriate groups.
Go deeper
For even more on SQL snippets, check out our detailed documentation:
So you need to provide analytics to your customers. You’d like to put dashboards and charts in your app, make sure nobody can see data that isn’t theirs, and put that in front of your customers or users. These customers may be other businesses like vendors, regulatory agencies, or other organizations. Regardless of the audience, they need reporting. How should you go about it, and how would you use Metabase to get the job done?
An example dashboard using the Sample Dataset included with Metabase.
Some context: a lot of companies come to Metabase saying that they want to embed analytics in their application, asking if Metabase can do that. The answer is yes, it can, and that might indeed be the right option for that company. But that’s not the right question.
In this article we’ll lay out the map for what’s possible with Metabase, and clarify some muddy vocabulary to help surface the options available for shipping analytics with Metabase.
The goal: delivering analytics
First, let’s be especially clear on the goal: what we are talking about is delivering analytics to customers. Can you give customers the data they need to understand their business and make better decisions? To that end, embedding is simply one strategy, not the goal. It can be easy to make the wrong (i.e., needlessly expensive) decision by conflating the two.
The bar for high quality external analytics
If delivering analytics is the goal, then what is the ideal delivery? That depends (and we’ll unpack our options below), but fundamentally the solution that ships should:
Give customers the ability to analyze their own data.
Restrict their analysis to only the data they’re authorized to see.
Provide a great user experience.
With Metabase Enterprise Edition, it takes surprisingly little effort to deliver on all three of the above requirements, but there’s more than one way to get the job done. We’ll go through four strategies, ranging from low effort with big payoffs to high effort with staggering customizability, so you can get a better understanding of the advantages and tradeoffs of each approach, and land on the right strategy for your use case.
For most companies, we’re also going to make a case for the simplest option, because for the most part, escalating the effort only really moves the needle on the user experience.
Four strategies for delivering customer-facing analytics with Metabase
The order below situates the strategies from low effort (1) to high (4):
We’re calling our first strategy the “sidecar” because Metabase sits alongside your application. You don’t have to embed Metabase in your app, nor are you setting up a standalone Metabase application just for the external folks you want to provide analytics to. Instead, this approach employs single sign-on, data sandboxing, and white labeling (branding) to provide the experience across your application and your existing Metabase instance.
Here’s how it works: you stand up an instance of Metabase, add your logo and brand colors, and hook up your databases. You then sandbox the data so that customers only see what they are authorized to see, and authenticate users with single sign-on (SSO) so that both your app and your Metabase instance can coordinate users’ permissions. (We’ll go more in depth about how these various technologies work together in an upcoming post.)
Metabase sidecar: instead of embedding Metabase in your app, you coordinate users between your app and your Metabase instance, and send users to your Metabase instance to view their dashboards, charts, and more.
With your branded Metabase sidecar set up, all you need to do is put links to your Metabase instance in your app, and your customers will have beautiful, on-brand dashboards and charts that they can explore themselves with Metabase’s easy-to-use query builders and drill-throughs. Your customers get great UX out of the box, and can do their own analysis on top of whatever stock dashboards you’ve set up for them. Everyone is the hero in this story, without any embedding.
The query builder in Metabase that your customers can use to ask questions of their data. The query above filters the Orders table by the Widget category, counts the rows, and groups those counts by month.
The advantages of the sidecar approach are speed of setup and deployment, and also (and perhaps more importantly) that you don’t have to anticipate the kinds of questions your customers will ask of their data. With Metabase, you’ll have delivered powerful analytics primitives via Metabase’s notebook editor, which your customers can use to define and save their own questions. You can still create stock dashboards to provide a starting point, or an editorial take on the data, but the point is you don’t have to.
Additionally, choosing to stand up a Metabase app as a sidecar to your app derisks the project. A Metabase instance is a prerequisite for options 2 and 3 anyway, and if, during the process of setting up your Metabase application, you find that it doesn’t fit your needs, you won’t have wasted any time figuring out how to incorporate Metabase into your app.
Advantages of a Metabase sidecar
Delivers on all three requirements for high quality analytics.
Quick to set up.
Easy handoffs.
Really powerful analytics primitives for your users.
Derisks the process.
Tradeoffs of a Metabase sidecar
What you don’t get: a lot of customizability. You’re just using Metabase’s UI and functionality.
2. Metabase full app embedding
Embed the full Metabase application in your app to integrate charts, dashboards, collections, query-builders, and more.
One rung up on the effort ladder, full app embedding is the most popular strategy among our customers. You get a lot more control over the experience when you embed Metabase in your app, because you can design the experience around specific dashboards and charts. You can specify a set of screens for customers to navigate through, each with embedded dashboards and charts, and you can link those screens or dashboards together. And because you have the full Metabase app at your disposal, you can also expose the query builder so that users can ask their own questions of their data.
The amount of effort you put into full app embedding can vary widely, depending on how extensive of an experience you want to build. If your set of screens is small and straightforward, it may not be that much more work than a sidecar implementation. But since you are in control of your app, you can (with more effort) create a potentially much richer experience for your customers.
Advantages of Metabase full app embedding
You can embed any Metabase screens, dashboards, or charts in your app, with full interactivity.
Much more customizable, as you can create richer experiences in your app by embedding and customizing links between Metabase dashboards, charts, or pages.
Tradeoffs of Metabase full app embedding
It’s more work: you’ll need to design and implement the pages that house the embedded Metabase screens.
For now, you can only change Metabase’s color scheme, so there may be some aesthetic variance between the design of your app and the embedded Metabase components (like the fonts on a chart).
3. Fork the Metabase source code
Metabase Enterprise Edition will soon be source available, so you’ll be able to fork the code and enjoy unlimited power. Don’t like the fonts? Chart design? Grid system conflicts with your grid? You’ll have access to the CCS, the graphing primitives - the full source code. Go nuts.
You may go this route if you really want to dial in the design of your embedded Metabase app to create a seamless experience. And even if you never take this route, it’s nice to know you have that runway available to change the design or add functionality if you ever have an analytics challenge that the stock Metabase doesn’t address.
Advantages of forking the Metabase source code
Granular control over the user experience for seamless integration with your app.
Ability to add functionality to dashboards, charts, and more.
Full control over the design of Metabase and its charts.
Tradeoffs of forking the Metabase source code
You have to maintain a fork; as new Metabase versions come out, you’ll have to integrate those changes into your fork.
It’s a lot of work. A lot less work that rolling your own analytics platform, but significantly more than full app embedding.
4. Roll your own analytics platform
Too many companies take on the work of building their own analytics software, often for reasons that are unclear. These companies spend a lot of time and money (on the order of millions to tens of millions of dollars) to build sub-par tools that Metabase outclasses out of the box, when they could be applying those resources to their core business and development.
With that said, there are legitimate reasons for building your own analytics platform. In this scenario, you likely already have a third-party analytics solution in place, but it’s falling short of meeting your needs. You know exactly what you need to build (and why you need to build it), and have the engineering resources to commit to the project. For example, you might have a permissions problem that you need to solve that Metabase (or another offering) can’t account for that is more than a simple matter of adding functionality and submitting a pull request. Just make sure the problems you intend to solve by writing your own platform are mission critical, and that existing offerings lack the features to handle them.
Advantages of rolling your own analytics platform
Full control over the experience, without having to maintain a fork.
Handle use cases other solutions (like Metabase) cannot solve for.
Tradeoffs of rolling your own analytics platform
Most expensive option, with murky return on investment.
Which option is right for your organization?
Most companies choose full app embedding, as they want the extra customizability to provide a more curated set of screens for their customers. But one purpose of this article is to make it clear that embedding is not the only option for delivering analytics with Metabase. It might not be the best place to begin, especially if you’re a startup trying to make the most out of limited resources. In that case, take a good look at the sidecar approach — setting up Metabase as a sidecar to your application is the fastest way to deliver value to your customers. You can always transition to an embedded solution as you iterate, but in the meantime you’ll already be giving your customers the ability to make data-driven decisions, and you’ll develop a much better understanding of the analytics they need.
For larger companies, forking the source code might make sense. It’s more of an investment, but you’ll be able to deliver a much more cost-effective solution than if you were to take on building an analytics platform from scratch.
And, of course, if you’re not up for maintaining a fork, Metabase accepts pull requests! Take advantage of Metabase’s built-in feature set, submit a PR for that last bit of functionality you need to complete your analytics offering, and you’ll have the custom solution you need at a fraction of the cost it would take to build your own platform, all while improving Metabase for everyone.
Stay tuned
In an upcoming post, we’ll provide an overview of the challenges to delivering multi-tenant, self-service analytics, and the technologies Metabase offers to solve for those challenges.
So, you’re considering embedding Metabase in your application to deliver analytics to your customers. How would that work?
In this article, we’ll:
Start with the target experience for your customers that you’ll deliver.
Give a brief tour of the challenges to providing self-service reporting to multiple tenants.
Talk through the relevant features Metabase offers, and how they work together to solve those problems.
The target user experience
To level-set, let’s start with what the finished analytics experience will look like from the perspective of your customers.
The players:
Your company: Megafauna Analytics
Customer 1: MammothCo
Customer 2: RhinoCo
The story:
Say you’re a B2B company, Megafauna Analytics, and you’re using Metabase to provide reporting to your customers by embedding Metabase in your app at megafauna-analytics.com.
Here’s an overview of the customer experience: Woolly from MammothCo logs into your app at megafauna-analytics.com, then navigates to the analytics page you’ve set up for MammothCo. There, Woolly is treated to a set of screens featuring polished dashboards and charts showing MammothCo orders, transactions, and other sandboxed data you’ve provided, all of which feature your - Megafauna’s - brand colors.
More than just viewing static dashboards, Woolly can click on data plotted on the charts and drill-through. For example, he could zoom in on a particular week of transactions, or click on a bar in a chart to pull up a table listing the unaggregated rows that populate that bar.
An example of drilling through a question in Metabase using the Sample Dataset included with your installation.
Plus, with the Metabase’s graphical query builders, Woolly can slice and dice MammothCo data to his heart’s content. He can use the editor’s powerful querying primitives, as well as custom expressions for more sophisticated queries, to find answers to questions your dashboards don’t address — all without seeing any data from RhinoCo or other customers.
The query builder in Metabase allows people to explore their data and create new questions. They can join data, filter it, summarize it, sort it, use custom expressions, and more.
So that’s the story. How do we get there?
The challenges to creating multi-tenant, self-service analytics
By multitenancy, we mean a shared application instance where groups of people (groups like MammothCo or RhinoCo) enjoy access to different permissions, and therefore to different data. And by self-service, we mean that the application should give people the ability to create their own reporting, in addition to whatever reporting administrators provide.
Before we get into how Metabase delivers multi-tenant, self-service analytics, let’s first identify the problems we’ll need to solve. To give Woolly the experience outlined above, we’ll need to:
Embed Metabase in your application (megafauna-analytics.com) in a way that gives people access to a query builder and drill-through functionality.
Make the dashboards and charts match your (Megafauna’s) brand colors.
Tie a given user in megafauna-analytics.com to a user in your embedded Metabase instance.
Limit the data the users can see to just the rows and columns they’re allowed to view.
Now let’s go through the four features Metabase offers that combine to solve these problems.
The four features necessary to set up multi-tenant, self-service reporting
We’ll go through each feature, but here they are at a glance:
Full app embedding. Add drill-through to embedded charts and dashboards, curate collections, expose a query builder pre-filled with a dataset, or even expose the entire Metabase interface.
White labeling (branding). Make the Metabase charts and dashboards in your app match its look and feel.
Single sign-on (SSO). Enable your app and Metabase to agree on who their users are and what data they can see.
Data sandboxing. Let your users explore their data, without seeing anyone else’s data.
Full Metabase app embedding
Though our focus will be on full app embedding, there are several ways to handle embedding Metabase in your app. And because “embedding” is such an overloaded term, it’s worth going through the different ways you can use Metabase to embed charts and dashboards in your app.
Let’s start with the open source options:
Metabase embedding options
Public link embedding
The dead simple option (and it may not even technically qualify as embedding). You put up a question or dashboard in a Metabase instance, send someone a link to it, and tell them not to share it.
Public embed
One step up, as the link is still public, but the chart is visible in your app via an embedded iframe.
Signed embed
Same as the public embed, but now it’s an iframe secured by a signed JSON Web Token (JWT). To set up secure embedding, you’ll need an iframe with a link to the question or dashboard on the front end, as well as server code on the back end to create the token. You can then sign tokens with a user ID, and do some cool things via parameterization.
The signed embedding wizard. Signed embeds require publishing. You can change the chart's style, and activate parameterization so people can filter the data based on a specific field (in this case, by Customer ID).
For example, you can create filters on dashboards that can accept parameters, and filter that data in the dashboard based on those parameters. You could have a parameterized dashboard that accepts a user ID, so when that user logs in, the dashboard would only show users data restricted to their user ID. In other words, you can create a generic dashboard for user data that any user can access, but each individual will only see data relevant to them.
The big drawback with secure embedding is that customers will not be able to drill through the data. And the reason drill-through is unavailable is because Metabase does not know what data permissions the user has access to. Metabase may know that the user is user_id: 13, but there’s no connection between that user 13 and a user account in Metabase (and therefore the groups and permissions the user has access to). To link accounts across your application and your Metabase instance, we’ll need full app embedding and single sign-on.
Full app embedding, in combination with SSO and data sandboxing, is what makes the self-service reporting possible. Like the simpler embedding options above, you’re still using an iframe to embed Metabase, only this time you can embed any Metabase screen you want, complete with full interactivity. Combined with SSO and data sandboxing, your embedded Metabase instance will know who Woolly is and what data he can view, which means Metabase can let Woolly drill-through his data without being able to see any data from RhinoCo users or other tenants.
And because you have the full Metabase app at your disposal, you can provide MammothCo users with collections, dashboards, and predefined questions, such as week over week transactions. You can also expose Metabase’s notebook editor, which would allow customers like Woolly to create their own questions and dashboards (and annotate them with Markdown) based on their evolving needs for data.
Metabase’s white labeling feature allows you to change the colors and branding of Metabase to match your company’s application. It’s one of those small, simple changes that can make for a much better user experience.
In the Admin panel, you can change Metabase's logo, colors, and more to match the design of your application.
The goal with single sign-on (SSO) is to associate users in your app with users in Metabase. By using SSO, you avoid the awkward login screens where your charts should be. Without SSO, a user would log into your app, but would still have to log into your Metabase instance to view individual embedded dashboards and charts – a less-than-ideal user experience.
How SSO works with Metabase
Let’s say you set up a MammothCo user in your app, megafauna-analytics.com, with name: Woolly; user_id: 13. To coordinate with your Metabase instance, you would then (manually or programmatically) set up a user account for Woolly in your Metabase app. Next, you need to associate Woolly’s ID in your app, megafauna-analytics.com, with Woolly’s account in your Metabase instance. To do that, you simply add an attribute to Woolly’s account in Metabase to indicate that Woolly’s user_id in your app is 13.
Add attributes to people in Metabase to coordinate user permissions between your app and your embedded Metabase instance.
With SSO set up, when Woolly logs into megafauna-analytics.com, a signed token is generated that attests to his identity and the attributes you’ve set for him in your Metabase instance. When Woolly navigates to pages with embedded Metabase charts or dashboards, Metabase can tell from the user_id attribute attached to the token that Woolly is user 13 and should only be able to see data you’ve made available to that user ID.
That’s the basic idea, and there are several ways to set it up:
For internal analytics, companies usually use SAML, LDAP, or Google Sign-in. For external analytics, companies usually prefer (and we recommend) JWT SSO.
How you want to handle authentication is up to you, but you must set up SSO in order to coordinate permissions across your application and your Metabase instance.
Now that your SSO is providing Metabase with details about your users, those attributes can be used to filter what each user sees, down to the columns and rows of specific tables. This is data sandboxing, and it’s a powerful tool, so we’ll only be able to give an overview of how it works.
For example, if your orders table contains all customer orders, you could filter that table based on a column, say user_id. You can assign an attribute, user_id, to your users in Metabase (and attach that attribute to their login tokens), so that when users log in and view orders, they’ll only see data with rows that match their user_id.
Sandboxing the Orders table by filtering the orders by the User ID column. Users will only see orders that correspond to their user_id attribute.
If you need even more control, you can create custom views of tables with saved questions. For example, let’s say you don’t want ANY users seeing a specific column in the orders table. In this case, you could create a custom view of that table with the relevant column omitted by asking and saving a question in Metabase.
We hope this article gave you a high-level understanding of how you could deliver self-service reporting with a multi-tenant Metabase instance embedded in your web app. Here’s a quick summary with links to our documentation if you’d like to learn more about a particular topic.
Set up SSO (SAML, JWT) to link users in your app with your embedded Metabase instance, and attach user attributes to signed tokens.
Use the attributes to sandbox the data to ensure users can only see the data they have permissions to view.
Full app embedding, as outlined above, is a great option for most companies, but it’s not the only option for providing self-service reporting. See how to use Metabase to deliver analytics to your customers for an overview of the different ways you can use Metabase – from a simple setup (no embedding necessary) to setups that offer staggering customizability – to find the approach that’s right for your organization.
Metabase is scalable, battle-hardened software used by tens of thousands of companies to deliver high quality, self-service analytics. It supports high availability via horizontal scaling, and it’s efficient out of the box: a single core machine with 4 gigs of RAM can scale Metabase to hundreds of users.
This article provides high-level guidance and best practices on how to keep Metabase running smoothly in production as the numbers of users and data sources increase.
A single Metabase instance connected to multiple databases, as well as to its application database, which stores questions, dashboards, and other data specific to Metabase. You can easily add more Metabase instances as you grow.
Hosted Metabase. Let Metabase handle application operation for you.
Each data system is different, so we can only discuss scaling strategies at a high level, but you should be able to translate these strategies to your particular environment and usage.
Factors that impact Metabase performance and availability
Metabase scales well (both vertically and horizontally), but Metabase is only one component of your data warehouse, and the overall performance of your system will depend on the composition of your system and your usage patterns.
Major factors that impact your experience using Metabase include:
The number of databases connected to Metabase.
The number of tables in each database.
The efficiency of your data warehouse.
The number of questions in your dashboards.
For example, it won’t matter how many instances of Metabase you run if a question needs to run a query that your database(s) takes 30 minutes to run. That’s just going to take 30 minutes. The solution in that case is either to re-evaluate your need for that data (do you really need all that info every time?), or to find ways to improve the performance of your database, such as reorganizing, indexing, or caching your data.
The number of databases and tables can also affect client performance, but only in large-scale situations where you’re managing hundreds of databases and/or thousands of tables, as the metadata itself can be a lot to query. To help keep performance smooth even at this scale, you can manage when Metabase syncs its metadata with your connected databases.
Now let’s make sure your Metabase application is well-tuned to scale.
Vertical scaling
Vertical scaling is the brute force approach. Give Metabase more cores and memory, and it will have more resources available to do its work. If you are experiencing performance issues related to the application itself (i.e., unrelated to the breadth and magnitude of your databases), running Metabase on a more powerful machine can improve performance.
That said, Metabase is already efficient out of the box. For example, for a starter Metabase instance on AWS, we recommend running Metabase using Elastic Beanstalk on a t3.small instance, and scaling up from there. That’s a dual core machine with 2 gigabytes of RAM. Machines with with 4-8 gigs of RAM should handle hundreds of users, and you can bump the number of cores and gigabytes of memory if needed.
While adding more cores and memory can be effective, you’re generally better off using horizontal scaling to support more users. The reason is because there are database connectivity limits built into each Metabase instance to prevent the instance from overwhelming your data warehouse with requests. You can increase the number of connections available for your instance, but we still recommend multiple instances.
Horizontal scaling
Horizontal scaling involves running multiple instances of Metabase, in combination with a load balancer, to direct traffic to the instances. Metabase is set up for horizontal scaling out of the box, so you don’t need any special configuration to run multiple instances of Metabase.
The primary use cases for horizontal scaling is to improve reliability (a.k.a. “high availability”), but horizontal scaling can also improve multi-user performance. When the load is balanced, a high-traffic, CPU-bound Metabase instance will perform better (faster) when some of its traffic is directed to other instances, as the CPU load will be distributed across multiple machines.
Metabase ships with a local H2 database to store your application data (all of your questions, dashboards, logs, and other Metabase data), but when running in production you should upgrade to a relational database (like PostgreSQL) running on a separate server. In fact, when scaling horizontally, you must use a relational database that runs on a separate server to store your application data. That way, all instances of Metabase can share a common database. We recommend an external database on a separate server for all production instances, even if you only ever run one instance of Metabase, so an external database is not an added cost for horizontal scaling.
Metabase uses the external application database to store user session data, so people don’t have to worry about losing saved work if one or all Metabase instances go down, and administrators don’t have to deal with configuring sticky sessions to ensure people are connected to the right Metabase instance. The load balancer will route people to an available instance so they can keep right on working.
Taking advantage of time-based horizontal scaling
Some customers adjust the number of Metabase instances based on the time of day. For example, some companies will spin up multiple instances of Metabase in the morning to handle a burst of traffic when people log in and run their morning dashboards, then spin the extra Metabase instances down in the afternoon (or at night, or on the weekends) to save money on cloud spend.
Load balancers direct traffic to multiple Metabase instances to ensure that each request gets the fastest response. If one instance of Metabase goes down temporarily, the load balancer will route requests to another available instance.
Setting up a load balancer with Metabase is simple. Metabase’s API exposes a health check endpoint, /api/health, that load balancers can call to determine whether a Metabase instance is up and responding to requests. If the instance is healthy, the endpoint will return an HTTP status code 200 OK. Otherwise, the load balancer will know to route the request to another instance.
Architecting a data warehouse is beyond the scope of this article, but you should know that your queries in Metabase will only be as fast as your databases can return data. If you have questions that ask for a lot of data that takes your database a long time to retrieve, those query times will impact your experience, regardless of how fast Metabase is.
Here are three ways you can improve data warehouse performance:
Structure your data in a way that anticipates the questions people will ask. Identify your usage patterns and store your data in a way that make its easy to return results for questions common in your organization. Compose ETLs to create new tables that bring together frequently queried data from multiple sources.
Tune your databases. Read up on the documentation for your databases to learn how to improve their performance via indexing, caching, and other optimizations.
Filter your data. Encourage people to filter data when asking questions. They should also take advantage of Metabase’s data exploration tools (including record previews) so they only query data relevant to the question they’re trying to answer.
Database or data warehouse? Folks often get started with Metabase using a transactional database like MySQL or Postgres. While these databases scale quite well, they often aren’t optimized for the type of analytical queries that Metabase will use. Operations like sum or max can slow down once you reach a certain scale. As analytics adoption grows, you may find the need to explore dedicated data warehouses like Amazon Redshift, Google BigQuery, or Snowflake.
Metabase application best practices
Here are some strategies to get the most out of your Metabase application:
If people are running a lot of queries that return a lot of records, it won’t matter that Metabase is fast: the users will get their data only as fast as your data warehouse can return the requested records.
And sometimes people go overboard with dashboards, loading them up with 50 questions or more. When a dashboard with 50 questions loads, it sends 50 simultaneous requests asking for data. And depending on the size of that database, it can be quite some time before those records return.
Example dashboard with filter widgets using data from the Sample Dataset included with Metabase.
But that’s not the whole story. Metabase doesn’t slow down simply because you put more questions in your dashboard. If your questions don’t pull a lot of data, or your data warehouse can return results in under a second, 50 questions will load quickly.
In general, however, encourage your users to keep their dashboards focused. Dashboards are meant to tell a story about your data, and you can tell a good story with just a handful of questions (or even a single question). Take advantage of Metabase’s data exploration tools to learn about your data (such as the ability to preview records in tables) so you can dial in on only the records you need to answer your questions.
So make sure each question is necessary to complete the dashboard, and be especially mindful when querying data across time or space, as you can filter out a lot of unnecessary data by restricting your question to a shorter timespan or a smaller area.
Example question using a pin map to visualize latitude and longitude. Geospatial data can add up quickly, slowing query times, so it's important to filter for only the data you need.
Use a managed relational database to store your Metabase application data
The application database stores all of your questions, dashboards, collections, permissions, and other data related to the Metabase application. You can use an relational database (like PostgreSQL or MySQL) to manage your application database, but we recommend a managed solution like AWS RDS. RDS will automate backups and make it easy for you to adjust storage and compute as you scale, giving you one less thing to worry about. Managed database solutions are especially useful for Enterprise customers who take advantage of Metabase’s auditing functionality, as enabling auditing will increase the amount of data Metabase stores in the application database.
Metabase will show users the timestamp for the results, and users can manually refresh the results of the question if they want to rerun the query. Caching is suitable for results that do not update frequently.
Look for bottlenecks
Metabase’s Enterprise Edition offers auditing tools for you to monitor the usage and performance of your application.
See your most popular (and slowest) questions using the auditing tools.
You can, for example, see how many questions are being asked, by whom, and how long the questions took to run, which can help identify any bottlenecks that need attention.
Increase the maximum number of connections to the Metabase application database
The default number of connections to the Metabase application database is 15 (MB_APPLICATION_DB_MAX_CONNECTION_POOL_SIZE). If your usage regularly consumes all 15 connections, you can improve performance by increasing the maximum number of connections. Alternatively, you can increase the number of connections via horizontal scaling (e.g., if you add an additional Metabase instance, you effectively add an additional 15 connections to the application database).
You can check the number of connections by viewing the logs, and checking for lines with ... App DB connections: 12/15. In that example, Metabase is using 12 out of the 15 available application database connections.
Increase the maximum number of connections to each database
Similarly, the default maximum number of connections for a single Metabase instance to each database is 15. That’s 15 for each database, so if you’ve connected Metabase to two databases, you’ll have a maximum of 30 connections.
You can increase the maximum number of connections to each database by changing the MB_JDBC_DATA_WAREHOUSE_MAX_CONNECTION_POOL_SIZE environment variable. As above with the application database connections, you can also increase the number of connections via horizontal scaling. Each additional Metabase instance would increase the maximum number of connections by 15 (or by whatever maximum you’ve set). To learn more, see our documentation on environment variables.
Sync with your databases only when you need to
By default, Metabase performs a lightweight sync every hour. The sync does not copy any of your data. Metabase merely checks to make sure the list of tables, columns, and rows it maintains in its application database is up to date with the tables, columns, and rows in your databases.
You can change when Metabase syncs with your databases, which can improve performance when you're connected to especially large databases.
For large databases, you might consider limiting the number of times Metabase performs the sync, and restricting those synchronizations to off-peak hours, especially if you aren’t frequently adding new tables to your database.
If you don’t want to deal with the care and feeding of a Metabase application, Metabase will soon offer a hosted solution. You’ll still have to ensure your data sources are performant, but you’ll no longer have to manage running the Metabase application.
Visit our contact page to get in touch with us about our upcoming hosted offering.
Getting help
You can check out our Operations Guide in our documentation for more information on maintaining a Metabase instance.
If you still have questions, chances are someone’s already had the same question. Check out the Metabase discussion forum and search for your issue. If you can’t find a solution, submit a question of your own.
Figure 1. A Field Filter is a special type of variable that can wire up a variable in your SQL code to a field (column) in a table, which enables it to create a 'smart' filter widget.
For Metabase questions written in SQL, we can use basic variable types — Text, Number, and Date variables — to create simple filter widgets. To create “smarter” filter widgets that can display options specific to the data in the filtered column, such as to create a dropdown menu of values, we can use a special variable type called a Field Filter.
Figure 2. To create a Field Filter, add a variable to your SQL code by enclosing the variable in double braces (Mustache style), and select Field Filter as the Variable type from the Variables sidebar.
Field Filters can initially confuse some people, because 1) Field Filters only work with certain fields, and 2) people expect Field Filters to behave like basic input variables (which they don’t).
Field Filters are well worth learning, as you can use them to create much more sophisticated filter widgets. In this article, we’ll go through Field Filters in depth, but first let’s discuss the main practical differences that distinguish Field Filter variables from basic Text, Number, and Date variables.
Distinguishing Field Filters from simple Text, Number, and Date variables
Field Filters are optional by default. If no value is given, the SQL query will run as if the Field Filter didn’t exist. You do, however, have the option to require a value.
Field Filters won’t work with SQL aliases. Since Field Filters rely on metadata about columns in your tables (and the specific names of those columns), the filters can’t “know” that you’ve aliased a table or column.
Field Filters use a special syntax so they can handle SQL code behind the scenes. You simply supply a Field Filter to a WHERE clause, and the Field Filter will manage the SQL code for you, which allows the code to account for multiple selections people make in the filter widget.
Point #3 can be especially confusing, so let’s unpack it with an example where we create a filter widget with a dropdown menu.
Creating a filter widget with a dropdown menu
Let’s say we want to create a SQL question that grabs all the orders from the orders table, but we want to give people the option to filter the results by category in the products table.
Now, we could create a products.category filter with a basic input variable, like so:
SELECT *
FROM orders
LEFT JOIN products
ON orders.product_id = products.id
[[WHERE products.category = {{category}}]];
In this case, we enclose the WHERE clause in double brackets to make the input optional, and use the Variables sidebar to set the Variable type to Text and the Filter widget label to Category.
This approach works, but it’s not ideal:
In order to filter the data, people would have to know which categories exist (and spell them correctly when they input them).
Plus, they can’t select multiple categories at a time, as the {{category}} variable only accepts a single value.
By contrast, a Field Filter will map the variable to the actual column data. The filter widget connected to the variable then “knows” which categories are available, and can present a dropdown menu of those categories, like so:
Figure 3. A filter widget created by a Field Filter that's connected to the Category column in the Products table. The Field Filter allows the widget to display a dropdown list of the available product categories.
Here’s the syntax for the products.category Field Filter (note the omission of the = assignment in the WHERE clause – more on Field Filter syntax below):
SELECT *
FROM orders
LEFT JOIN products
ON orders.product_id = products.id
WHERE {{category}};
With our variable in place in the WHERE clause, we can use the Variables sidebar to wire up our variable as a Field Filter. we’ll set:
Variable type to Field Filter.
Field to map to to Products -> Category. This setting tells Metabase to connect the variable in our SQL code to the Category column of the Products table.
Field widget type to Category.
Field widget label to Category.
Figure 4. Setting a variable's type as Field Filter, and mapping that Field Filter to the Category field (or column) of the Products table.
We won’t require the variable, so no default value is necessary. If the query runs without a specified value in the filter widget, the query will return records from all categories.
Note that the WHERE clause does not specify which column the variable should be equal to. The reason for this implicit syntax (the hidden SQL code) is so that the Field Filter can handle the SQL code behind the scenes to accommodate multiple selections. If, for example, the viewer only selects one category Gadget, the Field Filter would supply the SQL products.category = 'Gadget' to the WHERE clause (note the single quotes around the argument ‘Gadget’). If the viewer additionally selected Gizmo, the Field Filter widget would (under the hood) update the SQL code given to the WHERE clause to be products.category = 'Gadget' OR products.category = 'Gizmo'.
Creating sophisticated date filter widgets
You can create a basic input variable of type Date, which will add a filter widget with a simple date filter. But by instead using a Field Filter variable, you can connect that variable to a field (column) that contains dates, which unlocks a lot more options for configuring your filter widget.
Figure 5. Setting the Field to map to option to a field containing dates will open up a range of Filter widget types: Month and Year, Quarter and Year, Single Date, Date Range, Relative Date, and Date Filter.
Here are the different widget types for Field Filters mapped to Date fields:
Month and Year
Quarter and Year
Single Date
Date Range
Relative Date
Date Filter
Each widget type offers different ways for people to filter the results: Here are three examples:
Figure 6. The Month and Year widget type.Figure 7. The Relative Date widget type.Figure 8. The Date Filter widget type.
The Date Filter widget type offers the most flexibility, allowing people to filter by relative dates and ranges.
Field Filter gotchas
There are a few places that people typically get stuck when trying to implement Field Filters.
Field Filters are incompatible with aliasing
As noted above, Field Filters won’t work if you use aliases in your SQL query. For example, this code (with aliases) will not work:
AVOID
SELECT *
FROM orders AS o
LEFT JOIN products AS p
ON o.product_id = p.id
WHERE {{category}};
Whereas this code without aliases WILL work:
CORRECT
SELECT *
FROM orders
LEFT JOIN products
ON orders.product_id = products.id
WHERE {{category}};
The reason is because Field Filters work by analyzing metadata about your data (e.g., the column names of your tables), and that metadata does not include the aliases you create in your SQL code.
Omit the direct assignment in the WHERE clause
As stated above: the SQL code around Field Filters is not exactly street legal. You may be tempted to write:
AVOID
WHERE category = {{ category }}
because, in standard SQL, that is the correct syntax for a WHERE clause. But that syntax won’t work for Field Filters.
The correct syntax for Field Filters does not include the = operator:
CORRECT
WHERE {{ category }}
The reason for this shorthand is so that Metabase can, behind the scenes, handle the SQL code for situations like multiple selections, e.g., when a user selects multiple categories from a dropdown.
Only certain fields are compatible with Field Filters
Field types not included in that list are incompatible with Field Filters. For example, it wouldn’t make sense to have dropdown list for ranges with large numbers of different values, such as emails, descriptions, latitude/longitude, or image URLs.
Figure X. Some fields are incompatible with Field Filters. For incompatible fields, you'll see the notice: There aren't any filter widgets for this type of field yet.
We’ll cover how to include filters widgets on dashboards, and how to wire them up to multiple questions so users can filter data across multiple questions at once.
This article walks through how to create questions in Metabase using native SQL queries so that viewers of our questions can plug in values and filter the results. While Metabase makes it easy to summarize and visualize data without SQL, data analysts sometimes need to dig into complicated queries, which they can write using Metabase’s SQL editor.
For example, using the Sample Dataset included with Metabase, we might write a question in SQL that pulls product information with our orders, but we want viewers of that question to specify the category of products that they want to view. To give people the option to input values on saved SQL questions, we can write SQL queries with variables, and Metabase will automatically create filter widgets that people can use to enter values.
Figure 1. An example question, written in SQL, that uses a basic text variable to power a filter widget. The widget allows people to enter text to filter records for product titles containing that text.
For now, we’ll just be focusing on filters applied to questions written in SQL. We’ll cover dashboard filters in an upcoming guide.
But first: do you want to write your question using SQL, or would a custom question better suit your use case?
SQL questions versus custom questions
Before we dig into adding filter widgets, it’s worth considering how people will use our question. If we just want to give people the option to plug in values to simple filter widgets, writing a question in the native query editor and adding variables to our SQL code makes sense.
If, instead, we compose a simple or custom question using the notebook editor, filter widgets are unnecessary, as viewers of our question will have the full suite of query building primitives available to slice and dice the data however they like – by joining, filtering, and summarizing the data. For more sophisticated questions, they’ll also have custom expressions at their disposal, as well as have the ability to drill-through the data to zoom in on orders, or click on values to view individual records – functionality that does not apply to questions written in SQL. (In Enterprise edition, however, you can customize what happens when people click on a chart.)
Figure 2. The notebook editor interface for composing custom questions. You can join, filter, summarize, sort, use custom expressions, and more.
If you can answer your question using the functionality in the simple or custom question builders, we recommend you use those. If, however, you need custom SQL commands or functions, and you want your users to be able to filter the results of those questions…read on!
With that said, let’s move on to setting up filter widgets on our SQL questions.
The different types of variables available for native SQL queries
Figure 3. A SQL question with a basic input variable of type Date. The Variable type dropdown menu shows the four variable types you can include in your queries: Text, Number, and Date, as well as a special variable type called a Field Filter.
As in Figure 3 above, for questions composed using Metabase’s native sql query editor, there are four types of variables you can select:
Text
Number
Date
Field Filter
One of these variable types, Field Filters, is unlike the others. In fact, it’s better to think of the variable types falling into two main categories: basic filter variables and Field Filters.
Basic input variables create simple filter widgets where people can plug in values to filter the question’s results. Basic input variables comprise:
Field filter variables are special input variables. They are more sophisticated than basic input variables, and they behave differently. Field filters are “wired up” to columns, and can provide dropdown menus for people to select one or more values, like the dropdown shown in Figure 1 at the beginning of this article.
We’ll cover the three basic input variables - Text, Number, and Date – in this article, and Field Filters in another article. But first let’s get a sense of when to reach for one type of variable over the other.
Field filters or basic input variables?
When to use basic input variables
For simple text, number, and date filtering. For more flexible date filtering, use a Field Filter.
In general, for innumerable values that would not make sense to include in a dropdown (which would require a field filter).
For cases where you may need to do some data wrangling/munging on the input variable in your SQL.
To hook up a variable to a dashboard filter (more on dashboard filters in an upcoming guide).
Now let’s move on to how basic input variables work.
Basic input variables
Basic input variables take input like text, number, or dates. Basic input variables are good for when values are not predefined, or range widely, like order subtotals.
Dates, of course, are a special case; everyone knows the range of values, as time is just standard time. Metabase does not support alternative calendars, or at least not yet: Metabase is open source, so if you want to add support for the Mayan calendar or whatever, go for it.
Basic input variable: Number
Let’s use a basic input variable of type Number. Say we want to create a question that returns all of the records from the orders table, but we want to give people the option to filter out orders based on the order subtotal.
Let’s keep it simple and just give people the option to input a number so that the question will return records for orders with a subtotal greater than that number.
To include a variable in your query, simply wrap the variable name in double braces, like this: {{ variable }}. In this example, we’ll call our variable, {{subtotal_var}} . Here’s the SQL:
SELECT *
FROM orders
WHERE subtotal > {{subtotal_var}}
When we add a variable to a SQL query, Metabase will add a filter widget at the top of the question, and slide out a sidebar to present options for the variable.
Figure 4. A question using a basic variable of type Number, subtotal_var, that will add a filter widget, allowing people to filter for orders greater than the value they plug in.
As shown in figure 4, here are the options in the Variables sidebar:
Variable type. Types can be Text, Number, Date, or Field filter. The variable type determines the input interface for variable’s widget (e.g., for Date, the widget will present a date picker).
Filter widget label. The name of your variable as presented in the widget, which defaults to the variable name in the SQL query.
Required? When you make a variable required, Metabase will prompt you for a Default filter widget value to plug into the variable when the question first loads. If you don’t provide a default, Metabase won’t execute the query until a value is supplied.
In the case of subtotal_var, we’d want to:
Set the Variable type to be Number (since we’re dealing with subtotals).
Change the Filter widget label from subtotal_var to Subtotal (just to make it easier to read).
Toggle Required? to be true.
Set a Default filter widget value to 0. That way, when a question runs, it will return all results automatically; people can enter a higher subtotal if they wish to filter the results.
Now we’re ready to plug in values into our Subtotal widget, and filter for orders with Subtotals greater than that value:
Figure 5. Minimizing the SQL editor, and entering 100 into the Subtotal filter widget to filter for orders with Subtotals greater than 100.
Making a basic input variable optional
If we want to make the filter widget optional, we can enclose the WHERE clause in double brackets:
SELECT *
FROM orders
[[WHERE subtotal > {{subtotal_var}}]]
With the WHERE clause in brackets, if the viewer does not enter a subtotal, and no default is set, the query would simply return all records from the orders table (i.e., Metabase would only run SELECT * FROM orders).
Adding multiple filters
We can use multiple filters as well. If, for example, we’d prefer people filter the results by inputting a range of subtotal values, we can add two variables for lower and upper bounds:
SELECT *
FROM orders
WHERE subtotal BETWEEN {{subtotal_min}} AND {{subtotal_max}}
In this case, two widgets will appear, one for each variable.
Figure 6. Question with multiple widgets: one for Subtotal Min, and one for Subtotal Max. Both values are toggled as required, with a default filter widget value set to 0.
Basic input variable: Text
Let’s try an example using a simple text input variable. In this case, we want to create a question with a filter widget that allows people to search for product titles that contain the text they enter into the widget.
Here’s the code:
SELECT *
FROM products
[[WHERE UPPER(title) LIKE UPPER(CONCAT('%', {{search_term}},'%'))]]
We enclose the WHERE clause in brackets to make the widget input optional. We bookend the variable with the wildcard character % to indicate that the term could have zero or more characters to the left or right of the variable. Additionally, we guard against case sensitivity by using the UPPER function on both the title column and the {{search_term}}.
And here’s our filter:
Figure 7. A basic text filter widget, filtering for titles that contain the word 'computer'. The results contain the rare Lightweight Wool Computer.
Basic input variable: Date
When you select the Date variable type, the filter widget will present a simple date picker. Here’s a question with two basic date variables so that users can input a start and end date to return orders placed between those dates.
SELECT *
FROM orders
[[WHERE created_at BETWEEN {{start_date}} AND {{end_date}}]]
Figure 8. Basic date widgets, with simple date pickers connected to the start_date and end_date variables in the optional WHERE clause.
Now, something to note here: people would need to select dates for both widgets in order for the filter to activate, which could lead to unexpected behavior. For example, someone might leave the End date blank and expect the orders to be filtered from the Start date up to today - when in fact no filter would be applied.
For dates, consider instead using a Field Filter, which offers a lot more flexibility.
Up next: Field Filters
In upcoming articles, we’ll cover how you can create more sophisticated filter widgets using Field Filters, including how to hook those Field Filter variables up to dashboard filter widgets, which allow you to filter data across multiple questions. In the meantime, you can check out our documentation on SQL variables.
This article covers how to create dashboard widgets to filter data in SQL questions. Figure 1 shows the dashboard we’ll be building:
Figure 1. What we'll be building: a dashboard with two filters connected to two questions written in SQL.
This dashboard features:
two questions written in SQL,
two filter widgets: one Date filter and one State filter.
Selecting a value in either filter (or both filters) will update the results in both questions.
Our goal, or vision quest, is to understand how to connect dashboard filter widgets to a special variable type, called Field Filters, that we insert into the SQL code of our questions. Here’s our itinerary:
Let’s start with a simple question written in SQL that shows the orders per month from the Sample Dataset included with Metabase.
Figure 2. A question written in SQL showing orders per month, visualized as a line chart.
Here’s the SQL code:
SELECT
PARSEDATETIME(FORMATDATETIME(orders.created_at, 'yyyyMM'), 'yyyyMM') AS "Created at",
COUNT(*) AS "Number of orders"
FROM orders
GROUP BY "Created at"
ORDER BY "Created at" ASC
Now, we’re talking about filtering data, and astute readers might already recognize that we don’t have a filter statement in our SQL code. There is no WHERE clause. Meaning: even if we were to add this question to a dashboard, and add a filter widget to that dashboard, that filter would have no effect on our SQL question, because there would be no designated place in our code for the widget to insert its value.
Let’s fix that by adding a special variable to our SQL question known as a Field Filter.
Field Filters are a special type of variable that map the variable to a field (column) in a table.
Here’s the SQL code:
WHERE {{created_at}}
You may notice the lack of an = assignment in the WHERE clause. The reason for this abbreviated syntax is somewhat involved. The short answer is because Field Filters handle some SQL code for you under the hood. For the long answer: refer to our article on Field Filters.
With the WHERE clause in place, our code should now look like this:
SELECT
PARSEDATETIME(FORMATDATETIME(orders.created_at, 'yyyyMM'), 'yyyyMM') AS "Created at",
COUNT(*) AS "Number of orders"
FROM orders
WHERE {{created_at}}
GROUP BY "Created at"
ORDER BY "Created at" ASC
Now that we have our variable situated in our SQL code, we need to tell Metabase how to use the variable. When we add a variable to our code, Metabase will slide out the Variables sidebar. We’ll set the Variable type to Field Filter, then map that variable to a field in our database so Metabase can know what kind of filter widget it should add to the question. In this case, we’ll map the variable to the created_at field of the orders table.
Figure 3. Setting the Variable type for our SQL variable, created_at, to Field Filter, then setting the Field to map to option to the created_at field (column) of the orders table.
Note that we can call the SQL variable whatever we want, but we must map the variable to the appropriate field in order for the dashboard filter to work.
For now, we’ll leave the Filter widget label as “Created at”, and leave the Required? toggle alone. This way, if the variable’s filter widget does not supply a value, the question will run as if the WHERE clause did not exist.
We can also select a Filter widget type, though this widget will only apply to our question. Let’s select the Date Filter type, which is the most expressive of the widgets.
Let’s Save our question. We’ll call it Number of orders per month - SQL.
Next, we’ll need to:
create a dashboard
add our question to a dashboard,
add a different filter widget to that dashboard,
then connect that dashboard widget to the Field Filter variable in our SQL question.
But first, a detour.
A detour on data types
When we select the created_at field, Metabase knows that the Field type is a Creation timestamp (note the calendar icon). You can learn about the types of fields (columns) each table has by browsing your data.
Figure 4. View information about your data by clicking on Browse Data from the top navigation bar, selecting your database - in this case, Sample Dataset - clicking on the information icon next to a table, and clicking on the book icon to Learn about this table.Figure 5. Viewing information on the fields in the Orders table of the Sample Dataset.
Administrators can edit the field type, as well as other metadata settings, in the Data Model tab of the Admin Panel. To learn more, check out our documentation on metadata editing.
Connecting a dashboard filter widget to a Field Filter variable
So we have our SQL question (figure 6) with a Field Filter variable in a WHERE clause, and it’s time to add that question to a dashboard.
Figure 6. Our SQL question with our Field Filter variable, showing orders created per month.
Let’s create a dashboard (we’ll give our dashboard the wildly unimaginative name Dashboard with filter widgets).
Click on the filter icon to add a new filter widget to the dashboard.
Under What do we want to filter, we’ll select Time.
For What kind of filter?, we’ll select All options, which will add a Date filter to our dashboard.
Next, we’ll need to connect our widget to the Field Filter variable in our question. Click on the dropdown menu in the center of our question, and select our Created At Field Filter variable.
Click the Done button at the top of the screen.
Then Save the dashboard.
See the GIF in figure 7.
Figure 7. Adding a Date filter (Time -> All options) to a dashboard, and connecting the filter widget to the Field Filter variable, Created at in our SQL question. Remember to click on the Done button, then Save the dashboard.
Now we’re all wired up, and we’re ready to test out our new Date filter. This particular widget type gives us an abundance of options. Let’s choose the Between option to select a date range. In this case, we’ll select the orders for 2019 (between January 1, 2019, and December 31, 2019).
Figure 8. Using the Between option in our Date filter widget to select orders from January 1, 2019 to December 31, 2019.
With our dashboard filter set, our SQL question updates to filter for orders where the created_at column falls in the date range specified by the filter widget on the dashboard.
Figure 9. With the date range set in our filter widget, our SQL question, Number of orders per month - SQL, will only show orders from that range.
Filtering multiple SQL questions on a dashboard
Let’s add an additional SQL question to our dashboard. To keep it simple, let’s create a question that returns the new accounts created per month. The SQL for this question is similar to our first question:
SELECT
PARSEDATETIME(FORMATDATETIME(people.created_at, 'yyyyMM'), 'yyyyMM') AS "Created at",
COUNT(*) AS "Number of new accounts"
FROM people
WHERE {{created_at}}
GROUP BY "Created at"
ORDER BY "Created at" ASC
Once again, for our {{created_at}} variable, we’ll:
Set the Variable type to Field Filter.
Set the Field to map to option to the Created_At field of the People table (not the Orders table).
Leave the Filter widget type as is, or change it to Date Filter (just note that this setting only applies to the filter widget on the question, not the dashboard.)
Leave the Required? toggle off (as in: do not require the variable).
Save your question, title it Number of new accounts - SQL, and add your question to the dashboard.
Now, the existing Date filter does not automatically connect to the newly added question, Number of new accounts - SQL. If we update the value(s) in the Date filter widget, only the data in the connected question, Number of orders per month - SQL, will update to show the filtered data.
So let’s wire up the new question to the dashboard filter widget:
Click on the pencil icon to enter dashboard edit mode.
Select Edit on the Date filter widget.
Then select the Created At option (the only option) from the dropdown menu at the center of the Number of new accounts - SQL card (See figure 10).
Save the dashboard.
Figure 10. Connecting another question to the same Date filter widget.
With both Field Filter variables connected to our dashboard filter widget, when we select a value in the widget, both questions will update their charts with the filtered data.
Adding another filter widget to the dashboard
We now have two questions on our dashboard connected to a single filter. Let’s add another filter. And since we’ve already conquered Time, let’s control Space. Let’s add a widget to the dashboard that allows us to filter by state in the United States.
To start, we’ll need to add a {{state}} variable to the WHERE clause of our Number of orders per month - SQL question.
SELECT
PARSEDATETIME(FORMATDATETIME(orders.created_at, 'yyyyMM'), 'yyyyMM') AS "Created at",
COUNT(*) AS "Number of orders"
FROM orders
LEFT JOIN people
ON orders.user_id = people.id
WHERE {{created_at}} AND {{state}}
GROUP BY "Created at"
ORDER BY "Created at" ASC
Next, we’ll specify that our new {{state}} variable is a Field Filter, of type State (see figure 10).
Figure 10. Adding a Field Filter variable connected to the State field of the People table.
Now that our question has access to date from the state column in the people table, we can connect the question’s Field Filter variable, {{state}}, to the dashboard filter widget.
Figure 11. Connecting State Field Filters to the dashboard State filter widget.
And behold: our dashboard with two SQL questions and two filter widgets: one Date filter and one State filter.
Figure 12. Dashboard with two filter widgets: one Date filter and one State filter. Date filter for orders/accounts created in 2018 in California (CA).
All that’s left to do in order to match the dashboard in figure 1 is to add a text box.
Building effective dashboards often involves providing context for folks to give them insight into how or why a calculation exists. While precise titles, descriptions, and labeled axes can go a long way toward clarifying your visualizations, Metabase also allows you to add flexible text boxes to your dashboards. This post will show you how to take full advantage of this feature using Markdown syntax.
Adding text to dashboards
Select a dashboard you’d like to add text to, and click on the pencil icon to edit the dashboard. Click on the Aa button in the dashboard toolbar. A text box will appear, which you can move around just as you would a question box. To resize the text box, click and drag on its bottom right corner.
Writing in the text box
To interact with a text box, you’ll need to be in edit mode in a dashboard. Click on the pencil icon in the dashboard toolbar in the upper right of the dashboard.
Text boxes in Metabase have two modes.
Edit text (paper and pencil icon).
Rendered view (eye icon).
The text editor in Metabase employs a lightweight markup language called Markdown. If you’ve never used Markdown before, it takes some getting used to, but you’ll soon learn to appreciate its simplicity. Markdown can make writers feel like coders and coders feel at home.
How Markdown works
You write a line of Markdown:
# Heading
And Metabase converts your plain text into formatted HTML code.
<h1>Heading<H1>
Which renders like this:
Heading
You can access six levels of headings just by adding more #s.
You can use Markdown syntax to add links, images, gifs, lists, codeblocks, blockquotes, and more. Here’s a text box with heading, paragraph, blockquote, and code block:
You can do everything this post does and more using text boxes in Metabase. The big deal with Markdown is that you don’t have to write tedious HTML, and Markdown is human-readable even when it’s not rendered. Its minimalist feature set will keep you focused on the content, and provide a standardized look across your dashboards.
Here is an example dashboard with a question and text box:
Here is the Markdown code used in the text box above:
# Analysis
Although **Gadgets** outsold **Gizmos** in 2019, we only introduced **Gizmos** and **Doohickeys** in September of 2019. Additionally, both **Gadgets** and **Widgets** were heavily discounted during our spring, summer, and holiday sales.
We expect sales to continue to grow in the **Gizmo** and **Doohickey** product lines.
# SQL query
SELECT "PRODUCTS__via__PRODUCT_ID"."CATEGORY" AS "CATEGORY",
sum("PUBLIC"."ORDERS"."QUANTITY") AS "sum"
FROM "PUBLIC"."ORDERS"
LEFT JOIN "PUBLIC"."PRODUCTS""PRODUCTS__via__PRODUCT_ID" ON
"PUBLIC"."ORDERS"."PRODUCT_ID" = "PRODUCTS__via__PRODUCT_ID"."ID"
WHERE ("PUBLIC"."ORDERS"."CREATED_AT">= timestamp with time zone '2019-01-01 00:00:00.000Z'
AND "PUBLIC"."ORDERS"."CREATED_AT"< timestamp with time zone '2020-01-01 00:00:00.000Z')
GROUP BY "PRODUCTS__via__PRODUCT_ID"."CATEGORY"
ORDER BY "sum" ASC, "PRODUCTS__via__PRODUCT_ID"."CATEGORY" ASC
# Contact
If you have questions, reach out to us on the [\#product](https://fakemessageservice.com/product) channel.
The hypothetical analyst provided some context, code, and contact info: but you can include whatever context will help readers of your report.
Note: in the example above, the analyst pasted the raw SQL query generated by the notebook editor for reference. You can view the SQL composed by the notebook editor by clicking on the console icon to “View the SQL” when in editing mode.
One last pro tip for gif aficionados
The image syntax,
also works for gifs. Because there are far more important use cases for dashboard text boxes:
In fact, you don’t even need to include data in your dashboards; you could just go full gif:
You can use Metabase’s notebook editor to create questions with explorable charts. People can then click on values to pull up an Action menu, which will give them different options to drill through the chart’s data.
Figure 1. Questions composed in the notebook editor allow people to drill through charts. The GIF demonstrates zooming in on a particular category and timeframe, then viewing the orders that make up one of the bars on the chart.
The Action menu’s drill-through functionality is only available for certain question types, so let’s review those now.
The different ways you can ask questions of your data in Metabase
There are three different ways to ask questions in Metabase.
Figure 2. Simple and custom questions, and questions written in the native/SQL editor.
All methods share the same visualization features, but only simple and custom questions can provide the Action menu with the auto-generated drill-through functionality.
Question type
Description
Action menu?
Simple Question
Pick some data, view it, and easily filter, summarize, and visualize it.
YES
Custom Question
Use the advanced notebook editor to join data, create custom columns, do math, and more.
YES
Native Query
For more complicated questions, you can write your own SQL or native query.
NO
The reason that the Action menu is only available for simple and custom questions is because the functionality leverages metadata about your tables and columns. Under the hood, simple and custom questions use the same query building primitives (grouping functions, filtering functions, and so on). In fact, simple and custom questions share a continuum; you can start with a simple question to explore the data (affectionately known as Chill Mode in Metabase), then customize that question with the notebook editor. For even more advanced questions, you have the awesome power of custom expressions at your disposal.
If you prefer SQL (or you want to take advantage of Metabase’s SQL snippets feature), you can take questions composed in the notebook editor, and convert them to SQL at any time (figure 3).
Figure 3. You can convert a question composed in the notebook editor to a SQL question at any time.
SQL questions give a lot more freedom of expression — you have all of SQL at your disposal — but you lose the tight coupling with the metadata that makes drill-through possible.
An example question composed with the notebook editor
Let’s tour the drill-through functionality available in the Action menu. If you’d like to follow along, see figure 4 for a screenshot of the question in the notebook editor. The visualization has been set to a stacked bar chart (Figure 5).
Figure 4. The question, composed in the notebook editor, use in the following drill-through examples.Figure 5. Setting the visualization to a stacked bar chart.
The Action menu
Clicking on a value in your chart presents an Action menu with the different drill-through options you can take to explore your data.
Figure 7. Clicking on a value of a chart will bring up the action menu, presenting options to zoom in, view records, breakout, and X-ray.
We’ll step through each of the options in the popup menu you see above:
You can click and drag to select an area of a chart to zoom in on.
Figure 8. You can click and drag to select an area of a chart, and Metabase will zoom in on the values in that selected area.
Auto-zoom
You can left-click to bring up the action menu, and select Zoom in and Metabase will create a “close-up” of the data surrounding the value you selected.
Figure 9. Left-click on a chart to bring up the action menu and select Zoom in. Metabase will automatically create a close-up of the data surrounding the value.
The Zoom in option in the action menu will choose an appropriate range of values based on the full range of values in the chart. As shown in figure 9, Zoom in also focuses on the clicked category, so only orders with products in the Doohickey category are shown. If you’d prefer a different range, or wish to include additional groups, you have the notebook editor available to fine tune your chart (and save it as a new question).
View these records
You can click on a value on a chart and select the View these orders option to bring up a table with the individual records that compose the value.
Figure 10. Clicking on a value and selecting View these orders** will bring up a table of records that compose the value.
Breakouts
Break out by a category — this option lets us do things like see the banana cream pie orders in June 2017 broken out by the status of the customer (e.g., new or VIP, etc.) or other different aspects of the order. Different charts will have different breakout options, such as Location and Time.
Figure 11 shows an example breakout looking at orders from people.source.
Figure 11. Clicking on a value will present options to breakout the data. Different options will appear based on the data in the chart. The GIF shows a breakout by Category: User source.
X-rays
X-rays are auto-magic explorations of your data. You can click anywhere on a chart to perform an X-ray, and Metabase will generate a dashboard full of different questions about the data. You’ll have an option to save that X-ray as a dashboard, which you can then edit to your liking, by removing irrelevant questions, or adding new questions or text boxes to fill in the story you want to tell.
Figure 12. X-rays will create (a lot) of different charts based on data in your table(s).
Clicking on a point or a bar additionally gives you the option to compare data, which will give you another dashboard with auto-generated charts.
Drill-through is a great way to give viewers of your questions more insight into their data, which makes it an excellent feature for delivering self-service, embedded analytics.
But drill-through is also an excellent tool for exploring your data, especially when you’re starting out with a new dataset and you’re not sure what questions you should be asking. Start with a broad query, then experiment with the options on the Action menu. When you discover an insightful cut of the data, save it as a new question, and fine tune the visualization as needed, or convert it a SQL question if you’re more comfortable with code.
To learn more about asking questions, see our documentation.
SQL joins are a fact of life when it comes to analyzing data, and with Metabase we try to make getting the answers to your questions as easy possible – even if that requires some more complex tactics. In this article, we’ll cover how to join tables in Metabase using simple and custom questions in the notebook editor, and give some context on relational databases, tables, and keys, to give you a better understanding of how joins work.
Joins, in SQL, allow you to combine data from multiple tables, and where possible, Metabase will make those connections for you, making it easy for people to work with data across multiple tables. You’ll have to have those foreign key relationships defined in your data model, which allows Metabase to detect those relationships and connect those tables in the data model. If those foreign keys are not configured, administrators can always use the Data Model section to specify those relationships.
We’ll cover keys in depth below, but for now, let’s see an implicit join in action in a simple question. We’ll use the Sample Dataset included with Metabase as our data so you can try it out for yourself.
From the top navigation bar, select Ask a question.
Choose Simple Question.
Select the Sample Dataset as your data source.
Then choose the ORDERS table. Metabase will display a list of orders from the ORDERS table.
Click on the Settings button at the bottom left of the screen. As shown in figure 1, Metabase will slide out a sidebar of Table options, and present you with a list of columns to choose from.
Figure 1. ORDERS table with the Settings sidebar open to the Table options section. You can add, remove, and edit columns, including More columns from tables that Metabase can automatically join to the ORDERS table.
The Visible columns are the columns currently in the query of the question, which in this case includes all the columns from the ORDERS table. You can add and remove columns, and change column settings (which differ depending on the column’s field type).
In addition to the Visible columns, you’ll see a section called More columns. This section will include columns from two other tables: PRODUCTS and PEOPLE. (We’ll get to why the REVIEWS table is conspicuously absent later in the article).
Metabase has already “joined” the ORDERS table to the PRODUCTS and PEOPLE table, which allows you to add columns from these tables.
For example, from this Settings sidebar, scroll down to find the PRODUCTS table, and click on the + next to the CATEGORY column to add it to Visible columns. You’ll see a new column, Products -> Category, in the visible columns, with the PRODUCT -> prefix indicating that the column is not native to the ORDERS table.
By clicking on an ID column and selecting view data, you can check to see which tables are connected to that data. For example, from the ORDERS table, you can view the details on an entry in the Product Id column. In the case in Figure 2, Metabase tells us that the product, Lightweight Wool Computer (ID: 146), is connected to the ORDERS table and the REVIEWS table.
Figure 2. Clicking on an ID column and selecting View details will show you, among other data, which tables the record is connected to.
Note that the actual PRODUCT_ID value is 146, not “Lightweight Wool Computer”. Metabase allows administrators to change how column values are displayed.
Figure 3. The field settings for the PRODUCT ID field in the ORDERS Table Admins can use the Display Values setting to display a human-readable title instead of an ID.
As shown in figure 3, when viewing the ORDERS table, the foreign key, PRODUCT_ID, displays the value from the TITLE column of the PRODUCTS table.
Let’s try a simple join in the notebook editor. Start a new custom question with the Sample Dataset. In the Data section, select the ORDERS table. Next, click on the Join icon to add another table.
Figure 4. To add tables to your question, click on the Join icon (the Venn Diagram).
If we add the PRODUCTS table, Metabase will automatically fill in the foreign key relationship, since it knows that the field, PRODUCT_ID, contains foreign keys that reference values in the ID column (the entity key) of the PRODUCTS table.
Figure 5. Joining the ORDERS table to the PRODUCTS table by linking the foreign key (PRODUCT_ID) in the ORDERS table to the entity key (ID) in the PRODUCTS table.
Here’s what the query in figure 5 is saying: for each row in the ORDERS table, use the value in the PRODUCT_ID column to find the corresponding row(s) with the same value in the ID column of the PRODUCTS table, and return a new row with the values from the columns in both the ORDERS and PRODUCTS tables.
If you’re interested, here’s the basic SQL at work under the hood:
select * from ORDERS
left join PRODUCTS
on ORDERS.PRODUCT_ID = PRODUCTS.ID
(Note that if you convert the question to SQL, you’ll get a much more involved SQL query, but both queries return the same data.)
You may have noticed in the simple question mode above that you couldn’t add columns from one of the tables in the Sample Dataset, the REVIEWS table. To understand why, we’ll need some background on relational databases.
Relational databases
Relational databases (like PostgreSQL and MySQL), store data in tables (relations) that generally represent an entity of some kind, like orders or products. These tables comprise columns (attributes of the entity) and rows (sometimes called records). If you’ve ever worked with spreadsheet software before, a table is akin to a sheet with columns and rows.
Term
Description
Examples
Table
Entity
Person, Order, Product
Column
Attribute of that entity
Address, Description, ID
Row
Instance of that attribute
CA, 7, “Lightweight Wool Computer”
Some of these columns contain special attributes, called keys.
The keys to joins
Each table has a special column that contains distinct keys, known as entity keys, or primary keys, that uniquely identifies each row in the table. These columns typically contain ID numbers, and they can be automatically generated by the database, or by an application (e.g., an employee ID).
A second type of key, known as a foreign key, stores a reference to the entity key of a row in another table.
To join tables, you will use foreign and entity keys to tell the database how to combine data from multiple tables.
Let’s use Metabase’s Data browser to look at some keys with an example table in the Sample Dataset. From the top navigation bar in Metabase, go to Browse Data, and click on the Sample Dataset. As shown in figure 6, you’ll see cards for each table. Hover over a card to see additional options, and click on the book icon that appears to Learn about this table.
Figure 6. Go to Browse Data from the top navigation bar, select Sample Dataset, and click on the book icon to Learn about this table.
This table reference page contains several tabs:
Details
Fields in this table
Questions about this table
X-ray this table
Select the Fields in this table to view the Field name, Field type, and Data type for each field in the table, as shown in Figure 7.
Figure 7. Select Fields in this table to view the Field name, Field Type, and Data Type. The Orders table contains an entity key (ID) and two foreign keys, USER_ID and PRODUCT_ID.
The ORDERS table contains one entity key (a.k.a. primary key), ID, and two foreign keys: USER_ID and PRODUCT_ID:
The foreign key, USER_ID, is associated with the entity key, ID, of the PEOPLE table.
The foreign key, PRODUCT_ID, is associated with the entity key, ID, of the PRODUCTS table.
We can combine data from these tables — ORDERS, PEOPLE, PRODUCTS— by joining on these keys. Joining simply directs Metabase to line up the records from one table, and use the foreign key values in each row to combine data from the other table by finding the corresponding row (or rows) with the matching entity key value.
Metabase defaults to left joins, which brings us to types of joins.
Types of joins
Figure 8. The four types of joins: Left outer join, right outer join, inner join, full outer join.
There are four types of joins, which we’ll describe using an example of joining the ORDERS table (the left circle) to the PRODUCTS table (the right circle).
Left outer join. Select all records from the ORDERS table, along with records from the PRODUCTS table that meet the join condition, if any.
Right outer join. Select all records from the PRODUCTS table, along with records from the ORDERS table that meet the join condition, if any.
Inner join. Select only the records from ORDERS and PRODUCTS where the join condition is met.
Full outer join. Select all records from both tables, whether or not the join condition is met.
And by “join condition” we just mean: for each foreign key in the left table, is there a matching entity key in the right table?
Note that not all databases support all types of joins. As you can see in figure 7, the H2 database (the database that houses the Sample Dataset) does not support full outer joins.
Figure 9. Select a join: Left outer join, Right outer join, or Inner join.
Multiple joins
With that context on tables, keys, and joins, let’s see if we can link the ORDERS table to the REVIEWS table. Stated generally: if our starting table (the left table) doesn’t contain a foreign key that references the table we want to join to, how would we go about joining them?
If we try to join the ORDERS table to the REVIEWS tables, Metabase won’t know what to do (see Figure 10).
Figure 10. If there is no foreign key relationship, Metabase won't know what to do, and it will prompt you to select a foreign key relationship. Which, in the case of the ORDERS and REVIEWS tables, does not exist.
The ORDERS table does not have a foreign key for the REVIEWS table, which is why Metabase didn’t automatically connect the two tables.
Let’s turn to the data browser to find out which foreign keys the REVIEWS table includes (figure 11).
Figure 11. The only foreign key in the REVIEWS table is PRODUCT_ID.
So, here’s our situation:
The ORDERS table has foreign keys to the PRODUCTS and PEOPLE tables.
The REVIEWS table has a foreign key to the PRODUCTS table.
To connect the ORDERS table to the REVIEWS table, we’ll need to join them via the PRODUCTS table. We’ll use a custom question to specify the joins. Here’s our notebook:
Figure 12. Joining the ORDERS table to the REVIEWS table via the PRODUCTS table.
If you visualize the results, you’ll see a table of all of the columns from all three tables: ORDERS, PRODUCTS, and REVIEWS. Note that because a single product can have multiple reviews, we’ll see multiple rows for the same product and order, one for each review.
Column selection
In addition to joining our tables, we can be selective about which columns are visible in our question’s results. In the notebook editor, you can select the columns Metabase displays:
Figure 13. Selecting columns in the notebook editor.
As above, you can also select the visible columns from the Settings sidebar.
Figure 14. Click on Settings to add or remove columns from your question.
You’ll notice that Metabase helpfully makes columns from the PEOPLE table available to add as well, so you now have the full Sample Dataset at your disposal.
Have fun joining tables in your own datasets, and remember: if you get stuck, be sure to consult the data browser to learn which tables have the foreign keys you need to join them.
