Press "Enter" to skip to content

Curated SQL Posts

A Medallion Architecture Primer

Andy Brownsword takes us through a concept:

Medallion architecture is the go-to for handling analytical data, particularly with the prominence of lakehouses in Databricks and Fabric.

Whilst the layers are well defined, their definitions don’t tell you where particular tasks belong. So here I wanted to present my view from a more practical perspective of what goes into each layer.

What I’ve found interesting is that, despite almost everyone agreeing on the rules of what goes where, you can easily get into debates with other data architects and lakehouse practitioners once you start asking concrete questions around specific datasets and specific levels of transformation. The edges between silver and gold get really fuzzy.

Leave a Comment

Managing Resources via Azure Cloud Shell

Jordan Boich isn’t afraid of the command line:

As a DBA or any data professional, it is becoming more valuable and vital to have a well rounded understanding of your data estate and environment. However, sometimes it can be a bit cumbersome, especially when connecting to Azure resources via PowerShell. You have to fumble with authentication, connection cmdlets and more, which can be a bit discouraging to use at times.

There are some interesting and useful tools to help circumvent that overhead and manage your Azure resources. Azure Cloud Shell is a great quick tool to use if you want to have a lightweight, and easily accessible way to help manage and administer your Azure resources.

Read on to see how you can connect and some of the things you can do with it. You also get your choice of bash versus PowerShell.

Leave a Comment

Clustering in Fabric Data Warehouse

Louis Davidson does some exploring:

Clustering is one of the data terms that could not have been created by a database design person. Why? Because it has multiple meanings. In PostgreSQL it means the same thing as an instance in SQL Server. In server architecture it means connecting multiple servers together to form a “single” unit of some form to protect against failure. Then it means an index that determines the physical storage of a table in some manner. This usage of the word means the former.

The weirdest part of using a Fabric Data Warehouse when you first try to look at a query is the lack of indexes. So (currently as of writing this is late 2026), you only have this one tool to affect your performance. The tool allows you to order the data physically in your files so query processing can be reduced when searching the physical file.

Read on for an introduction of how it works, with the caveat that Louis is about two pages ahead of most people in the book.

Leave a Comment

Tracking Microsoft Fabric Capacity Operation Events

Gilbert Quevauvilliers stores some data:

This blog post will explain how to capture the data from real-time events for Fabric Operation Events and store the results in a Lakehouse.

My approach is to use a Python notebook to query Eventhouse.

The reason for this is the Python notebook runs very quickly and because it consumes very little Capacity Units, it is a very efficient way to store the Fabric Operational Events to be analyzed over a long period of time.

Read on for the instructions and the notebook that Gilbert used.

Leave a Comment

Checking archive_mode on pgBackRest

Stefan Fercot shares some advice:

A recent question on the community channels described a difficult situation: a standby had been promoted with archive_mode=off, and restarting the new primary was something the team wanted to avoid. Could they enable archiving on a downstream standby and take a pgBackRest backup from there instead?

Their attempt failed with archive_mode must be enabled, even with archive-mode-check=n. Removing the checks from pgBackRest’s source code allowed their proof of concept to succeed, but was that enough to trust the approach?

My recommendation was to return to a supported configuration, either through a restart or a controlled switchover. But I wanted to take a closer look and see whether archive-mode-check=n might allow the standby-based approach.

Read on for that answer, as well as some of the issues you might run into.

Leave a Comment

Running Postgres in Docker

Grant Fritchey spins up a container:

The goal of this guide is simple: get PostgreSQL up and running in the fastest, easiest way. You’ll then learn how to create databases in PostgreSQL and the objects that live within them.

The two easiest ways to get started working in PostgreSQL are to either use containers, or create an account with a cloud provider. In this case, we’ll be using containers.

This post covers a lot of the basics of pulling a Docker container image and how you interact with that running Postgres container.

Leave a Comment

Page Splits and Fill Factor

Brent Ozar differentiates page splits:

You’re looking at page split numbers in a monitoring tool or Perfmon, and you’ve heard that page splits are bad, so you’re lowering fill factor, expecting your page splits to go down.

You’re monitoring the wrong number.

Jeff Moden has a good comment in there as well that what Brent’s saying is often true, but there can be edge cases. Though that’s part of the point: it’s an edge case, not a primary case. And I like Erik Darling’s comment on the post as well, as it’s often fretting about the color of the furniture when the house is burning down.

Leave a Comment

Using the Azure Resiliency Agent

Reitse Eskens gives a copilot a spin:

And that isn’t limited to services like storage, web portals, user management, and data stuff. It’s also location, backups, and disaster recovery. But there’s a catch: you need to make sure this is configured. I always compare the cloud (Azure in my daily work) with a huge box of Lego. You have all the pieces and elements to make something cool, but you have to build it. Even when you automate it, you still need to think about what you want.

Now, before I continue my Azure Fundamentals training, let’s move on to what this post is about: resiliency. Or, how your environment is set up for disaster recovery.

Let’s use the Azure Copilot to guide the process, create the necessary resource and review the advice.

Click through to see how it works. Because it’s in preview right now, we don’t know how much it will cost later. But in the meantime, you can give it a try.

Leave a Comment

Storage Engines in MySQL versus MariaDB

Lukas Vileikis compares storage engines:

The big difference between MySQL and MariaDB largely revolves around database storage engines and anything related to them. In this article, I’ll walk you through some of the key differences between storage engines in MySQL and MariaDB, explain which is best for your use case, and detail how to change your database’s storage engine if necessary. 

Most of you will do just fine with InnoDB on either of the database management systems – however, those of you who are building something more wild and exotic might want to look specifically into MariaDB. 

Read on to learn more.

Leave a Comment

Building Daily Averages in DAX

Marco Russo and Alberto Ferrari take generative AI tooling for a spin:

Computing an average daily sales amount is straightforward: you should iterate over the dates, compute sales for each day, and average the results. An AI assistant can generate DAX code that works in just a few seconds; whether it computes the number you need is a different question.

In this article, we ask AI to generate every measure, function, and validation query. We start with a simple average and test it against more demanding scenarios. Our job is to explain the business requirements, challenge the proposed code, and verify the results. Each test reveals a missing detail from the original request.

The prompts below show a conversation with an AI agent. They describe how to get the DAX code from the AI. They are not a verbatim transcript: a different agent might produce different expressions.

This shows the duality of generative AI outcomes: you can end up with a good solution, but unless you know the ins and outs of what you’re doing, you can end up in a sub-optimal situation.

Leave a Comment