Press "Enter" to skip to content

Curated SQL Posts

Azure SQL DB Hyperscale versus Distributed SQL Solutions

Anna Hoffman clarifies:

In my last post, I broke down the Socrates paper, the research that became Azure SQL Database Hyperscale. Writing it made me want to tackle a question I think trips a lot of people up when they’re choosing a database: is Azure SQL Database Hyperscale “distributed SQL”? 

It’s a fair question. “Distributed SQL” has become shorthand for “the relational database that scales in the cloud,” and anything outside that bucket tends to get filed under “the relational one my company already has.” I’d rather not leave that to assumption, so here’s how I think about it. 

Read on for Anna’s take.

Leave a Comment

Lakehouse Design and Direct Lake Performance

Nikola Ilic shares some numbers:

When a Direct Lake report is slow, people open the semantic model first. They check relationships, rewrite DAX, look at the capacity metrics. In my experience, the cause usually sits one layer down, in how the Delta tables under the model were written.

That sounds like a bold claim, so I wanted to test it properly. I took one dataset of 500 million rows and wrote it to seven Delta tables. Each table differs from a baseline in exactly one way: the number of files, the row group size, V-Order, or partitioning.

The performance matters more for cold data, though when you’re dealing with a data lake, that risk goes up considerably versus a relational database platform where we try to keep as much data in cache as possible.

Leave a Comment

Preparing a Domain Controller for a Home Lab

Lee Markum continues a series on building a SQL Server 2025 home lab on Linux-based hardware:

In part 1 I described laying the foundation for the lab. This post will describe building the Windows domain controller and other machines on my Linux Mint host.

My goal was to have a realistic, production-like configuration. As a result, I wanted a domain controller, two SQL Servers in a Windows cluster and an Availability Group.

Looking back through the ChatGpt thread, I apparently started cloning the Windows Server template for the SQL Servers before I made a domain controller. I don’t know why I did that, but I wouldn’t do it that way again. I’ll give you the instructions in a better order.

Please note that domain names (example.com), subdomain names (lab), and IP addresses shown in this article are examples. Some identifying details have been changed, but the configuration and procedures remain the same.

Click through for Lee’s notes.

Leave a Comment

Removing a 4TB Table in PostgreSQL

Radim Marek drops a table:

GitLab has a public story about deleting the wrong thing. In January 2017 an engineer who was trying to restore replication wiped a data directory they believed was on the secondary. It was on the primary. The postmortem they published afterwards is still one of the better things written about a database outage.

At PGConf.EU in Valencia last week a few people from GitLab came to me with the inverse problem. They have one table, 4 TB, on a primary with ten replicas, and they want it gone, on purpose, with as little impact as they can manage. It’s an archival table: nothing writes to it, and nothing is supposed to read it.

Read on for a variety of thoughts around different ways to get to the same end state at differing levels of user impact.

Leave a Comment

Securing Microsoft Fabric Gen2 Dataflows API Keys

John Kerski saves some keys:

If you’ve ever developed a Gen1 dataflow to connect to an external API (Application Programming Interface), you know the awkward moment when you have to paste an API key into the Power Query script.

There it sits, in plain text, inside an artifact (the proper name for something like a semantic model, report or dataflow created in the service), with limited options for secret rotation or access control.

Thankfully, Gen2 dataflows improve the story considerably – especially when Azure Key Vault, Microsoft Fabric pipelines, and the newer parameters feature are combined.

In this guide, I’ll show you the pattern I now reach for when I need to pass API secrets into a Fabric Gen2 dataflow without leaving them in code or configuration.

Click through to see how.

Leave a Comment

Load Testing Azure SQL GP and BC

Reitse Eskens continues a series on load testing. First up is Azure SQL Database General Purpose edition:

My previous post on load testing Azure SQL covered the Premium Edition; today I’m moving to the General Purpose Tier.

The GP tier comes in two flavours: regular and serverless. Serverless has only one hardware tier: Gen5. With provisioned, you can also select memory-optimised or confidential compute. In earlier tests, I never found significant differences between serverless and provisioned, so I skipped testing both SKUs. Given the nature of these tests, going memory-optimised didn’t seem to make much sense either. Also, I’ve never seen any other deployment than the regular Gen5 General Purpose database.

And then there’s Business Critical:

My previous post on load testing Azure SQL covered the General Purpose Edition, let’s move it up to the Business Critical Tier

The Business Critical tier comes in two flavours: General and confidential compute. For my testing, I’ve stuck with the General SKU. I haven’t seen confidential compute with customers, and it has very specific use cases: you want to protect your threads on the processor itself. A lot happens behind the scenes to make that work, and it costs more. There are scenarios where this makes sense, but not in my work. For example, using 8 cores for regular business-critical workloads costs about 3,650 euros each month. Confidential Compute will cost about 6,000 euros each month. So yeah, you need a use case for that.

Click through for results on each.

Leave a Comment

The Limits of CHOOSE()

Steve Jones performs a test:


I was looking over the CHOOSE() function while editing an article at SQL Server Central. I realized there isn’t a documentation note on how many parameters can be included. I decided to test.

This post looks at the tests and results.

Click through to see how many parameters Steve was able to cram in before the whole thing fell apart. This is definitely a place where Swart’s 10% Rule comes into play.

Leave a Comment

Clean Up the Housekeeping Jobs

Aaron Bertrand shares some advice:

When I inspect a new SQL Server instance for the first time, I am often surprised by the number of maintenance and housekeeping jobs. There can be a lot going on, and it’s not uncommon to find several of these jobs scheduled to start at the same time, usually at exactly midnight. This can create compounded resource usage and practically invites the processes to interfere with each other.

For many of these routine tasks, I would much rather have a single job that runs them sequentially. This avoids concurrency issues and gives us a much cleaner view of the job list, current activity, and execution history. And no, I don’t mean putting every maintenance operation into one ginormous job, but there are definitely some consolidation opportunities worth pursuing, particularly for things that run once a day or less.

Click through for a series of examples and what Aaron would do with each.

Leave a Comment

The Shape of a Shape Profile

Jonathan Stewart continues a series on Shape:

Read it top down:

  • Title and row count. 1 table(s) · 5,000 rows profiled. Check this first. If it’s not the population you meant to profile, nothing below answers your question.
  • Capture line. captured safe (k=5). A safe capture shows statistics and formats only for a sensitive column, and lists a category only if at least 5 rows have it.
  • Table line. 5,000 rows · 8 columns · primary key: order_id. The primary key is detected from the data, not read from a schema.
  • Sampling. read whole: 5,000 rows; adequacy adequate. Shape read every row. Two analyses sample inside: pattern detection on 1,000 rows and distribution fitting on 2,000. The adequacy line says a value that appears in at least 0.06% of rows is seen with 95% probability.

Click through for the demo.

Leave a Comment

Table Distributions in Azure Synapse Analytics Dedicated SQL Pools

Andy Brownsword has a revelation:

When designing tables in a Synapse Dedicated SQL Pool, I initially focussed on the ‘SQL’ and not so much the ‘pool’. How tables are stored is fundamentally different to other Azure SQL offerings, so it’s easy to get tripped up and impact efficiency and performance.

Let’s see how the SQL pools work differently and how to tailor your tables for optimal performance.

Click through for an explanation of the three distribution strategies and when you might use each. As a quick note, the REPLICATED option sounds like it’d be really great, but in practice, there are some major drawbacks as table sizes get larger. That’s a big part of why we save them for small, rarely-changing dimensions.

Leave a Comment