Press "Enter" to skip to content

Curated SQL Posts

What to DO When Out of Capacity in Microsoft Fabric

James Serra hits the ceiling:

Microsoft Fabric makes it wonderfully easy to put many analytics workloads on one platform. Power BI, data engineering, warehousing, data science, real-time analytics, Copilot, and other experiences can all share the same Fabric capacity. That is a big advantage, but it also creates an architectural question that does not get much attention until something goes wrong: what should you actually do when a capacity starts running out of room? The answer is not always “buy a bigger capacity.” Sometimes you should optimize, sometimes scale up, sometimes scale out, and sometimes isolate the workload causing the problem.

Read on for an explanation of what it means to “run out of capacity” and a quick overview of four options available to you.

Leave a Comment

Working in Batches in SQL Server

John Deardurff has some advice:

I’ve been reviewing Azure SQL Database performance guidance recently and came across Microsoft’s documentation on How to Use Batching to Improve Application Performance. While the article focuses on Azure SQL Database, the same principles of batching transactions for better performance apply equally well to SQL Server and Azure SQL Managed Instance.

The reason this topic caught my attention is that batching doesn’t just improve performance. It can reduce blocking, minimize rollback pain, improve transaction log efficiency, and potentially lower costs in cloud environments. That’s a pretty good return on investment for a relatively simple coding change. Here is the SQL Script that I use for this demonstration. Feel free to test for yourself. (It is a text file, so you will have to save it as a .sql file.)

Batching is especially important on delete operations against larger tables, where you don’t remove enough data to make TRUNCATE TABLE a viable alternative (or where you don’t have permissions to truncate). But one thing to keep in mind is that index design matters for batch operations. If you don’t have a good index, your first batches will be fast but they will gradually slow down as SQL Server needs to scan an increasingly large range to find the next set of rows to update. I wrote about this quite a while ago when putting together a talk on near-zero downtime T-SQL operations.

Leave a Comment

The Pain of Microsoft Fabric Deployment Pipelines

Meagan Longoria has a list:

Fabric deployment pipelines look like they solve CI/CD for Fabric content, especially for people who prefer a GUI over writing code to handle deployments, but the implementation has enough structural gaps that they fall apart for several real deployment workflows. Even when they do work, the UI isn’t always intuitive.

Every piece of active software carries a backlog of feature requests and known limitations, and deployment pipelines get new capabilities on a regular basis. Everything below reflects how deployment pipelines behave as of August 2026. Some of it may have changed by the time you’re reading this, so check Microsoft’s docs for the current state before you plan around any of these.

Click through for the list, as well as a few alternatives that come with their own trade-offs.

Leave a Comment

Validating Selective Deployments in Microsoft Fabric

Matt Collins shares some advice:

Selective deployments with the fabric-cicd python library are highly useful for shipping just the items that actually changed in your CI/CD process. Unfortunately, by default, it also tells you that a deployment succeeded when nothing was deployed at all.

This blog showcases an example where we selectively deployed Fabric items to upper environments, not realising that the dev ops pipeline reported as “successful” but did not contain our intended changes. We will then dig into some quality checks that highlighted a bigger error in the way Microsoft Fabric handles item naming in a repository.

You’ll learn how to help safeguard from human error in selective Fabric deployments, as well as create useful build validations in Azure DevOps that help to keep your repository resource names clean and fit-for-purpose. All this is achieved through two simple CI/CD pipelines.

Read on to learn more.

Leave a Comment

Semantic Link Labs UI Updates

Chris Webb takes a look:

There’s so much going on in the Fabric community that it can be hard to keep up with it all. Semantic Link Labs is a great example: in the six months or so since I last had a proper look at it my colleague Michael Kovalsky has done a whole load of cool things and it wasn’t until I had a chat with him recently that I realised how much had changed. Most importantly, for someone old-fashioned like me who still likes tools with a UI, a lot of new functionality has been added which has a UI and is usable with minimal coding.

Click through to see what’s available.

Leave a Comment

Visual Design and Generative AI

Cole Nussbaumer Knaflic guides the non-thinking machine:

Now that we have the story planned, it’s time to start developing the content that will support our message and narrative. When data is part of that, a good first step is choosing a visual that aids in comprehension. The right graph makes your point immediately clear. The wrong one makes your audience spend their mental energy decoding the graph instead of understanding your message.

This is where people sometimes stumble. They use the first chart that comes to mind—or simply carry forward the one they used during exploratory analysis. But a graph that works for exploring data isn’t necessarily the best for communicating it. Your audience and takeaway should drive the choice. By this point, you’ve already done that work: you know your audience, you’ve planned your story, and you’ve written takeaway titles that tell you exactly what each graph needs to show. Let those sentences guide your design.

As you’d expect, there’s some good advice on choosing specific types of visuals. But the majority of this article is around nudging the language model to spit out the correct visual for the right reasons.

Leave a Comment

Reviewing Older Guidance on Postgres

Christopher Winslett takes a look back:

Over the years we have written a lot about how data gets into Postgres, how it sits on disk, and how indexes help you find it again. Some of that advice was written against Postgres 10 or 11. A surprising amount of it is still exactly what we would tell you for the upcoming Postgres 19 release. Functionality described here is based on current betas; minor details may still change before GA.

This post revisits Crunchy posts in the “load, storage, indexes, and partitioning” bucket: what we wrote, which version moved the needle, and what we would tell you to do now. Along the way: async I/O, more resilient COPY, LZ4 by default, richer BRIN shapes, skip scan, and smoother partition operations.

I like this sort of article because it helps old-timers in an industry recalibrate their expectations based on the current state of technology.

Leave a Comment

Fun with Podman

Deepak Vohra explains why you might want to look into Podman:

Podman is a relatively latecomer to containerization. Some of you may have invested heavily in the Docker stack of technologies (containerd, dockerd, and Docker Swarm). You may be wondering if there are any benefits to learning about Podman.

Click through for that answer. One thing that does not show up but might influence your decision is licensing: Podman is Apache 2.0 licensed and there are no commercial costs.

Leave a Comment

Asynchronous Snapshot Replication from ActiveCluster Pods to Other Arrays

Anthony Nocentino digs into some very neat and very expensive things:

I’ve been rebuilding my three-site SQL Server demo lab, and I ran into something I’ve wanted for a long time. If you’ve ever designed a SQL Server environment on ActiveCluster, you know the pattern: two FlashArrays running a synchronously replicated pod for zero RPO between sites, and a third array somewhere else for a longer retention, disaster recovery copy. The problem was that you couldn’t get the data to that third array directly from the pod. Protection groups inside a stretched pod simply couldn’t have an array target.

That’s changed, and it’s been possible longer than a lot of us realize. You can create a protection group inside an ActiveCluster pod, add a third FlashArray as a target, and asynchronously replicate snapshots to it on a schedule. Your synchronously replicated data gets a third copy, and you don’t have to build a parallel set of non-pod volumes to make it happen.

In this post, I’m going to show you how to configure this end to end with the Pure Storage PowerShell SDK2, so you can automate it. Let’s go.

I mean, sure, you need multiple FlashArrays to do this. But who doesn’t have a few of those floating around?

Leave a Comment

The State of JSON Indexing in SQL Server 2025

Greg Low shares some thoughts:

SQL Server 2025 finally gives developers a native JSON data type and, with it, a purpose-built way to index JSON documents with the new CREATE JSON INDEX statement. Before this, indexing JSON meant exposing individual properties through computed columns and building standard indexes on top.

It’s a major step toward closing the gap with databases like PostgreSQL, long praised for its JSON and JSONB support. However, as a preview feature, JSON indexing comes with real constraints DBAs and developers should understand before adopting it.

This guide has everything you need to know about JSON indexing in SQL Server 2025: what it is, how it works, and current limitations.

Click through to learn more.

Leave a Comment