Press "Enter" to skip to content

Curated SQL Posts

Power BI Projects Now GA

John Kerski takes note:

Do you know where you were on June 15, 2023? Do not fret, neither do I. But I do remember a blog post written by Rui Romano announcing the preview of the Power BI Project (PBIP) format back then.

Many in the Power BI world rejoiced at this news. Microsoft was finally taking steps toward better practices seen in software development and applying them to the analytics space.

As it happens, June 15, 2023 was just the first step. Since then, PBIP has evolved into the format we have today and I am happy to see that, as of September 2026, PBIP is generally available as a fully-supported feature for Power BI by Microsoft.

Before I explain the mechanics of PBIP, let me outline the real problems it was designed to solve.

Not included in John’s article: “Power BI Report Server.” Because Power BI Report Server users learn early on not to attempt to rise above their station.

Leave a Comment

SQL Server on Azure Local Now GA

Raj Pochiraju announces a product:

Some of the world’s most critical databases run in places where cloud connectivity cannot be assumed. A remote industrial site, for example, may need production systems to continue operating when external connectivity is unavailable. A regulated organization may need sensitive data and processing to remain within sovereign boundaries.

Today, SQL Server on Azure Local is generally available for connected and disconnected operations. With this release, customers can modernize where their data resides while maintaining control over infrastructure, connectivity, and data placement. They can bring AI closer to their data with Foundry Local on Azure Local, currently in preview, and use eligible existing SQL Server licensing investments.

Azure Local is rather limiting in terms of the available hardware, though there are specific environments that I think could be well-suited for this.

Leave a Comment

DiskANN Vector Index and Search Now GA

Pooja Kamath makes an announcement:

Today, we are announcing the general availability of DiskANN Vector Index & Search across Azure SQL Database, Azure SQL Managed Instance with the always-up-to-date update policy, and SQL database in Microsoft Fabric.

DiskANN brings scalable approximate nearest-neighbor search directly to the SQL engine. Developers can store vectors alongside relational data and combine vector similarity with the filters, joins, security policies, and transactional data their applications already rely on.

It does support INSERT, UPDATE, and DELETE, and they say there’s asynchronous index maintenance. If that works the way it should, it gets rid of one of the biggest pain points of using DiskANN-based vector indexes in the SQL Server universe today.

Leave a Comment

Flat File Ingestion Options with Microsoft Fabric

Andy Brownsword loads some data:

For this example I’ll use NHS monthly prescribing data which provides us with a large CSV, and we’ll copy it into a Lakehouse. For the Bronze layer we want to retain data as close to source as possible, so simply retrieving the file and retaining the CSV is the goal.

I’ll compare the options for this specific use case, with multiple runs for each. So, let’s see how the options stack up.

Click through to see how long each of the four options takes and what Andy recommends.

Leave a Comment

Ten Decisions to Make Prior to Creating a Fabric Workspace

Meagan Longoria has a list:

Creating a Fabric workspace takes about 30 seconds. Restructuring workspaces after people have built content in them takes a lot longer. Moving content to another workspace usually means redeploying or recreating it, and anything bound to the old item IDs has to be rebound: reports connected to a semantic model, a notebook’s default lakehouse, pipeline activities, and shortcuts. Item-level shares, app content, and links people saved don’t carry over either.

Most of that rework is avoidable if you make a handful of design decisions before anyone starts building. None of these decisions has a single right answer. The right choice depends on your organization’s size, skills, security requirements, and how much self-service you want to support. But I’d much rather see them made deliberately, before the first workspace exists, than left at the defaults and discovered later. For each decision, I’ll cover the options, what should drive the choice, and the platform constraints that narrow it down.

Read on for that list.

Leave a Comment

Queues in Postgres

Jobin Augustine builds a queue:

In many real production databases, we often see some tables receiving high-frequency updates (millions per day) and autovacuum running repeatedly (hundreds of times per day). Additionally, those tables can sometimes become heavily bloated. The end effect is poor performance, poor concurrency, and an unmanageable database.

In case we are using methods pg_gather for diagnosis, such anomalies are easy to spot by clicking on the table header to sort.

Read on to learn more about the PgQue extension and how it works.

Leave a Comment

Tracking Object History in Flyway Desktop

Steve Jones looks at an update to Flyway:


It’s a small change, but a handy one. Flyway Desktop (FWD) now includes the object history for different schema changes, so as you are evaluating how your changes might fit in with others, or you are trying to determine where something broke, you can see a list of historical changes. This post looks at checking history quickly in FWD.

Click through for a demo.

Leave a Comment

Extracting Lineage Information from Microsoft Fabric Spark Notebooks

Gerhard Brueckl retrieves some information:

When building data platforms and data pipelines to populate them, one of the most challenging parts is orchestration. You need to figure out dependencies between your pipelines and activities, know the run times/duration and make sure to run them only when all upstream pipelines succeeded. From a technical point of view, this can be easily accomplished within Microsoft Fabric using runMultiple(). (I really dont know why Databricks doesn’t offer something similar out of the box?!?)

Now the next question that comes up is: “Where do I get those dependencies from?” – and this is exactly what this blog post is about!

Click through to learn how.

Leave a Comment

Window Functions and Virtual Tables

Dualcore DBA pushes a predicate:

In this post, we’ll look at some interesting behaviour I’ve stumbled across with window functions and derived datasets in SQL Server.

I came across a query in a workload that was looking to get the most recent thing per group, a representative example in the Stack Overflow 2010 Database is below. The database is provided under cc-by-sa 4.0 licence from Stack Exchange Data Dump, I am running in compatibility level 150 on a SQL Server 2022 instance installed on a VM with 8 cores and 35GB RAM, though the issue being illustrated also occurs in earlier compatibility levels.

Click through for the demonstration and to see what happens when you switch to compatibility level 160.

Leave a Comment

A Quick Primer on Microsoft Entra ID

Jordan Boich gives us a high-level overview:

As a DBA with on-prem and cloud experience, I feel confident in saying that I’m fairly well versed on the “DBA Domain” part of the shared responsibility model. However, if we really want to do our jobs right as data professionals, having an understanding of the entire infrastructure does wonders when it comes to making architectural decisions that can impact the entire system.

The struggle I’ve experienced being a DBA is that hearing things like “Entra ID” or “Microsoft Entra Domain Services” has me sitting in a position where I know of those things because I’ve been around it for years but not really having as deep of an understanding as I’d like to. In this post, I’m going to talk a bit about authorization, and authentication from the cloud perspective, specifically Azure, so that other fellow DBAs and data professionals can gain some insight, see what it looks like to operate in the Entra Admin Portal, and show that it’s not as overwhelming as I thought it would be.

Read on to learn more.

Leave a Comment