Press "Enter" to skip to content

Curated SQL Posts

When Additional Data Doesn’t Shrink Confidence Intervals

John Cook follows Betteridge’s Law of Headlines:

In general, new information reduces your uncertainty regarding whatever you’re estimating. The posterior distribution becomes more concentrated as more data are collected.

That’s what happens “in general” but does it necessarily happen every time you get new data? Conceivably if you get surprising data, data that is very unlikely given your current prior, posterior uncertainty might increase.

Click through for an example, as well as a pair of good comments on the post.

Leave a Comment

Power BI: Database Was Evicted to Balance the CPU Load

Chris Webb diagnoses a case of passive voice:

A few months ago I wrote about a rare error – the “Maximum allowable memory allocation” error – that may occur when the physical machine, or node, that a semantic model is running on in the Power BI Service comes under memory pressure. Recently, someone I was working with who was doing some load testing showed me a related error:

The operation was canceled and the database was evicted to balance the CPU load on the node. Please try again later.

Read on to see what causes this.

Leave a Comment

Ways Data Apps Improve Microsoft Fabric Semantic Models

Ruben Van de Voorde digs into data apps:

Fabric data apps are the most interesting thing to happen to data visualization in Power BI and Microsoft Fabric, ever. They provide limitless potential for how to visualize and interact with your data. However, they also indirectly provide a major benefit to your semantic models by allowing them to be simpler to build, use, and maintain.

In this article, we argue that data apps provide a reporting layer that is separate from the semantic layer, something that wasn’t true in Power BI due to tight coupling between reports and semantic models.

Click through to see where they fit vis-a-vis Power BI reports and semantic models.

Leave a Comment

Ghost Records in SQL Server

Martyn Jones sees dead records:

When a delete operation is executed, or when an update causes a page split, a marker is set in the page header, and each row is marked to show that the record is to be deleted; this is the process of creating ghost records. The concept is to improve performance as the data is not immediately removed, and, with row-level locking, it helps minimise blocking and supports efficient concurrency under typical isolation levels such as Read Committed and above. If a transaction is later rolled back, the cost of effectively restoring the row is considerably reduced as the data doesn’t need to be reinserted, only the marker removed. Later demos will show this in action.

This is the first part of a longer series on ghost records and ghost cleanup.

Leave a Comment

Building a Lakehouse Table Health Pipeline

Gilbert Quevauvilliers builds a pipeline:

In this post I will show how I used GitHub Copilot CLI / Agent mode in VS Code to create a Microsoft Fabric pipeline that checks Lakehouse table health and only optimizes tables that require maintenance. I’ll also show the prompts I used, the issues I ran into, and how Copilot helped me resolve them.

Recently Microsoft announced Lakehouse table health. The post showed how you can check can check the health of your Lakehouse table in a very simple way.

Read on to see what this feature can do, as well as how Gilbert was able to gin up (with the help of GitHub Copilot) a pipeline to optimize lakehouse tables. What I appreciate is that Gilbert also included the errors in the process, rather than making it seem like everything with perfectly the first time around.

Leave a Comment

SSMS Tools in VS Code

Denny Cherry has a new VS Code extension:

Are you a user of VS Code, but you wish that the MSSQL extension had more of the SQL Server Management Studio features? Good news, there’s now a VS Code Plugin that gives you those capabilities in VS Code by using the “SSMS Tools for SQL Server” extension.

Click through to see what it includes. There’s a pretty decent amount of existing SSMS functionality in this extension, especially on the database administrator side, where Azure Data Studio and VSCode historically have not focused.

Leave a Comment

Error 1408 with Availability Group Automatic Seeding

Sean Gallardy troubleshoots an error:

This used to be a very hot topic around SQL Server 2016 when automatic seeding first came out, then everyone learned how automatic seeding was just manual seeding but done for you and the topic went away. A decade later, it’s somehow back to being a hot topic. I guess all that is old is new again.

Error 1408 states that the database doesn’t have enough log to find a starting point with the primary/principal – which, let’s just use AG terminology from this point on because mirroring has been dead for 14 years and counting.

Mirroring has been dead and yet it lives on in undead form. Regardless of that, Sean provides some good information around what automatic seeding actually means and what you can do if you get stuck with Error 1408.

Leave a Comment

An Introduction to Spindle

Jonathan Stewart has an open-source tool:

A client wanted to see a feature that I had previously built for another customer. I obviously couldn’t just use real client data. When I used Faker and other tools like that, I got data that looked fine on its own but fell apart when I needed relational integrity such as needing an order to point to a customer which points to an address. AdventureWorks, Contoso, WideWorldImporters and the like are great for what they are, but they are either too small, too clean or not relevant to the demo.

I needed data that would not only look the part for the subject, but also follow relationships, have proper distributions, and so on.

So I built Spindle.

I’ve used Spindle for work purposes. It’s a good product for generating artificial data, either from one of the default data domains (e.g., HR, insurance, health care, manufacturing) or based on an existing database schema.

Leave a Comment

Too Many Tables and PostgreSQL

Laurenz Albe uncovers an oddity:

Recently, I helped a customer investigate database problems. It turned out that these problems could be traced back to too many tables in the database. Since this may come as a surprise to many users, I thought it worth the while to write about it.

Click through to see what was happening, as well as a demonstration of the problem. Granted, when you start talking about tens of thousands of tables, something has gone catastrophically wrong in your database architecture.

Leave a Comment

Share Groups and Sub-Optimal Performance

Jack Vanlightly creates a problem:

In this post we’re going to see how share.acquire.mode=record_limit combined with:

  • fewer consumers than partitions
  • and various cases of “partition skew”

…can result in subpar performance with share groups. 

I stumbled on these issues when running large sets of dimensional tests with Dimster’s explore-limits mode, which finds the highest sustainable throughput while staying within a target end-to-end latency target. There was a specific subset of the tests that explore-limits mode would consistently fail to complete, and they all happened to be with record_limit and a consumer count lower than the partition count. In this test, we’ll understand why Dimster had such a hard time with this combination.

Click through for the details, as well as how to mitigate this sort of scenario.

Leave a Comment