|
by Mark Callaghan (noreply@blogger.com)
Murat Demirbas
In a recent viral post, Matt Shumer declares dramatically that we've crossed an irreversible threshold. He asserts that the latest AI models now exercise independent judgment, that he simply gives an AI plain-English instructions, steps away for a few hours, and returns to a flawlessly finished product that surpasses his own capabilities. In the near future, he claims, AI will autonomously handle all knowledge work and even build the next generation of AI itself, leaving human creators completely blindsided by the exponential curve. This was a depressing read. The dramatic tone lands well. And by extrapolating from progress in the last six years, it's hard to argue against what AI might achieve in the next six. I forwarded this to a friend of mine, who had the misfortune of reading it before bed. He told me he had a nightmare about it, dreaming of himself as an Uber driver, completely displaced from his high-tech career.
Someone on Twitter had a come back: "The thing I don't get is: Claude Code is writing 100% of Claude's code now. But Anthropic has 100+ open dev positions on their jobs page?" Boris Cherny of Anthropic replied: "The reality is that someone has to prompt the Claudes, talk to customers, coordinate with other teams, and decide what to build next. Engineering is changing, and great engineers are more important than ever." This is strongly reminiscent of the Shell Game podcast I wrote about recently. And it connects to my arguments in "Agentic AI and The Mythical Agent-Month" about the mathematical laws of scaling coordination. Throwing thousands of AI agents at a project does not magically bypass Brooks' Law. Agents can dramatically scale the volume of code generated, but they do not scale insight. Coordination complexity and verification bottlenecks remain firmly in place. Until you solve the epistemic gap of distributed knowledge, adding more agents simply produces a faster, more expensive way to generate merge conflicts. Design, at its core, is still very human. Trung Phan's recent piece on how Docusign still employs 7,000 people in the age of AI provides useful context as well. Complex organizations don't dissolve overnight. Societal constructs, institutional inertia, regulatory frameworks, and the deeply human texture of business relationships all act as buffers. The world changes slower than the benchmarks suggest.
So we are nowhere near a fully autonomous AI that sweeps up all knowledge work and solves everything. When we step back, two ways of reading the situation come into view. The first is that we are all becoming butlers for LLMs: priming the model, feeding it context in careful portions, adding constraints, nudging tone, coaxing the trajectory. Then stepping back to watch it cook. We do the setup and it does the real work. But as a perennial optimist, I think we are becoming architects. Deep work will not disappear, rather it will become the only work that matters. We get to design the blueprint, break down logic in high-level parts, set the vision, dictate strategy, and chart trajectory. We do the real thinking, and then we make the model grind.
In anyway, this shift brings a real danger. If we delegate execution, it becomes tempting to delegate thought gradually. LLMs make thinking feel optional. People were already reluctant to think; now they can bypass it entirely. It is unsettling to watch a statistical prediction machine stand in for reasoning. Humbling, too. Maybe we're not as special as we assumed. This reminds me Ted Chiang's story "Catching Crumbs from the Table" where humanity is reduced to interpreting the outputs of a vastly superior intellect. Human scientists no longer produce breakthroughs themselves; they spend their careers reverse-engineering discoveries made by "metahumans". The tragedy is that humans are no longer the source of the insight, they are merely trying to explain metahumans' genius. The title captures the feeling really well. We're not at the table anymore. We're just gathering what falls from it. Even if things come to that, I know I'll keep thinking, keep learning, keep striving to build things. As I reflected in an earlier post on finding one's true calling, this pursuit of knowledge and creation is my dharma. That basic human drive to understand things and build things is not something an LLM can automate away. This I believe. I recently launched a free email newsletter for the blog. Subscribe here to get these essays delivered to your inbox, along with behind-the-scenes commentary and curated links on distributed systems, technology, and other curiosities.
by Murat (noreply@blogger.com)
Phil Eaton
This is an external post of mine. Click
here
if you are not redirected.
February 14, 2026
Small Datum - Mark Callaghan
This has results for HammerDB tproc-c on a small server using MySQL and Postgres. I am new to HammerDB and still figuring out how to explain and present results so I will keep this simple and just share graphs without explaining the results. tl;dr - Modern Postgres is faster than old Postgres
- Modern MySQL has large perf regressions relative to old MySQL, and they are worst at low concurrency for CPU-bound worklads. This is similar to what I see on other benchmarks.
- Modern Postgres is about 2X faster than MySQL at low concurrency (vu=1) and when the workload isn't IO-bound (w=100). But with some concurrency (vu=6) or with more IO per transaction (w=1000, w=2000) they have similar throughput. Note that partitioning is used at w=1000 and 2000 but not at w=100.
Builds, configuration and hardware
I compiled Postgres versions from source: 12.22, 13.23, 14.20, 15.15, 16.11, 17.7 and 18.1.
I compiled MySQL versions from source: 5.6.51, 5.7.44, 8.0.44, 8.4.7, 9.4.0 and 9.5.0. The server is an ASUS ExpertCenter PN53 with an AMD Ryzen 7 7735HS CPU, 8 cores, SMT disabled, and 32G of RAM. Storage is one NVMe device for the database using ext-4 with discard enabled. The OS is Ubuntu 24.04. More details on it are here.
For versions prior to 18, the config file is named conf.diff.cx10a_c8r32 and they are as similar as possible and here for versions 12, 13, 14, 15, 16 and 17. For Postgres 18 the config file is named conf.diff.cx10b_c8r32 and adds io_mod='sync' which matches behavior in earlier Postgres versions.
For both Postgres and MySQL fsync on commit is disabled to avoid turning this into an fsync benchmark. The server has an SSD with high fsync latency.
The benchmark is tproc-c from HammerDB. The tproc-c benchmark is derived from TPC-C. The benchmark was run for several workloads: - vu=1, w=100 - 1 virtual user, 100 warehouses
- vu=6, w=100 - 6 virtual users, 100 warehouses
- vu=1, w=1000 - 1 virtual user, 1000 warehouses
- vu=6, w=1000 - 6 virtual users, 1000 warehouses
- vu=1, w=2000 - 1 virtual user, 2000 warehouses
- vu=6, w=2000 - 6 virtual users, 2000 warehouses
The w=100 workloads are less heavy on IO. The w=1000 and w=2000 workloads are more heavy on IO.
- stored procedures are enabled
- partitioning is used for when the warehouse count is >= 1000
- a 5 minute rampup is used
- then performance is measured for 120 minutes
Results
My analysis at this point is simple -- I only consider average throughput. Eventually I will examine throughput over time and efficiency (CPU and IO).
On the charts that follow y-axis does not start at 0 to improve readability at the risk of overstating the differences. The y-axis shows relative throughput. There might be a regression when the relative throughput is less than 1.0. There might be an improvement when it is > 1.0. The relative throughput is: (NOPM for some-version / NOPM for base-version)
I provide three charts below: - only MySQL - base-version is MySQL 5.6.51
- only Postgres - base-version is Postgres 12.22
- Postgres vs MySQL - base-version is Postgres 18.1, some-version is MySQL 8.4.7
Results: MySQL 5.6 to 8.4 Legend:
- my5651.z12a is MySQL 5.6.51 with the z12a_c8r32 config
- my5744.z12a is MySQL 5.7.44 with the z12a_c8r32 config
- my8044.z12a is MySQL 8.0.44 with the z12a_c8r32 config
- my847.z12a is MySQL 8.4.7 with the z12a_c8r32 config
- my9400.z12a is MySQL 9.4.0 with the z12a_c8r32 config
- my9500.z12a is MySQL 9.5.0 with the z12a_c8r32 config
Summary - Perf regressions in MySQL 8.4 are smaller with vu=6 and wh >= 1000 -- the cases where there is more concurrency (vu=6) and the workload does more IO per transaction (wh=1000 & 2000). Note that partitioning is used at w=1000 and 2000 but not at w=100.
- Perf regressions in MySQL 8.4 are larger with vu=1 and even more so with wh=100 (low concurrency, less IO per transaction).
- Performance has mostly been dropping from MySQL 5.6 to 8.4. From other benchmarks the problem is from new CPU overheads at low concurrency.
- While perf regressions in modern MySQL at high concurrency have been less of a problem on other benchmarks, this server is too small to support high concurrency.
Results: Postgres 12 to 18 Legend:
- pg1222.x10a is Postgres 12.22 with the x10a_c8r32 config
- pg1323.x10a is Postgres 13.23 with the x10a_c8r32 config
- pg1420.x10a is Postgres 14.20 with the x10a_c8r32 config
- pg1515.x10a is Postgres 15.15 with the x10a_c8r32 config
- pg1611.x10a is Postgres 16.11 with the x10a_c8r32 config
- pg177.x10a is Postgres 17.7 with the x10a_c8r32 config
- pg181.x10b is Postgres 18.1 with the x10b_c8r32 config
Summary - Modern Postgres is faster than old Postgres
Results: MySQL vs Postgres Legend:
- pg181.x10b is Postgres 18.1 with the x10b_c8r32 config
- my847.z12a is MySQL 8.4.7 with the z12a_c8r32 config
Summary - MySQL and Postgres have similar throughput for vu=6 at w=1000 and 2000. Note that partitioning is used at w=1000 and 2000 but not at w=100.
- Otherwise Postgres is 2X faster than MySQL
by Mark Callaghan (noreply@blogger.com)
Franck Pachot
Relational database joins are, conceptually, a cartesian product followed by a filter (the join condition). Without that condition, you get a cross join that returns every possible combination. In MongoDB, you can model the same behavior at read time using $lookup, or at write time by embedding documents.
Example
Define two collections: one for clothing sizes and one for gender-specific fits:
db.sizes.insertMany([
{ code: "XS", neckCm: { min: 31, max: 33 } },
{ code: "S", neckCm: { min: 34, max: 36 } },
{ code: "M", neckCm: { min: 37, max: 39 } },
{ code: "L", neckCm: { min: 40, max: 42 } },
{ code: "XL", neckCm: { min: 43, max: 46 } }
]);
db.fits.insertMany([
{
code: "MEN",
description: "Straight cut, broader shoulders, narrower hips"
},
{
code: "WOMEN",
description: "Tapered waist, narrower shoulders, wider hips"
}
]);
Each collection stores independent characteristics, and every size applies to every fit. The goal is to generate all valid product variants.
Cross join on read: $lookup + $unwind
In order to add all sizes to each body shape, use a $lookup without filter condition and, as it adds them as an embedded array, use $unwind to get one document per combination:
db.sizes.aggregate([
{
$lookup: {
from: "fits",
pipeline: [],
as: "fit"
}
},
{ $unwind: "$fit" },
{ $sort: { "fit.code": 1, code: 1 } },
{
$project: {
_id: 0,
code: { $concat: ["$fit.code", "-", "$code"] }
}
}
]);
Here is the result:
Application-side
For such small static reference collections, the application may simply read both and join with loops:
const sizes = db.sizes.find({}, { code: 1, _id: 0 }).sort({ code: 1 }).toArray();
const fits = db.fits.find({}, { code: 1, _id: 0 }).sort({ code: 1 }).toArray();
for (const fit of fits) {
for (const size of sizes) {
print(`${fit.code}-${size.code}`);
}
}
While it's good to keep the reference in a database, such static data can stay in cache in the application.
Cross join on write: embed the many-to-many
Because sizes are inherently tied to body shapes (no size exists without a body shape), embedding them in the fits documents is often a better model:
db.fits.aggregate([
{
$lookup: {
from: "sizes",
pipeline: [
{ $project: { _id: 0, code: 1, neckCm:1 } },
{ $sort: { code: 1 } }
],
as: "sizes"
}
},
{
$merge: {
into: "fits",
on: "_id",
whenMatched: "merge",
whenNotMatched: "discard"
}
}
]);
Here is the new shape of the single collection:
Once embedded, the query becomes straightforward, simply unwind the embedded array:
db.fits.aggregate([
{ $unwind: "$sizes" },
{
$project: {
_id: 0,
code: {
$concat: ["$code", "-", "$sizes.code"]
}
}
}
]);
You may embed only the fields required, like the size code, or all fields like I did here with the neck size, and then remove the size collection:
Although this may duplicate the values for each body shape, it only requires using updateMany() instead of updateOne() when updating it. For example, the following updates one size:
db.fits.updateMany(
{},
{ $set: { "sizes.$[s].neckCm": { min: 38, max: 40 } } },
{
arrayFilters: [
{ "s.code": "M" }
]
}
);
Duplication has the advantage of returning all required information in a single read, without joins or multiple queries, and it is not problematic for updates since it can be handled with a single bulk update operation. Unlike relational databases—where data can be modified through ad‑hoc SQL and business rules must therefore be enforced at the database level—MongoDB applications are typically domain‑driven, with clear ownership of data and a single responsibility for performing updates.
In that context, consistency is maintained by the application's service rather than by cross‑table constraints. This approach also lets business rules evolve, such as defining different sizes for men and women, without changing the data model.
Conclusion
In a fully normalized relational model, all relationships use the same pattern: a one-to-many relationship between two tables, enforced by a primary (or unique) key on one side and a foreign key on the other. This holds regardless of cardinality (many can be three or one million), lifecycle rules (cascade deletes or updates), ownership (shared or exclusive parent), navigation direction (and access patterns). Even many-to-many relationships are just two one-to-many relationships via a junction table.
MongoDB exposes these same concepts as modeling choices—handled at read time with $lookup, at write time through embedding, or in the application—instead of enforcing a single normalized representation. The choice depends on the domain data and access patterns.
by Franck Pachot
February 13, 2026
Supabase Blog
A detailed account of the February 12 outage in us-east-2, what caused it, and the steps we are taking to prevent it from happening again.
February 12, 2026
AWS Database Blog - Amazon Aurora
In this post we introduce the blue/green deployment plugin for the AWS JDBC Driver, a built-in plugin that automatically handles connection routing, traffic management, and switchover detection during blue/green deployment switchovers. We show you how to configure and use the plugin to minimize downtime during database maintenance operations during blue/green deployment switchovers.
by Jason Pedreza
Murat Demirbas
Academic writing has long been criticized for its formulaic nature. As I wrote about earlier, research papers are unfortunately written to please 3 specific expert reviewers who are overwhelmingly from academia. Given this twisted incentive structure (looking impressive for peer-review), the papers end up becoming formulaic, defensive, and often inpenetrable. Ironically, this very uniformity makes it trivially easy for LLMs to replicate academic writing. It is easy to spot LLM use in personal essays, but I dare you to do it successfully in academic writing. Aside: Ok, I baited myself with my own dare. In general, it is very hard to detect LLM usage at the paragraph level in a research paper. But LLM usage in research papers becomes obvious when you see the same definition repeated 3-4 times across consecutive pages. The memoryless nature of LLMs causes them to recycle the same terms and phrases, and I find myself thinking "you already explained this to me four times, do you think I am a goldfish?" I have been reviewing a lot of papers recently, and this is the number one tell-tale sign. A careful read by the authors would clean this up easily, making LLM usage nearly undetectable. To be clear, I am talking about LLM assistance in polishing writing, not wholesale generation. A paper with no original ideas is a different beast entirely. They are vacuous and easy to spot.
Anyway, as LLM use become ubiquitous, conference/journal reviewing is facing a big crisis. There are simply too many articles being submitted, as it is easy to generate text and rush half-baked ideas into the presses. I am, of course, unhappy about this. Writing that feels effortless because an LLM smooths every step deprives you of the strain that produces "actual understanding". That strain in writing is not a defect; it creates the very impetus for discovering what you actually think, rather than faking/imitating thought. But here we are. We are at an inflection point in academic publishing. I recently came across this post, which documents an experiment where an LLM replicated and extended a published empirical political science paper with near-human fidelity, at a fraction of the time and cost. I have been predicting the collapse of the publishing system for a decade. The flood of LLM-aided research might finally break its back. And here is where I want to take you in this post. I want to imagine how academic writing may change in this new publishing regime. Call it a 5-10 year outlook, because at this day and age, who can predict anything beyond that. I claim that costly signals of genuine intelligence will become the currency of survival in this new environment. Costly signals work because they are expensive to fake, like a peacock’s tail or an elk’s antlers. And I claim academic writing will increasingly demand features that are expensive to fake. Therefore, a distinctive voice becomes more valuable precisely because it cannot be generated without genuine intellectual engagement. Personal narratives, peculiar perspectives, unexpected conceptual leaps, and field-specific cultural fluency are things that require deep immersion and creative investment that LLMs lack. These are the costly signals that will make a paper worth publishing. Literature reviews are cheap to automate, so they will shrink --as we are already seeing. But reviews with distinctive voice and genuine insight, ones that reflect on the author's own learning and thought process, will survive. Work that builds creative frameworks and surprising connections, which are expensive to produce, will flourish. When anyone can generate competent prose, only writing that screams "a specific human spent serious time thinking about this" will cut through. So, LLMs may accidentally force academia toward what it always claimed to value: original thinking and clear communication. The costliest signal of all is having something genuinely new to say, and saying well. I am an optimist, as you can easily tell, if you are a long time reader of this blog. “Simplicity and elegance are unpopular because they require hard work and discipline to achieve and education to be appreciated.” -- Edsger W. Dijkstra
by Murat (noreply@blogger.com)
February 11, 2026
AWS Database Blog - Amazon Aurora
In this post, we demonstrate how to efficiently migrate relational-style data from NoSQL to Aurora DSQL, using Kiro CLI as our generative AI tool to optimize schema design and streamline the migration process.
by Ramesh Raghupathy
Franck Pachot
Prisma is an ORM (Object-Relational Mapper). With MongoDB, it acts as an Object Document Mapper, mapping collections to TypeScript models and providing a consistent, type-safe query API.
MongoDB is a document database with a flexible schema. Prisma does not provide schema migrations for MongoDB, but it supports nested documents and embedded types to take advantage of MongoDB’s data locality.
This article walks through a minimal “Hello World” setup on a Docker environment:
- Run MongoDB as a replica set
- Connect to it using Prisma
- Insert a "Hello World" document
- Read and display all documents
Start MongoDB as a replica set
Prisma requires MongoDB to run as a replica set. While MongoDB supports many operations without transactions, Prisma relies on MongoDB sessions and transactional behavior internally, which are only available on replica sets.
Start MongoDB in a Docker container with replica set support enabled:
docker run --name mg -d mongo --replSet rs0
Initialize the replica set (a single‑node replica set is sufficient for local development and testing):
docker exec -it mg mongosh --eval "rs.initiate()"
Start a Node.js container
Start a Node.js container that can access MongoDB using the hostname mongo:
docker run --rm -it --link mg:mongo node bash
Prepare the Node.js environment
Update the package manager, install an editor, update npm, disable funding messages, and move to the working directory:
apt-get update
apt-get install -y vim
npm install -g npm@11.9.0
npm config set fund false
cd /home
Install Prisma Client and enable ES modules
Install Prisma Client and enable ES modules by adding "type": "module" to package.json:
npm install @prisma/client@6.19.0
sed -i '1s/{/{\n "type": "module",/' package.json
Using ES modules enables standard import syntax and aligns the project with modern Node.js tooling.
Install Prisma CLI and TypeScript tooling
Install the Prisma CLI and supporting tooling, and generate the initial Prisma configuration:
npm install -D prisma@6.19.0 @types/node
npm install -D tsx
npx prisma init
Configure the Prisma schema
Edit prisma/schema.prisma, change the provider from postgresql to mongodb, and define a minimal Message model:
generator client {
provider = "prisma-client"
output = "../generated/prisma"
}
datasource db {
provider = "mongodb"
url = env("DATABASE_URL")
}
model Message {
id String @id @default(auto()) @map("_id") @db.ObjectId
content String
createdAt DateTime @default(now())
}
Prisma maps MongoDB’s _id field to a String backed by an ObjectId.
The prisma-client generator produces TypeScript output in a custom directory to avoid using @prisma/client.
Configure the database connection
Define the MongoDB connection string in .env:
DATABASE_URL="mongodb://mongo:27017/test"
Prisma reads DATABASE_URL at generation time, while the application reads it at runtime. Importing dotenv/config ensures both environments are consistent.
Generate the Prisma client
Generate the Prisma client from the schema:
This produces TypeScript client files in generated/prisma.
Write the “Hello World” program
Create prisma/index.ts:
import 'dotenv/config'
import { PrismaClient } from '../generated/prisma/client.ts'
const prisma = new PrismaClient()
async function main() {
await prisma.$connect()
console.log('Connected to MongoDB')
await prisma.message.create({
data: {
content: 'Hello World',
},
})
const messages = await prisma.message.findMany()
console.log('Messages in database:')
for (const message of messages) {
console.log(`- ${message.content} at ${message.createdAt}`)
}
}
main()
.catch(console.error)
.finally(() => prisma.$disconnect())
This program connects to MongoDB, inserts a “Hello World” document, and prints all stored messages.
Run the program
For running TypeScript directly in modern Node.js projects, tsx is generally preferred over ts-node due to better ESM support and faster startup.
Execute the TypeScript file:
Output:
Connected to MongoDB
Messages in database:
- Hello World at Wed Feb 11 2026 17:36:08 GMT+0000 (Coordinated Universal Time)
Conclusion and final note on schemas in MongoDB
This example shows a minimal Prisma + MongoDB setup:
- MongoDB running as a replica set
- Prisma configured for MongoDB
- A single model with one insert and one read
From here, you can add schema evolution, indexes, and more complex queries while keeping the same core configuration.
MongoDB is often called schemaless, but that’s misleading in practice, as we started to declare the database schema in schema.prisma and generate the client for it. Real‑world MongoDB applications are schema‑driven, with structure defined in the application layer through models, validation rules, and access patterns.
Unlike relational databases—where the schema is enforced in the database and then mapped into the application—MongoDB uses the same document structure across all layers: in‑memory cache, on‑disk storage, and application models. This preserves data locality, avoids ORM overhead and migration scripts, and simplifies the development.
Prisma makes this explicit by defining the schema in code, providing type safety and consistency while keeping MongoDB’s document model flexible as your application evolves.
by Franck Pachot
Murat Demirbas
The crux of this chapter is how to schedule tasks without perfect knowledge. If you remember from the previous chapter, the core tension in CPU scheduling is these two conflicting goals: - Minimizing Turnaround Time: Usually achieved by running shorter jobs first (SJF).
- Minimizing Response Time: Usually achieved by Round Robin scheduling (RR). Essential for interactive users.
Unfortunately, the OS does not have a crystal ball. It doesn't know if a process is a short interactive job or a massive number-crunching batch job. The Multi-Level Feedback Queue (MLFQ) solves this by encoding/capturing information from history of the job, and assumes that if a job has been CPU-intensive in the past, it likely will be in the future. As we'll see below, it also gives a chance for jobs to redeem themselves through the boosting process. I really enjoyed this chapter. MLFQ, invented by Corbato in 1962, is a brilliant scheduling algorithm. This elegant solution served as the base scheduler for many systems, including BSD UNIX derivatives, Solaris, and Windows NT and subsequent Windows operating systems. (This is part of our series going through OSTEP book chapters.)
How MLFQ Works: The Basic RulesThe chapter constructs the MLFQ algorithm iteratively, starting with a basic structure involving distinct queues, each with a different priority level. - Rule 1: If Priority(A) > Priority(B), A runs.
- Rule 2: If Priority(A) = Priority(B), they run in Round-Robin.
But how does a job get its priority? - Rule 3: New jobs start at the highest priority.
- Rule 4 (Initial Version): If a job uses up its time allotment, it moves down a queue. If it gives up the CPU (e.g., for I/O) before the time is up, it stays at the same priority.
This setup cleverly approximates Shortest Job First. Because the scheduler assumes every new job is short (giving it high priority), true short jobs finish quickly. Long jobs eventually exhaust their time slices and sink to the bottom queues, where they run only when the system isn't busy with interactive tasks.
Patching the initial MLFQ rulesHowever, this basic version has fatal flaws. - If too many interactive jobs flood the system, low-priority background jobs might starve.
- A clever user could rewrite a program to yield the CPU (say through I/O: writing to a dummy file) just before its time slice ends. This resets the job's allotment, allowing it to monopolize the CPU at the highest priority.
- A job that starts CPU-intensive but becomes interactive later (like a compiler finishing and waiting for input) would be stuck at the bottom priority.
To fix these issues, the chapter introduces two crucial modifications. The Priority Boost: To prevent low-priority jobs from starving, the scheduler employs Rule 5: After a set time period (S), all jobs are moved back to the topmost queue. This "boost" ensures that CPU-bound jobs get at least some processing time and allows jobs that have become interactive to return to a high-priority state. Better Accounting: To stop users from gaming the system, the scheduler rewrites Rule 4 regarding how it tracks time. Rule 4: Instead of resetting the allotment every time a job yields the CPU, the scheduler tracks the total time a job uses at a given priority level. Once the allotment is used up (regardless of how many times the job yielded the CPU) it is demoted.
Tuning MLFQThe remaining piece of the puzzle is parameterization. An MLFQ requires choosing the number of queues, the time slice length for each, and the frequency of the priority boost. There are no easy answers to these questions, and finding a satisfactory balance often requires deep experience with specific workloads. For example, most implementations employ varying time-slice lengths, assigning short slices (e.g., 10 ms) to high-priority queues for responsiveness and longer slices (e.g., 100s of ms) to low-priority queues for efficiency. Furthermore, the priority boost interval is often referred to as a "voodoo constant" because it requires magic to set correctly; if the value is too high, jobs starve, but if it is too low, interactive performance suffers. MLFQ is a milestone in operating systems design. It delivers strong performance for interactive jobs without prior knowledge of job length, while remaining fair to long-running tasks. As noted earlier, it became the base scheduler for many operating systems, with several variants refining the core idea. One notable variant is the decay-usage approach used in FreeBSD 4.3. Instead of using fixed priority tables (as in Solaris), it computes priority using a mathematical function of recent CPU usage. Running increases a job’s usage counter and lowers its priority, while the passage of time decays this counter. Decay plays the same role as periodic priority boosts. As usage fades, priority rises, ensuring long-running jobs eventually run and allowing jobs that shift from CPU-bound to interactive to regain high priority.
TLA+ model I used Gemini to write a TLA+ model of the MLFQ algorithm here. To run this MLFQ TLA+ model at Spectacle for visualization, click this link and it will open the model on your browser, no installation or plugin required. What you will see is the initial state. Click on any enabled action to take it, you can go back and forward on the right pane to explore the execution. And you can share a URL back with anyone to point to an interesting state or trace, just like I did here.
by Murat (noreply@blogger.com)
February 10, 2026
AWS Database Blog - Amazon Aurora
In this post, we discuss configuring AWS DMS tasks to migrate HierarchyID columns from SQL Server to Aurora PostgreSQL-Compatible efficiently.
by Shashank Kalki
Franck Pachot
MongoDB guarantees durability—the D in ACID—over the network with strong consistency—the C in the CAP theorem—by default. It still maintains high availability: in the event of a network partition, the majority of nodes continue to serve consistent reads and writes transparently, without raising errors to the application.
A consensus protocol based on Raft is used to achieve this at two levels:
- Writes are directed to the shard's primary, which coordinates consistency between the collection and the indexes. Raft is used to elect one replica as primary, with the others acting as secondaries.
- Writes to the shard's primary are replicated to the secondaries and acknowledged once a majority has guaranteed durability on persistent storage. The equivalent of the Raft log is the data itself—the transaction oplogs.
It's important to distinguish the two types of consensus involved: one for controlling replica roles and one for the replication of data itself. By comparison, failover automation around monolithic databases like PostgreSQL can use a consensus protocol to elect a primary (as Patroni does), but replication itself is built into PostgreSQL and does not rely on a consensus protocol—a failure in the middle may leave inconsistency between replicas.
Trade-offs between performance and consistency
Consensus on writes increases latency, especially in multi-region deployments, because it requires synchronous replication and waiting on the network, but it guarantees no data loss in disaster recovery scenarios (RPO = 0). Some workloads may prefer lower latency and accept limited data loss (for example, a couple of seconds of RPO after a datacenter burns). If you ingest data from IoT devices, you may favor fast ingestion at the risk of losing some data in such a disaster. Similarly, when migrating from another database, you might prefer fast synchronization and, in case of infrastructure failure, simply restart the migration from before the failure point. In such cases, you can use {w:1} write concern in MongoDB instead of the default {w:"majority"}.
Most failures are not full-scale disasters where an entire data center is lost, but transient issues with short network disconnections. With {w:1}, the primary risk is not data loss—because writes can be synchronized eventually—but split brain, where both sides of a network partition continue to accept writes. This is where the two levels of consensus matter:
- A new primary is elected, and the old primary steps down, limiting the split-brain window to a few seconds.
- With the default
{w:"majority"}, writes that cannot reach a majority are not acknowledged on the side of the partition without a quorum. This prevents split brain. However, with {w:1}, those writes are acknowledged until the old primary steps down.
Because the failure is transient, when the old primary rejoins, no data is physically lost: writes from both sides still exist. However, these writes may conflict, resulting in a diverging database state with two branches. As with any asynchronous replication, this requires conflict resolution. MongoDB handles this as follows:
- Writes from the new primary are preserved, as this is where the application has continued to make progress.
- Writes that occurred on the old primary during the brief split-brain window are rolled back, and it pulls the more recent writes from the new primary.
Thus, when you use {w:1}, you accept the possibility of limited data loss in the event of a failure. Once the node is back, these writes are not entirely lost, but they cannot be merged automatically. MongoDB stores them as BSON files in a rollback directory so you can inspect them and perform manual conflict resolution if needed.
This conflict resolution is a Recover To a Timestamp (RTT).
Demo on a Docker lab
Let's try it. I start 3 containers as a replica set:
docker network create lab
docker run --network lab --name m1 --hostname m1 -d mongo --replSet rs0
docker run --network lab --name m2 --hostname m2 -d mongo --replSet rs0
docker run --network lab --name m3 --hostname m3 -d mongo --replSet rs0
docker exec -it m1 mongosh --eval '
rs.initiate({
_id: "rs0",
members: [
{ _id: 0, host: "m1:27017", priority: 3 },
{ _id: 1, host: "m2:27017", priority: 2 },
{ _id: 2, host: "m3:27017", priority: 1 }
]
})
'
until
docker exec -it m1 mongosh --eval "rs.status().members.forEach(m => print(m.name, m.stateStr))" |
grep -C3 "m1:27017 PRIMARY"
do sleep 1 ; done
The last command waits until m1 is the primary, as set by its priority. I do that to make the demo reproducible with simple copy-paste.
I insert "XXX-10" when connected to m1:
docker exec -it m1 mongosh --eval '
db.demo.insertOne(
{ _id:"XXX-10" , date:new Date() },
{ writeConcern: {w: "1"} }
)
'
{ acknowledged: true, insertedId: 'XXX-10' }
I disconnect the secondary m2:
docker network disconnect lab m2
With a replication factor of 3, the cluster is resilient to one failure and I insert "XXX-11", when connected to the primary:
docker exec -it m1 mongosh --eval '
db.demo.insertOne(
{ _id:"XXX-11" , date:new Date() },
{ writeConcern: {w: "1"} }
)
'
{ acknowledged: true, insertedId: 'XXX-11' }
I disconnect m1, the current primary, and reconnect m2, and immediately insert "XXX-12", still connected to m1:
docker network disconnect lab m1
docker network connect lab m2
docker exec -it m1 mongosh --eval '
db.demo.insertOne(
{ _id:"XXX-12" , date:new Date() },
{ writeConcern: {w: "1"} }
)
'
{ acknowledged: true, insertedId: 'XXX-12' }
Here, m1 is still a primary for a short period before it detects it cannot reach the majority of replicas and steps down. If the write concern was {w: "majority"} it would have waited and failed, not able to sync to the quorum, but with {w: "1"} the replication is asynchronous and the write is acknowledged when written to local disks.
Two seconds later, a similar write fails because the primary stepped down:
sleep 2
docker exec -it m1 mongosh --eval '
db.demo.insertOne(
{ _id:"XXX-13" , date:new Date() },
{ writeConcern: {w: "1"} }
)
'
MongoServerError: not primary
I wait that m2 is the new primary, as set by priority, and connect to it to insert "XXX-20":
until
docker exec -it m2 mongosh --eval "rs.status().members.forEach(m => print(m.name, m.stateStr))" |
grep -C3 "m2:27017 PRIMARY"
do sleep 1 ; done
docker exec -it m2 mongosh --eval '
db.demo.insertOne(
{ _id:"XXX-20" , date:new Date() },
{ writeConcern: {w: "1"} }
)
'
{ acknowledged: true, insertedId: 'XXX-20' }
No nodes are down, it's only a network partition, and I can read from all nodes as long as I don't connect through the network. I query the collection on each side:
docker exec -it m1 mongosh --eval 'db.demo.find()'
docker exec -it m2 mongosh --eval 'db.demo.find()'
docker exec -it m3 mongosh --eval 'db.demo.find()'
The inconsistency is visible, "XXX-12" is only in m1 and "XXX-20" only in m2 and m3:
I reconnect m1 so that all nodes can communicate and synchronize their state:
docker network connect lab m1
I query again and all nodes show the same values:
"XXX-12" has disappeared and all nodes are now synchronized to the current state. When it rejoined, m1 rolled back the operations that occurred during the split-brain window. This is expected and acceptable, since the write used a { w: 1 } write concern, which explicitly allows limited data loss in case of failure in order to avoid cross-network latency on each write.
The rolled back operations are not lost, MongoDB logged them in a rollback directory in the BSON format, with the rolled back document as well as the related oplog.
I read and decode all BSON in the rollback directory:
docker exec -i m1 bash -c '
for f in /data/db/rollback/*/removed.*.bson
do
echo "$f"
bsondump $f --pretty
done
' | egrep --color=auto '^|^/.*|.*("op":|"XXX-..").*'
The deleted document is in /data/db/rollback/0ae03154-0a51-4276-ac62-50d73ad31fe0/removed.2026-02-10T10-40-58.1.bson:
{
"_id": "XXX-12",
"date": {
"$date": {
"$numberLong": "1770719868965"
}
}
}
The deleted oplog for the related insert is in /data/db/rollback/local.oplog.rs/removed.2026-02-10T10-40-58.0.bson:
{
"lsid": {
"id": {
"$binary": {
"base64": "erR2AoFXS3mbcX4BJSiWjw==",
"subType": "04"
}
},
"uid": {
"$binary": {
"base64": "47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=",
"subType": "00"
}
}
},
"txnNumber": {
"$numberLong": "1"
},
"op": "i",
"ns": "test.demo",
"ui": {
"$binary": {
"base64": "CuAxVApRQnasYlDXOtMf4A==",
"subType": "04"
}
},
"o": {
"_id": "XXX-12",
"date": {
"$date": {
"$numberLong": "1770719868965"
}
}
},
"o2": {
"_id": "XXX-12"
},
"stmtId": {
"$numberInt": "0"
},
"ts": {
"$timestamp": {
"t": 1770719868,
"i": 1
}
},
"t": {
"$numberLong": "1"
},
"v": {
"$numberLong": "2"
},
"wall": {
"$date": {
"$numberLong": "1770719868983"
}
},
"prevOpTime": {
"ts": {
"$timestamp": {
"t": 0,
"i": 0
}
},
"t": {
"$numberLong": "-1"
}
}
}
Conclusion: beyond Raft
By default, MongoDB favors strong consistency and durability: writes use { w: "majority" }, are majority-committed, never rolled back, and reads with readConcern: "majority" never observe rolled-back data. In this mode, MongoDB behaves like a classic Raft system: once an operation is committed, it is final.
MongoDB also lets you explicitly relax that guarantee by choosing a weaker write concern such as { w: 1 }. In doing so, you tell the system: "Prioritize availability and latency over immediate global consistency." The demo shows what that implies:
- During a transient network partition, two primaries can briefly accept writes.
- Both branches of history are durably written to disk.
- When the partition heals, MongoDB deterministically chooses the majority branch.
- Operations from the losing branch are rolled back—but not discarded. They are preserved as BSON files with their oplog entries.
- The node then recovers to a majority-committed timestamp (RTT) and rolls forward.
This rollback behavior is where MongoDB intentionally diverges from vanilla Raft.
In classic Raft, the replicated log is the source of truth, and committed log entries are never rolled back. Raft assumes a linearizable, strongly consistent state machine where the application does not expect divergence. MongoDB, by contrast, comes from a NoSQL and event-driven background, where asynchronous replication, eventual consistency, and application-level reconciliation are sometimes acceptable trade-offs.
As a result:
- MongoDB still uses Raft semantics for leader election and terms, so two primaries are never elected in the same term.
- For data replication, MongoDB extends the model with Recover To a Timestamp (RTT) rollback.
- This allows MongoDB to safely support lower write concerns, fast ingestion, multi-region latency optimization, and migration workloads—without silently corrupting state.
In short, MongoDB replication is based on Raft, but adds rollback semantics to support real-world distributed application patterns. Rollbacks happen only when you explicitly allow them, never with majority writes, and they are fully auditable and recoverable.
by Franck Pachot
Supabase Blog
The Hydra team, maintainers of pg_duckdb, is joining Supabase to focus on Postgres + Analytics and Open Warehouse Architecture.
|