<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>DiffyPick Blog</title><description>Articles on database schema diffs, drift detection, and safe schema sync — from the maker of DiffyPick.</description><link>https://diffy-pick.com/</link><language>en</language><item><title>Migration files and Git do not mix</title><link>https://diffy-pick.com/blog/migration-files-and-git-do-not-mix/</link><guid isPermaLink="true">https://diffy-pick.com/blog/migration-files-and-git-do-not-mix/</guid><description>Two conflicting schema changes never produce a merge conflict, because they live in separate files. Being stored in Git is not the same as being managed by it.</description><pubDate>Wed, 02 Sep 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Your repository contains two version control systems.&lt;/p&gt;
&lt;p&gt;One is Git. The other is your migration directory — a timestamped, append-only sequence of schema changes with its own applied-state tracking in the database. Two systems, two timelines, living in the same repo. And they do not synchronize.&lt;/p&gt;
&lt;p&gt;Because migrations are files, it’s natural to assume they inherit the protections Git gives files: reviewable diffs, merge conflicts where work collides, meaningful revert, meaningful checkout. This post walks through those protections one by one and shows that, for migrations, each is quietly absent — not because Git is deficient, but because &lt;strong&gt;being stored in Git is not the same as being managed by Git&lt;/strong&gt;. I think the pattern is underdiagnosed: a lot of “weird database problems” in day-to-day development are really this one mismatch wearing different costumes.&lt;/p&gt;
&lt;h2 id=&quot;the-control-group-declarative-schema-files&quot;&gt;The control group: declarative schema files&lt;a class=&quot;heading-anchor&quot; href=&quot;#the-control-group-declarative-schema-files&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;To see what’s missing, look at a kind of file Git genuinely manages. A declarative schema file — &lt;code&gt;schema.prisma&lt;/code&gt;, &lt;code&gt;models.py&lt;/code&gt;, &lt;code&gt;schema.rb&lt;/code&gt;, an Ent or sqlc definition — cooperates with Git almost perfectly:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Diffs are readable.&lt;/strong&gt; Open the PR and you see which columns changed, in place, in context.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Conflicts happen when they should.&lt;/strong&gt; Two people touch the same table, they touch the same lines, Git flags it. A human gets involved.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Revert means something.&lt;/strong&gt; Revert the commit and the declared state is restored.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Checkout means something.&lt;/strong&gt; Switch branches and you’re looking at that branch’s schema.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is no accident. Git is built to manage &lt;em&gt;stateful files evolving over time&lt;/em&gt;. A declarative schema is exactly that.&lt;/p&gt;
&lt;p&gt;A migration directory is not that. It’s an event log wearing a file system costume. The costume is convincing — text files, in a directory, in the repo — and every Git operation degrades the moment it touches what’s underneath.&lt;/p&gt;
&lt;h2 id=&quot;editing-is-forbidden&quot;&gt;Editing is forbidden&lt;a class=&quot;heading-anchor&quot; href=&quot;#editing-is-forbidden&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Git’s core verb is “change this file.” Applied migrations must never change — edit one and every environment that already ran it now disagrees with your repo (your migration tool’s checksum validation will make this loudly clear).&lt;/p&gt;
&lt;p&gt;So the migration directory prohibits, by convention, the fundamental operation of the system that stores it. Append-only. Git will happily let you violate this at any time, of course. It has no idea the convention exists — the convention isn’t expressible in anything Git understands.&lt;/p&gt;
&lt;h2 id=&quot;the-diff-shows-the-change-not-the-result&quot;&gt;The diff shows the change, not the result&lt;a class=&quot;heading-anchor&quot; href=&quot;#the-diff-shows-the-change-not-the-result&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;A schema PR’s diff says: one new file. Inside it, an imperative step — &lt;code&gt;ALTER TABLE users ADD COLUMN plan_id bigint REFERENCES plans&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;What the reviewer actually needs to evaluate is the &lt;em&gt;resulting&lt;/em&gt; table. Does &lt;code&gt;users&lt;/code&gt; already have an index that covers this? Is there a conflicting nullable pattern? What does the table even look like now? None of that is in the diff — and none of it &lt;em&gt;can&lt;/em&gt; be, because the event-log format only ever states deltas. Code review works because the diff shows the future state in context. Migration review inspects a delta against a state that isn’t on the screen, and the reviewer reconstructs the current schema from memory — or, more honestly, doesn’t.&lt;/p&gt;
&lt;h2 id=&quot;conflicts-dont-happen-when-they-should&quot;&gt;Conflicts don’t happen when they should&lt;a class=&quot;heading-anchor&quot; href=&quot;#conflicts-dont-happen-when-they-should&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;This is the one that costs real incidents.&lt;/p&gt;
&lt;p&gt;Two developers, two branches, both modify the &lt;code&gt;users&lt;/code&gt; table in incompatible ways. If schemas were declarative files, they’d edit the same lines, and merging would produce a conflict. Someone would look.&lt;/p&gt;
&lt;p&gt;With migrations, each developer creates a &lt;strong&gt;new file&lt;/strong&gt;. Different filenames. Textually, there is nothing to conflict. Both branches merge clean. Both migrations apply. The changes are semantically incompatible, and no merge machinery — Git’s or anyone’s — will ever see it, &lt;strong&gt;because the migration format guarantees that concurrent changes never meet in the same file&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Git protects you when concurrent work collides in the same place. Migrations arrange for concurrent work never to collide in the same place. The safety mechanism isn’t weakened; it’s unplugged — by the format, not by the tool.&lt;/p&gt;
&lt;h2 id=&quot;merge-order-and-apply-order-disagree&quot;&gt;Merge order and apply order disagree&lt;a class=&quot;heading-anchor&quot; href=&quot;#merge-order-and-apply-order-disagree&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Branch A is cut first and adds a migration timestamped Monday. Branch B is cut later, migration timestamped Friday. B merges first.&lt;/p&gt;
&lt;p&gt;The repository now shows Monday → Friday. Your shared environment applied Friday first, then Monday arrived. History as recorded and history as executed disagree — and since most tools track applied migrations as a &lt;em&gt;set&lt;/em&gt;, nothing notices. The repo carries one timeline, each database carries its own, and the format has no way to state which is authoritative.&lt;/p&gt;
&lt;p&gt;(Fairness requires a caveat: Django models migration dependencies as an explicit DAG and forces branch divergence to be resolved with a merge migration. This is a genuinely better design, and it narrows this particular failure. Maturity here varies a lot across ecosystems — most timestamp-ordered tools have nothing equivalent.)&lt;/p&gt;
&lt;h2 id=&quot;revert-doesnt-revert&quot;&gt;Revert doesn’t revert&lt;a class=&quot;heading-anchor&quot; href=&quot;#revert-doesnt-revert&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;git revert&lt;/code&gt; on a code commit restores the previous behavior. &lt;code&gt;git revert&lt;/code&gt; on a migration commit deletes a file. The database keeps the change; the history table keeps the record; the repo now claims a migration that the database remembers applying doesn’t exist. You haven’t undone anything — you’ve manufactured a new inconsistency.&lt;/p&gt;
&lt;p&gt;The verb worked. It did what it does to files. It’s just that what you needed reverted was never in the files.&lt;/p&gt;
&lt;h2 id=&quot;checkout-doesnt-check-out&quot;&gt;Checkout doesn’t check out&lt;a class=&quot;heading-anchor&quot; href=&quot;#checkout-doesnt-check-out&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;The everyday one. You run migrations on a feature branch, then switch back to &lt;code&gt;main&lt;/code&gt;. The code time-travels back. Your local database stays in the future.&lt;/p&gt;
&lt;p&gt;Bounce between a few feature branches for a week and your local database reaches a state that corresponds to no branch at all — a personal, unreproducible accumulation. I’d wager a meaningful fraction of “works on my machine / breaks only on my machine” traces back to exactly this. And because the state is unreproducible, nobody can share or verify it.&lt;/p&gt;
&lt;h2 id=&quot;the-snapshot-patch&quot;&gt;The snapshot patch&lt;a class=&quot;heading-anchor&quot; href=&quot;#the-snapshot-patch&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;The ecosystem noticed. That’s why Rails maintains &lt;code&gt;schema.rb&lt;/code&gt; / &lt;code&gt;structure.sql&lt;/code&gt;, and why Prisma treats the declarative definition as the source that migrations are &lt;em&gt;generated from&lt;/em&gt;. The fix is telling: put a &lt;strong&gt;declarative file&lt;/strong&gt; back in the repo — reshape the schema into the kind of file Git actually manages — and diffs become readable again, conflicts fire again.&lt;/p&gt;
&lt;p&gt;It’s a real improvement and worth using. Two limits keep it a patch rather than a fix:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;The snapshot is &lt;strong&gt;the output of replaying migrations&lt;/strong&gt; — it describes what the files claim, not what any actual database is. Drift is invisible to it by construction.&lt;/li&gt;
&lt;li&gt;It’s one file everyone’s changes flow through, which makes it a conflict magnet of its own.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The snapshot repairs the repo’s relationship with &lt;em&gt;intended&lt;/em&gt; schema state, precisely by giving Git a file in the shape it was built for. The relationship between intended and actual state — the drift problem — is untouched, and no file in the repository can touch it, because the information isn’t in the repository. It’s in the database.&lt;/p&gt;
&lt;h2 id=&quot;what-to-take-from-this&quot;&gt;What to take from this&lt;a class=&quot;heading-anchor&quot; href=&quot;#what-to-take-from-this&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Nothing here says migrations are wrong. Append-only event logs are how you replay changes across environments; that job is real and migrations do it. And nothing here says Git is deficient — every operation above did exactly what it does to files.&lt;/p&gt;
&lt;p&gt;The point is narrower and more actionable: &lt;strong&gt;a workflow doesn’t earn Git’s protections by being made of files.&lt;/strong&gt; Git’s guarantees attach to a specific kind of file — mutable, stateful, edited in place — and migrations are deliberately none of those things. For schema changes, what Git supplies is storage and ordering. Conflict detection, meaningful review context, revert, state synchronization: every protection you rely on for code is absent here, and was never on offer.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;So the safety has to come from somewhere Git can’t see: the databases themselves.&lt;/strong&gt; The repo states what the schema should be; only the live databases know what it is. Compare the two — the &lt;a href=&quot;https://diffy-pick.com/blog/how-to-compare-two-postgresql-schemas/&quot;&gt;drift check from earlier in this series&lt;/a&gt; is one concrete way — and treat the repo’s version as a claim to be tested, not a fact.&lt;/p&gt;
&lt;p&gt;Your migrations are in Git. That is storage, not management — and the difference is where the incidents come from.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;&lt;em&gt;I build &lt;a href=&quot;https://diffy-pick.com/&quot;&gt;DiffyPick&lt;/a&gt;, a desktop tool for comparing what your databases actually are — the part no file in your repo can tell you.&lt;/em&gt;&lt;/p&gt;
</content:encoded><category>database-migration</category><category>git</category></item><item><title>What Flyway validate actually checks</title><link>https://diffy-pick.com/blog/what-flyway-validate-actually-checks/</link><guid isPermaLink="true">https://diffy-pick.com/blog/what-flyway-validate-actually-checks/</guid><description>Flyway validate compares file checksums, not your database. What Flyway and Liquibase each verify, what drift detection costs, and what falls through.</description><pubDate>Mon, 17 Aug 2026 00:00:00 GMT</pubDate><content:encoded>&lt;blockquote&gt;
&lt;p&gt;Tool capabilities and edition boundaries described here are accurate as of publication. Both vendors move these lines; check current documentation before making decisions based on this post.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;There’s a specific moment this post is written for: your pipeline runs &lt;code&gt;flyway validate&lt;/code&gt;, everything is green, and you conclude that your database matches your migrations.&lt;/p&gt;
&lt;p&gt;That conclusion doesn’t follow. Not because Flyway is broken, but because &lt;code&gt;validate&lt;/code&gt; answers a different question than the one you’re asking.&lt;/p&gt;
&lt;h2 id=&quot;what-validate-does&quot;&gt;What validate does&lt;a class=&quot;heading-anchor&quot; href=&quot;#what-validate-does&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;When Flyway applies a migration, it computes a checksum of the file (CRC32 for SQL migrations) and stores it in the &lt;code&gt;flyway_schema_history&lt;/code&gt; table alongside the version, description, and execution details.&lt;/p&gt;
&lt;p&gt;&lt;code&gt;validate&lt;/code&gt; recomputes checksums for the local migration files and compares them against what’s stored in the history table. It also checks that applied migrations still exist locally and flags pending or missing ones.&lt;/p&gt;
&lt;p&gt;In other words, &lt;code&gt;validate&lt;/code&gt; answers: &lt;strong&gt;“Have the migration files changed since they were applied, and is the file set consistent with the recorded history?”&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;It’s a file-integrity check. A good one — it catches the classic incident where someone edits an already-applied migration (even just fixing a typo in a comment) and every environment that applied the original version now disagrees with the repository.&lt;/p&gt;
&lt;h2 id=&quot;what-it-never-looks-at&quot;&gt;What it never looks at&lt;a class=&quot;heading-anchor&quot; href=&quot;#what-it-never-looks-at&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Notice what’s absent from that description: &lt;strong&gt;the schema itself.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;&lt;code&gt;validate&lt;/code&gt; reads exactly one thing from your database — the history table. It never inspects your actual tables, columns, indexes, or constraints. So:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Run a manual &lt;code&gt;ALTER TABLE&lt;/code&gt; in production → files unchanged, history unchanged → &lt;strong&gt;validate passes&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Another tool adds an index → &lt;strong&gt;validate passes&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Someone drops a constraint during an incident and forgets → &lt;strong&gt;validate passes&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;None of this is a bug. The tool is doing precisely what it’s scoped to do. The problem is the widespread belief that it’s scoped to do more.&lt;/p&gt;
&lt;p&gt;A useful way to hold it in your head:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Question&lt;/th&gt;
&lt;th&gt;Does &lt;code&gt;validate&lt;/code&gt; answer it?&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Were my migration files edited after being applied?&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Are there migrations recorded as applied that no longer exist locally?&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Are there pending migrations not yet applied?&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Does my database schema match what the migrations describe?&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;No&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Did anyone change the schema outside of Flyway?&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;No&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The last two rows are schema drift, and they are outside the jurisdiction of every command in the default Flyway workflow.&lt;/p&gt;
&lt;p&gt;A related footnote: &lt;code&gt;repair&lt;/code&gt; doesn’t fix your schema either. It updates the history table — realigning stored checksums with current files, removing failed migration entries. It’s history bookkeeping, same jurisdiction as &lt;code&gt;validate&lt;/code&gt;.&lt;/p&gt;
&lt;h2 id=&quot;what-flyway-actually-offers-for-drift&quot;&gt;What Flyway actually offers for drift&lt;a class=&quot;heading-anchor&quot; href=&quot;#what-flyway-actually-offers-for-drift&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Flyway does address real drift detection — outside the free core workflow.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;check -drift&lt;/code&gt;&lt;/strong&gt; compares a target database against the state your migrations should have produced, using snapshots, and generates a report. Recent versions can store a snapshot inside the target database on each successful deploy (&lt;code&gt;-saveSnapshot&lt;/code&gt;), so the next run can detect anything that changed out-of-band since. It can also gate deployments: drift found, deployment aborted. This lives in the paid tier — at the time of writing, Flyway’s editions have consolidated such that drift detection is an Enterprise capability.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Community Drift Check&lt;/strong&gt; exists for PostgreSQL specifically: it compares schema state after your last migration against the state found before the next run, and surfaces out-of-process changes. The nuance: results are viewed through Flyway Pipelines, Redgate’s hosted service — so “free” here means free-with-an-account, wired into their platform.&lt;/p&gt;
&lt;p&gt;Both are real capabilities. Neither is the thing most teams have running.&lt;/p&gt;
&lt;h2 id=&quot;the-liquibase-side&quot;&gt;The Liquibase side&lt;a class=&quot;heading-anchor&quot; href=&quot;#the-liquibase-side&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Liquibase draws its lines differently, and more generously at the free tier.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;&lt;code&gt;diff&lt;/code&gt;&lt;/strong&gt; compares two databases (or a database against a snapshot) and reports differences. &lt;strong&gt;&lt;code&gt;diff-changelog&lt;/code&gt;&lt;/strong&gt; goes further and emits changesets that would resolve them. Both are in the open-source distribution.&lt;/p&gt;
&lt;p&gt;Two caveats from the practical side:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Liquibase’s own documentation is direct about &lt;code&gt;diff-changelog&lt;/code&gt; output needing human inspection before deployment — some objects and dependencies can’t be represented automatically. That matches everything I’ve written elsewhere about diff generation being only part of the job.&lt;/li&gt;
&lt;li&gt;The operational conveniences around drift — like severity flags that turn “unexpected object found” into a configurable CI exit code — are Pro features. OSS gives you the comparison; making it enforce policy in a pipeline is where the paid tier starts.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Also worth knowing if you’re evaluating today: Liquibase has split OSS and Pro into distinct distributions (the 5.0 restructuring), with the OSS core slimmed down and extensions moved to on-demand installation. The capability boundaries above are exactly the kind of thing that shifts during such transitions — hence the note at the top of this post.&lt;/p&gt;
&lt;h2 id=&quot;the-gap-that-remains&quot;&gt;The gap that remains&lt;a class=&quot;heading-anchor&quot; href=&quot;#the-gap-that-remains&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Put the pieces together and the landscape looks like this:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The command everyone runs (&lt;code&gt;validate&lt;/code&gt;) checks files, not schemas.&lt;/li&gt;
&lt;li&gt;The commands that check schemas exist, but sit behind paid tiers, hosted platforms, or changelog-centric workflows with real setup cost.&lt;/li&gt;
&lt;li&gt;The result, in most teams: &lt;strong&gt;drift detection is technically available and practically absent.&lt;/strong&gt; Nobody runs it nightly. Nobody runs it before deployments. It gets run for the first time during the incident retrospective.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If you want the checking habit without the licensing conversation, the previous post in this series builds a &lt;a href=&quot;https://diffy-pick.com/blog/how-to-compare-two-postgresql-schemas/&quot;&gt;pg_dump-based drift check&lt;/a&gt; you can put in CI today. And the &lt;a href=&quot;https://diffy-pick.com/blog/why-schema-drift-goes-undetected/&quot;&gt;first post&lt;/a&gt; covers why this gap exists at the level of the workflow itself, not any particular tool.&lt;/p&gt;
&lt;p&gt;The one-sentence takeaway: &lt;strong&gt;green &lt;code&gt;validate&lt;/code&gt; means your files are consistent with your history — it is not evidence about your database.&lt;/strong&gt; Treat it as such, and the mystery production column stops being a mystery.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;&lt;em&gt;I build &lt;a href=&quot;https://diffy-pick.com/&quot;&gt;DiffyPick&lt;/a&gt;, a desktop tool for comparing live database schemas directly — the “look at the actual database” half of this problem.&lt;/em&gt;&lt;/p&gt;
</content:encoded><category>database-migration</category><category>flyway</category><category>liquibase</category></item><item><title>How to compare two PostgreSQL schemas</title><link>https://diffy-pick.com/blog/how-to-compare-two-postgresql-schemas/</link><guid isPermaLink="true">https://diffy-pick.com/blog/how-to-compare-two-postgresql-schemas/</guid><description>Comparing Postgres schemas with pg_dump and diff, a normalization script that removes most of the noise, and where the DIY approach stops.</description><pubDate>Tue, 28 Jul 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;You have two Postgres databases that are supposed to have the same schema — local and staging, staging and production, or two customer deployments — and you need to know whether they actually do.&lt;/p&gt;
&lt;p&gt;This post walks through doing it with nothing but &lt;code&gt;pg_dump&lt;/code&gt; and standard Unix tools, gets the approach to a genuinely usable state, and is honest about where it stops being enough.&lt;/p&gt;
&lt;h2 id=&quot;the-naive-version&quot;&gt;The naive version&lt;a class=&quot;heading-anchor&quot; href=&quot;#the-naive-version&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;pre class=&quot;astro-code github-dark&quot; style=&quot;background-color:#24292e;color:#e1e4e8; overflow-x: auto;&quot; tabindex=&quot;0&quot; data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;pg_dump&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --schema-only&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;$DB_A&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; &amp;gt;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; a.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;pg_dump&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --schema-only&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;$DB_B&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; &amp;gt;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; b.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;diff&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -u&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; a.sql&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; b.sql&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This works, in the sense that real differences will appear in the output. The problem is what appears alongside them. On any two databases with separate histories, you’ll see:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Ownership and privilege lines&lt;/strong&gt; (&lt;code&gt;ALTER TABLE ... OWNER TO ...&lt;/code&gt;) differing because the role names differ&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Version-dependent preamble&lt;/strong&gt; — &lt;code&gt;SET&lt;/code&gt; statements that vary with the server and pg_dump version&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Object ordering differences.&lt;/strong&gt; pg_dump orders output by dependency and OID, and OIDs reflect each database’s creation history. The same table can appear at a different position in each dump&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Comment lines&lt;/strong&gt; (&lt;code&gt;-- Name: users; Type: TABLE; ...&lt;/code&gt;) that shift with the ordering&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Representation differences&lt;/strong&gt; — a column created as &lt;code&gt;serial&lt;/code&gt; versus one created as an identity column dumps as structurally different SQL even when the runtime behavior is equivalent&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The signal is in there, but you’re reading dozens of hunks to find the three that matter. And attention degrades: after a screen of noise, humans start scrolling past real differences too.&lt;/p&gt;
&lt;h2 id=&quot;step-1-cut-the-flags&quot;&gt;Step 1: cut the flags&lt;a class=&quot;heading-anchor&quot; href=&quot;#step-1-cut-the-flags&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Two options remove entire noise categories at the source:&lt;/p&gt;
&lt;pre class=&quot;astro-code github-dark&quot; style=&quot;background-color:#24292e;color:#e1e4e8; overflow-x: auto;&quot; tabindex=&quot;0&quot; data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;pg_dump&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --schema-only&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --no-owner&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --no-privileges&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;$DB_A&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; &amp;gt;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; a.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;pg_dump&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --schema-only&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --no-owner&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --no-privileges&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;$DB_B&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; &amp;gt;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; b.sql&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you don’t intend to compare grants and ownership — and across environments you usually don’t, since roles legitimately differ — remove them from the dump rather than filtering them later.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Be clear about what this trades away, though.&lt;/strong&gt; With these flags, privilege drift is invisible by construction: a &lt;code&gt;GRANT&lt;/code&gt; quietly missing in production — or an extra one quietly granted — will never appear in this comparison, and a permissions gap can matter more than a mystery column. If privileges are part of what you need to verify, keep &lt;code&gt;--no-owner&lt;/code&gt;, drop &lt;code&gt;--no-privileges&lt;/code&gt;, and absorb the legitimate role-name differences with the allow-list from Step 3 instead. (Credit to a commenter on the previous post for calling out that the original version of this article took the noise reduction without stating its cost.)&lt;/p&gt;
&lt;h2 id=&quot;step-2-normalize&quot;&gt;Step 2: normalize&lt;a class=&quot;heading-anchor&quot; href=&quot;#step-2-normalize&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;The remaining big two are comments/preamble and ordering. Both can be handled with a short script:&lt;/p&gt;
&lt;pre class=&quot;astro-code github-dark&quot; style=&quot;background-color:#24292e;color:#e1e4e8; overflow-x: auto;&quot; tabindex=&quot;0&quot; data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#6A737D&quot;&gt;#!/usr/bin/env bash&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#6A737D&quot;&gt;# normalize-schema.sh — make pg_dump output diff-friendly&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt;set&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -euo&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; pipefail&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;normalize&lt;/span&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;() {&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;  grep&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -vE&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &apos;^(--|SET |SELECT pg_catalog|\\(un)?restrict)&apos;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt;$1&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; |&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;    sed&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -E&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &apos;s/[[:space:]]+$//&apos;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; |&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;    awk&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &apos;BEGIN{blank=0} /^$/{blank++; if(blank&amp;gt;1) next} /./{blank=0} {print}&apos;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; |&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;    awk&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &apos;BEGIN{RS=&quot;&quot;; ORS=&quot;\0&quot;} {print}&apos;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; |&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;    sort&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -z&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; |&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;    tr&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &apos;\0&apos;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &apos;\n&apos;&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;}&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;normalize&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt;$1&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; &amp;gt;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt;${1&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt;%&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;.&lt;/span&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;sql&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt;}&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;.normalized.sql&quot;&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Line by line: drop comment lines, &lt;code&gt;SET&lt;/code&gt; statements, catalog selects, and &lt;code&gt;\restrict&lt;/code&gt;/&lt;code&gt;\unrestrict&lt;/code&gt; markers (recent pg_dump versions emit these with a random token that differs on every run — a guaranteed false positive if left in); strip trailing whitespace; collapse blank-line runs; then treat each blank-line-separated block as a record and sort the blocks.&lt;/p&gt;
&lt;pre class=&quot;astro-code github-dark&quot; style=&quot;background-color:#24292e;color:#e1e4e8; overflow-x: auto;&quot; tabindex=&quot;0&quot; data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;./normalize-schema.sh&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; a.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;./normalize-schema.sh&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; b.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;diff&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -u&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; a.normalized.sql&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; b.normalized.sql&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The interesting step is the last one. Treating each blank-line-separated block as a record and sorting the blocks means every &lt;code&gt;CREATE TABLE&lt;/code&gt;, &lt;code&gt;CREATE INDEX&lt;/code&gt;, and &lt;code&gt;ALTER TABLE ... ADD CONSTRAINT&lt;/code&gt; statement lands at a deterministic position regardless of the order pg_dump emitted it. Ordering noise disappears entirely; what’s left in the diff is actual structural difference.&lt;/p&gt;
&lt;p&gt;To put numbers on it: I tested this against two databases with three planted differences (a &lt;code&gt;NOT NULL&lt;/code&gt; mismatch, an extra column, a production-only index) plus deliberately different object creation order. The naive diff was 141 lines. Normalized: 71 lines, all three differences clearly visible, and — verified separately with two identically-schemed databases built in different order — &lt;strong&gt;zero false positives from ordering.&lt;/strong&gt; The output is also deterministic: same input, byte-identical output, every run.&lt;/p&gt;
&lt;p&gt;This gets you surprisingly far. For a periodic “are these two databases still the same” check, it’s most of the way there.&lt;/p&gt;
&lt;h2 id=&quot;what-it-still-cant-see-through&quot;&gt;What it still can’t see through&lt;a class=&quot;heading-anchor&quot; href=&quot;#what-it-still-cant-see-through&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Some noise is semantic, and text processing can’t normalize it:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;serial&lt;/code&gt; vs. identity.&lt;/strong&gt; One database was built with &lt;code&gt;serial&lt;/code&gt;, the other migrated to &lt;code&gt;GENERATED BY DEFAULT AS IDENTITY&lt;/code&gt;. The dumps differ substantially; the behavior barely does. Whether this counts as drift is a judgment call — it may be a migration in progress.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Default expression spelling.&lt;/strong&gt; &lt;code&gt;now()&lt;/code&gt; vs. &lt;code&gt;CURRENT_TIMESTAMP&lt;/code&gt;, &lt;code&gt;&apos;0&apos;::integer&lt;/code&gt; vs. &lt;code&gt;0&lt;/code&gt;. Same effect, different text.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Type aliases.&lt;/strong&gt; &lt;code&gt;timestamp without time zone&lt;/code&gt; is stable in dumps, but defaults and check constraints echo whatever spelling they were created with.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Constraint names.&lt;/strong&gt; Auto-generated names (&lt;code&gt;users_email_key&lt;/code&gt;, &lt;code&gt;fk_orders_user_id&lt;/code&gt;) can differ between databases even when the constraints are identical, and every dependent line differs with them.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Each of these produces a diff hunk that a human has to look at and classify. Which is fine at low volume — the point of normalization is to get the volume low enough that classification is feasible.&lt;/p&gt;
&lt;h2 id=&quot;step-3-make-it-a-drift-check&quot;&gt;Step 3: make it a drift check&lt;a class=&quot;heading-anchor&quot; href=&quot;#step-3-make-it-a-drift-check&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Once the comparison is cheap, run it on a schedule. The most useful variant compares &lt;strong&gt;what migrations claim&lt;/strong&gt; against &lt;strong&gt;what production is&lt;/strong&gt;:&lt;/p&gt;
&lt;pre class=&quot;astro-code github-dark&quot; style=&quot;background-color:#24292e;color:#e1e4e8; overflow-x: auto;&quot; tabindex=&quot;0&quot; data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#6A737D&quot;&gt;# 1. Build the expected schema: replay all migrations on a scratch database&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;createdb&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; scratch_expected&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;migrate&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -database&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;$SCRATCH_URL&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; up&lt;/span&gt;&lt;span style=&quot;color:#6A737D&quot;&gt;   # your migration tool here&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#6A737D&quot;&gt;# 2. Dump both sides and normalize&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;pg_dump&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --schema-only&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --no-owner&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --no-privileges&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;$SCRATCH_URL&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; &amp;gt;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; expected.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;pg_dump&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --schema-only&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --no-owner&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --no-privileges&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; &quot;&lt;/span&gt;&lt;span style=&quot;color:#E1E4E8&quot;&gt;$PROD_URL&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt;&quot;&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt;    &amp;gt;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; actual.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;./normalize-schema.sh&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; expected.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;./normalize-schema.sh&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; actual.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#6A737D&quot;&gt;# 3. Fail loudly on any difference&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;diff&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -u&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; expected.normalized.sql&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; actual.normalized.sql&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Wire that into CI as a nightly job and you’ve closed the biggest gap in a migration-based workflow: drift now surfaces in days, not on the next incident.&lt;/p&gt;
&lt;p&gt;One thing you will immediately need: an &lt;strong&gt;allow-list&lt;/strong&gt;. Some differences are intentional — the production-only index, the replica-only tuning — and they’ll show up every night. Grep them out with a maintained pattern file, and accept the trade-off you’ve just made: the allow-list is itself a small new convention someone has to maintain. In exchange, your intentional drift is now written down for the first time, which is more than most teams have.&lt;/p&gt;
&lt;h2 id=&quot;where-diy-stops&quot;&gt;Where DIY stops&lt;a class=&quot;heading-anchor&quot; href=&quot;#where-diy-stops&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Everything above answers &lt;em&gt;detection&lt;/em&gt;. If detection is all you need — you review the nightly diff and fix things through your normal migration flow — the scripts on this page are honestly enough, and you should use them.&lt;/p&gt;
&lt;p&gt;What they don’t give you:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Applying the difference.&lt;/strong&gt; The diff shows &lt;code&gt;name character varying(255)&lt;/code&gt; on one side and &lt;code&gt;NOT NULL&lt;/code&gt; on the other. Producing the &lt;code&gt;ALTER TABLE&lt;/code&gt; statements — in a form that accounts for what’s actually there — is still on you.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Ordering.&lt;/strong&gt; Creating a table with a foreign key requires the referenced table first. Dropping a column requires dropping dependent indexes and constraints first. With dozens of selected changes, dependency ordering becomes a real graph problem.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Safety.&lt;/strong&gt; Adding &lt;code&gt;NOT NULL&lt;/code&gt; to a column that contains NULLs fails — or worse, a type change succeeds and mangles data. Text diffs know nothing about the data the schema holds.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Semantic equivalence.&lt;/strong&gt; The &lt;code&gt;serial&lt;/code&gt;-vs-identity problem above never fully goes away in text.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;In the open-source ecosystem, &lt;a href=&quot;https://github.com/djrobstep/migra&quot;&gt;migra&lt;/a&gt; deserves a mention: it compares two live Postgres databases at the catalog level rather than the text level, and emits the &lt;code&gt;ALTER&lt;/code&gt; statements for the difference. If you live entirely in Postgres and entirely in the terminal, it solves a good chunk of items 1 and 4.&lt;/p&gt;
&lt;p&gt;Items 2 and 3 — and doing any of this across MySQL, SQL Server, or SQLite, or with someone on the team who won’t run a CLI — are the point where I stopped scripting and built &lt;a href=&quot;https://diffy-pick.com/&quot;&gt;DiffyPick&lt;/a&gt;: a desktop tool (macOS / Windows / Linux) that does the extraction, classification, dependency ordering, and pre-flight data checks, with the diff presented one line per difference. The free tier covers SQLite with no limits if you want to see the approach.&lt;/p&gt;
&lt;p&gt;But start with the script. If the nightly diff stays quiet, you may never need anything else — and you’ll know within a week either way.&lt;/p&gt;
</content:encoded><category>postgresql</category><category>schema-drift</category></item><item><title>Why schema drift goes undetected</title><link>https://diffy-pick.com/blog/why-schema-drift-goes-undetected/</link><guid isPermaLink="true">https://diffy-pick.com/blog/why-schema-drift-goes-undetected/</guid><description>Migration history records intent, not the current state of your database. Why environments diverge and nothing catches it.</description><pubDate>Sun, 26 Jul 2026 00:00:00 GMT</pubDate><content:encoded>&lt;p&gt;Here is a scenario from a project in its third year of operation. Migrations are managed properly. CI is green. &lt;code&gt;validate&lt;/code&gt; passes.&lt;/p&gt;
&lt;p&gt;Then an error shows up in production only. The production table has a column that doesn’t exist anywhere else. Nobody remembers who added it, when, or why. There is no migration file for it.&lt;/p&gt;
&lt;p&gt;The important part is not the mystery column. It’s that &lt;strong&gt;this divergence passed every check the team had&lt;/strong&gt;. Nothing was broken. The tooling worked exactly as designed — and, as designed, it does not detect this class of problem.&lt;/p&gt;
&lt;p&gt;This post is about why.&lt;/p&gt;
&lt;h2 id=&quot;code-and-databases-are-not-symmetric&quot;&gt;Code and databases are not symmetric&lt;a class=&quot;heading-anchor&quot; href=&quot;#code-and-databases-are-not-symmetric&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Git made deployment reproducible. Check out any commit and you get exactly the same tree, every time. If CI is green, the contents of that commit are settled.&lt;/p&gt;
&lt;p&gt;Migrations brought the same idea to databases: record every change as a timestamped file, apply them in order, and every environment converges on the same schema.&lt;/p&gt;
&lt;p&gt;There is an asymmetry hiding in that analogy, and it’s rarely stated outright.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Replaying history is deterministic for code.&lt;/strong&gt; The repository’s state is fully determined by its history. If someone hand-edits a file on a server, the next deploy overwrites it. The working tree is disposable; the truth lives in the history.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Replaying history is not deterministic for databases.&lt;/strong&gt; A database is a persistent entity you cannot throw away and rebuild from scratch. And it changes through paths that never touch your migration files:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;An emergency &lt;code&gt;ALTER TABLE&lt;/code&gt; run directly in a production console during an incident&lt;/li&gt;
&lt;li&gt;Changes made by another team or another tool with write access — GUI clients, BI tools, monitoring agents, batch jobs&lt;/li&gt;
&lt;li&gt;Implicit behavioral differences between engine versions: default collations, implicit type coercions, index length limits&lt;/li&gt;
&lt;li&gt;An index added by hand in production because the data volume demanded it&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;None of these leave a trace in migration history.&lt;/p&gt;
&lt;p&gt;A migration directory is an &lt;strong&gt;append-only log of intent&lt;/strong&gt;. It records what someone meant to change. It is not a representation of what the database currently is.&lt;/p&gt;
&lt;p&gt;For code, history is the truth. For a database, the truth is only in the database.&lt;/p&gt;
&lt;h2 id=&quot;two-questions-migrations-cannot-answer&quot;&gt;Two questions migrations cannot answer&lt;a class=&quot;heading-anchor&quot; href=&quot;#two-questions-migrations-cannot-answer&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;This asymmetry has practical consequences. There are two questions a migration-based workflow structurally cannot answer.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;1. What does the schema look like right now?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Migration files are a sequence of imperative steps, not a declaration. If you want to know the current definition of a table, you’d have to mentally fold hundreds of files in order. Nobody does that. By year three, everyone runs &lt;code&gt;SHOW CREATE TABLE&lt;/code&gt; or checks &lt;code&gt;\d&lt;/code&gt; in psql instead — because the live database is more trustworthy than the files.&lt;/p&gt;
&lt;p&gt;And that’s the problem. The moment “the database is more reliable than the migration files” becomes shared team knowledge, the files stop functioning as documentation. Files nobody reads decay. Decayed files get read even less.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2. When two environments differ, which one is correct?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;There is no arbiter. “Whatever you get by replaying all migrations against an empty database” sounds like an answer, but it’s circular — the whole problem is that this result and production disagree.&lt;/p&gt;
&lt;h2 id=&quot;validate-is-not-looking-at-your-database&quot;&gt;&lt;code&gt;validate&lt;/code&gt; is not looking at your database&lt;a class=&quot;heading-anchor&quot; href=&quot;#validate-is-not-looking-at-your-database&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;A common misconception is worth clearing up precisely.&lt;/p&gt;
&lt;p&gt;Flyway’s &lt;code&gt;validate&lt;/code&gt; checks that &lt;strong&gt;the checksums of applied migrations still match the files on disk&lt;/strong&gt;. It answers the question “has anyone edited a migration file after it was applied?” It does not answer “does the actual schema match what the migrations describe?”&lt;/p&gt;
&lt;p&gt;Run a manual &lt;code&gt;ALTER&lt;/code&gt; in production and &lt;code&gt;validate&lt;/code&gt; stays green. The files didn’t change, so from its point of view, nothing happened. This is correct behavior — for what the tool is scoped to do.&lt;/p&gt;
&lt;p&gt;To be fair: tooling for the other question does exist. Liquibase ships &lt;code&gt;diff&lt;/code&gt; and &lt;code&gt;diffChangeLog&lt;/code&gt; in its open-source tier. Flyway offers drift detection in its paid editions. But the former assumes a changelog-centric CLI workflow with real setup cost, and the latter sits behind a license. The practical result across most teams is the same: &lt;strong&gt;the capability exists somewhere, and nobody runs it routinely.&lt;/strong&gt; “Technically possible” and “actually happening every day” are very different states.&lt;/p&gt;
&lt;p&gt;While we’re being honest about tooling: most codebases carry down migrations that have never been executed even once. Untested code doesn’t work. “We can roll back if we need to” is, in most shops, a comforting fiction that costs writing effort on every change.&lt;/p&gt;
&lt;h2 id=&quot;the-discipline-problem&quot;&gt;The discipline problem&lt;a class=&quot;heading-anchor&quot; href=&quot;#the-discipline-problem&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Everything so far was mechanical. The deeper problem is organizational.&lt;/p&gt;
&lt;p&gt;A migration workflow looks like a technical mechanism, but what actually holds it together is a &lt;strong&gt;rule&lt;/strong&gt;: every schema change goes through a migration file. No exceptions, no direct changes, everyone, always.&lt;/p&gt;
&lt;p&gt;Rules like this have three properties that determine their fate:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;They can be broken.&lt;/li&gt;
&lt;li&gt;Things keep working when they’re broken.&lt;/li&gt;
&lt;li&gt;Breaking them is not detected.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Any process with all three properties degrades. It’s only a question of when.&lt;/p&gt;
&lt;p&gt;And this particular rule has an extra cost: it must be &lt;strong&gt;transmitted to every future participant for the lifetime of the project&lt;/strong&gt;. It doesn’t matter that the current team understands it perfectly. The engineer who joins in six months, the team that inherits the system in two years, the contractor called in for a 2 a.m. incident five years from now — the rule binds all of them, and any one of them can silently break it.&lt;/p&gt;
&lt;p&gt;The higher your staff turnover, the faster the history rots.&lt;/p&gt;
&lt;p&gt;This is the trade migrations quietly make: &lt;strong&gt;less friction today, paid for with a discipline debt assigned to people who weren’t in the room.&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;the-assumption-that-all-environments-are-identical&quot;&gt;The assumption that all environments are identical&lt;a class=&quot;heading-anchor&quot; href=&quot;#the-assumption-that-all-environments-are-identical&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;There’s one more assumption baked into migration workflows, and it’s worth dragging into the light: &lt;em&gt;every environment should have the same schema.&lt;/em&gt; Apply the same files in the same order, get the same result. Any divergence is an accident.&lt;/p&gt;
&lt;p&gt;Except some divergence is deliberate.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;An index that exists only in production, because production has 400× the data. Locally it would just slow the tests down.&lt;/li&gt;
&lt;li&gt;Partitioning applied only in production.&lt;/li&gt;
&lt;li&gt;Read-optimized indexes that exist only on replicas.&lt;/li&gt;
&lt;li&gt;Collation or encoding differences mid-migration.&lt;/li&gt;
&lt;li&gt;Debug views and seed tables that exist only in staging.&lt;/li&gt;
&lt;li&gt;Per-customer schema variations in on-premise or packaged deployments.&lt;/li&gt;
&lt;li&gt;A foreign key constraint deliberately dropped in production for operational reasons.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Try to express these in migrations and you get conditional branches keyed on environment variables — which makes the files even harder to read, which means they get read even less.&lt;/p&gt;
&lt;p&gt;So there is a category of real, legitimate schema state that &lt;strong&gt;migrations cannot express even in principle&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;To be clear, none of this is an argument against migrations. Versioned migration history is the correct practice. It just has a smaller jurisdiction than we usually assume.&lt;/p&gt;
&lt;h2 id=&quot;only-a-human-can-classify-a-difference--and-weve-given-humans-nothing-to-work-with&quot;&gt;Only a human can classify a difference — and we’ve given humans nothing to work with&lt;a class=&quot;heading-anchor&quot; href=&quot;#only-a-human-can-classify-a-difference--and-weve-given-humans-nothing-to-work-with&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;p&gt;Once you accept that intentional drift exists, a task becomes unavoidable: when a difference between environments surfaces, someone has to decide — &lt;strong&gt;is this intentional, or is it an accident?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;That decision cannot be automated. Intent lives in people’s heads. No tool can know whether an extra index is a performance fix someone deliberately applied to production, or something an engineer added during an investigation and forgot to remove.&lt;/p&gt;
&lt;p&gt;I think it’s important to say this plainly, because it marks the hard boundary of every “fully automated schema management” pitch: classification requires human judgment, full stop.&lt;/p&gt;
&lt;p&gt;The real problem is not that humans have to look. It’s that &lt;strong&gt;we’ve made looking nearly impossible.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Today your options are roughly:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Open two database clients side by side.&lt;/strong&gt; Works for three tables. Does not work for eighty. This is a limitation of human attention, not effort.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Dump both schemas and diff the text.&lt;/strong&gt;&lt;/p&gt;
&lt;pre class=&quot;astro-code github-dark&quot; style=&quot;background-color:#24292e;color:#e1e4e8; overflow-x: auto;&quot; tabindex=&quot;0&quot; data-language=&quot;bash&quot;&gt;&lt;code&gt;&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;pg_dump&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --schema-only&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -h&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; host-a&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; mydb&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; &amp;gt;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; a.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;pg_dump&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; --schema-only&lt;/span&gt;&lt;span style=&quot;color:#79B8FF&quot;&gt; -h&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; host-b&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; mydb&lt;/span&gt;&lt;span style=&quot;color:#F97583&quot;&gt; &amp;gt;&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; b.sql&lt;/span&gt;&lt;/span&gt;
&lt;span class=&quot;line&quot;&gt;&lt;span style=&quot;color:#B392F0&quot;&gt;diff&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; a.sql&lt;/span&gt;&lt;span style=&quot;color:#9ECBFF&quot;&gt; b.sql&lt;/span&gt;&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Reasonable in theory. In practice, most of what comes out is noise: object ordering differences, &lt;code&gt;AUTO_INCREMENT&lt;/code&gt; counters, formatting variations in defaults, type spelling differences (&lt;code&gt;serial&lt;/code&gt; vs. identity columns), ownership lines, version-dependent &lt;code&gt;SET&lt;/code&gt; statements. You end up hunting for the one meaningful line buried in dozens of irrelevant ones — and humans who scan noisy output for long enough start missing the meaningful lines too.&lt;/p&gt;
&lt;p&gt;That’s the structural gap: &lt;strong&gt;the judgment can only be made by a human, and the output was never designed for human judgment.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Here’s the part that makes it tractable. Intentional drift is &lt;em&gt;stable&lt;/em&gt;. That production-only index shows up as the same difference every single time you compare. If the comparison output is structured and noise-free, stable differences become recognizable patterns — after the third look, it’s “ah, that one again,” and the real anomalies stand out against a familiar background.&lt;/p&gt;
&lt;p&gt;People don’t miss differences because there are too many. They miss them because &lt;strong&gt;the same difference looks different every time.&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;where-this-leaves-us&quot;&gt;Where this leaves us&lt;a class=&quot;heading-anchor&quot; href=&quot;#where-this-leaves-us&quot; aria-label=&quot;Link to this section&quot;&gt;&lt;/a&gt;&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;Migration history records intent. It does not describe the present. The present exists only in the database itself.&lt;/li&gt;
&lt;li&gt;The verification built into migration tools checks files, not databases. Nothing in the default workflow detects drift.&lt;/li&gt;
&lt;li&gt;The workflow’s integrity rests on a convention that is breakable, silently breakable, and must be re-taught forever.&lt;/li&gt;
&lt;li&gt;Some drift is intentional, which means classification is unavoidable, which means human eyes are unavoidable.&lt;/li&gt;
&lt;li&gt;So the leverage point is not more automation of intent — it’s better material for judging results: a clean, structured, repeatable comparison of what your databases actually are.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Manage history with migrations. Verify the present with diffs. These are different jobs, and conflating them is how a column ends up in production that nobody can explain.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;&lt;em&gt;I build &lt;a href=&quot;https://diffy-pick.com/&quot;&gt;DiffyPick&lt;/a&gt;, a desktop tool for comparing and syncing database schemas — it exists because of everything above.&lt;/em&gt;&lt;/p&gt;
</content:encoded><category>schema-drift</category><category>database-migration</category></item></channel></rss>