A source table changing shape underneath a running pipeline, a new column, a dropped one, a type that quietly changed, without anyone updating the pipeline to match.
No credit card. 14 days. Cancel in one click.
Quick answer: Schema drift is when a source table's structure changes after a pipeline was built against it: a column gets added or dropped, a type changes, or a column gets renamed. Some managed ETL vendors market automatic drift handling, silently adapting the pipeline. QueryFlow's Data Sync takes the opposite, more visible approach: an existing field mapping doesn't update itself, so when you reopen a sync after a schema change, the current schema is shown and you re-map deliberately rather than the tool guessing which new column replaced which old one.
Schema drift describes a source changing shape after something downstream was built to expect its old shape. A source table gains a column your pipeline never mapped, so that data simply isn't captured. A column gets dropped, and any pipeline still referencing it starts failing. A column's type changes, a string that used to hold plain numbers starts including currency symbols, and a downstream calculation that assumed numeric values breaks or silently produces wrong results. None of this requires anyone to intend a breaking change; it's just what happens when the team that owns the source system and the team that owns the pipeline aren't the same people, coordinating in real time.
The risk isn't the change itself, it's that a pipeline can keep running and reporting success while quietly missing new data or misinterpreting changed data. A hard failure is annoying but visible. Silent drift is the more expensive failure mode, because nobody notices until a number downstream looks wrong and someone has to trace it back.
QueryFlow doesn't auto-adapt a Data Sync mapping when the source schema changes. When a source table gains a new column or a target table's column gets renamed, the existing mapping doesn't update itself: open the sync, and the field list reflects the current schema, so you add a line for the new column or re-drag the line that pointed at an old name. That's a manual step by design, guessing that a renamed column is really the same field as the one it replaced is exactly the kind of silent assumption a pipeline shouldn't make on its own.
This is a real tradeoff, not a missing feature dressed up as a choice. A vendor that markets automatic schema drift handling is promising less manual upkeep when a source changes; the cost is trusting the tool's guess about what a rename or a type change means. QueryFlow's Data Sync makes you look at the current schema and confirm the mapping yourself, which takes a few minutes after a real schema change but never silently reinterprets your data on your behalf.
A Postgres customers table you sync into Snowflake gains a new preferred_locale column. Your existing Upsert sync, mapped on customer_id, keeps running successfully and keeps updating every field it was already mapped to. preferred_locale simply isn't written to Snowflake, because it isn't in the mapping, not an error, just a silently missing column until someone notices the target table lacks it and opens the sync to add the line.
Since QueryFlow won't catch a new column for you automatically, the practical mitigation is watching for it elsewhere: a periodic manual review of source schemas for pipelines that matter, or a row count or value watch on the target table that would flag an obviously wrong downstream number if a type change broke a calculation. Neither replaces re-checking the mapping after a known schema change on the source side; they just shrink the window before someone notices.
Related: the field mapping tool for re-mapping mechanics, and dry run a sync before trusting a re-mapped job on real data.
No. It doesn't silently adapt a mapping when a source schema changes. Reopen the sync after a known change and the field list reflects the current schema for you to re-map.
The sync will fail on that field the next time it runs, since the column it's mapped to no longer exists. That's a visible failure, unlike an added column, which is silently just not captured.
No. A duplicate key error is about a MATCH ON column not being unique per row. Schema drift is about the shape of the table itself changing, independent of matching logic.
Some managed platforms market automatic schema drift handling as a feature, meaning they adapt pipelines on your behalf when a source changes. That trades manual upkeep for trusting the tool's guess about what changed.
14-day free trial, no card. Try Data Sync's field mapping on a real table.
No credit card. 14 days. Cancel in one click.