04Validation, Parallel Runs & Financial Reconciliation

I make the highest-stakes hours scripted, timed, and reversible.

Running zero-tolerance financial checks and managing executive sign-offs — I didn't improvise any of this, because nothing here is worth gambling on.

De-risking trust before I needed it

Executing Dual-Run Parallel Testing Cycles

I run both platforms in parallel and reconcile daily through every wave, not just in a final test window. By the time cutover arrives, the target has already matched the source for weeks — so the switch removes the old system instead of gambling on the new one.

Why I choose dual-run over big-bang

I want trust earned before go-live, not asked for after. A big-bang cutover means the first real test happens in front of the business; dual-run means I've already seen weeks of clean reconciliation before anyone else even notices the switch happened.

Hypercare, on my terms

I open an elevated-support window at go-live: a daily war-room I run myself, fast-track defect SLAs, and champions fielding questions inside each function. I set the exit criteria myself too — defect rate below threshold and adoption climbing — before I let hypercare close and support go back to normal.

The three tiers I reconcile on

Row counts alone don't prove a migration; matching business aggregates does. So reconciliation runs on three tiers — structural, then financial, then human sign-off — and all three have to pass before cutover.

Artifact · Three-tier reconciliationstructural → financial → sign-off
1 · Automated row & hash checks
Source-to-target row counts, distinct-key checks, and checksum/hash validation across both historical and incrementally loaded tables — the structural layer that catches dropped or duplicated rows automatically.
2 · Financial & metric parity
Through the 4-week dual-run, key business aggregates — ARR, pipeline revenue, churn, and usage billing — reconciled daily between production SQL Server and target Snowflake. This is the tier that surfaced the trailing-12-month timezone break.
3 · Documented sign-off authority
Final cutover required formal, documented sign-off from the Director of Enterprise Data/BI alongside the lead business owners in Finance Operations and Sales Ops — parity had to be accepted by the people accountable for the numbers, not just green in a script.

The gate I wouldn't move on

Proving Zero-Variance Financial Accuracy

I agree every go/no-go criterion before the meeting happens, so the decision is reading the evidence, not debating the bar. I keep one named decider — the executive sponsor — but the numbers have to pass before it ever reaches them.

Why this artifact

Agreeing the bar after the evidence is in is how "close enough" quietly becomes the new standard under deadline pressure. Setting the criteria before the meeting is what lets me hold the line on one pending UAT sign-off instead of waving it through.

Artifact · Go/No-Godecider: Exec Sponsor
CriterionThreshold I setStatus
Financial reconciliationZero variance on all financial metricsPass
Open defectsNo Sev-1 / Sev-2 openPass
Report parity100% of in-scope reports signed offPass
Rollback testedRehearsed in a non-prod dry runPass
UAT sign-offAll function SMEs signed1 pending
Support readinessHypercare team & runbook staffedPass

The runbook I scripted minute by minute

Every step sequenced against a T-clock, with an owner and a checkpoint. The freeze protected data integrity; the reconciliation gate was my point of no return.

Why this artifact

Cutover weekend is the worst possible time to improvise a decision. A minute-by-minute runbook with a pre-agreed rollback trigger meant nobody had to find me to ask permission when the reconciliation gate mattered most.

Artifact · Runbookcutover window · weekend
T-clockStepOwnerCheckpoint
T-48hComms sent; freeze announced; support on standbyChange LeadAcknowledged
T-2hSource freeze — no writes to legacy EDWData EngFreeze confirmed
T-0Final incremental load; catch-up to freeze pointData EngLoad complete
T+1hFinal reconciliation — revenue to the pennyTesting / meGo point
T+2hRe-point reports & connections to SnowflakeBI LeadSmoke test pass
T+3hOpen access to users; announce liveMeLive
AnyRollback trigger I set: reconciliation fails or Sev-1 found → revert to legacy, unfreeze, stand downMeLegacy restored

Golden thread · go-live

At the T+1h gate, CLOSE_AMOUNT_USD reconciled to the penny against the frozen source — the same field that ran 6% high in test now matched exactly, because I'd already fixed the grain and definition upstream. That single green check is what let the sponsor say go.

What actually fought me

The friction the plan didn't predict

A clean reconciliation framework doesn't mean a clean run. Two problems cost real velocity before I got ahead of them.

Friction · Legacy T-SQL that wouldn't port

Several critical reports depended on nested, recursive T-SQL stored procedures that didn't translate one-to-one into Snowflake SQL. It stalled sprint velocity for about two weeks.

Decision

Refactored the recursive logic into modular dbt / Snowflake views rather than force a literal translation — rebuilding the logic to be cloud-native instead of carrying the legacy pattern forward.

Outcome

The reports came back with their numbers intact, on a model that was actually maintainable in Snowflake afterward.

Friction · Month-end warehouse contention

During parallel testing, unoptimized BI extract queries collided at month-end and caused compute-warehouse contention in Snowflake — slow dashboards exactly when finance needed them.

Decision

Segregated workloads into dedicated virtual warehouses for ETL versus ad-hoc BI consumption, and tuned the auto-suspend and scaling policies so the two stopped competing for the same compute.

Outcome

Extract performance recovered under peak load, and the separation kept ETL jobs and analyst queries from throttling each other going forward.

The judgment call

What I decide here

What I Decide Here

Decision: In week 2 of the 4-week dual-run, business reconciliation surfaces a trailing-12-month renewal discrepancy — traced to a UTC/timezone offset between the on-prem SQL Server sources and the cloud ingestion layer. I hold the cutover gate until it is root-caused and patched, not just flagged.

Friction: The fix sits on the critical path to a cutover planned ahead of quarter close. Raw staging was already inside our 0.003% automated anomaly threshold, so it would have been easy to call it “close enough” and move on.

Outcome: I isolate the root cause in the transformation layer, patch the timezone handling, and re-run the dual-run to exact parity on ARR and renewals before greenlight — keeping the planned cutover date without trading away the sign-off Finance and Sales Ops require.