Data
Build an ETL pipeline
Sources joined with validation, and lineage you can trace.
Fill it in
The files or feeds.
Reject bad rows loudly rather than coercing them.
So a wrong number can be traced to its row.
Your prompt
Build a pipeline over . Validate at the BOUNDARY and reject loudly. The tempting alternative is to coerce: parse the date if you can, default the missing amount to zero, skip the row that fails. Every one of those produces a pipeline that never errors and quietly changes the answer. Make it idempotent. A rerun after a partial failure must not double-count, and the only reliable way is a key you can upsert on. Track lineage to the row. When a number is wrong, the question is always which source row produced it, and a pipeline that cannot answer that is debugged by rerunning it with print statements. Report counts at every stage: in, transformed, rejected, out. They should tie.
Continue and pick your folderOpens with everything above already filled in.
Why this works
The tempting failure is to coerce: parse the date if you can, default the missing amount, skip the bad row. Each one produces a pipeline that never errors and quietly changes the answer. This rejects at the boundary and reports counts at every stage.