# Is our ingestion process more complicated than necessary?

Prompt · Writes and ingestion · PG07

The import pipeline is long, slow to correct, or does the same work at several stages.

## Prompt

```text
Trace one complete ingestion cycle from the original source to the final database records. Identify repeated parsing, unnecessary intermediate states, duplicate validation, repeated reads, redundant transformations and writes that could be consolidated. Recommend the simplest workflow that preserves accuracy, traceability, failure recovery and the ability to process corrections.
```

**What it returns:** One real trace from source to final rows, the work done twice, the checkpoints, a shorter path, and checks that the shorter path gives the same results and still replays.

**Skip it when:** The pipeline is already one short step from source to table.

**Avoid:** Removing evidence, provenance or checkpoints that recovery depends on.

**Tools:** Repository and read-only database access.

**Works on:** Any PostgreSQL host: Supabase, Neon, RDS, Cloud SQL or self-hosted.

**Changes:** Changes nothing

**Sessions:** Look first, then fix in a new session

## Related

- [Imports that fail quietly](https://atlasscope.dev/prompts/imports-that-fail-quietly): Finds rows that go missing or land on the wrong record during import, with no error.
- [Map every place that writes to the database](https://atlasscope.dev/prompts/map-every-place-that-writes-to-the-database): Several jobs, functions, APIs or agents change the same data, or nobody is sure who owns it.
- [Stop rewriting data that hasn't changed](https://atlasscope.dev/prompts/stop-rewriting-data-that-hasnt-changed): Upserts and updates run at high volume, imports are costly, or vacuum and write-ahead log churn look high.
- [Can we rerun everything without damaging the data?](https://atlasscope.dev/prompts/can-we-rerun-everything-without-damaging-the-data): Imports get retried, schedulers overlap, files are replayed, or runs fail halfway.
- [Find where write logic should live](https://atlasscope.dev/prompts/find-where-write-logic-should-live): Business rules are split across SQL, triggers, edge functions, server routes and workers.

Source: https://atlasscope.dev/prompts/is-our-ingestion-process-more-complicated-than-necessary