How do people balance the engineering benefits of ...
# random
g
How do people balance the engineering benefits of backfills (testability / idempotence) against the runtime benefits of having a different flow for an initial load? In my case a huge psycopg2 copy_from to insert into Postgres and MongoDB collection call without any filter (executed in large batches). Or are backfills more often just used when a job's been failing for a few days?