What’s the minimum delay I could expect between triggering a Job run and the first Op starting execution? And what’s the minimum delay I could expect between a Op finishing execution and the next Op in the Job starting execution?
Every time a new user of my application is created, I want to trigger a Job run that will scrape this user’s data from multiple sources and populate our database. I need to populate my database within 30 seconds of user creation. This is achievable with my current implementation of orchestrating this pipeline via a python function.
I want to make sure that migrating my orchestration logic to Dagster will not add significant time to this process. Ideally I’d keep the additional time under 5 seconds. Is this achievable with Dagster?