Hi all! I find myself having quite a few jobs (not ops) that need to run in a specific order (dbt seed, then dbt run, then dbt test, then kick off jobs that send data to stripe for invoicing, or do reverse ELT). Am I correct in thinking the new asset-based way of working will fix most of this? What would it look like if the outcome of a job is not a very clean asset (like with dbt test)?

For managing cross-job dependencies currently, a lot of users are using asset sensors, where they are manually materializing an asset at the end of each job, which then kicks off a downstream job. This model translates pretty cleanly to the asset-based model. For your dbt test job, it might not be creating a physical asset, but it could be represented in dagster as having created a separate “verified” form of the physical asset. cc @sandy, @owen who might be thinking about this differently.

Yeah, that’s what I figured - the sensors might get kind of nasty if there’s more than one dependency going on, or is that OK?

I think sensors have an experimental parameter that can take in a list of jobs to target. You could then could then emit a

RunRequest

with a

job_name

argument for each job that you want to target.

oh nice, didn’t notice that one, thanks!

Hi Huib! @owen just responded to a very similar question re: dbt_test here: https://dagster.slack.com/archives/C01U954MEER/p1651772216465939?thread_ts=1651770836.171329&cid=C01U954MEER Hope that’s helpful!

That’s actually super helpful, thanks!