https://dagster.io/ logo
#dagster-support
Title
# dagster-support
a

Anil

08/17/2022, 4:19 AM
Hello, Sorry if this is a very basic and silly question. First time using Dagster. I am trying to use Dagster with our dbt project and i was able to construct the jobs to run them sucessfully. Now, I would like to run these jobs one after the other. Basically making the second job depend on first job with successful run. Not sure how to achieve this. Below is my code. #Asset for dbt raw models excluding crm dbt_raw_wo_crm_assets = with_resources( load_assets_from_dbt_project(DBT_PROJECT_DIR,DBT_PROFILES_DIR), { "dbt": dbt_cli_resource.configured( {"project_dir": DBT_PROJECT_DIR,"profiles_dir": DBT_PROFILES_DIR,"target": RAW, "models": ["tag:raw"],"exclude":["tag:crm"]}, ) }, ) #Asset for dbt stage models excluding crm dbt_stage_wo_crm_assets = with_resources( load_assets_from_dbt_project(DBT_PROJECT_DIR,DBT_PROFILES_DIR), { "dbt": dbt_cli_resource.configured( {"project_dir": DBT_PROJECT_DIR,"profiles_dir": DBT_PROFILES_DIR,"target": TRANSFORM, "models": ["tag:stage"],"exclude":["tag:crm_analytics"]}, ) }, ) # Run Raw Jobs @job(resource_defs={"dbt":dbt_cli_resource.configured(dbt_raw_wo_crm_assets)}, name='Job_to_run_dbt_raw_models', description='This job loads the raw models excluding CRM') def run_dbt_raw_models(): dbt_raw_models = dbt_run_op() # Run Staging Jobs @job(resource_defs={"dbt":dbt_cli_resource.configured(dbt_stage_wo_crm_assets)}, name='Job_to_run_dbt_stage_models', description='This job loads the stage models excluding CRM') def run_dbt_stage_models(): dbt_stage_models = dbt_run_op() I want to run the staging jobs after successful finish of raw models job. Any help?
j

Jacek Chmielewski

08/17/2022, 8:29 AM
I think you'll have to redefine your jobs as ops and then create a job that creates appropriate workflow
that is - your staging op should take raw models output as it's input
s

sandy

08/17/2022, 5:15 PM
Hey @Anil - if you're asset-based dbt APIs like
load_assets_from_dbt_project
(which is recommended), it usually doesn't make sense to also use
@job
or
dbt_run_op
. Instead, you'd want to use
define_asset_job
, and you could use model job dependencies using a run status sensor. I can send an example that shows what that looks like. Before I do that, I'm curious, why not combine the raw models and the staging models into a single job?
a

Anil

08/17/2022, 6:25 PM
@sandy I want to be able to run the job independent of each other to have more control. Thanks for the suggestion and the links. I will check them out. Could you please send me the example you were talking about.
10 Views