19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
I have 3 jobs:
1. Load data to s3 2. move data fr...
# ask-communitya
Alec Ryan
04/21/2022, 12:50 AMI have 3 jobs:
1. Load data to s3
2. move data from s3 to snowflake
3. build dbt models
Jobs 1,2 rely on partition dates to run. dbt cannot rely on a partition_date directly. How can I pass the date from job 2 to dbt? Can I role them all into the same job?
daggy success 1
s
Stephen Bailey
04/21/2022, 1:54 AMim curious what dbt uses the partition date for?
Stephen Bailey
04/21/2022, 1:55 AMmy first thought was using a sensor that listens for Job 2, and yields a
RunRequesta
Alec Ryan
04/21/2022, 1:55 AMI'm using it as a runtime vars to build incremental models one day at a time
👍 1
s
Stephen Bailey
04/21/2022, 1:56 AMyeah, looks like you could pass the run_config into the job config with runrequests if you were okay using a sensor to trigger it: https://docs.dagster.io/_apidocs/schedules-sensors#run-requests
Stephen Bailey
04/21/2022, 1:57 AMhere's some discussion on how to do that: https://dagster.slack.com/archives/C01U954MEER/p1648680715531009?thread_ts=1647374649.161559&cid=C01U954MEER
Stephen Bailey
04/21/2022, 1:57 AMfwiw, i've been using this sensor-trigger pattern for the past couple weeks and it works quite well.
a
Alec Ryan
04/21/2022, 1:58 AMDo you know of examples where assets are used?
Alec Ryan
04/21/2022, 1:58 AMGenerally, seems like the same concept regardless
s
Stephen Bailey
04/21/2022, 1:58 AMno, not aware of any specific examples
2 Views