Hey, some of our dbt models have tables ingested u...
# ask-community
h
Hey, some of our dbt models have tables ingested using AWS Batch as upstream asset. They have certainly been picked up because of the jinja blocks
{{source(schema,table)}}
, and appear in grey dagit. I would like to trigger the ingestion and then get the downstream assets materialised. One solution I could come up with is: 1. create an op that will send the AWS cli command to execute the Batch job 2. create an op that will wait for a notification of success 3. find a way to materialise the op as an asset, and play with AssetKey so that it is seen on Dagit as the model parent. Do you have a better approach in mind ? I guess the solution could be useful to anyone with models whose parents are ingested 'differently'. Since the batch job consists in executing containerised Python code, should I just import and execute the python script directly with dagster instead?
r
Yeah this sounds right. You want to line up the asset keys from the your dbt sources with the asset keys of the tables that you’re materializing from AWS Batch. We also have APIs that we’re releasing in 1.4 that will make this easier for you: https://github.com/dagster-io/dagster/discussions/14964. You can try them out today! If you have any other questions, feel free to ask in #dagster-dbt