19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
I'm trying to get started with software-defined as...
# ask-communityd
Danny Fraser
05/24/2022, 10:58 AMI'm trying to get started with software-defined assets and dbt in dagster and having trouble mixing Python assets with dbt.
I'm populating a source table with an asset definition in Python - just a simple dummy pandas DataFrame that gets persisted to a database table. The dbt project uses this table as a source, but I can't figure out how to make dagster aware that the Python asset and the dbt source table are the same thing. I end up with one asset graph showing the source table being populated, and another graph showing the dbt job, but ideally I'd like to see both as part of the same graph.
Can someone point me in the right direction please?
✅ 1
🤖 1
Danny Fraser
05/24/2022, 12:25 PMFor context here's what the asset graph looks like at the moment.
Danny Fraser
05/24/2022, 12:28 PMMy overall aim is to have an orchestrated process that brings in data from a bunch of sources (other databases, Excel files, web sources etc.), loads them into a central database (initially SQLite but will probably move to postgres), and then have dbt manage the transformations. I'd like to be able to see the overall lineage from source data to transformed tables including where tables populated by Python scripts are used as sources in dbt.
Danny Fraser
05/24/2022, 12:36 PMSolved! The secret was to use
namespace@asset@asset(compute_kind*=*"python", namespace*=*"main")❤️ 1
o
owen
05/24/2022, 4:05 PMglad you got it solved! your solution is what I would have recommended, but just in case it's relevant to you, you can customize exactly how we generate asset keys from dbt models using the
node_info_to_asset_keyload_assets_from_dbt...❤️ 1
a
Alec Ryan
05/28/2022, 1:42 AMHey @owen @Danny Fraser, I'm having a similar issue where my dbt models are in an analytics dev namespace. I can't seem to connect dbt though. Any thoughts?
3 Views