https://dagster.io/ logo
#ask-community
Title
# ask-community
d

Danny Fraser

05/24/2022, 10:58 AM
I'm trying to get started with software-defined assets and dbt in dagster and having trouble mixing Python assets with dbt. I'm populating a source table with an asset definition in Python - just a simple dummy pandas DataFrame that gets persisted to a database table. The dbt project uses this table as a source, but I can't figure out how to make dagster aware that the Python asset and the dbt source table are the same thing. I end up with one asset graph showing the source table being populated, and another graph showing the dbt job, but ideally I'd like to see both as part of the same graph. Can someone point me in the right direction please?
1
🤖 1
For context here's what the asset graph looks like at the moment.
My overall aim is to have an orchestrated process that brings in data from a bunch of sources (other databases, Excel files, web sources etc.), loads them into a central database (initially SQLite but will probably move to postgres), and then have dbt manage the transformations. I'd like to be able to see the overall lineage from source data to transformed tables including where tables populated by Python scripts are used as sources in dbt.
Solved! The secret was to use
namespace
in the
@asset
decorator, i.e.
@asset(compute_kind*=*"python", namespace*=*"main")
The graph looks like this now, so I can delete the template assets and start moving towards my actual data.
❤️ 1
o

owen

05/24/2022, 4:05 PM
glad you got it solved! your solution is what I would have recommended, but just in case it's relevant to you, you can customize exactly how we generate asset keys from dbt models using the
node_info_to_asset_key
parameter on
load_assets_from_dbt...
(basically just a function from a dictionary of dbt metadata to an asset key)
❤️ 1
a

Alec Ryan

05/28/2022, 1:42 AM
Hey @owen @Danny Fraser, I'm having a similar issue where my dbt models are in an analytics dev namespace. I can't seem to connect dbt though. Any thoughts?
2 Views