https://dagster.io/ logo
#dagster-support
Title
# dagster-support
g

geoHeil

08/13/2022, 2:12 PM
Why is dagster-DBT not showing the column types? The column mentadata already seems to be inferred nicely from the yaml, however, type information is missing. Would I manually need to add: https://docs.getdbt.com/reference/resource-configs/column_types
<https://docs.getdbt.com/reference/resource-configs/column_types|column_types>
to the YAML file for this to work? Shouldn`t Dagster/DBT be able to retrieve these details from the compiled SQL scripts (DDL)?
s

sandy

08/15/2022, 4:33 PM
Hey @geoHeil - that's right that you'd need to add them manually. Inferring types from compiled SQL is non-trivial, and in many cases it would require calling out to the database to find the type of a column that the SQL is referencing
g

geoHeil

08/16/2022, 8:11 AM
Is that true? Isn`t DBT doing that automatically? and only parsing the manifest.json of DBT doesn`this also contain the neccessary DDL?
s

sandy

08/16/2022, 3:38 PM
to my knowledge, dbt isn't aware of the column types. have you observed otherwise? I thought they just do a "create table as select", which delegates column types to the database
g

geoHeil

08/17/2022, 12:46 PM
Perhaps I also got confused with the metadata collector of OpenMetadata and datahub project. They could somehow discover it. But they might also potentially simply extract the DDL from the DB
4 Views