https://dagster.io/ logo
#integration-dbt
Title
# integration-dbt
j

Joel Olazagasti

04/25/2023, 5:12 PM
Are there any good examples floating about RE: creating an observable source asset for a DBT source that generates a
DataVersion
based on a query to the table it's generated from? For example, getting the number of rows.
t

Tim Castillo

04/25/2023, 5:38 PM
Hi! I can't find one right now, but this is an interesting use case! Mind if I ask a question to help out with this? What would you like to do by having a dbt source as a separate observable source asset?
j

Joel Olazagasti

04/25/2023, 5:40 PM
Hi Tim, sure thing! We have a number of manually loaded data sources in Snowflake that end up as DBT sources, and our ideal scenario is to have the downstream DBT assets rematerialize whenever those sources are changed.
t

Tim Castillo

04/25/2023, 5:46 PM
That makes sense and it's an interesting (and logical) push for how to connect observable source assets + dbt models work together. Let me take it back to the team and get their thoughts.
Update: It should work. There are a couple ways to figure out that data version. I'd lean away from row count and instead try to track when it was last updated. ex. if you have an
updated_on
column, to calculate the data version off of
max(updated_on)
.
j

Joel Olazagasti

04/25/2023, 6:19 PM
Great to hear! I have a follow up question, but I think that will pertain more to the Snowflake team, so I'll take it to the appropriate channel
Appreciate the help
d

denes

04/25/2023, 8:00 PM
you can get the timestamp of the last update from the TABLES information schema view:
Copy code
select last_altered 
from information_schema.tables 
where table_name = '<TABLE_NAME>'
🤩 1
3 Views