Hello - curious if anyone has seen this before: I...
# ask-community
Hello - curious if anyone has seen this before: I'm using Dagster with dbt on Dagster Serverless connected to Snowflake. When running dbt, the queries resolve in <1s, but are consistently taking ~30s to resolve in
dbt build
. This doesn't occur when running dbt locally. E.g. in the image attached.
cc @rex
@jamie @rex any thoughts here?
Could you run your dbt commands in Serverless with
enabled? Our serverless tasks should have more than enough CPU/MEM to handle dbt tasks, esp since the transformation is happening in the data warehouse anyways.
You could also try turning off usage stats from being sent on `dbt`’s side to see if that alleviates some latency: https://docs.getdbt.com/reference/global-configs#send-anonymous-usage-stats
also, dbt uses some caching to help out with running models in local development, and that cache might not be hit when run in an isolated environment (like serverless): https://docs.getdbt.com/reference/global-configs#cache-database-objects-for-selected-resource
@rex looping back around on this - I'll take a look at both above. However, I've been able to correlate the change with moving from
. Was there, perhaps, a change to the dockerfile for serverless deploys in that interval?
@rex @jamie could you advise on how to access dbt's
flag in the
integration? It doesn't appear to be exposed as an option: https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-dbt/dagster_dbt/cli/constants.py