https://dagster.io/ logo
#ask-ai
Title
# ask-ai
i

Ismael Rodrigues

02/09/2024, 7:25 PM
@Scout My dbt_model is running tyhe dbt CLI command twice per run and it only materializes when two assets are done, can it be that it is running one per asset materialized in which the execution depends on?
s

Scout

02/09/2024, 7:25 PM
Based on the documentation, when integrating dbt with Dagster, the dbt CLI commands are optimized to run as efficiently as possible. This means that Dagster attempts to group dbt model runs into as few dbt CLI commands as necessary. If you are observing that the dbt CLI command is running twice per run, it could be due to a few reasons: 1. Partitioned dbt Models: If you have partitioned dbt models, Dagster might be passing additional variables to tell the model what partition it's running for, which could result in separate dbt CLI commands for different partitions. 2. Dependencies Between Models: If there are dependencies between models, Dagster should resolve this by staggering the runs. However, if you're experiencing incomplete data due to two separate runs, it could be a sign that the dependencies between the models are not being correctly resolved, potentially indicating a bug. 3. Configuration Overhead: Each dbt model does not appear as a unique asset in a run by default because Dagster's dbt integration is designed to create as few dbt runs as possible. This is different from other orchestrators that may make a task for each dbt model, which can introduce additional overhead time. 4. Selection Criteria Override: If you are using multiple CLI invocations in one dbt assets definition, Dagster might override your selection criteria and run all models in your project. This could lead to multiple dbt CLI commands being executed if the selection criteria are not being respected. If you are experiencing issues with dbt models being materialized incorrectly or dbt CLI commands running more than expected, it could be due to one of the reasons mentioned above. It's also possible that there might be a specific issue with your Dagster and dbt integration that requires further investigation. For a more detailed analysis, you would need to review the configuration of your dbt models, the dependencies between them, and how you have set up the integration with Dagster. If necessary, you can refer to the documentation or seek help from the community or support channels to resolve the issue.