Hi Everyone 👋 I'm trying to implement a solid that runs the "dbt run" command that can be configured to run against dev or production based on a different profile file passed via the config. The solid runs successfully but the dbt run command doesn't create the views in bigquery. I have a feeling that it's to do with putting "dbt_cli_run.configured: within a solid definition. Any help would be appreciated. Thank you 🙂

@solid(config_schema={"profiles-dir": str, "project-dir": str})
def run_dbt_models(context):

    config_test = {
        "profiles-dir": context.solid_config["profiles-dir"],
        "project-dir": context.solid_config["project-dir"]
    }

    dbt_cli_run.configured(
        config_or_config_fn=config_test,
        name="run_dbt_models",
        )

hi @Jazzy you're correct about the issue here! dbt_cli_run is itself a solid. The configured() call just "bakes in" that metadata to the solid, returning a solid itself, which is why it doesn't do anything in your pipeline (the code doesn't get executed). Assuming you're not doing anything else in this solid body, you could replace your run_dbt_models solid with the dbt_cli_run solid.

@owen thanks so much for the response! I'll swap it out 🙂

@Jazzy ahh I see the issue now. yes, the configured API only really works if you're going to use the same configuration for all modes in your pipeline. Changing env variables would work (but might be annoying depending on your setup), so the other main option with configurable solids is just to run the pipeline with a different run config depending on what environment you're in. This pattern has been a source of confusion for a lot of users (and is not all that ergonomic), so for 0.12.0 (this week), we're releasing a dbt resource that should be a lot easier to configure differently for different modes. I think this pattern will feel a lot more natural for your use case.

That all makes sense @owen. I'll look out for the dbt resource 🙌

Just following up on this -- the resource is available now in the 0.12.0 release 🙂

Thanks @owen already given it a try and it's really great 😄