How can I pass (arbitrary variables i.e. lets say ...
# integration-dbt
g
How can I pass (arbitrary variables i.e. lets say a date variable) to DBT run on DBT core orchestrated via dagster?
r
You can supply a
partitions_def
and
partition_key_to_vars_fn
to the function that generates your dbt assets
a
Are you using assets or ops?
g
assets
no not for the partitions def - but it might be a way to parse arbitrary variables here
a
@rex’s answer is the way, even if the assets don’t need to be partitioned. There isn’t currently another way.
Note that you can use a StaticPartitionsDefinition with a single partition to make this work
👍🏽 1
👍 1
o
Another option would be to define a config-mapping over the dbt_cli_resource to fill in the values that won't change, while allowing you to configure the vars parameter after the fact:
Copy code
@configured(dbt_cli_resource, config_schema={"vars": dict})
def custom_dbt_cli_resource(config):
    return {
        "vars": config["vars"],
        "project-dir": DBT_PROJECT_DIR,
        "profiles-dir": DBT_PROFILES_DIR
    }
a
@owen After making this dbt_cli_resource how to make a job out of it Please help me in this
o
You can do something like: https://docs.dagster.io/integrations/dbt/using-dbt-with-dagster/part-two#step-2-define-a-dagster-code-location, but instead of the default
dbt_cli_resource
, you can pass in your
custom_dbt_cli_resource
(i.e.
resources = {"dbt": custom_dbt_cli_resource}
)
a
@owen Great , This step worked After this , i want to make it as a job and then pass the parameter using launch pad . Please assist
o
if you create a job to materialize your assets using
define_asset_job(...)
then attempting to launch a run for that job in the UI should prompt you to fill in configuration for the vars parameter
g
Does this mean if I have daily and monthly partitioned tables I need to tag them and load them separately? Or how can I map the partitionsdef to potentially > 1 type of partitioning in the partition_key_to_vars_fn function? I.e. I would not want to trigger the monthly one on a daily basis.
o
ah yeah good question -- the short answer is that yes you'd need to invoke
load_assets_from_dbt...
multiple times, once for each subset of assets with a different partitions definition