Hi all! I’m using software-defined assets (specifically the AssetGroup().build_job() methods) to import my data generated from DBT into my dagster graph. I would like to find a way for users of the script to provide arbitrary variables to my job (similar to the functionality provided by a config.yaml file. The

AssetGroup()

allows me to pass in

resource_defs

as shown below - but I do not know how to allow users to pass arbitrary values into the “vars” dictionary:

my_job = AssetGroup(
    dbt_assets + csv_assets,
    resource_defs={
        "dbt": dbt_cli_resource.configured(
            {
                "project_dir": DBT_PROJECT_DIR,
                "profiles_dir": DBT_PROFILE_DIR,
                "vars": {"test_split": 0.2, "train_split": 0.5},
            }
        )
    },
).build_job()

I am able to run the

my_job

in a new script with the

my_job.execute_in_process()

method but I would like to pass new values for the

train_split

and

test_split

variables instead of hard-coding them.

"vars": {"test_split": 0.2, "train_split": 0.5},

How can I pass values to jobs defined by

AssetGroup()

’s

resource_defs

?

Hi Harpal. You can omit the

.configured

on the DBT resource and instead provide the config at job-level. For example:

# run config yaml
resources:
  my_dbt_resource:
    config:
      vars:
        test_split: 0.2
        train_split: 0.5
      ...

You could also provide this on the

config

argument of the job

Hi @claire and thanks for the suggestion! I’ve followed the docs and have configured the resource in a separate

config.yaml

file (see below):

#config.yaml
resources:
  dbt_cli_resource:
    config:
      project_dir: "./dbt"
      profiles_dir: "./dbt/config"
      vars: {"test_split": 0.2, "train_split": 0.5}

I think I understand what you mean by removing the

.configured

DBT resource. But I don’t quite understand what you mean by “configure this at the job-level”. Do I have to pass this

config.yaml

file to the

AssetGroup()

,

build_job()

or

my_job.execute_in_process()

methods? Because it’s not clear which one of these would accept such an argument. Thanks in advance dagsir

Ahh right. I realized that we haven't enabled config on asset jobs yet, though it is a top priority for us and a feature that will come out in the next couple of months. Sorry about that