it is possible to add a group_name direct in the c...
# integration-dbt
e
it is possible to add a group_name direct in the config of an dbt model?
q
Use the parameter
node_info_to_group_function
e
@Qwame thanks!
@Qwame i`m using the following config, but it`s not working
Copy code
{{
  config(
    materialized="incremental",
    schema="assets",
    tags=["asset", "places", "freshness_sensor", "staging"],
    unique_key = 'place_id',
    incremental_strategy = 'merge',
    node_info_to_group_fn = {
        "group_name": "staging"
    },
    dagster_freshness_policy={
        "maximum_lag_minutes": (60 * 7),
        "cron_schedule": "*/40 6-23 * * *",
        "cron_schedule_timezone": "America/Sao_Paulo"
    })
}}
q
this will not work. You have to add it as a tag and then when importing into dagster using
load_assets_from_dbt_project
, you use the
node_info_to_group_fn
to pull that tag as a group name
Copy code
{{
  config(
    materialized="incremental",
    schema="assets",
    tags=["asset", "places", "freshness_sensor", "staging"],
    unique_key = 'place_id',
    incremental_strategy = 'merge',
    dagster_freshness_policy={
        "maximum_lag_minutes": (60 * 7),
        "cron_schedule": "*/40 6-23 * * *",
        "cron_schedule_timezone": "America/Sao_Paulo"
    })
}}
Then in dagster, (you can also use
load_assets_from_dbt_manifest
)
Copy code
def _node_info_to_group(node_info):
    return node_info["tags"][3]

load_assets_from_dbt_project(
   project_dir='..',
   profiles_dir='..',
   select="tag:staging",
   node_info_to_group_fn=_node_info_to_group
)
This assumes that in your tags config list, the
staging
tag is always the 4th item in the list
2
daggy love 2
I haven't tested to see if
dbt
will accept additional configs like you have in the
dagster_freshness_policy
, I know you can put anything in under
meta
. But if dbt accepts
dagster_freshness_policy
and
group_name
like you have above, then you can have the change the node_info function: You can inspect the
manifest.json
to see how the additional parameters you put under
config
to see how it shows in the manifest and then pull it like you will access a python dictionary object. Hope this makes sense
e
@Qwame thanks alot!!! it makes sense