The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hello,
How can we assign run_tags to new `AutoMaterializePolicy` ?

In `1.2.X` , we used `build_asset_reconciliation_sensor`  with `run_tags` to ensure reconciliation/automaterialize runs follows our run tags concurrency limits. I do not see how to do something similar with AutoMaterializePolicy.

Other question regarding reconciliation sensor and freshness policy: how can I enforce child asset with Freshness policy are rematerialized if any parent asset has changed ?

It seems like reconciliation sensor only change status of child asset to "upstream data changed" without rematerialize it if a Freshness policy has been set on this child asset.

The behavior I would like to have is to enforce both Freshness policy and rematerialization when any parent asset has changed.

<@U03QBTYQNKA> this will be possible in our upcoming release this week: <https://docs.dagster.io/master/concepts/assets/asset-auto-execution#run-tags> (link is to unreleased docs)

&gt; Other question regarding reconciliation sensor and freshness policy: how can I enforce child asset with Freshness policy are rematerialized if any parent asset has changed ?
Have you had a chance to look at the docs? <https://docs.dagster.io/concepts/assets/asset-auto-execution>

Basically, if you use an eager AutoMaterializePolicy, it will do it immediately, not based on the freshness policies.

&gt; this will be possible in our upcoming release this week: <https://docs.dagster.io/master/concepts/assets/asset-auto-execution#run-tags> (link is to unreleased docs)
<@U011CET83FG> from this documentation it seems like run tags can be set in `dagster.yaml` file which means that these tags would be global to all auto_materialize policies.
Am I misunderstanding this or can auto_materialize runs be triggered by <https://docs.dagster.io/_apidocs/schedules-sensors#dagster.build_asset_reconciliation_sensor|asset_reconciliation_sensor> and associated run tags ?

&gt; from this documentation it seems like run tags can be set in `dagster.yaml` file which means that these tags would be global to all auto_materialize policies.
so your issue is that you want different concurrency limits for different assets?

Yes,
Different groups of assets within our code locations are data extracted by their specific source. Concurrency is per data-source (i.e. API_1 concurrency limit is 5 while API_2 is 20).

I managed this with reconciliation_sensor using run_tags parameter and defining a reconciliation sensor per data source (linked to one or multiple asset groups for this source).