Josh
02/26/2024, 12:34 PMScout
02/26/2024, 12:34 PMAutoMaterializePolicy
to your assets. Here's how you can do it according to the documentation: 1. Define an AutoMaterializePolicy: You can start with the built-in AutoMaterializePolicy.eager()
and customize it if necessary. This policy includes all supported rules except for materialize_on_cron
and skip_on_not_all_parents_updated
. 2. Assign the Policy to Assets: You can assign the AutoMaterializePolicy
to your assets using the auto_materialize_policy
argument in the @asset
decorator. Here's an example from the documentation: python from dagster import AutoMaterializePolicy, asset @asset def asset1(): # Your asset computation logic ... @asset(auto_materialize_policy=AutoMaterializePolicy.eager(), deps=[asset1]) def asset2(): # Your asset computation logic that depends on asset1 ...
In this example, asset2
is configured to auto-materialize eagerly, meaning it will be materialized whenever possible, following the rules defined in the AutoMaterializePolicy.eager()
policy. Remember that runs triggered by auto-materialize policies are tagged with dagster/auto_materialize: true
, and you can configure additional tags in your dagster.yaml
file or deployment settings. For more advanced use cases, you can also customize the auto-materialize policies generated by Dagster for dbt assets by defining meta config on your dbt node or by implementing a custom DagsterDbtTranslator
. Please note that to see these changes reflected in the UI and to have the assets automatically materialized, you need to ensure that the auto-materialization feature is enabled in your Dagster deployment. This can typically be done through the Dagster UI under the "Deployment" section.Josh
02/26/2024, 12:36 PMScout
02/26/2024, 12:36 PMmarkdown This is now possible by setting the following in your dagster.yaml: auto_materialize: enabled: True Sorry folks - it looks like I actually misunderstood the behavior of the aforementioned config param and this functionality is not yet present :(. Reopening this. However, we are currently working on changes that will make this possible in a not-far-away release.
This indicates that the feature was considered but not available at the time of the discussion. It's possible that there may have been updates or changes since then. To get the most current information, I recommend checking the latest Dagster documentation or the GitHub repository for any updates on this feature. If you're managing your Dagster instance programmatically and looking for a workaround, you might consider using the Dagster API to enable auto-materialization as part of your deployment process. However, this would be an unofficial solution and would require custom scripting on your part.