The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

How would I go about scheduling a `DailyPartitionedAsset`to materialize only the last 30 days if they haven't been materialized already? Do I need to make a custom `job` and manually check each partition before returning a `RunRequest`?

Hi <@U04V89UAKCZ>! That'd work, and another option would be to use an `AutoMaterializePolicy`. The API is still experimental / subject to change, and right now the default behavior of the policy is to not materialize any partition other than the last one, but you could override this with
```AutoMaterializePolicy.eager()._replace(
    time_window_partition_scope_minutes=60*24*30,
    max_materializations_per_minute=30,
)```


So, I did this for about 40 assets and it completely crashes the user code server and doesn't produce any logs that I can find.

seem like the only log i get is:
``[32m2023-06-14 18:38:49 +0000[0m - dagster.code_server - [34mINFO[0m - Code server was interrupted`

I dropped `max_materializations_per_minute` to 5 and now the user code server stays up but the autmaterialization daemon is stuck in `not running` and no runs are launched

hm is there a large history of materializations for these assets? there's a known startup performance issue (where the first iteration of the daemon takes much longer than subsequent ones) in some cases that you might be running into (we're working to fix that right now) -- sorry you're running into it!

the worst offenders are probably hourly assets that have a few years of potential  materializations but they haven't been materialized