https://dagster.io/ logo
#ask-community
Title
# ask-community
h

Henri Blancke

05/31/2023, 6:24 PM
👋 I'm trying out declarative scheduling and auto materialization and am running into some issues. We configured all our assets with the
AutoMaterializationPolicy.eager()
policy except for 3-4 dbt models that we want to have materialized only a handful of times a day, they are configured with a
FreshnessPolicy
and
AutoMaterializationPolicy.lazy()
(in the screenshot below
loan/transaction
is one of those assets). The expected behavior is that the assets with a freshness policy materialize when the policy is overdue yet the auto materializer doesn't seem to kick off an update when they are. Is there something here that I'm overlooking? The ideal for us is that the handful of assets that have a freshness policy wait until all their upstream assets are up to date, let me know if there's a better way to accomplish that. Thanks for the help!
r

R. Amaral Vieira

05/31/2023, 8:41 PM
I‘ve hit the same problem while testing. and for me it happens when the time window defined by
maximum_lag_minutes
in the
FreshnessPolicy
doesn‘t catch any already existing materialization in its upstream asset. This is for assets with a
lazy
auto-materialize policy, and I‘ve only tested with non-partitioned assets… Does it work for you when it is set to
eager
, or when you increase the
maximum_lag_minutes
parameter?
c

claire

06/01/2023, 9:53 PM
Hi Henri. Based on the screenshot, it looks like all of these assets are unpartitioned and the upstream parents are up to date (no unincorporated upstream materializations have occurred). I think transaction should be able materialize, I'm also confused a run of
transaction
hasn't been kicked off? Will reach out to the team about this
Something that might be helpful is upgrading to version 1.3.7, since that version adds a UI that explains reasoning for why materializations were / were not kicked off
s

sandy

06/01/2023, 9:58 PM
‘ve hit the same problem while testing. and for me it happens when the time window defined by
maximum_lag_minutes
in the
FreshnessPolicy
doesn‘t catch any already existing materialization in its upstream asset. This is for assets with a
lazy
auto-materialize policy, and I‘ve only tested with non-partitioned assets…
@R. Amaral Vieira does this behavior seem reasonable to you? Is there different behavior you would expect or prefer?
o

owen

06/01/2023, 10:04 PM
hey @Henri Blancke! definitely hard to tell what's happening here from the immediate screenshot, and I'd agree that it seems like transaction should be getting updated. looks like there might be some further assets upstream that aren't displayed here, which may or may not help explain this behavior -- would it be possible to share the full upstream graph (click the "Upstream" button, and select "All" as the Graph depth)?
oh and also what are the parameters of the FreshnessPolicy attached to that asset?
h

Henri Blancke

06/01/2023, 10:28 PM
@owen thanks! weird that shows up in the corner, all upstream assets should be displayed in the screenshot though... The FreshnessPolicy params
maximum_lag_minutes=30
which may mean that what Sandy and R. Amaral say could be the main issue here. I'm going to give 1.3.7 a try tomorrow to help diagnose this further. Thanks for all the help here!
We enabled auto materialization in our production environment and are definitely experiencing some issues/oddities. For example we often see multiple materializations being kicked off within a minute for the same asset. We're also still seeing freshness policies that are out of date (policies with a max lag minutes of 24 hours). For context, most of our assets have an eager auto materialization policy with the exception of a couple of assets at the bottom of the DAG having freshness policies defined with a lazy policy. We're on dagster 1.3.9. Any ideas of what could be causing 1) multiple materializations per minute 2) freshness policies rendering stale for hours? Thanks again for the help here 🙏 🙇
We also have some cases where eager materialization is not being kicked off, like in this example where it detects upstream data has changed yet the asset is not getting materialized:
o

owen

06/16/2023, 11:56 PM
Gotcha, I believe the issue w/ the asset getting stuck is due to the observable source asset issue mentioned in the other thread, and we're also looking into the multiple materialization issue as well
❤️ 2
h

Henri Blancke

06/20/2023, 1:22 PM
Thanks for following up @owen!
@owen do you all have a timeline on when we can expect the observable asset issue to be fixed?
o

owen

06/21/2023, 8:53 PM
Just merged in a fix that will go out in this week's release!
2 Views