clay02/14/2023, 5:00 PM
In my case, let's say I have
. As with the example,
ends up being
and is passed to
in the same way that the assets are in the linked example. If I go into dagit and launch a run to materialize all of the
assets, which are in different groups, all works fine.
However, if I modify the code to
and then update the git repository and restart dagster/dagit, the system no longer knows that the
assets were materialized, in spite of the fact that
did not change at all. I can understand that
would reset, however, I'd like to figure out how to set this up so that only the assets associated with the package that changed would need to be rematerialized for dagster to know about them.
Most of my work/assets is being pushed to Snowflake. Is this simply a matter of dagster not knowing how to find them on Snowflake? Or will all of the assets associated with all of the packages always reset when I push a code change to a single package?
owen02/14/2023, 7:03 PM
clay02/14/2023, 8:12 PM
for the module changes? For instance, the additional of a job to the list of jobs?
owen02/14/2023, 8:54 PM
clay02/14/2023, 8:55 PM
owen02/14/2023, 9:02 PM
clay02/14/2023, 9:06 PM
, then the history for all of them would be wiped out. Or, if I added a 4th subfolder called
or something, and then added new assets/groups/jobs related to it, then the history for
assets also would disappear upon restarting docker. Or... I think that's what's happening.
Thanks for the input. I will poke around with it more to see if I can determine what's going on.
owen02/14/2023, 9:09 PM
, then checking the number of the rows in the
table should help determine if anything is actually getting deleted. I'm biased towards thinking that's very unlikely but I've been surprised before on these things
clay02/14/2023, 9:13 PM
daniel02/14/2023, 9:25 PM
clay02/14/2023, 9:27 PM
daniel02/14/2023, 9:27 PM
clay02/14/2023, 9:28 PM
daniel02/15/2023, 3:22 PM