Hello guys, Is there a way to have a staging feat...
# dagster-serverless
p
Hello guys, Is there a way to have a staging feature branch whilst persisting assets, runs, sensor cursors etc between deployments, whilst using serverless? We don’t want to rematerialize all of our assets each time our staging branch is updated. Thanks! Pierre @Misa Nogly Aleksic @Reid Falconer @Alan Hutcheson @Oliver Gate
j
hey @Pierre Cadman its not uncommon for users to have two normal deployments:
prod
and
staging
and have branch deployments for smoke-testing changes before merging to staging. If you do have this setup your
staging
shouldn't be getting cleared unless you have some dynamic renaming of the sensors/assets
m
This potentially relates to this discussion if I’m reading your problem correctly: https://github.com/dagster-io/dagster/discussions/12561#discussioncomment-5326467 It’s possible to create dummy materialisations of assets, especially if you are using something like snowflake and can create clones of prod without any overhead. That said, this does consume credits and there isn’t currently a way to say “assume there is a copy of all of my prod asssets pre-materialised at the paths my branch deployment is targeting”
p
Thanks guys, @Joe does this require enterprise pricing? @Matt Clarke this could point us in the right direction, our issue is mostly that the assets produced in staging are expensive and long to compute, and are used downstream so we’d like to not rematerialize more than required when we deploy to staging
j
does this require enterprise pricing?
it does but enterprise pricing might not be much more than what you are already paying with pay-go
p
We had investigated that route but couldn’t budget it 😕 Thanks though