👋 Hello all, catching up a bit on run isolation.
It looks like a really cool feature, but I have a few questions:
1. Will there be reduced cost in the future for using non-isolated runs outside of the faster startup time? Seems like going from 4cpu, 16gb ram -> .25cpu, 1gb ram should be cheaper, but, perhaps I’m misunderstanding how this works together 🤔
a. EG: Non-isolated runs should be much cheaper for short running jobs, where lots of the cost will be from the ~3 minute startup time. However, many of our jobs are in the 30+ minutes to 2 hours range. Currently, we don’t have much a reason to want to switch over since the dagster cost is a small portion of the overall job cost, and we have still other jobs are running and taking advantage of the increased compute resources of isolated runs (even if this isn’t ‘best practice’).
2. Do we have access to said PEX files? Something I’m looking into is creating a PexStepLauncher and a great starting point could be to just deploy said pex objects to an EC2/ECS instance. This is almost like a “psuedo” hybrid deployment :blob_thinking: