Hello team - I've done quite a bit of work with bu...
# ask-community
z
Hello team - I've done quite a bit of work with building out custom StepLaunchers in order to be able to run jobs on various compute platforms, which has required delving into quite a few of the 'private' methods in dagster. It appears that in the latest 0.14.15 release the StepLauncher and associated methods are still not importable from the top-level dagster - are there plans to make the StepLauncher functionality public? The ability to swap StepLaunchers is amazing and seems unnecessarily undocumented / hidden - it's really helping me achieve my dream of platform-independent DAGs and I've been hoping it gets a little more visibility in general
1
s
Hi Zach-- thanks for pointing this out,
StepLauncher
should be public. We’ll address this in the next release.
@Zach, we’re aiming to address this issue this week. Here are the names we’re adding as of now: https://github.com/dagster-io/dagster/pull/7945/files Anything else you’re using?
z
@sean - thanks for reaching out and being accommodating. Those additions will be super helpful. There's only a couple other small things I'm using:
Copy code
from dagster.core.errors import (
    raise_execution_interrupts,
    DagsterExecutionInterruptedError,
)
in order to be able to catch Dagster terminations and properly shut down external compute resources
Copy code
from dagster.serdes import deserialize_value
- for deserializing pickled step event records please do let me know if there are better interfaces to rely on for these purposes
s
Cool, we’ve added these (also
serialize_value
) to the PR, which was merged into the release 0.14.16 going out today. Don’t hesitate to reach out again if you discover other private stuff you think should be public, we’re expecting some growing pains as we go through the formalization process.