I’m in the middle of attempting to convert from ho...
# ask-community
k
I’m in the middle of attempting to convert from homegrown workflow system to Dagster. When resuming runs, Dagster seems to generate a new
run_id
. The existing code expects that resumes maintain the same run id (which determines working directories and so on). Is there an id in Dagster that uniquely identifies a run across resumes? I see references to a `root_run_id`…
p
Hi Keith. When a failed run is resume / retried, it gets a new run id, but it becomes part of a run group, which has the originating run id as the
root_run_id
(as you have identified). If you keep retrying runs, each new run will have the last attempted run as a
parent_run_id
, while keeping the original run id as the root. This run group shows up in the right side panel on the Run view, showing the run lineage. I’m not sure if that will serve your purpose. There is not a run id that uniquely identifies the entire group, but I can get the set of runs from the run group based on the common root run id.
k
Thanks for explaining, @prha. So it sounds like that root run id should be what I want. I was getting the
run_id
from the Context and setting it in the environment for the called process. I don’t see
root_run_id
documented for Contexts (https://docs.dagster.io/_apidocs/execution#contexts) but I’ll experiment and see if it’s on there.
Looks like the Context doesn’t have the
root_run_id
, but
context.pipeline
does, but it’s null when starting out. So, the parent run is the immediately preceding run to this one in a chain, and all can be part of a group that’s represented by the root run id, but that’s only set if there’s been a resume?
p
Yes, I think that’s right
k
Ok thanks. So if the
root_run_id
== the original
run_id
, then that should get me what I need. Experimenting.
Yeah looks like that works that way. So I’ll be able to use the original working directory based on root_run_id || run_id. Thanks!
It seems like
root_run_id
should always be set, and should == the
run_id
if it’s the first run?
p
I think that sounds reasonable… mind filing an issue?
k
Sure will do!
Here’s that ticket: https://github.com/dagster-io/dagster/issues/6356 I also opened https://github.com/dagster-io/dagster/issues/6357 to describe a problem with using environment variables in shell ops in
dagster_shell
.