https://dagster.io/ logo
Title
a

Anoop Sharma

03/05/2022, 9:42 AM
Is there a way to have a shared variable available across all the ops within a job, whose value can also be updated by a certain op in the job? Essentially, an editable environment variable for a job.
a

Anoop Sharma

03/05/2022, 3:54 PM
Yes but I also need the ability to update that resource's value from each op. For example, consider a STATUS resource for which I want value to be updated as it moves through different stages - first op will set it as Started, second op will update it to InProgress and third will update it to Completed.
p

prha

03/07/2022, 5:38 PM
@Anoop Sharma there’s no mechanism for that currently. I think your best bet at the moment would be to have some resource that’s backed by a persistent store, since there’s no guarantees that the ops would execute in the same process or even on the same machine (depends on the executor / run launcher).
a

Anoop Sharma

03/08/2022, 9:57 AM
Got it. Thanks.