Mark Fickett03/30/2022, 4:04 PM
in the pipeline started? I know Dagster has an internal event log and a GraphQL API (looks like the Python API client doesn't touch on events). Would it be crazy to get the current run ID from context, query GraphQL to get the timestamp of an earlier op start, and use that to compute/log elapsed time for some portion of the DAG? Is there a simpler way to access events from within a running job?
owen03/30/2022, 4:22 PM
property, which can be used to interact with the active DagsterInstance. So for your use case you could probably do something using
, then filter for the specific op you're interested in from there
Mark Fickett03/31/2022, 2:34 PM
will work to get records of events from `@op`s run previously on different Kubernetes nodes? I've been testing it with local python multithreading, just want to make sure it'll keep working in our planned deployment environment.