Team, hello. We have our own triggering and retry logic in our data pipeline, but we only want to use the visualisation part of the Dagstar. to understand what stage the pipelines have been at so far. can we do this using dagster?

hi @Yeshwanth LN! how are your pipelines currently defined? In order to visualize progress, you'll need to break each step into a separate op (as dagster generally shows progress in terms of per-op status). It is possible to programmatically generate a dagster job from a different system, but without knowing more I'm not sure if I'd recommend that