I’m not sure if there is another way I’m supposed to do this. What I would like to do is in my pipeline I am checking to see if an op returns data. If it does then I want it to continue processing the rest of the ops in the graph, else do nothing. I tried to use conditional branching, where I have a no data op branch and a received data op branch that executes the rest of the ops. The problem I’m having is I don’t see the ops in dagit. I only see the parent op received data op and not all the ops that it’s executing. Please let me know if there is a better way to do this or how to be able to see all the ops in dagit if this is the correct method
dagster bot responded by community 1
12/09/2022, 7:26 PM
You can configure an Op to have optional outputs, which makes anything downstream from it in the graph not be executed.