When using op selection syntax is it possible to exclude one

Question

When using op selection syntax is it possible to exclude one or more ops and include the rest

yuhan · Answer

Hi Leo we don t have a syntax supporting exclude rules at the moment but curious to hear your thoughts do you have anything in mind that what syntax you d be writing to exclude one or more ops

Leo Kell · Answer

dbt has an <https docs getdbt com reference node selection exclude|exclude> flag that has the same section rules as its `select` flag Maybe dagster can do something similar For reference I was thinking this could be useful for a case where I am creating a weekly schedule based on an hourly schedule The weekly schedule does a full refresh of a subset of some of the hourly tasks I don t want it to target all of the hourly tasks because some of the them would take too long to fully refresh So being able to exclude some tasks from the weekly schedule would make it easier to define and maintain the schedule

yuhan · Answer

Got it Thanks for the content I m currently working on improving op selection Will spend some more time exploring the exclude options while doing that

Leo Kell · Answer

Sounds good thanks Let me know if you have any additional questions as you explore this

Leo Kell · Answer

< yuhan> I have an additional question If I have a Job that contains nested graphs is it possible to select specific subgraphs in a schedule definition If this doesn t exist I think could be helpful to have a `graph selection` argument similar to the `solid selection` argument

yuhan · Answer

We don t have that functionality yet This is what I m actively working on building Will keep you posted with the progress

Leo Kell · Answer

Perfect slightly smiling face

Leo Kell · Answer

< yuhan> I just saw that the latest dagster release includes op selection inside subgraphs This is really great so I would like to thank you and the team slightly smiling face I m also wondering have you had a chance to explore exclude options for op selection

yuhan · Answer

Thanks < Leo Kell> I haven t had time to explore the exclude but it s def on the plan Here s the tracking issue

yuhan · Answer

< Leo Kell> one more follow up question are you using dagit s selection text input or op selection api i was prioritizing the dagit flow under the assumption that the dagit flow of this has more usage but i could be wrong

Leo Kell · Answer

< yuhan> sorry for the late reply I am actually using both the dagit interface and the `op selection` API particular in the Graph `to job` method I use the dagit selection for development and troubleshooting certain ops and I the `op selection` API to create specific pipelines run specific op subsets In terms of the exclude feature I would have more immediate usage for it within the op selection API

yuhan · Answer

got it thanks for the info