Hi team, is it possible to set different tag values for different runs of the same pipeline? If yes, is there a way to search and filter the runs in the Dagit by the tag values?

yes and yes

Thanks Alex. Do we have any examples on how to set the tag values while creating a pipeline run from sensors and dagit?

in dagit there is a tags editor in the playground in sensors

RunRequest

has a tags arg

Ah got it. Thanks a lot for your help. Would also love to get your additional inputs on our pipeline design. Context: For our usecase, we have a set of entities stored in DB that needs to be processed and I am currently creating a separate pipeline to process each entity by dynamically generating the pipeline with the config set at definition using

configured

(instead of creating a single pipeline and creating a new run for each entity). I thought this would give us separate visibility into each entity process and also make it easier to manage them. But with this design, we might have to update the user-code frequently (al least daily) to include the new entities in the DB. Now I am thinking if we should just create a single configurable pipeline and pass the entity details through the config and tags and create different runs for different entities. With this design, the sensors can directly read the entities from the DB and trigger the runs and the user code need not be changed to include new entities.

yea this direction you are heading sounds good, a more static pipeline that varies its behavior based on config should go smoother for the reasons you mentioned of reducing user code changes

Thanks Alex, this is helpful. I might have to request multiple pipeline runs, because each entity has some dependencies which must be satisfied before starting the process on it. I am going to have a single sensor which will check the dependencies for all of the entities by calling an API on each and triggering a pipeline only for those that are satisfied. I was initially concerned about the performance of having a single sensor looping through all the entities, but I think it should be fine.