Hello! I am new to dagster. I have a question. Is there a way to run a pipeline multiple times based on a list of values that I can pass in somehow? For example, I slightly modified the hello world example from the main github page where I am executing 2 tasks that don't depend on each other.

from dagster import (

    

execute_pipeline,

      

pipeline,

      

solid

)

@solid

def get_name(_, plant_code: str):

    

print(f"Plant code: {plant_code}")

@solid

def hello(context, plant_code: str):

<http://context.log.info|context.log.info>('Hello, {name}!'.format(name=plant_code))

@pipeline

def hello_pipeline():

    

get_name()

    

hello()

if __name__ == "__main__":

    

run_config = {

        

"solids": {

            

"get_name": {"inputs": {"plant_code": {"value": "Plant1"}}},

            

"hello": {"inputs": {"plant_code": {"value": "Plant1"}}}

        

}

    

}

    

execute_pipeline(hello_pipeline, run_config=run_config)

# want to do something like this:     value = ["Plant1", "Plant2", "Plant3"]     for plant_code in value:         execute_my_pipeline(plant_code) etc... I can create a pipeline for one plant code "Plant1", but let's say I want to execute this set of tasks in this pipeline for each additional plant code. With the run_config, not sure if it is possible to loop through a list of values passed into the "value" parameter. I hope my question makes sense. Any help is much appreciated!

Hi Daniel - is it not an option to use different run config for each execution to accomplish this? i.e. you could take in plant_code as an argument to your function that calls execute_pipeline, and set the run config that's passed into execute_pipeline based on the value of plant_code?

Hi @daniel I see what you mean. I can do what you suggest for now. My list of values or plant codes could be dynamic or a user may wish to be able to pass in any number of values for the pipeline input. I realize that the proper terminology for what I think I am after is perhaps called creating "dynamic pipelines". I think I found a GH issue related to what I'm after: https://github.com/dagster-io/dagster/issues/2077 I'll take a deeper look at Nick's suggestion and let it sink in for me for a while. Thanks for your response!

Ah great - you might also find https://github.com/dagster-io/dagster/discussions/2902 interesting, that's a bit more recent and outlines future work to make pipelines parametrizable in the same way that solids are