https://dagster.io/ logo
Title
d

Dylan Hunt

08/23/2021, 7:16 AM
Hi All, Right now I have two solids to hit a URL and return the response, It's actually same code blocks with different naming since I'm using execute_pipeline to run the Dagster from python, I'm passing two different URL's from the run_config. So the question is, Is there a way to avoid these code duplications to perform same tasks(solids) with different configs in a pipeline? I might reuse the same like 5-10 times in a complex scenario. TIA.
d

Daniel Kim

08/23/2021, 11:03 AM
c

chris

08/23/2021, 11:59 AM
^configured is one option, but I think in your use case you could also call the solid x number of times within your pipeline, alias it each time, and then provide the different run config for each alias
❤️ 1
This would be advantageous in case you didn’t know config until execution time. Otherwise, configured is probably cleaner.
❤️ 1
d

Dylan Hunt

08/23/2021, 4:51 PM
Thanks for the replies @chris and @Daniel Kim I'll tryout the mentioned options and get back here, If I need further clarifications. Again, I'm trying to build a REST endpoint which will dynamically generates solids and pipeline then execute them by another endpoint.