Hey! I want to generate a pipeline using Yaml, is ...
# announcements
j
Hey! I want to generate a pipeline using Yaml, is it required to generate the python file or is there a dagit API which could be directly plugged into?
d
Hi Jaikant - currently there needs to be some function in Python that returns a RepositoryDefinition containing PipelineDefinitions (but that function could absolutely create the definitions from a YAML file)
👍 1
j
Thanks Daniel!
dagstir 1
j
Thank you for the great links @alex
@alex @daniel this isn’t the recommended way, because you guys have something better in mind or just keeping possibilities open? I am referring to code generation for pipelines. Any wisdom to share on code generation for pipelines?
Is it better to generate the code for the pipeline python files and the python repo files or use the RepositoryDefinition / PipelineDefinition? Whats your opinion?
d
I think it depends on your reason for wanting to generate them from YAML files - but generating the Definition classes from YAML seems more straightforward to me than generating code.
(since the @pipeline decorator implementation just outputs a PipelineDefinition - so if I wasn't starting from code I'd probably default to skipping the middleman and creating a PipelineDefinition)
👍 1
j
Thanks Daniel. As you mentioned, it is more direct using the Definition classes.