I am trying to create a few repositories dynamically using a for loop. For each repository, I have one job in each pipeline. When I am trying to do this using the

@repository

decorator, it is only creating the repository for the last iteration of the loop. Not sure why that is happening. Probably have something to do with the decorated function namespace. Wondering if there is a better way to achieve this e.g. creating a repository using

RepositoryDefinition

or something. Following is a glimpse of what I am trying to do:

# pipelines = ["preprocessing","training","inference"]
for pipeline in pipelines:
    ....
    # creating ops and dependencies for that pipeline dynamically here 
    ....

    job = GraphDefinition(name=pipeline, node_defs=ops, dependencies=deps).to_job()

    @repository(name=pipeline)
    def my_repository():
        return [job]

    my_repository.__name__ = pipeline

Currently, it is only showing the repository for

inference

pipeline in the

dagit

ui. Is it even possible to load multiple repositories from a single python file?

you certainly can have multiple repos in a python file. the way you have it right now though you're saving over your repository variable

my_repository

every loop, so you only ever have one repository. I don't know exactly how to do it, but it does seem like you might need to use

RepositoryDefinition

to do this dynamically, as the

@repository

decorator is going to assign to an object with the name of the function it's wrapping, and function names / signatures can't be changed dynamically.

Agreed. Although, I thought since I was explicitly mentioning the

name

of pipeline as a parameter of

@repository

decorator and also changing the

__name__

attribute of

my_repository

function object, the namespace and overwriting issues would be taken care of. But I guess not.