Hi all, what is the best way to setup Dagster for multi-tenants use case? To be easier to understand, let looks at this example. We have 3 tenants: Tenant1, Tenant2 Tenant3 We have this DAG : opA -> opB -> opC opA -> opD -> opE We want to use this same DAG for all 3 tenants, but with different configuration for each tenant. Then I want the Dagit UI to show the jobs like this: Tenant1_DAG: … Tenant2_DAG: … Tenant3_DAG: … I hope that is clear enough. Can we do things like this example?

Hi @Son Giang! If I understand your use case correctly, I believe you should be able to make a graph of your three ops, then create multiple jobs from that graph using the

<http://GraphDefinition.to|GraphDefinition.to>_job

function. You can pass unique resource config and run config to the

to_job

function. In Dagit, the jobs will get the names you provide to them in

to_job

. So your code might look something like

@graph 
def my_graph():
    opA()
    opB()
    opC()

tenant1_dag = my_graph.to_job(resource_defs, run_config, ...)
tenant2_dag = my_graph.to_job(resource_defs, run_config, ...)
tenant3_dag = my_graph.to_job(resource_defs, run_config, ...)

In dagit you should see three jobs

tenant1_dag

,

tenant2_dag

,

tenant3_dag

https://docs.dagster.io/concepts/ops-jobs-graphs/jobs-graphs#from-a-graph https://docs.dagster.io/_apidocs/graphs#dagster.GraphDefinition.to_job

Hi @jamie, thank you for the help, that’s what I need. However, can you tell me what is the best practice for loading configuration from database and use that for creating job? Should it be defined in

__init__.py

in jobs folder? And can it be reused from resource/config inside the op/job? Like I have a database connection that contains both the configuration to create a job and the run config of a job.