https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jake Kagan

08/28/2023, 12:26 PM
hey everyone question about automating job creation: let's say i setup a database table thats lists out all the jobs i want to create... beyond just writing a script that will generate files on dagster side - is there a way to tap into the api or something to loop through the table and create those jobs? i really don't want to start generating text if i dont have to, because im worried that it will stop jobs that are running or something
i dont see anything about creating a job...
s

sandy

08/28/2023, 3:09 PM
hi @Jake Kagan could you have the code that builds your
Definitions
object iterate over the values in the database table?
j

Jake Kagan

08/28/2023, 10:49 PM
@sandy hey thanks for getting back to me! ok i figured out what you meant:
Copy code
from dagster import op, configured, get_dagster_logger, job, Definitions

## -- NAMER

def name_job(job_name: str):
	def decorator(func):
		func.__name__ = job_name
		return func
	return decorator


## -- OP SETUP

@op(config_schema={"passed_arg": str})
def op_names(context):
    arg = context.op_config['passed_arg']
    get_dagster_logger().info(arg)

def op_configger(arg):
    configged_op = configured(op_names, name=f'{arg}_op')(
        {"passed_arg": arg}
    )
    return configged_op


## -- JOB FACTORY

def job_configged(name, job_ops):
    @job()
    @name_job(name)
    def build():
        job_ops()
    return build


def job_factory(names):
    final_jobs = []
    for name in names:
        job_ops = op_configger(name)
        final_jobs.append(job_configged(name, job_ops))
    return final_jobs
         

## -- DEFINITIONS

names = ["job1", "job2", "job3", "job4"]
db_jobs= job_factory(names)
jobs = Definitions(jobs=db_jobs)
@sandy question for you though, if i load new data into the db table, i then need to restart the webserver and daemon right? so how do i ensure that it doesn't stop in the middle of any jobs - and that if i run it in between jobs, it wont prevent others from running if there's a lag? do i setup some designated time or something, like a window - or is there a better approach