Hello everyone, I have a python function for migrating tables from a database to another. So I want to create a job calling the same migration function, but with a different argument value defining the schema name. For this I've used the following doc: https://docs.dagster.io/concepts/configuration/configured#-reusing-an-op-definition-with-configured

migrate__1 = configured(migration_func,
                        name="migrate__1")({"db_source": "database_name",
                                            "schema_list": ["schema_name"]})
migrate__2 = configured(migration_func,
                        name="migrate__2")({"db_source": "database_name",
                                            "schema_list": ["schema_name"]})
@job
def job_db_migration():
    migrate__1()
    migrate__2()

It works just fine. However I cant access/set the variables "_db_source_" and "_schema_list_" from the launchpad. If I write the following structure, it doesn't change the execution of the job:

{

ops: {

migrate__1: {

config: {

db_source: "db_name",

schema_list: ["schema_name_new"]

}

}

}

}

What am I doing wrong? Thank you so much !

Hi @Davi the

configured

API sets the values for the configuration in code, so in your code snippet you are setting

db_source

to

database_name

. If you want to set values in the launchpad, instead you can define your job like this

@job  
def job_db_migration():
    migrate.alias("migrate__1")
    migrate.alias("migrate__2")

Hello :) Am I able to set one of the arguments via code and let only one to set via launchpad? On the same line where I choose an alias, I could as well, somehow, set one of the arguments ? Thank you so much !

yeah you can use configured but only set one of the config values and then the other should still be configurable in the launchpad

migrate__1 = configured(migration_func,
                        name="migrate__1")({"db_source": "database_1"})
migrate__2 = configured(migration_func,
                        name="migrate__2")({"db_source": "database_2"})
@job
def job_db_migration():
    migrate__1()
    migrate__2()

Thank you so much @jamie ! I really appreciate your help. :)

that's a bit odd because the error message is referencing an op called

apac_environment

. is there anything with that name anywhere in your code?

You are right ! I changed the names on the code to post it here, but schema_1 actually is called

apac_environment

, that's way on the error it uses the actual name on our production env. We have a bunch of different schemas on our migration process, but I called schema_1, schema_2, etc... to make it easier to understand here. :)

ok thanks for the clarification. we must have a straggling error message referring to solids somewhere. i'll look around for it. not sure how i missed this in the docs before, but to partially specify config you have to use the

@configured

decorator specifically, like this

@configured(
    migration_func,
    config_schema={"schema": str}
)
def migrate__1(config):
    return {"db_source": "database_1", "schema": config["schema"]}

@configured(
    migration_func,
    config_schema={"schema": str}
)
def migrate__2(config):
    return {"db_source": "database_1", "schema": config["schema"]}
    
@job
def job_db_migration():
    migrate__1()
    migrate__2()

i tried this out myself and it worked for me. sorry i missed this before and it took a while to get the correct code snippet

No problem! The most important is to solve the problem. haha 🙂