Hi Everyone, I'm able to pass arguments when invok...
# ask-community
j
Hi Everyone, I'm able to pass arguments when invoking an op as per examples.
Copy code
# pass config as arg
my_op(build_op_context(), 1, 2, MyConfig(z=3))
But how to do this when calling the op from a job? When attempting to do the above in a job I get an error when starting up Dagster:
Copy code
Must pass the output from previous node invocations or inputs to the composition function as inputs when invoking nodes during composition.
🤔
👍 1
🤖 1
z
could you provide an example of what you're trying to do when you get this error? generally, when passing inputs to an op they need to be passed from another op's outputs. in Dagster you don't hard-code arguments in your graph, and the
context
is passed automatically under the hood - you won't need to use
build_op_context
when constructing a graph or calling ops (it's generally used for testing ops that rely on an
OpExecutionContext
. to pass arguments to your job to kick it off you'll generally use run config, which will allow you to pass initial configuration to each op that defines a config schema or pythonic config.
s
What Zach said is correct-- have you done the Tutorial? Highly recommended to get acquainted with the basics: https://docs.dagster.io/tutorial
j
thanks for the feedback guys, the code example above is from https://github.com/dagster-io/dagster/pull/13459
just thinking if I have an op using a config, is there a way I can call the op from a job using different config values? (example below)
Copy code
class MyConfig(Config):
    z: int

@op
def my_op(context, config: MyConfig):
    return config.z

# pass config as arg --> works
my_op(MyConfig(z=3))
my_op(MyConfig(z=4))

# how to invoke op from job with different config values? --> doesn't work
@job
def my_job():
    my_op(MyConfig(z=3))
    my_op(MyConfig(z=4))
using a hardcoded configuration for example works but then you can obviously only call the op once in the job with one set of configuration values
Copy code
my_config = RunConfig(
    ops={"my_op": MyConfig(z=3)}
)

@job(config=my_config)
def my_job():
  my_op()
I'm busy with a POC and I mainly use SDAs, but in this case I've got an op that's basically doing a reverse ETL, preparing & posting journals from a Data Warehouse to Oracle NetSuite via REST API
s
Multiple invocations of the same op get aliased automatically (you can also choose the aliases). See here: https://docs.dagster.io/concepts/ops-jobs-graphs/graphs#that-re-use-ops This allows you to set separate config for each call:
Copy code
from dagster import Config, RunConfig, job, op


class AddNConfig(Config):
    n: int


@op
def return_one(context) -> int:
    return 1


@op
def add_n(context, number, config: AddNConfig) -> int:
    return number + config.n


@op
def adder(context, a: int, b: int) -> int:
    result = a + b
    print(result)
    return result


@job
def inputs_and_outputs():
    value = return_one()
    a = add_n(value)
    b = add_n(value)
    adder(a, b)


inputs_and_outputs.execute_in_process(
    run_config=RunConfig({
        "add_n": AddNConfig(n=1),
        "add_n_2": AddNConfig(n=1)
    })
)
j
didn't know about the aliasing, I'll give that a go thank you