https://dagster.io/ logo
#ask-ai
Title
# ask-ai
l

Lucas Humbert

02/21/2024, 10:36 AM
How to call an op function in the definition of another op function. How to give the parameter asked in the config schema of the called op ?
s

Scout

02/21/2024, 10:36 AM
To call one op from within another in Dagster, you typically wouldn't invoke one op directly inside the compute function of another op. Instead, you would define the dependencies between ops within a graph or job, which orchestrates how data flows between ops. However, if you need to use the result of one op as a parameter for another op, you would define the flow of data using inputs and outputs. Each op can have its own configuration schema, and you can pass the necessary parameters through the config when you execute the job or graph that contains the ops. Here's a step-by-step guide based on the documentation: 1. Define the ops with their respective input and output definitions. Use the
@op
decorator to define each op. 2. If an op requires configuration, you can define a config schema for that op using the
config_schema
parameter. 3. In the graph or job definition, you would connect the ops by passing the output of one op as the input to another. 4. When executing the graph or job, you provide the config values that match the defined config schemas for the ops. Here's an example of how you might define two ops where one op's output is used as a parameter for another op:
Copy code
python from dagster import op, job, Out, In, ConfigSchema  @op(config_schema={"param": str}) def op_with_config(context):     # Use the config parameter in the op's logic     param = context.op_config["param"]     # ... perform some operation with param ...     return result  @op def downstream_op(context, input_param):     # Use the input parameter in this op's logic     # ... perform some operation with input_param ...  @job def my_job():     downstream_op(op_with_config())
When executing
my_job
, you would provide the config for `op_with_config`:
Copy code
python result = my_job.execute_in_process(     run_config={         "ops": {             "op_with_config": {                 "config": {                     "param": "value"                 }             }         }     } )
In this example,
op_with_config
has a config schema that expects a parameter named "param". When
my_job
is executed, the
run_config
provides the value for "param", which is then used within
op_with_config
. The output of
op_with_config
is passed as an input to
downstream_op
. Remember that the actual invocation of ops and the passing of parameters are handled by Dagster's execution engine when you run the job or graph, not by directly calling one op from another in the compute function code.
10 Views