Andras Somi
06/28/2023, 9:31 AMAn op only starts to execute once all of its inputs have been resolved. Inputs can be resolved in two ways:
• The upstream output that the input depends on has been successfully emitted and stored.
• The input was stubbed through config.https://docs.dagster.io/concepts/ops-jobs-graphs/ops#inputs How does one “stub” an input “through config”? Is it somehow possible to have a simple op
@op
def my_op(my_input:str):
# do something with my_input
# ...
and provide my_input
through a Pythonic config object without having to do something like this:
@op(ins={"my_input": In(Optional[str], default_value=None)})
def my_op(config: MyConfig, my_input:str | None = None):
my_input = my_input if my_input is not None else config.configured_input
I wouldn’t consider the latter as “stubbing” the input, given that all the gymnastics around finding the actual value is done in the body of the function. Am I missing something here?Harry Park
06/28/2023, 11:42 AMmy_input
to that.
Here’s an example op with an array of Outputs passed to it
@op(ins={"start": In(Nothing)})
def OrderPipeline(context, nums: List[Output]):
shell_script_runner(context, "MyOrderPipeline.ksh")
sean
06/28/2023, 1:22 PMfrom dagster import job, op
@op
def foo(a: int) -> int:
return a + 1
@job
def foo_job():
foo()
foo_job.execute_in_process(
run_config={"ops": {"foo": {"inputs": {"a": 1}}}}
)
I realize the terminology is confusing, but this is separate from the “config” for the op, (i.e. what the op’s config schema defines).Andras Somi
06/28/2023, 1:46 PMsean
06/28/2023, 2:00 PMAndras Somi
06/28/2023, 2:45 PM