https://dagster.io/ logo
Join Slack
Powered by
Hi, is it possible to pass an `op` that returns `N...
# ask-community
i
Hi, is it possible to pass an 
op
that returns 
None
to another 
op
?
I am getting the following error when trying to do this:
Copy code
dagster._core.errors.DagsterExecutionLoadInputError: Error occurred while loading input "_" of step "spark_submit_op":
  File "/app/.venv/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_plan.py", line 265, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
  File "/app/.venv/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 344, in core_dagster_event_sequence_for_step
    for event_or_input_value in ensure_gen(
  File "/app/.venv/lib/python3.10/site-packages/dagster/_core/execution/plan/inputs.py", line 523, in load_input_object
    yield from _load_input_with_input_manager(input_manager, load_input_context)
  File "/app/.venv/lib/python3.10/site-packages/dagster/_core/execution/plan/inputs.py", line 866, in _load_input_with_input_manager
    with op_execution_error_boundary(
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/app/.venv/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 85, in op_execution_error_boundary
    raise error_cls(
The above exception was caused by the following exception:
botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: Unknown
  File "/app/.venv/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 55, in op_execution_error_boundary
    yield
  File "/app/.venv/lib/python3.10/site-packages/dagster/_core/execution/plan/inputs.py", line 873, in _load_input_with_input_manager
    value = input_manager.load_input(context)
  File "/app/.venv/lib/python3.10/site-packages/dagster_aws/s3/io_manager.py", line 74, in load_input
    obj = pickle.loads(self.s3.get_object(Bucket=self.bucket, Key=key)["Body"].read())
  File "/app/.venv/lib/python3.10/site-packages/botocore/client.py", line 530, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/app/.venv/lib/python3.10/site-packages/botocore/client.py", line 960, in _make_api_call
    raise error_class(parsed_response, operation_name)
j
i
Hmm, do you know what would happen if 
Nothing
is used but the value passed is not 
None
?
j
Not sure, as io_manager you are using have something like that:
Copy code
if context.dagster_type.typing_type == type(None):
            check.invariant(
                obj is None,
                (
                    "Output had Nothing type or 'None' annotation, but handle_output received"
                    f" value that was not None and was of type {type(obj)}."
                ),
            )
            return None
4 Views
Open in Slack
PreviousNext
Powered by