https://dagster.io/ logo
#ask-community
Title
# ask-community
s

Simrun Basuita

05/15/2023, 5:29 PM
Hi there, I've just deployed our local dagster to dagster cloud, but am getting a strange error running our pipeline on cloud which doesn't happen on local. I'm confused because I can't reproduce the error on my local machine. The dagster version should be identical (and I checked it: 1.3.4). I've also fixed the python version to be the same in cloud as local (3.10). Any ideas? :S
Copy code
dagster._core.errors.DagsterExecutionLoadInputError: Error occurred while loading input "company_returns" of step "archetype_returns":
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_plan.py", line 262, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 335, in core_dagster_event_sequence_for_step
    for event_or_input_value in step_input.source.load_input_object(step_context, input_def):
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_core/execution/plan/inputs.py", line 187, in load_input_object
    yield from _load_input_with_input_manager(loader, load_input_context)
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_core/execution/plan/inputs.py", line 797, in _load_input_with_input_manager
    with op_execution_error_boundary(
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 84, in op_execution_error_boundary
    raise error_cls(
The above exception was caused by the following exception:

dagster._check.CheckError: Failure condition: Tried to access partition key for asset 'AssetKey(['company_returns'])', but the number of input partitions != 1: 'DefaultPartitionsSubset(subset={'DT55', 'DT30', 'DT60', 'DT45', 'DT25', 'DT40', 'DT50', 'DT15', 'DT10', 'DT35', 'DT20'}, partitions_def='DT10', 'DT15', 'DT20', 'DT25', 'DT30', 'DT35', 'DT40', 'DT45', 'DT50', 'DT55', 'DT60')'.
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 54, in op_execution_error_boundary
    yield
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_core/execution/plan/inputs.py", line 804, in _load_input_with_input_manager
    value = input_manager.load_input(context)
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster_cloud/serverless/io_manager.py", line 87, in load_input
    key = self._get_path(context)
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster_cloud/serverless/io_manager.py", line 52, in _get_path
    path = context.get_asset_identifier()
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_core/execution/context/input.py", line 453, in get_asset_identifier
    return [*self.asset_key.path, self.asset_partition_key]
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_core/execution/context/input.py", line 341, in asset_partition_key
    check.failed(
  File "/venvs/893b35886e6f/lib/python3.10/site-packages/dagster/_check/__init__.py", line 1669, in failed
    raise CheckError(f"Failure condition: {desc}")
For context,
company_returns
is a static partitioned asset.
archetype_returns
is not partitioned.
c

claire

05/15/2023, 5:54 PM
Hi Sim. Which IO manager are you using for each asset?
s

Simrun Basuita

05/15/2023, 5:54 PM
fs_io_manager
c

claire

05/15/2023, 6:02 PM
Hm... looks like you're running on serverless. Not too familiar with the details and will follow up with the team, but I think dagster is replacing the
fs_io_manager
with the serverless IO manager which can't handle loading multiple inputs. I think you'll need to explicitly define another IO manager that can handle these interactions i.e. the
s3_pickle_io_manager
s

Simrun Basuita

05/15/2023, 6:02 PM
oh 😕
Yeah if you could follow up with the team that would be much appreciated
c

claire

05/15/2023, 6:18 PM
Hi Sim. Followed up with the team--this is a bug with the default
serverless_io_manager
used in serverless. It doesn't yet handle loading from multiple input partitions, which is why you're seeing this error. I'd recommend filing an issue to fix this. In the meantime, I would recommend that you explicitly define another IO manager to use.
s

Simrun Basuita

05/15/2023, 6:18 PM
Okay, thank you.
How do I file an issue?
Presumably I can't explicitly set
fs_io_manager
in serverless? It gets replaced with
serverless_io_manager
right?
c

claire

05/15/2023, 6:23 PM
You can file an issue here: https://github.com/dagster-io/dagster/issues I think you cannot set
fs_io_manager
in serverless because there is no explicit file directory to persist outputs to, it would have to be an IO manager that writes to cloud storage i.e. s3, gcs, snowflake, etc.
s

Simrun Basuita

05/15/2023, 6:23 PM
okay, thanks
2 Views