https://dagster.io/ logo
#ask-community
Title
# ask-community
d

Dante Oz

06/01/2023, 10:16 PM
I am running into an error with
defs.load_asset_value()
where the FileNotFoundError shows a different file path (
/var/folders/ks/9mzlnwgs5dg5zzkyy07r25380000gn/T/tmp7wtsmlvu/storage/seed_channels
) than what is displayed in dagit (
/Users/rexor/projects/fastsearch/pipeline/tmpash1tyfe/storage/seed_channels
). Not sure how to go about debugging this. I am running dagster 1.3.7 inside of a conda environment.
Copy code
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[2], line 1
----> 1 asset = defs.load_asset_value("seed_channels")

File ~/mambaforge/envs/fastsearch/lib/python3.11/site-packages/dagster/_core/definitions/definitions_class.py:452, in Definitions.load_asset_value(self, asset_key, python_type, instance, partition_key)
    426 @public
    427 def load_asset_value(
    428     self,
   (...)
    433     partition_key: Optional[str] = None,
    434 ) -> object:
    435     """Load the contents of an asset as a Python object.
    436 
    437     Invokes `load_input` on the :py:class:`IOManager` associated with the asset.
   (...)
    450         The contents of an asset as a Python object.
    451     """
--> 452     return self.get_repository_def().load_asset_value(
    453         asset_key=asset_key,
    454         python_type=python_type,
    455         instance=instance,
    456         partition_key=partition_key,
    457     )

File ~/mambaforge/envs/fastsearch/lib/python3.11/site-packages/dagster/_core/definitions/repository_definition/repository_definition.py:313, in RepositoryDefinition.load_asset_value(self, asset_key, python_type, instance, partition_key, resource_config)
    308 from dagster._core.storage.asset_value_loader import AssetValueLoader
    310 with AssetValueLoader(
    311     self.assets_defs_by_key, self.source_assets_by_key, instance=instance
    312 ) as loader:
--> 313     return loader.load_asset_value(
    314         asset_key,
    315         python_type=python_type,
    316         partition_key=partition_key,
    317         resource_config=resource_config,
    318     )

File ~/mambaforge/envs/fastsearch/lib/python3.11/site-packages/dagster/_core/storage/asset_value_loader.py:165, in AssetValueLoader.load_asset_value(self, asset_key, python_type, partition_key, resource_config)
    140 io_manager_config = get_mapped_resource_config(
    141     {io_manager_key: io_manager_def}, io_resource_config
    142 )
    144 input_context = build_input_context(
    145     name=None,
    146     asset_key=asset_key,
   (...)
    162     instance=self._instance,
    163 )
--> 165 return io_manager.load_input(input_context)

File ~/mambaforge/envs/fastsearch/lib/python3.11/site-packages/dagster/_core/storage/upath_io_manager.py:217, in UPathIOManager.load_input(self, context)
    215 if not context.has_asset_key or not context.has_asset_partitions:
    216     path = self._get_path(context)
--> 217     return self._load_single_input(path, context)
    218 else:
    219     asset_partition_keys = context.asset_partition_keys

File ~/mambaforge/envs/fastsearch/lib/python3.11/site-packages/dagster/_core/storage/upath_io_manager.py:172, in UPathIOManager._load_single_input(self, path, context, backcompat_path)
    170             raise e
    171     else:
--> 172         raise e
    174 context.add_input_metadata({"path": MetadataValue.path(str(path))})
    175 return obj

File ~/mambaforge/envs/fastsearch/lib/python3.11/site-packages/dagster/_core/storage/upath_io_manager.py:160, in UPathIOManager._load_single_input(self, path, context, backcompat_path)
    158 context.log.debug(self.get_loading_input_log_message(path))
    159 try:
--> 160     obj = self.load_from_path(context=context, path=path)
    161 except FileNotFoundError as e:
    162     if backcompat_path is not None:

File ~/mambaforge/envs/fastsearch/lib/python3.11/site-packages/dagster/_core/storage/fs_io_manager.py:273, in PickledObjectFilesystemIOManager.load_from_path(self, context, path)
    272 def load_from_path(self, context: InputContext, path: UPath) -> Any:
--> 273     with path.open("rb") as file:
    274         return pickle.load(file)

File ~/mambaforge/envs/fastsearch/lib/python3.11/pathlib.py:1044, in Path.open(self, mode, buffering, encoding, errors, newline)
   1042 if "b" not in mode:
   1043     encoding = io.text_encoding(encoding)
-> 1044 return io.open(self, mode, buffering, encoding, errors, newline)

FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/ks/9mzlnwgs5dg5zzkyy07r25380000gn/T/tmp7wtsmlvu/storage/seed_channels'
Screenshot 2023-06-01 at 6.18.30 PM.png
c

claire

06/01/2023, 10:28 PM
Looks like your code snippet is attempting to read from a temporary directory, which makes me think that you're not reading from your instance. Do you have a DAGSTER_HOME env var set? You can also add an instance arg to
defs.load_asset_value("seed_channels", instance=...)
d

Dante Oz

06/02/2023, 1:10 AM
I reran the with DAGSTER_HOME set and ran into the same error.
c

claire

06/08/2023, 10:09 PM
That's odd, I'm unable to repro. When my DAGSTER_HOME env var is set,
load_asset_value
reads from the configured dagster instance. Are you sure that the path is set correctly in your environment?
3 Views