Martin Peak
03/01/2024, 3:26 AMScout
03/01/2024, 3:26 AMUPathIOManager
to use the default Dagster storage path, you don't need to specify a base_dir
explicitly. The default storage path for the UPathIOManager
is determined by a hierarchy of possible locations, as described in the documentation: 1. The base_dir
configuration value of the IO manager, if specified. 2. A storage/
directory underneath the value for local_artifact_storage
in your dagster.yaml
file, if specified. 3. A storage/
directory underneath the directory that the DAGSTER_HOME
environment variable points to, if that environment variable is specified. 4. A temporary directory, if none of the above are specified. If you want to use the default Dagster storage path, ensure that you do not provide a base_dir
when configuring the UPathIOManager
. Also, make sure that you have set the DAGSTER_HOME
environment variable to the desired location, or have configured local_artifact_storage
in your dagster.yaml
to point to the desired directory. If neither is set, Dagster will use a temporary directory for storage. Here's an example of how you might configure the UPathIOManager
without specifying a base_dir
, allowing it to fall back to the default storage path: python from dagster import io_manager, UPathIOManager @io_manager def default_upath_io_manager(): return UPathIOManager()
By not providing a base_dir
to UPathIOManager
, it will follow the hierarchy described above to determine the storage path.