Writing my first IOManager and my first Dagster unittest (and thus my first attempt at using the

build_context

helper functions. Using dagster 1.3.1. My IOManager is subclassing

UPathIOManager

, which I think should automatically provide

path

to

dump_to_path

and

load_from_path

that incorporates the

asset_key

and the

partition_key

. I'm buidling my output context like this:

output_context = build_output_context(
        name="abc",
        step_key='mystep',
        asset_key="myasset_name",
        partition_key="my_partition_name",
        dagster_type=XarrayDagsterType,
    )
    input_context = build_input_context(
        upstream_output=output_context,
    )

And the UPath manager is this:

class XArrayNetcdfIoManager(UPathIOManager):
    """IOmanager that stores xarray as netcdf"""

    extension: str = ".nc"

    def dump_to_path(self, context: OutputContext, obj:xr.Dataset, path: UPath):  #: xr.Dataset,
        print(f"path is {path}")
        obj.to_netcdf(path, group=MYGROUP)

    def load_from_path(self, context: InputContext, path: UPath) -> xr.Dataset:
        print(f"path is {path}")
        ds = xr.open_dataset(path, group=MYGROUP)
        return ds

I set

base_path

equal to "iomanager_base_path" for the test. When I tests for

manager.handle_output

and

manager.load_input

, the print statements say the path for output is "iomanager_base_path/myasset_name.nc" and "iomanager_base_path/__EPHEMERAL_RUN_ID/mystep/abc.nc" respectfully. My questions are: • Why aren't the paths the same? • Why isn't the partition key represented?

you need to set the

asset_key

and

partition_key

on the InputContext too here

Thanks. So why don't they need to do that in the example under testing IOManagers? https://docs.dagster.io/concepts/io-management/io-managers#testing-an-io-manager

honestly our examples in this department are definitely lacking blob cry but that example is written op-centric, no partitions. We should fix that

Thanks! So, it looks like

build_output_context

can take either an

AssetKey

or a

str

for the

asset_key

argument.. However,

build_input_context

requires an

AssetKey

object explicitly? Is that true?

blob facepalm yea the asset key bit is an oversight. Just threw up a PR to fix it: https://github.com/dagster-io/dagster/pull/13974

Okay, thanks. From printouts, it seems that the has_asset_partitions flat is False, which seems to be checked before adding partition keys to the path (by my quick skimming of the code).

blob facepalm blob facepalm another oversight https://github.com/dagster-io/dagster/pull/13975

Okay, thanks.