Hello, I am having an issue with my `IOManager` wi...
# ask-community
j
Hello, I am having an issue with my
IOManager
with
MultiPartitionsDefinition
I think was caused changes in the latest release. Details in 🧵
I am using
context.asset_partition_key
I see that
context.asset_partitions_def
is an instance of
MultiPartitionsDefinition
, however,
context.asset_partition_key
is a string of format
key_1|key_2
and not
MultiPartitionKey
. This was previously working, but after I upgraded from 1.10 to 1.14, it does this behavior now.
j
cc @claire do you know if the key representation for multi partitions changed recently?
j
I ran the below code, which works:
Copy code
from dagster import StaticPartitionsDefinition, ResourceDefinition, asset, AssetIn, materialize, io_manager, IOManager, MultiPartitionKey, MultiPartitionsDefinition

dummy_multi_partition = MultiPartitionsDefinition(
    {
        "part_a": StaticPartitionsDefinition(["abc", "def"]),
        "part_b": StaticPartitionsDefinition(["123", "456"]),
    }
)

@io_manager()
def mock_delta_io_manager(_):
    class TestIOManager(IOManager):
        def handle_output(self, context, obj):
            print(f"TYPE: {type(context.asset_partition_key)}")
            print(context.asset_partition_key)
            assert isinstance(context.asset_partition_key, MultiPartitionKey)

        def load_input(self, context):
            pass

    return TestIOManager()


@asset(
    name="dummy_asset_upstream",
    metadata={"dagster_partition_cols": ["foo", "date"], "additional_partition_cols": ["test_col"]},
    io_manager_def=mock_delta_io_manager,
    partitions_def=dummy_multi_partition
)
def dummy_asset_upstream(context):
    return 1234

materialize(
    [dummy_asset_upstream], partition_key=MultiPartitionKey({"part_a": "abc", "part_b": "123"})
)
However, when I ran my actual asset through dagit, I encountered the above issue.
I noticed now in the run configuration, I see tags
partition: abc|123
in addition to
partition/part_a: abc
and
partition/part_b: 123
. In 1.10, I only see
partition/part_a: abc
and
partition/part_b: 123
.
c
Hi Jeff, thanks for reporting this. Yes, this is a recent regression caused in the latest release. It'll be fixed in next week's release
ty spinny 1
m
@Audrey Kervella