https://dagster.io/ logo
#dagster-support
Title
# dagster-support
e

Eric Larson

06/24/2022, 4:58 PM
Bug in 0.15.1 related to
Copy code
Experimental#
    If two assets are in the same group and the upstream asset has a multi-segment asset key, the downstream asset doesn't need to specify the full asset key when declaring its dependency on the upstream asset - just the last segment.
I am specifying the
key_prefix
for my multi segment asset key in the
@asset(ins=)
and it worked in 0.15.0 but 0.15.1 is not respecting it, and picking up other assets with the same base
asset_key
🤖 1
👀 1
s

sandy

06/24/2022, 5:30 PM
hey Eric - thanks for reporting. would it be possible for you to share code that reproduces the issue?
e

Eric Larson

06/24/2022, 5:49 PM
here is important parts
Copy code
@asset(
    compute_kind='SFTP',
    key_prefix='l2_commercial',
    group_name='l2',
)
def sftp_files():
    return ['files']

@asset(
    partitions_def=state_parts,
    key_prefix='l2_commercial',
    ins={
        "sftp_files": AssetIn(key=sftp_files.asset_key),
    },
    group_name='l2',
)
def sftp_filename(context, sftp_files: List[str]) -> str:
    return 'filename'


# In another Python module

@asset(
    compute_kind='SFTP',
    key_prefix='VM2Uniform',
    group_name='l2',
)
def sftp_files(context) -> List[str]:
    return ['files']


# in another module

sftp_assets = with_resources([
        vm2uniform.sftp_files,

    ],
    resource_defs = l2_resources,
)

vm2_assets = with_resources([
        vm2uniform.sftp_filename,

    ],
    resource_defs = l2_resources,
)

l2_com_assets = with_resources(
    definitions=[
        l2_commercial.sftp_filename,

    ],
    resource_defs = l2_resources,
)

l2_commercial_job = define_asset_job(
    name='Pipeline_Commercial', 
    selection=AssetSelection.assets(*l2_com_assets),
    partitions_def=state_parts,
    tags = {
        "ecs/cpu": "512",
        "ecs/memory": "2048",
    },
)


@repository
def helm_repository():
    return [
        l2_commercial_job,

    ] + sftp_assets + vm2_assets + l2_com_assets
I would expect if my asset declars
asset_in
then it uses that, otherwise it would use this new experimental convience that is in 0.15.1
s

sandy

06/24/2022, 6:21 PM
thanks for the example - I was able to reproduce this problem locally - currently investigating
here's a fix - going to see if it can make it into a patch release today: https://github.com/dagster-io/dagster/pull/8609
e

Eric Larson

06/24/2022, 7:17 PM
🎉
s

sandy

06/24/2022, 10:27 PM
0.15.2 should get released today, with this fix in it
❤️ 1
3 Views