The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hey! Ive run into a somewhat unique issue.  I have a `@graph_multi_asset` that accepts a set of Asset Keys as ins.  There are some cases where two of the ins are the same  Asset/AssetKey.  In this case, the <https://github.com/dagster-io/dagster/blob/1a853f994f681bad4ab974ed36aade7d30c7111c/python_modules/dagster/dagster/_core/definitions/decorators/asset_decorator.py#L594|build_asset_ins> method will eliminate one of the two keys in the mapping.  Thus causing the dagster invariant checker to not see one of the inputs.

Any help or workaround is much appreciated!

Maybe I'm missing something, I thought asset keys are unique within the system. If you have multiple entries pointing to the same asset key wouldn't you just use the same reference?

but since it’s a key in the dictionary the method I linked returns, only one reference goes through

actually i think i can do a copy of the AssetKey to ensure its unique

Edit: did not fix it

after further looking into it, <https://github.com/dagster-io/dagster/blob/1a853f994f681bad4ab974ed36aade7d30c7111c/python_modules/dagster/dagster/_core/definitions/decorators/asset_decorator.py#L636-L649|its this loop> and <https://github.com/dagster-io/dagster/blob/1a853f994f681bad4ab974ed36aade7d30c7111c/python_modules/dagster/dagster/_core/definitions/decorators/asset_decorator.py#L653-L655|more specifically these lines assigning the dictionary key>. Which makes me believe a deepcopy would work, but it is a NamedTuple and python but be doing some magic there. Yeah even after a deep copy, they are equal

Hi Jordan, could you elaborate on what you’re aiming for with multiple asset copies of the same asset key in the inputs-- this is outside of Dagster’s model, where an asset has a _set_ of dependencies on other assets (<@U053DKZ39MG> is correct), which is why the keys are de-duped.

So I’d like to keep a singular asset copy as the input for multiple keys. It is sometimes the case that two inputs to the graph asset will be the same file based on the functionality. So letting two of the inputs be the same graph asset is the main goal. It isn’t always the case, but it sometimes will be. 

Would you mind posting some simple example code so I can make sure we’re on the same page?

Untitled.py

broadly where the issue comes up, however we are using a graph asset factory, so the asset inputs are variable, and sometimes the same.

Hmmm-- I am pretty sure that the assumption of a one-to-one relationship between input names and asset keys is baked in at a fairly deep level.

It seems to me that if you are using a factory, you shouldn’t need to define multiple inputs with the same key-- can you dedup at the top of the factory to determine the number of unique inputs and construct the graph asset accordingly?