Hello! One of my Definitions has almost a hundred assets and I’m expecting to add more. For organization’s sake I thought it would make sense to have multiple Definitions where each Definition is a collection of assets that are at different stages of our full data pipeline. These Definitions would be connected using SourceAssets. Is there an easy way to convert a collection of AssetDefinitions to SourceAssets? My solution is to gather the assets from the upstream Definition and create SourceAssets from the Assets’ key and io_manager. Is there a more graceful way to do this? Is this a reasonable way to structure large dagster projects?

It’s hard to know whether that’s a reasonable organization strategy without knowing the use case, but you can also make use of asset groups to help organize things - since having multiple definitions will require additional operational burden (as you’ve identified with the need to have source assets bridging the gap)

Thanks! Maybe I’ll keep all of the assets in one definition for now. If think get unruly I’ll explore creating a second Definition with SourceAssets. A utility function that creates source assets from asset definitions sounds nice!

you can call

to_source_assets()

on an asset to get a list of `SourceAsset`s

^

to_source_assets

isn’t public tho right?

ah sorry missed what you said @chris. yeah, we should probably make it public, but you can still use it and I think it's unlikely that it will change

Great thanks y’all! I’ll create an issue. Is

to_source_assets

used for this situation where you want to link assets from different Definitions?

I think that's a very legitimate way to use it

Sounds good. Is it common to have > 100 assets in a definition?

definitely

Oh great. thanks for the help!