Hi all, I'm trying to get to grips with SDA's. I was wondering what's the best practice in structuring your assets? For example, I have three categories of assets - 1) raw assets, which are datasets of the raw data, 2) stg assets, which are validated datasets of the raw data passed thru panderas, 3) clean assets, which are the stg assets with cleaning and transformation. Should I have a separate module for each of the three categories? In doing so, I was getting an error that an AssetKey was defined multiple times.

you can use namespaces!

sorry for my ignorance, but how would I use namespaces?

you can configure namespaces when defining an asset. So you could use namespaces raw, stg and clean to distinguish assets across layers. so e.g. an asset

@assset(name="my_asset", namespace=["raw"])
def raw_asset():
   # do your thing

can be referenced with asset key

["raw", "my_asset"]

hope it is clearer and helps 😄

wow thanks, that looks really useful! I'll give it a go 😄