Hi, I have the exact same question about partitions as this poster: https://stackoverflow.com/questions/75703813/reduce-a-partitioned-asset-to-a-single-data-frame . Can we get an answer to this question (either here or on the site). Thanks! 🙂

If you use your own IO manager you can then define whatever behaviour you want in

load_input()

for loading a partitioned upstream into a non-partitioned downstream. It has the benefit of flexibility, but you might have to reimplement stuff that is otherwise available in the built-in IO managers (the partitions as a dict thing is actually a feature of the built-in IO manager, not a default thing — you can create whatever loading logic you need in a custom IO manager).

thanks Andras

Yes, what Andras said is exactly right.