12/14/2022, 9:07 PM
I would assume that having abstraction for data analysts/scientist is a good thing because each could end up using different methods saving data to snowflake or S3, etc for pickup by other tools causing a lot of inefficiencies or just bad implementations. My hope is that by having data eng provide these very common tasks (saving and getting data via io manager) that scientist/analyst have one less thing to worry about BUT it does require a change in mindset which some people will definitely not want (or can’t) do.