I've been struggling with this for a few days... I...
# ask-community
I've been struggling with this for a few days... I'm trying to use the IOManager for outputting pandas dataframes as parquet files - as shown in the screenshot. I'm loading the dataframe
in successfully (by passing the
into the logger) but when I run this `context.resources.io_manager.handle_output(context, obj=df)`I get the error
AttributeError: 'NoneType' object has no attribute 'to_parquet'
. Which suggests that when I pass the dataframe into the io_manager, it becomes a NoneType. How does this work?
🤖 1
The weird thing is, although it's throwing out the error, it actually does output the parquet 🙄
hi @Barry Sun -- in general, you shouldn't be manually calling
(this will get automatically called by framework code to store the outputs of your ops). so if you have an op defined as
Copy code
def do_some_output(context):
    # ...
    return some_pandas_dataframe
then the io manager will automatically be invoked on some_pandas_dataframe. So I think what's happening in your case is that you're manually calling this handle_output function inside the body of your op (which will succeed), and then not returning a value from your op, so then when dagster invokes the handle_output function automatically, it will be invoked on the return value of your op (which is None).
Hi Owen, thanks a lot! That makes a lot of sense 🙂 I'm not sure where I got that code from... must be somewhere obscure in the docs 😅
no problem! 🙂