https://dagster.io/ logo
#ask-community
Title
# ask-community
c

clay

02/15/2023, 3:41 PM
I have a custom DataFrame type, like:
MyDataFrame = create_dagster_pandas_dataframe_type(...)
And an asset that produces that type, for storage in Snowflake. This all works as expected. The asset signature is like:
Copy code
@asset
def my_asset(context: OpExecutionContext) -> MyDataFrame:
    ...
    return df
Validation passes, etc. Downstream, I have an asset that needs to understand the types specified for columns in
MyDataFrame
. I tried to do this for the downstream asset's signature:
Copy code
@asset
def my_downstream_asset(context: OpExecutionContext, my_asset: MyDataFrame) -> None:
    ...
    # some stuff happens
    ...
    return None
However, I cannot materialize
my_downstream_asset
because of the error:
Copy code
dagster._check.CheckError: SnowflakeIOManager does not have a handler for type 'typing.Any'. Has handlers for types '<class 'pandas.core.frame.DataFrame'>'. Please add <class 'pandas.core.frame.DataFrame'> type hints to your assets and ops.
I had this same error yesterday when I did not specify a type at all for
my_asset
in the signature of
my_downstream_asset
. How can I have
my_downstream_asset
load
my_asset
and apply the column types specified (and validated) in
MyDataFrame
?
o

owen

02/15/2023, 7:50 PM
hi @clay! this is an interesting case -- I think the issue is coming from the fact that when you use
create_dagster_pandas_dataframe_type
, it erases the information that the SnowflakeIOManager uses to determine what specific type of input object you want your table loaded as. I think this should be an easy enough fix on our end, but until that fix goes out I'd recommend just leaving the annotation off the input. If you still want to manually validate things, you can do
MyDataFrame.type_check(None, my_asset)
within the body of the asset
❤️ 1
actually just merged in the fix for this behavior, it'll get into tomorrow's release 🙂
c

clay

02/15/2023, 8:10 PM
Awesome, thanks @owen!
5 Views