Ok, I think this is a Snowflake question. 🙂 Let's say this is an asset that returns a DataFrame and that the asset is called

my_asset

. I can do this and create a DataFrame without a problem, which I return and from which Dagster creates a table in Snowflake.

@asset
def my_asset(context) -> pd.DataFrame:
    with context.resources.snowflake.get_connection() as conn:
            with closing(conn.cursor()) as cursor:
                q2 = "SELECT * FROM SANDBOX.MY_SCHEMA.MY_TABLE"
                cursor.execute(q2)
                df = cursor.fetch_pandas_all()
                return df

However, if the following step has

my_asset

as a dependency, like this...

@asset
def my_next_asset(context, my_asset) -> pd.DataFrame:
    stuff...

I get an error:

dagster._check.CheckError: SnowflakeIOManager does not have a handler for type 'typing.Any'. Has handlers for types '<class 'pandas.core.frame.DataFrame'>'. Please add <class 'pandas.core.frame.DataFrame'> type hints to your assets and ops.

Why is it that I can pull the data in to create a DataFrame but the SnowflakeIOManager cannot do it without hints?

So the snowflake io manager can be set up to handle different types at once. for example you could make a snowflake io manager that can handle both pandas and pyspark dataframes. In that case we require the typing hints to know which type to load the input as. However, in the case when the io manager is only set up to handle one type, we should default to using that type if there are no hints. I’ve been mentally tracking this issue, but haven’t had a chance to fix it yet. if you want to open a github issue with your code sample and assign it to me we can officially track it

ok, thanks!