https://dagster.io/ logo
#ask-community
Title
# ask-community
c

Chaitya

02/20/2023, 7:30 PM
Hi, apologies if this question is trivial, but I can't seem to find a clear answer in the docs. Are dataframe outputs supported by graph-backed assets?
Copy code
dagster._core.errors.DagsterInvalidDefinitionError: @graph 'graph' returned problematic dict entry under key my_df of type <class 'pandas.core.frame.DataFrame'>. Dict values must be outputs of invoked solids
To clarify a bit, my graph is composed of two ops. One is a non-asset-producing op (which writes files to disk somewhere). The other is an asset producing op, which loads some files from disk and returns two dataframes.
They must run sequentially, so my second op has a Nothing dependency as described here
I guess my error was related to something else in my graph definition. Now I'm hitting:
Copy code
dagster._check.CheckError: Invariant failed. Description: All leaf nodes within graph 'graph' must generate outputs which are mapped to outputs of the graph, and produce assets. The following leaf node(s) are non-asset producing ops: {'parquet_writer'}. This behavior is not currently supported because these ops are not required for the creation of the associated asset(s).
This statement seems to conflict with the information on the Graphs page that explicitly calls out how to run non-asset-producing ops within a graph...
c

claire

02/21/2023, 10:52 PM
Hi Chaitya, this error might be happening because you don't have a return statement in your graph? Materializing the asset in the code snippet below works for me:
Copy code
@op
def my_op():
    return 1


@op(ins={"input_1": In(Nothing)})
def my_op_2() -> pd.DataFrame:
    return pd.DataFrame([])


@graph
def my_graph():
    return my_op_2(input_1=my_op())


defs = Definitions([AssetsDefinition.from_graph(my_graph)])
3 Views