Charles Lariviere
02/08/2021, 9:09 PMNone
or an empty list?
I have an extract
solid that pulls data from a REST API as a list of dicts, which I take as input in a to_df
solid to format as a dataframe. The API is not guaranteed to return results for a given partition, which then causes issues downstream with my IO manager and Dagster’s type validation. I could address that in the to_df
solid, but curious if there was a better pattern to use — some kind of conditional execution logic in the pipeline definition?OutputDefinition(Type, "name", is_required=False)
sandy
02/08/2021, 9:50 PM