Quy
06/14/2023, 7:42 PM@op(required_resource_keys={"analytic_db"},)
def query(context) -> pd.DataFrame:
engine = create_engine(context.resources.analytic_db)
query = """
select * from foo;
"""
df = pd.read_sql_query(query, con=engine.connect())
context.log.debug(df.shape)
return df
@graph
def main():
result = query()
print(result.shape)
context.log.debug
can show my data frame shape, but outside said AttributeError: 'InvokedNodeOutputHandle' object has no attribute 'shape'
Not sure what I missed? I used all default configsowen
06/14/2023, 10:25 PM@graph
decorated function is purely for defining dependencies between ops (or other graphs). so when the code inside main
is executed, it's not actually executing the ops themselves, it's just telling dagster which outputs should be fed into which inputs.owen
06/14/2023, 10:26 PMresult
and then print out result.shape