Noah Ford
02/13/2023, 5:58 PM@asset(resource_keys={"bigquery_write_read")
def write_new_partition(context) -> None:
pandas_df = pull_recent_rows()
write_new_partition_to_bq(pandas_df, 'append')
@asset(io_manager_key="html_io_manager")
def viz_on_full_table_upload_to_s3(context) -> str:
df = read_full_df()
html_viz = do_viz(df)
return html_viz
The second function only needs the first function to run, it doesn't need anything from the first directlyrex
02/13/2023, 5:59 PMAlternatively, you can define dependencies where data from an upstream asset doesn’t need to be loaded by Dagster to compute a downstream asset’s output. When used,defines the dependency between assets but doesn’t pass data through Dagster.non_argument_deps
Noah Ford
02/13/2023, 6:01 PM