https://dagster.io/ logo
#ask-community
Title
# ask-community
j

Jake Kagan

03/14/2023, 4:26 PM
getting:
Copy code
ExperimentalWarning: "resource_defs" is an experimental argument to function "asset". It may break in futur
e versions, even between dot releases. To mute warnings for experimental functionality, invoke warnings.filterwarnings("ignore", category=dagster.ExperimentalWarning) or use one of the other methods described at <https://docs.python>.
org/3/library/warnings.html#describing-warning-filters.
i'm trying to give an io manager to my asset; kinda confused about this: the io manager uploads a parquet file into s3. so what other way would i be able to do this with an asset if not through a resource def like so:
Copy code
# ----------------------------------------------- VARS ----------------------------------------------- #
RESOURCE_DEFS = {"io_df_to_s3": common_io_df_to_s3,
                 }
# ----------------------------------------------- JOB ----------------------------------------------- #

@asset(resource_defs={**RESOURCE_DEFS}, io_manager_key="io_df_to_s3")
def beg_inv_excel_to_s3():
	# INTO S3:
	BUCKET = "my_bucket/"
	FILE_PATH = "my_path/"
	FILE_FORMAT = 'PARQUET'
	FILE_NAME = 'my_excel_file'
	FILE_NAME = f"{FILE_NAME}.{FILE_FORMAT.lower()}"
	EXCEL_FILE = r'L:my_excel_file.xlsx'
	EXCEL_SHEET = r'my_sheet'

	excel_data = pl.read_excel(file=EXCEL_FILE, sheet_name=EXCEL_SHEET)
	return {"file_name": FILE_NAME, "file_path": BUCKET + FILE_PATH, "polars_dataframe": excel_data}
y

yuhan

03/17/2023, 12:44 AM
the non-experimental way would be
Copy code
@asset(required_resource_keys={"api"})
def beg_inv_excel_to_s3():
and then thread it in using
with_resources
Copy code
with_resources(
    definitions=[beg_inv_excel_to_s3],
    resource_defs={"api": foo_resource},
)
or directly apply it on definition level:
Copy code
defs = Definitions(
    assets=[asset_requires_resource],
    resources={"foo": foo_resource},
)
but what you’re doing is the newest way to specify resources on assets, which will be unexperimentalized in our next release (v1.3), so i’d actually recommend sticking with what you have so far. you can find detailed docs here: https://docs.dagster.io/guides/dagster/pythonic-resources