If I wanted to pass two config params to all asset...
# ask-community
c
If I wanted to pass two config params to all assets in a job, called
start_date
and
end_date
, how would I configure that as the
config
param for the job without specifying it for each asset? Also, how would I specify it in a job so that the end user was required to enter the values in Launchpad before running the job?
Copy code
my_job = define_asset_job(
    name="Adhoc_Stuff_Updating_Job",
    selection=AssetSelection.groups(MY_ASSETS),
    config= # ? What goes here?,
)
o
hi @clay! the direct answer to that question is that you could specify that each asset has a required_resource_key of "my_values", then supply the make_values_resource as the implementation of that resource in your Definitions object, i.e.:
Copy code
defs = Definitions(..., resources={"my_values": make_values_resource(start_date=str, end_date=str))
then, any job that includes assets with that required resource key will require the user to supply config values for start_date and end_date before launching the job
the less direct answer is that you might want to consider using partitions for this use case. you can declare assets as time-partitioned, e.g.
Copy code
@asset(partitions_def=DailyPartitionsDefinition(start_date="2023-01-01"))
c
Thanks, Owen. There are some cases where partitions work but I also have need to support more ad-hoc (strange partitioning) jobs. I think your first suggestion would be good! I was wondering whether it could be specified in the job so that I could create multiple jobs that provided different time range via partitions. Monthly, Quarterly, Calendar Year, Fiscal Year, etc. without needing to change the assets for each -- just the job definition. I'm still sorting through how this all work. 🙂
o
would all of those assets be writing to the same location? or would they just be doing similar things with different aggregation windows?
if the second, then i'd recommend an asset factory pattern, where you can make multiple copies of a similar asset with different partitions_defs (they'll have distinct keys)
👍 1
177 Views