Barry Sun
06/20/2022, 5:21 AMload_assets_from_modules
is not working for me - I want to be able to load all the assets from a module (e.g. raw_assets) and apply the 'group_name'. I seem to be able to run this function, but how do I input it into with_resources
? I get the error AttributeError: 'list' object has no attribute 'with_resources'
.
2. Be able to build job from assets. I've tried using define_asset_job
but I just can't find clear documentation on the parameters inside, and how it fits into the final repository definition. Looking at the API docs, I'm not sure how to input my multi-key assets.
I'm not sure if it's me but I'm having a hard time finding enough documentation around the new functions 😕geoHeil
06/20/2022, 6:02 AMsandy
06/20/2022, 4:17 PMdefine_asset_job
with `with_resources`:
from dagster import with_resources, define_asset_job, load_assets_from_modules, repository
assets = load_assets_from_modules(..., group_name=...)
my_asset_job = define_asset_job("my_job", selection=...)
@repository
def prod():
assets_with_prod_resources = with_resources(assets, prod_resources)
return [*assets_with_prod_resources, my_asset_job]
An advantage of this approach is that, if you want to use different resources on the same assets in production / staging / dev, you don't need to load your assets and define your job separately for each environment.sandy
06/20/2022, 4:18 PMBe able to build job from assets. I've tried usingIt looks like we missed includingbut I just can't find clear documentation on the parameters inside, and how it fits into the final repository definition.define_asset_job
AssetSelection
in our apidocs. I'll get that fixed.
Looking at the API docs, I'm not sure how to input my multi-key assets.Is the there a way you were doing this with the pre-0.15.0 APIs?
Barry Sun
06/20/2022, 11:29 PMsandy
06/20/2022, 11:50 PMRe the multi-key assets, I was using namespaces before and just putting the assets together in a job.if you were doing this before:
build_assets_job(name=..., assets=[asset1, ...])
you can do this now:
define_asset_job(name=..., selection=AssetSelection.assets(asset1, ...))
would that work for you?Barry Sun
06/21/2022, 3:41 AM