https://dagster.io/ logo
Join Slack
Powered by
Hi, I'm getting an `DagsterInvalidDefinitionError`...
# ask-community
m
Hi, I'm getting an 
DagsterInvalidDefinitionError
when I run code samples from the documentation. Details in thread.
Copy code
dagster._core.errors.DagsterInvalidDefinitionError: Asset key AssetKey(['downstream_asset']) is defined multiple times. Definitions found in modules: test.
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_grpc\server.py", line 227, in __init__
    self._loaded_repositories = LoadedRepositories(
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_grpc\server.py", line 101, in __init__
    loadable_targets = get_loadable_targets(
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_grpc\utils.py", line 33, in get_loadable_targets
    else loadable_targets_from_python_file(python_file, working_directory)
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_core\workspace\autodiscovery.py", line 27, in loadable_targets_from_python_file
    return loadable_targets_from_loaded_module(loaded_module)
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_core\workspace\autodiscovery.py", line 113, in loadable_targets_from_loaded_module
    asset_group_from_module_assets = AssetGroup.from_modules([module])
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_core\definitions\asset_group.py", line 336, in from_modules
    assets, source_assets = assets_and_source_assets_from_modules(modules, extra_source_assets)
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_core\definitions\load_assets_from_modules.py", line 61, in assets_and_source_assets_from_modules
    raise DagsterInvalidDefinitionError(
test.py
Copy code
dagit==1.0.2
dagster==1.0.2
dagster-aws==0.16.2
dagster-gcp==0.16.2
dagster-graphql==1.0.2
dagster-pandas==0.16.2
a
What happens if you try renaming the file to something else? (Looks like there may be some conflict based on the name of the file)
m
I just renamed test.py to different_name.py and I get the same error
a
Yep, sorry - my bad. I get the same error if I try and run the code snippet - looks it it should work so I’m at a loss. 😞
c
Hi Mike. This is a nuance with 
with_resources
--this function will actually return your assets, but bound to the resources provided. This is the cause of the error, because the file now contains both the original assets and the assets bound to resources. One thing you can do here is put your 
with_resources
call within a repository:
Copy code
@repository
def my_repository():
    return [
        with_resources(
            [upstream_asset, downstream_asset],
            resource_defs={"io_manager": s3_pickle_io_manager, "s3": s3_resource},
        )
    ]
Or just bind the resources directly to your asset:
Copy code
@asset(resource_defs={...})
def my_asset():
     ...
👍 1
For now, I can file an issue to clarify this error message.
@Dagster Bot clarify "asset defined multiple times" error
d
Invalid command. Did you mean to create an issue or a discussion? Try 
@dagster_bot <issue|docs|discussion> <title>
c
@Dagster Bot issue clarify "asset defined multiple times" error
d
m
Makes sense. Thank you
f
this is part of the sample code on the io page, maybe we should update the documentation? https://docs.dagster.io/concepts/io-management/io-managers#applying-io-managers-to-assets also the solutions provided aren’t really elegant 😞
c
Thanks @Frank Lu for pointing this out, I've updated the issue to point out the IO manager page. Which solutions do you mean are inelegant?
f
i’m referring to stuffing the resource_defs in the decorators. If i have 20 assets it’ll be a lot of repeated code. I can also put that in repository.py with your other suggestion but it’s not very intuitive because we have all assets under an “asset” folder in our projects, but we’ll probably go with the second option! thanks! Ideally if we can do the docs code it’ll be really nice.
c
Makes sense, thanks for the feedback
s
not sure if this is helpful, but you could also consider using 
with_resources
in a module at the top level of your asset folder and then importing the assets with resources into your repository?
f
Ooooh let me try that!
118 Views
Open in Slack
PreviousNext
Powered by