https://dagster.io/ logo
#dagster-support
Title
# dagster-support
m

Mike Jones

08/10/2022, 6:28 PM
Hi, I'm getting an
DagsterInvalidDefinitionError
when I run code samples from the documentation. Details in thread.
Copy code
dagster._core.errors.DagsterInvalidDefinitionError: Asset key AssetKey(['downstream_asset']) is defined multiple times. Definitions found in modules: test.
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_grpc\server.py", line 227, in __init__
    self._loaded_repositories = LoadedRepositories(
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_grpc\server.py", line 101, in __init__
    loadable_targets = get_loadable_targets(
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_grpc\utils.py", line 33, in get_loadable_targets
    else loadable_targets_from_python_file(python_file, working_directory)
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_core\workspace\autodiscovery.py", line 27, in loadable_targets_from_python_file
    return loadable_targets_from_loaded_module(loaded_module)
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_core\workspace\autodiscovery.py", line 113, in loadable_targets_from_loaded_module
    asset_group_from_module_assets = AssetGroup.from_modules([module])
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_core\definitions\asset_group.py", line 336, in from_modules
    assets, source_assets = assets_and_source_assets_from_modules(modules, extra_source_assets)
  File "C:\Users\mike.jones\Projects\dagster\venv\lib\site-packages\dagster\_core\definitions\load_assets_from_modules.py", line 61, in assets_and_source_assets_from_modules
    raise DagsterInvalidDefinitionError(
Copy code
dagit==1.0.2
dagster==1.0.2
dagster-aws==0.16.2
dagster-gcp==0.16.2
dagster-graphql==1.0.2
dagster-pandas==0.16.2
a

Aaron

08/10/2022, 6:34 PM
What happens if you try renaming the file to something else? (Looks like there may be some conflict based on the name of the file)
m

Mike Jones

08/10/2022, 6:36 PM
I just renamed test.py to different_name.py and I get the same error
a

Aaron

08/10/2022, 6:45 PM
Yep, sorry - my bad. I get the same error if I try and run the code snippet - looks it it should work so I’m at a loss. 😞
c

claire

08/10/2022, 6:46 PM
Hi Mike. This is a nuance with
with_resources
--this function will actually return your assets, but bound to the resources provided. This is the cause of the error, because the file now contains both the original assets and the assets bound to resources. One thing you can do here is put your
with_resources
call within a repository:
Copy code
@repository
def my_repository():
    return [
        with_resources(
            [upstream_asset, downstream_asset],
            resource_defs={"io_manager": s3_pickle_io_manager, "s3": s3_resource},
        )
    ]
Or just bind the resources directly to your asset:
Copy code
@asset(resource_defs={...})
def my_asset():
     ...
👍 1
For now, I can file an issue to clarify this error message.
@Dagster Bot clarify "asset defined multiple times" error
d

Dagster Bot

08/10/2022, 6:48 PM
Invalid command. Did you mean to create an issue or a discussion? Try
@dagster_bot <issue|docs|discussion> <title>
c

claire

08/10/2022, 6:48 PM
@Dagster Bot issue clarify "asset defined multiple times" error
d

Dagster Bot

08/10/2022, 6:48 PM
m

Mike Jones

08/10/2022, 6:52 PM
Makes sense. Thank you
f

Frank Lu

09/15/2022, 10:09 PM
this is part of the sample code on the io page, maybe we should update the documentation? https://docs.dagster.io/concepts/io-management/io-managers#applying-io-managers-to-assets also the solutions provided aren’t really elegant 😞
c

claire

09/15/2022, 10:13 PM
Thanks @Frank Lu for pointing this out, I've updated the issue to point out the IO manager page. Which solutions do you mean are inelegant?
f

Frank Lu

09/16/2022, 7:23 PM
i’m referring to stuffing the resource_defs in the decorators. If i have 20 assets it’ll be a lot of repeated code. I can also put that in repository.py with your other suggestion but it’s not very intuitive because we have all assets under an “asset” folder in our projects, but we’ll probably go with the second option! thanks! Ideally if we can do the docs code it’ll be really nice.
c

claire

09/16/2022, 9:16 PM
Makes sense, thanks for the feedback
s

sandy

09/16/2022, 9:26 PM
not sure if this is helpful, but you could also consider using
with_resources
in a module at the top level of your asset folder and then importing the assets with resources into your repository?
f

Frank Lu

09/16/2022, 9:26 PM
Ooooh let me try that!
21 Views