https://dagster.io/ logo
Join Slack
Powered by
I have an asset defined as such - ```# __init__.py...
# ask-community
a
I have an asset defined as such -
Copy code
# __init__.py

from ..assets import asset_1

defs = Definitions(
    assets=[
        *load_assets_from_modules([asset_1], group_name='asset_1_assets')
    ]
)
I load dagit, and I can see defined in my "Asset Groups" - 
asset_1_assets
. Now, I reference an asset in a different module, and add the job to my 
__init__.py
-
Copy code
##### job_1.py
from ..assets.asset_1 import asset_1

@job
def job_1():
  op_1(asset_1.to_source_asset())

##### __init__.py
from ..assets import asset_1

defs = Definitions(
    assets=[
        *load_assets_from_modules([asset_1], group_name='asset_1_assets')
    ],
    jobs=BindResourcesToJobs([job_1]),
    schedules=[ScheduledDefinition(...)]
)
reload dagit and now I can no longer see the "Asset Group" 
asset_1_assets
, it is now listed under the 
default
"Asset Group." -- What is the reason behind this?
y
Hi! Let me try repro!
๐Ÿ‘ 1
re:
Copy code
from ..assets import asset_1
whatโ€™s asset_1? is it a python module or package?
and does 
..assets
have an 
__init__
file?
a
a module within the same package
y
mind sharing the file tree? this seems have something to do with how python handles module/package imports
Copy code
from ..assets.asset_1 import asset_1
this line might be the thing confuses python. is your asset and the file the same name?
a
No the name is slightly different actually it's more like -
Copy code
from ..assets.asset_1 import asset_file_1
working on the tree structure 1 sec
Copy code
data_pipeline/
โ”œโ”€ assets/
โ”‚  โ”œโ”€ asset_1_name/
โ”‚  โ”‚  โ”œโ”€ asset_1.py
|  |  โ”‚  โ”œโ”€ def asset_1_job()
โ”‚  โ”‚  โ”œโ”€ __init__.py
โ”‚  โ”œโ”€ __init__.py
โ”œโ”€ jobs/
โ”‚  โ”œโ”€ foo_jobs/
โ”‚  โ”‚  โ”œโ”€ foo_job_defs.py
|  |  โ”‚  โ”œโ”€ def job_1()
โ”‚  โ”œโ”€ index.html
โ”‚  โ”œโ”€ robots.txt
โ”œโ”€ __init__.py
It is probably the way python handles packages... might be my lack of knowledge on that front
probably the order in which the module is loaded might also affect it
y
this is how my tree looks like and i got the same init.py as yours. it loads fine for me
so i may suggest moving stuff one by one and reload dagit every time you move so to make sure python package loading doesnt get in the way ๐Ÿ˜…
a
ok, thanks. Will take a look at that. Technically my jobs.py file is under a jobs folder at the same level of asset_group_module
Good morning @yuhan, I did some testing, and am able to reproduce. The asset group disappears when I BindResourcesToJobs - If you pull the repo and run it, with the job commented out, it will show the asset group, but once the job is bound it seems to push the asset into the default assets - https://github.com/adjit/dagster_testbed/blob/master/my_module/__init__.py#L14-L16
y
oohh so if you are already on v1.3+, you can remove 
BindResourcesToJobs
as in:
Copy code
defs = Definitions(
    assets=[
        *load_assets_from_modules([asset_1_file], group_name='test_group_name')
    ],
    jobs=[my_job.my_job]
)
it was for 1.2-1.3 where the new config and resources were still experimental so we made that opt-in.
in my repro, i didnโ€™t use 
BindResourcesToJobs
and it worked fine for me.
a
This still happens in my test repo. I removed the BindResourcesToJobs, but the behavior is still the same
image.png
v1.3.2
Hi @yuhan, hate to keep bugging you about this - have you checked out the above? removing the 
BindResourcesToJobs
didn't have any affect
y
ok i was finally able to repro! the 
jobs=[โ€ฆ]
on 
Definitions
seems doing something off
as a stopgap, while im looking into this issue, you can apply 
group_name
on the asset itself, here:
Copy code
@asset(group_name="test_group_name")
def asset_1(context):
    return 123
a
Cool thanks! Do you need me to open a git issue?
y
Yes please that would be great for tracking. Thanks!
a
y
thank you thank you
5 Views
Open in Slack
PreviousNext
Powered by