geoHeil
01/04/2023, 7:59 PM# old repository:
CMD ["dagster", "api", "grpc", "-h", "0.0.0.0", "-p", "4000", "-f", "FOO/repository.py", "--attribute", "myrepo"]
# ==> works fine
# new definitions:
CMD ["dagster", "api", "grpc", "-h", "0.0.0.0", "-p", "4000", "-m", "package_name"]
# ==> fails with: =>
DagsterInvariantViolationError: No repositories, jobs, pipelines, graphs, asset groups, or asset definitions found in "package_name"
CMD ["dagster", "api", "grpc", "-h", "0.0.0.0", "-p", "4000", "-m", "package_name.package_name"]
# ==> fails with: => No module named 'package_name/package_name'` while importing module package_name/package_name
Similarly to https://github.com/slopp/dagteam/blob/main/workspace.yaml:
my workspace.yaml looks like:
load_from:
- grpc_server:
host: package_name
port: 4000
location_name: "package_name"
inside docker. But outside docker (during local development):
load_from:
- python_package:
package_name: package_name.package_name
Outside docker a pip install -e .
has been executed - and the modules are resolvable. Inside docker, it would be convenient to not first volume-map/copy the file and then additionally pip install -e . it.
It would be uesful (I think also for speedier reloading) to be able to pass a file like before: "-f", "FOO/repository.py".
Is there any chance I can get this convenient behavior back when using definitions?
My package (scaffolded from dagster cli as package_name/package_name/assets.py and package_name/package_name/__init__.py) contains in the init:
from . import assets
defs = Definitions(assets=(load_assets_from_modules([assets])))
However, I do not find any good way to specify the init file in the -f
parameter. There, dagster also fails to resolve it due to missing modules.-f
parameter as a file. And for both, One MUST omit relative imports in the initial file (irrespective if it is the __init__
or repository
file. With this both variants are now working for meSean Lopp
01/04/2023, 8:54 PMdefinitions.py
you could use it similar to your repository.py
file and include in __init__.py
something like from .definitions import defs
I believe what you are seeing is expected differences in how python resolves relative imports in installed packages vs un-installed packagesgeoHeil
01/05/2023, 7:16 AM