https://dagster.io/ logo
#announcements
Title
# announcements
p

paul.q

11/24/2020, 10:11 AM
We're trying to load our repositories from a python package.  Our package structure is like so:
Copy code
my_package:
  workspace.yaml
  subfolder:
     repository.py
The
workspace.yaml
file under
my_package
looks like this:
Copy code
load from:
    - relative_path: subfolder/repository.py
We start dagit from a folder with another
workspace.yaml
in it that contains:
Copy code
load_from:
  - python_package: my_package
We get this error:
Copy code
dagster.core.errors.DagsterUserCodeProcessError: (DagsterInvariantViolationError) - dagster.core.errors.DagsterInvariantViolationError: No pipelines or repositories found in "my_package".
I've changed some names and paths to simplify this.  Also, this loads properly when we do it from our local environments where
my_package
is not packaged (i.e. we set our
PYTHONPATH
to the
my_package
parent folder and our
workspace.yaml
in the folder where dagit starts points at relative path
my_package/subfolder/repository.py
So, is this meant to work?  We'd like to be able to have a
workspace.yaml
file in our deployment target that has pointers to packages as they're developed and these packages will have repositories inside them.  In that way we can allow different teams to develop their own dagster packages and deploy them to a central dagit server.
a

alex

11/24/2020, 3:40 PM
are the
@respository
objects in the top level
__init__.py
of
my_package
? We don’t crawl the whole package, just the entry point. For example, i am guessing if you did
python_pacakge: my_package.subfolder.repository
it would work
p

paul.q

11/24/2020, 9:28 PM
I thought I'd already moved the module with the
@repository
object up to the top level of the package (same folder as
__init__.py
without luck. Tried your idea and still no good, BUT, I must not have tested the first idea properly. Package structure is now:
Copy code
my_package:
  workspace.yaml
  repository.yaml
  subfolder:
     ...
The workspace.yaml file looks like:
Copy code
load_from:
  - python_file:
      relative_path: repository.py
And everything works! Thanks
a

alex

11/24/2020, 9:31 PM
just to be clear, i meant that the @repository object needs to be imported in the
__init__.py
file itself
❤️ 1
p

paul.q

11/24/2020, 9:33 PM
OK, I'll give that a try as well @alex
9 Views