How can I target a specific repository when the workspace contains all prod, staging and local resources? I.e. in particular how can I have a prod/staging/local instance of dagster which only tries to load the prod resources (and does not kick off sensors for staging and local) .

Hey @geoHeil have you tried using the

attribute

keyword (link to docs)? I believe if you updated your

workspace.yaml

to

load_from:
  - python_package: 
      package_name: hacker_news_assets
      attribute: staging

it you just load the staging repo

Errors while loading workspace config from /Users/geoheil/development/cybersio/poc/CYBERSIO/workspace_staging.yaml. Error 1: Received unexpected config entry "attribute" at path rootload from[0]grpc_server. Expected: "{ host?: (String | { env: String }) location_name?: (String | { env: String }) port?: (Int | { env: String }) socket?: (String | { env: String }) ssl?: Bool }".

@geoHeil attribute doesn't work as an argument to grpc_server for the same reason that python_file and python_package doesn't - it expects you to have separately spun up a grpc server and passed in those arguments to

dagster api grpc

I thought in the most naive example

dagit

internally a) reads the workspace.yaml file and b) spins up an

grpc

server (process local). So far, I do not understand why it cannot use this flag when provided in the configuration or phrased differently: why this flag only works when applied to the CLI when manually launching a grpc server.

yeah there are two cases. In one, dagit is responsible for spinning up the server as a local subprocess. In another, you're responsible for spinning up the server and dagit just connects to it. Extending the attribute flag to also work when you're providing a grpc server is a reasonable feature request - it's just different than how its currently implemented, which is as an argument to the grpc server startup command. So when dagit isn't responsible for running the grpc server startup command, there's no way currently to apply the argument.

When I have the GRPC server in docker (and supply the the workspace.yaml file as an argument) it would be really nice/useful if it would directly take the attribute parameter from the configuration. Also in the naive case where it is spun up directly - this would be useful. Do you think you can make a ticket to track this? Furthermore: Given that this is currently not implemented what would you suggest to switch the repository (wich each points to one environment for the pipeline).

I'm not totally following what you mean by "Also in the naive case where it is spun up directly"

Yeah, attribute works there

Understood. With this I can find a solution wich should be working well enough for me now.

What I would love to have long-term is the ability to have a workspace.yaml that tells dagster to spin up the gRPC servers in docker containers for you instead of subprocesses, so that you don't have to choose between the naive way and the grpc_server way

indeed - this sounds nice.