Hi. I want to test Dagster as our workflow orchestrator. We have general experience with Airflow, Prefect, and Flyte. One concept I’m having trouble internalizing is how to “deploy” code into Dagster. I’m reading up on the concepts of Workspaces and Definitions which seem to be different concepts. My use case is as envisioned: I have Dagster deployed in our ECS compute environment. We have a number of users who want to run jobs/ops in ECS. Ideally, they would have their code versioned in a GitHub repo. What is the canonical way to deploy the code to Dagster? If some code has complicated dependencies (e.g. requiring specific libraries or a different Python version than what Dagster is running), can we specify an image/container? TIA! 🙏

Are you using Dagster Cloud or the OSS version? The OSS version you specify a

workspace.yaml

with locations of your code. You probably want multiple gRPC servers with your code in it since you have multiple code locations. Check out this as well For cloud, all it needs is to be able to pull a Docker image from a repository. You can have multiple code-locations to a deployment.

OSS version (assuming that is the only way I can install on my ECS infrastructure).

Well, the other way would be to use Hybrid cloud. Basically Dagster manages your code versions, auth, full/branch deployments and other things. All you have to do is run a single agent in ECS and point Dagster at your code locations. The agent pulls those containers from ECR and starts new tasks using the code in them.

Thanks. From those docs, it seems that if I want to deploy new code, I would have to build a new image with the code and deploy it as a gRPC server.

Yep, then change the

workspace.yaml

to reflect that. BTW: If you want to hot-reload the workspace without restarting Dagit there’s a GraphQL mutation you can use.

No, the info is very much appreciated and the docs can be a little confusing.