https://dagster.io/ logo
Title
m

Marcel M

01/20/2022, 11:53 AM
Does any have experience with running Dagster in Digitalocean’s app platform? It would possible since Dagster is just a python app. dagster-daemon can run as a worker. A possible problem is that there is no way of password protecting dagit?
j

jordan

01/20/2022, 3:14 PM
Everything you say so far seems accurate to me - I think one thing you’ll need to figure out is how to handle launching runs. You could continue to use the
DefaultRunLauncher
to launch each run in process. But if you want some level of isolation between runs, you’d need to write a custom run launcher to interact with the App Platform API (maybe launching each new run as a job? just based at glancing at the docs - I’m not super familiar with App Platform’s concepts) Password protecting dagit isn’t unique to deploying to Digital Ocean - it’ll be true of wherever you deploy it. Some possible solutions include running in a network only accessible via VPN or putting Dagit behind an auth proxy.
m

Marcel M

01/20/2022, 5:04 PM
Wouldn't Dagster daemon handle launching runs (on a schedule)? Yes, I'd like to avoid additional moving parts like an auth proxy. Pity Dagster doesn't offer this out of the box.
j

jordan

01/20/2022, 5:07 PM
It’ll launch the runs, but always in process. Whereas many of Dagster’s other deployment models have their own run custom run launchers to allow you to isolate runs from each other (and from the Daemon) which aids in concurrency, resource contention, etc. For example, K8s can launch each run in its own pod, ECS can launch each run in its own task, Docker can launch each run in its own container.
m

Marcel M

01/20/2022, 5:57 PM
True, DO app platform is more for a single python app (with some background processes). So it might be an option for simple deployments, but then would be very easy.