Mitchell Hynes12/01/2022, 7:24 PM
Stephen Bailey12/01/2022, 8:39 PM
file in a git repo and manually run them when i need to update the agent. 4. I would recommend against auto-scaling at first. This will prevent surprise errors where jobs are evicted due to scaling up/down the cluster, especially as you start 5. Would recommend connecting into some observability tooling like Datadog to monitor the cluster health. After writing this, I wonder if ECS wis your better bet 😂
Zach12/01/2022, 10:42 PM
Mitchell Hynes12/01/2022, 11:43 PM
Dagster Cloud serverless for a bit?I would love to use it but with the sensitivity of our data and compliance reasons all of our data must stay in Canada + on our own servers. I do lean towards Serverless most times, for my own sanity
I would advise against prematurely jumping to kubernetesI agree, this and other things kinda changed my mind about the whole thing.
After writing this, I wonder if ECS wis your better betYeah, EC2 or ECS are quickly becoming the solution that makes the most sense here 😆 Thanks guys!
Sean Lopp12/02/2022, 3:42 PM