https://dagster.io/ logo
#dagster-cloud
Title
# dagster-cloud
l

Linton Ye

03/05/2024, 11:13 PM
Hi, I’m using EC2 to host a single agent in the hybrid setup, and I’d appreciate any feedback on my setup. 1. Running a t3a.small instance where the agent and user code run as separate docker containers 2. The user code container only makes http requests and accesses a postgres db 3. When a run starts, a larger container instance is created (via Fargate). All heavy-lifting work, which requires more memory and CPU, is done here. The result is either saved to db or S3 4. When a run finishes or errors out, the large instance is stopped. What do you think? Are there any potential issues with this setup? At which point should I start exploring multiple agents (for fault tolerance and scalability)?
🤖 1
sounds like I’m over-engineering this. 😆 I could have just use
EcsUserCodeLauncher
instead of splitting it into two nodes…
3 Views