https://dagster.io/ logo
#dagster-cloud
Title
# dagster-cloud
s

Stopher

12/14/2022, 11:34 PM
Good day! I am currently using Dagster Cloud hybrid running on AWS Fargate and I have a question about how to control the compute resources that Fargate tasks use to execute. I know that it is possible to customize the CPU and memory of every run for a job using
@job(tags={"ecs/cpu": "256", "ecs/memory": "512"})
. However this only sets the resources of the Fargate task executing runs (the
run_*
task definitions). Is it possible to customize the CPU and memory of the task that is loading the repository as well (i.e.: the
server_*
task definition)? Thank you!
d

daniel

12/15/2022, 12:19 AM
Hi @Stopher - we don’t have a hook for this yet but it would be quick and easy to add one. Are you running into the memory limit? It hasn’t come up yet in ECS yet (or at least nobody has asked for it) I think because the server typically uses way less memory than the runs do
s

Stopher

12/15/2022, 2:54 PM
Hi @daniel, I am not running into memory limits, but based on my experience so far, reloading a job repository in a hybrid Fargate deployment takes much longer than reloading the same job repository in my self-hosted OSS deployment (we're talking minutes vs seconds) and I am wondering what I can do to speed this up. This is pretty important since my application involves frequent repository reloading.
d

daniel

12/15/2022, 2:56 PM
I see, sounds like it’s CPU rather than memory. That should be no problem to add then
Probably will be the first week of January
🎉 2
s

Stopher

12/15/2022, 2:57 PM
Awesome, thank you very much!
d

daniel

01/06/2023, 2:55 PM
Hi, this is now live if you upgrade your agent, there's a 'server_resources' and 'run_resources' key: https://docs.dagster.io/dagster-cloud/deployment/agents/amazon-ecs/configuration-reference#user_code_launcher-properties
🎉 1
❤️ 1
s

Stopher

01/24/2023, 3:56 PM
Fantastic! We just got back from our holidays+company offsite and we are looking forward to using this new feature! Thank you!