The start time for my tasks range from 20 60 seconds Does an dagster #deployment-ecs

The start time for my tasks range from 20-60 secon...

Mitchell Hynes

01/30/2023, 11:31 PM

The start time for my tasks range from 20-60 seconds. Does anyone have any Dagster-specific* advice on ways to get this number down?

Mike Atlas

01/30/2023, 11:49 PM

You can try to shrink your container image (we use

python:3.10-slim-bullseye

as a base) to shave some cold-boot time off, but I'm not sure this is something Dagster can fix, presuming you're using ECS with Fargate.

👍 1

Mike Atlas

01/30/2023, 11:52 PM

I think if you use ECS with EC2, you can set up image caching and that would speed up startup

Mike Atlas

01/30/2023, 11:54 PM

You might even get some benefit using ECR public gallery for the base image, eg:

public.ecr.aws/docker/library/python:slim-bullseye

https://gallery.ecr.aws/docker/library/python

Mike Atlas

01/30/2023, 11:55 PM

https://github.com/aws/containers-roadmap/issues/696

Mitchell Hynes

01/30/2023, 11:57 PM

I’m not sure this is something Dagster can fix, presuming you’re using ECS with Fargate.

Ohh yeah, this is not dagster’s fault ofc. But it’s good to ask in here regardless 🙂

Mike Atlas

01/31/2023, 4:32 PM

Now that I think on it, using the ECR public gallery base image isn't going to help with deployment/cold boot times from Fargate, only help with build times for your image. The extra layers on the image to make it customized with your user code have to be pulled from your (private ECR) repo anyways, which is also within the aws intranet, so the network transfer speed is really about the image size and not public/private ECR base image.

12 Views

Open in Slack

Previous Next