We currently run our Dagster instance on a beefy EC2 instance however we would like to execute our jobs in AWS Fargate. Is there a specific run launcher for Fargate or would we just use the K8's launcher and utilize Fargate profiles within our EKS cluster?

Hi Timothy, one option is to use the ECS run launcher and configure the tasks to launch in fargate:

run_launcher:
  module: "dagster_aws.ecs"
  class: "EcsRunLauncher"
  config:
    include_sidecars: true
    run_task_kwargs:
      launchType: "FARGATE"

Watch out for Fargate instance startup time, though -- we were going to use Fargate, but it turned out that since each instance starts up fresh and has to pull new Docker images without a cache, it's slower than we wanted. May work for your use case though.

Thanks, yeah for us it might actually be an improvement. The department that will be using this was originally using Airflow and they would send their jobs to the K8's API that would then pass it off to a Fargate profile in EKS so there was quite a bit of latency there as well, so hopefully going directly to fargate and pulling from ECR will be a boost.