The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Having trouble with some performance issues on the tasks started by ecs run launcher.

Here is the error I'm seeing:

Multiprocess executor: child process for step ln2_tank_n2252_lbm was terminated by signal 9 (SIGKILL). This usually indicates that the process was killed by the operating system due to running out of memory. Possible solutions include increasing the amount of memory available to the run, reducing the amount of memory used by the ops in the run, or configuring the executor to run fewer ops concurrently.
dagster._core.executor.child_process_executor.ChildProcessCrashException

I have the following values set in my dagster.yaml for the launcher:

run_launcher:
  module: "dagster_aws.ecs"
  class: "EcsRunLauncher"
  config:
    run_resources:
      cpu: 512
      memory: 1024
      ephemeral_storage: 1

did you try increasing the memory in your dagster.yaml file?

Yes I have cpu and memory settings in my dagster yaml. I have also tried setting op tags on my asset, which, according to the documentation, should override settings elsewhere. Tasks that are launched still have cpu of 1024 and memory of 2048 despite this.