19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Channels
announcements
dagster-airbyte
dagster-airflow
dagster-bigquery
dagster-cloud
dagster-cube
dagster-dask
dagster-dbt
dagster-de
dagster-ecs
dagster-feedback
dagster-kubernetes
dagster-machine-learning
dagster-noteable
dagster-releases
dagster-serverless
dagster-showcase
dagster-snowflake
dagster-spatial
dagster-support
dagster-wandb
dagstereo
data-platform-design
events
faq-read-me-before-posting
gigs-freelance
github-discussions
introductions
jobs
random
tools
豆瓣酱帮
Title
n
Nick Cleveland
05/12/2023, 9:58 PMHaving trouble with some performance issues on the tasks started by ecs run launcher.
Here is the error I'm seeing:
Multiprocess executor: child process for step ln2_tank_n2252_lbm was terminated by signal 9 (SIGKILL). This usually indicates that the process was killed by the operating system due to running out of memory. Possible solutions include increasing the amount of memory available to the run, reducing the amount of memory used by the ops in the run, or configuring the executor to run fewer ops concurrently.
dagster._core.executor.child_process_executor.ChildProcessCrashException
I have the following values set in my dagster.yaml for the launcher:
run_launcher:
module: "dagster_aws.ecs"
class: "EcsRunLauncher"
config:
run_resources:
cpu: 512
memory: 1024
ephemeral_storage: 1
z
Zach
05/15/2023, 5:20 PMdid you try increasing the memory in your dagster.yaml file?
n
Nick Cleveland
05/15/2023, 11:38 PMYes I have cpu and memory settings in my dagster yaml. I have also tried setting op tags on my asset, which, according to the documentation, should override settings elsewhere. Tasks that are launched still have cpu of 1024 and memory of 2048 despite this.