Hi Team, My name in Chris Sarakasidis - Data Engineer, Sainsburys. We use dagster (v 0.11.11) on a daily basis to orchestrate our pipelines. Yesterday we cancelled a production pipeline from the UI (see image attached below), but unfortunately it didn’t terminate the job. The job continued and completed causing various issues. This wasn’t the first time it happened, so we are wondering if this is a known bug that was fixed in subsequent versions or there is something else happening that we should be aware of.

Hi Christos. Can you share what you’ve configured for your run launcher? Are you using the

DefaultRunLauncher

?

thanks for the reply @prha, we use

DockerRunLauncher

(runs on AWS ECS Fargate)

Also, do you have a run_coordinator configured?

yes, here it is:

run_coordinator:
  module: dagster.core.run_coordinator
  class: QueuedRunCoordinator
  config:
    tag_concurrency_limits:
      - key: "pipeline"
        value: "product_matching"
        limit: 1

cc @daniel

Hi Christos - when you terminated the run, did you check the 'force terminate' checkbox? That "the computational resources may not be fully cleaned up" message is reflected in the fact that the run kept going

these computational resources refer to AWS resources corresponding to the solids. But the pipeline on dagster why wasn’t terminated?

It's actually referring to the run worker as well (the docker container spun up by the run launcher) - if you want to make sure that all the resources spun up by your run get cleaned up (including the container) you don't want to click that checkbox

so when we actually want to cancel the pipeline (i.e., kill the container launched by the run launcher) what should we do?

the terminate button should give you an option to terminate it (without the checkbox that says "Force Terminate"). Are you only seeing the option to force terminate?