Hi all! We restart both dagit and dagster api when updating the code. Both dagit and the api are launched from the same container. After a restart, tasks that were running prior to the restart remain hanging in 'in progress'. Why does this happen and how can we fix it?

What's your storage backend (SQLite or Postgres)? It likely is because it wrote to the DB that the run started, but bringing it down doesn't give it the opportunity to clean up those pending runs. You can do something similar to this, but rather than filtering to old records, you either forcefully abandon or delete the stuck runs.