Do

run_keys

get persisted when running dagster 'in production'? At the moment I'm experimenting with a sensor locally, executing the script with

dagster dev -f myfile.py

. When I run the sensor first time against some files, it will run a job, subsequent invocations of that sensor skip the files due to the key already existing. If I kill and restart the dagster process, the run keys have disappeared, then the same files get parsed again. https://docs.dagster.io/concepts/partitions-schedules-sensors/sensors#idempotence-using-run-keys

yes the run_keys get persisted when not running via the default

dagster dev

this is because

dagster dev

defaults to temp dir as the storage. To designate a more permanent home for your runs or logs, you can set the

DAGSTER_HOME

environment variable to a folder on your filesystem. Dagster will then use the specified folder for storage on all subsequent runs of

dagster dev

. more details here: https://docs.dagster.io/guides/running-dagster-locally#asset-and-run-storage

Great, thanks for the info