Hi, new to dagster and looking to use it as our co...
# announcements
j
Hi, new to dagster and looking to use it as our computation pipeline library. Currently (pre-dagster) we store intermediate results in file and in the next pipeline step load those in again. This is very similar to Dagster's SerializationStrategy, nice! Our next need is: is it possible to re-execute parts of the pipeline based on the serialized data after restarting dagit, or is this possible with the CLI tools? I can re-execute single steps in dagit, but when I restart dagit, the previous runs are not shown anymore and I have to re-run the entire pipeline before I can select parts.
I could of course use hydrations, but then I'd have to manually set the path while I like the fact that dagster manages the locations of results with the storagefilesystembase_dir system
Ah, the solution is to set $DAGSTER_HOME env variable (https://docs.dagster.io/docs/deploying/instance)