thanks man

appreciate all the hard work

hmm sounds great folks–I think re: last run vs. caching the run ID, the only thing I was concerned about is that if you have two “sessions” (tabs) with different config and reload the page, clicking in to a previous tab you were working on should show the previous run, not whatever the current “last run” is on the server

yeah that's a good point

maybe the issue is that we don't have config on server

the root issue

yeah - is storing the run ids client side bad? It seemed from the graphQL implementation that the server was keeping all the runs not just the last one, so querying by runID didn’t seem like a problem

but yeah if we keep the runs on the server side and everything else cached in the client

it’s a bit split all over

maybe we should have this stored on disk

dunno

hmm yeah - really depends how you want to handle the runs. I think localstorage is preferable overall because we can iterate on the stored data / add persisted state without writing a whole graphQL layer for it but maybe logs should go on disk

would def prefer not to go to disk client side

or are you talking about server side

yeah I don’t think disk client side is an option

right

TIL i don’t know how browsers work lol

unless y’all want an Electron app haha which I’m more than happy to help with but…

questionable

i think we need a persistent store of log messages server-side

in a prod context they would certainly be streamed to kafka or similar

my q is what is the most convenient message queue esque system to run locally

could just do files i suppose

“/tmp/dagit/<<runid>>”

yeah considering it’s an append-only data structure a file would work pretty well

however if we are running pipelines concurrently then we may have race conditions

redis / sqlite are pretty dependency-free and cross platform, but I’m not sure those are really better for this use case

writing to two files?

multiple processes writing to a single file

oh gotcha