anyone see similar issues with a sensor doing this to your code location pod?
it's building resources using secrets from that code location and running some external queries, but something appears to be persisting in memory after each tick and eventually running it out of memory
adding some manual garbage collection (
) seems to have fixed it
still a mystery of what wasn't getting flushed, but no longer crashing
01/29/2023, 7:17 PM
This might depend a lot on the exact code of your sensor - do you have something that could be in a contextmanager but isnt?
01/29/2023, 11:27 PM
good idea, yeah I'm using an tunnel from dagster_ssh but calling stop and start explicitly
i'll test out if using it as a contextmanager changes anything
01/29/2023, 11:28 PM
Alternately maybe calling stop() in a finally: block would have a similar effect - if the sensor raises an exception you’d still want stop() to be called
01/29/2023, 11:37 PM
yeah the SSHTunnelForwarder.start() method doesn't actually seem to support a context manager usage