https://dagster.io/ logo
#ask-community
Title
# ask-community
d

David Merritt

01/12/2023, 7:52 PM
Hi - I am running into an unexpected error using dagster-serverless:
Copy code
Multiprocess executor: child process for step historical_fire_perimeters_geojson unexpectedly exited with code -9
dagster._core.executor.child_process_executor.ChildProcessCrashException
It seems to occur after a successful yield of an
op
result in a job, halting the flow of the job.
cc @Gabe Schine - trying out your branch
j

jordan

01/12/2023, 7:58 PM
Generally that’s an out of memory error. You might be launching the run in “non-isolated” mode meaning it’s sharing resources with some of your other processes. Switching to “isolated” mode might help get around this.
d

David Merritt

01/12/2023, 8:18 PM
hmm let me try that, but it seems like the isolated run takes 3+ minutes extra to start?
Any idea why that would happen after a successful yield on an op? I wouldn’t expect > 1gB of memory usage in that op. It’s downloading/parsing some json but its not THAT much data.
g

Gabe Schine

01/12/2023, 8:23 PM
I think it's on the order of 50mB, IIRC
d

David Merritt

01/12/2023, 8:24 PM
Copy code
They have fewer compute resources: 0.25 vCPU cores and 1GB of RAM. These resources are shared with other processes for a code location like sensors. As a result, it's recommended to use isolated runs for compute intensive jobs and asset materializations.
50mB of json @Gabe Schine? even if that doubles when parsed into the FS model, we’re well under gB but I’m not sure the basic overhead of the dagit runner/baseline level
isolated run did fix the issue thanks @jordan
g

Gabe Schine

01/12/2023, 8:25 PM
Yes, 50mB of uncompressed JSON string
d

David Merritt

01/12/2023, 8:26 PM
looks like ~1minute of spin up time, I think that’s acceptable. Looks like this won’t be an issue for scheduled jobs @Gabe Schine.
Copy code
When launching runs manually, select Isolate run environment in the Launchpad to launch an isolated runs. Scheduled, sensor, and backfill runs are always isolated.
g

Gabe Schine

01/12/2023, 8:27 PM
yep my read of the docs as well. does the UI let you do a manual asset materialization but isolate its job?
d

David Merritt

01/12/2023, 8:29 PM
yes - checkbox in the “materialization config” modal.
✔️ 1
29 Views