https://dagster.io/ logo
#dagster-cloud
Title
# dagster-cloud
g

Gabe Schine

08/03/2023, 2:04 PM
Gooood morning. I'd like to run an observable source asset job that is kicked off by a sensor in a non-isolated env. It's a really simple HTTP HEAD call that finishes in like 1 second. Is the correct thing to do to add the
tag
"dagster/isolated": "disabled"
to the
define_asset_job
that is passed to the sensor?
I've seen some references to this tag in chats, but not much in the way of documentation
r

rex

08/03/2023, 2:50 PM
Our documentation for run isolation is here: https://docs.dagster.io/dagster-cloud/deployment/serverless#run-isolation. @Shalabh Chaturvedi can this be configured by using tags?
g

Gabe Schine

08/03/2023, 3:05 PM
@rex thanks - I did find that documentation. It implies to me that I can change the setting for the entire deployment. As a dev, I know best what jobs are safe to run in the shared env (small jobs), and what needs an isolated env.
s

Shalabh Chaturvedi

08/03/2023, 4:25 PM
Hi Gabe, you should be able to add
tags={"dagster/isolation": "false"}
to the
define_asset_job
to make that job always run non-isolated. I'm going to verify this and fix our documentation.
g

Gabe Schine

08/03/2023, 5:06 PM
Great! And it's definitely "false" and not "disabled"?
s

Shalabh Chaturvedi

08/03/2023, 5:16 PM
Unfortunately I just verified that this doesn't work because it is a system tag. Let me see if there's another way we can mark a job to always run non-isolated.
g

Gabe Schine

08/03/2023, 5:32 PM
Copy that thanks
s

Shalabh Chaturvedi

08/03/2023, 5:46 PM
While you can't add the tag in the
define_asset_job
it does work in the
RunRequest
returned from the sensor itself. Eg - this worked for me:
Copy code
@sensor(job=my_job)
def my_sensor():
    yield RunRequest(
        tags={"dagster/isolation": "disabled"},
    )
g

Gabe Schine

08/03/2023, 6:27 PM
I'll try this. Thank you!
s

Sean Lopp

08/03/2023, 7:54 PM
@Gabe Schine if this works would you mind opening a GitHub issue for us to document this? I've starred this thread for my future use but agree we should document
👍 1
g

Gabe Schine

08/04/2023, 5:17 PM
How would you recommend going about testing this? It looks like my only option might be to deploy to prod, since local dev doesn't do isolation the same way, and branch deployments disable sensors.
When testing the sensor in a branch deployment, I am able to tell it to open the requested job in the Launchpad, which does apply the correct tag, however there is also the checkbox to select an isolated env or not:
image.png
Looks like hitting "Launch Run" did override the checkbox, though, so that's encouraging
Should the checkbox be disabled in the case of the tag being present?
s

Sean Lopp

08/04/2023, 7:19 PM
yea I am not surprised the UI is not necessarily aware of the manually set tag, but this does sound promising. Also confirmed the best way to test would be what you've done + a run in prod
s

Shalabh Chaturvedi

08/07/2023, 4:56 PM
Hi Gabe - we have also made a change to make this system tag editable: https://github.com/dagster-io/dagster/pull/11917. It should go out with the next release later this week. This will allow setting tag directly on the job.
g

Gabe Schine

08/07/2023, 5:37 PM
Nice! That'll be a little cleaner than the current approach.