my agent ECS task is running

dagster/dagster-cloud-agent:1.2.4

and I set

DAGSTER_HOME

env var on the container to

/opt/dagster/dagster_home

. The image for my code location does not have the

/opt/dagster

directory and so I set

DAGSTER_HOME

in

dagster.yaml

env_vars

to

/home/dagster

- but it still seems to be trying to store output in

/opt/dagster

which doesn't exist (and it does not have permission to create)

dagster._core.errors.DagsterExecutionHandleOutputError: Error occurred while handling output "result" of step "hello_redshift":
...
The above exception was caused by the following exception:
PermissionError: [Errno 13] Permission denied: '/opt/dagster'
  File "/site/venv/lib/python3.10/site-packages/dagster/_core/execution/plan/utils.py", line 54, in op_execution_error_boundary
    yield
  File "/site/venv/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 439, in iterate_with_context
    next_output = next(iterator)
  File "/site/venv/lib/python3.10/site-packages/dagster/_core/execution/plan/execute_step.py", line 594, in _gen_fn
    gen_output = output_manager.handle_output(output_context, output.value)
  File "/site/venv/lib/python3.10/site-packages/dagster/_core/storage/upath_io_manager.py", line 239, in handle_output
    path.parent.mkdir(parents=True, exist_ok=True)

We can likely fix the place that it's writing to on the filesystem - but are you considering writing these outputs somewhere other than the filesystem at some point by setting up a different io manager (like the s3 io manager)? That will give you some nice things like the ability to re-execute runs from failure, and should fix this as well

This is just a "hello world" job that just checks connectivity to our DB. But if it stores outputs in DAGSTER_HOME by default, why is it not using the modified DAGSTER_HOME from my dagster.yaml?

it uses that local_artifact_storage key that I referenced above - I think modifying that will do what you want

oh, and that key was initially set based on the agents DAGSTER_HOME I guess

yeah

does the

local_artifact_storage

key go under

user_code_launcher.config

in my dagster.yaml?

local_artifact_storage is a sibling of user_code_launcher

thanks