Kyle Montag
03/01/2023, 12:07 AMdagster_shell
from within an ECS task that was created by dagster, I am getting the following error: AWS_CONTAINER_CREDENTIALS_RELATIVE_URI: unbound variable
when trying to do a spot check on the task’s execution role permissions. is there a bug related to this on ECS somewhere? am i doing something wrong?
i am not seeing the IAM credentials being passed to the subprocess that is being kicked off as part of dagster_shell.execute_shell_script
, both when trying to use the credentials via the aws cli, or when doing curl 169.254.170.2$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
which is recommended by the ECS docs
EDIT: i solved this, it turns out that the environment from the operating system is not forwarded to the subprocess created by execute_shell_script
by default. this means that the aforementioned environment variable does not get loaded in. to remedy i simply changed my call to execute_shell_script
to include dict(os.environ)
in the value being sent to the env
parameterdaniel
03/01/2023, 1:52 AMexecute_shell_script
takes in an optional env dictionary argument, does it work if you pass in
{"AWS_CONTAINER_CREDENTIALS_RELATIVE_URI": os.getenv("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI")}
there? I think this is an oddity of execute_shell_script where it doesn't pass through the env from the parent process, i'll see about changing thisdaniel
03/01/2023, 1:52 AMdaniel
03/01/2023, 1:57 AMKyle Montag
03/01/2023, 1:59 AMKyle Montag
03/01/2023, 1:59 AMdaniel
03/01/2023, 2:00 AMKyle Montag
03/01/2023, 2:01 AMKyle Montag
03/01/2023, 2:01 AMthis function does not forward the OS environment by default, if you need it be sure to include
or something like that