https://dagster.io/ logo
Title
k

Koby Kilimnik

10/10/2021, 11:51 AM
there is a credential issue when using aws sso login , its looking for the ~/.aws/credentials path
d

daniel

10/11/2021, 1:18 PM
Hi Koby - could you share more about where exactly you're seeing this error? Do you have a full stack you could share?
k

Koby Kilimnik

10/11/2021, 1:19 PM
there is no full stack trace only the go error
d

daniel

10/11/2021, 1:19 PM
Could you give me a set of steps I could follow to reproduce the error myself?
k

Koby Kilimnik

10/11/2021, 1:19 PM
sure, let me check my history
setup:
git clone <https://github.com/dagster-io/dagster.git>
cd dagster/examples/deploy_ecs
docker context create ecs dagster-ecs
# interactive >> select an existing aws profile (default)
then: input
docker --context dagster-ecs compose --project-name dagster up
output:
NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
we are using aws sso login for credentials
made sure i am logged in and that ecs context is present
in docker context ls
d

daniel

10/11/2021, 1:34 PM
got it, this is very helpful context, thanks! cc @jordan who knows more of the ins and outs of ECS. I'm not actually sure if this error message is specific to dagster or not, but he may be
j

jordan

10/11/2021, 3:07 PM
Does
aws sts get-caller-identity
return the IAM user/role you expect? What versions of the aws sdk and docker compose are you running?
aws --version
docker compose version
k

Koby Kilimnik

10/11/2021, 3:08 PM
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019
sts caller identity is right
aws version
aws-cli/2.2.7 Python/3.8.8 Linux/5.13.19-200.fc34.x86_64 exe/x86_64.fedora.34 prompt/off
j

jordan

10/11/2021, 3:20 PM
what about
docker compose version
(not
docker-compose version
- the ecs deployment docs use the new compose v2 https://docs.docker.com/compose/cli-command/#compose-v2-and-the-new-docker-compose-command)
iirc the ecs context isn’t available with docker-compose but perhaps they’ve backported it since I last looked
k

Koby Kilimnik

10/11/2021, 3:23 PM
that might be the issue, looks like the docker compose command doesnt exist
weird that i got this far without it
was sure i installed it
j

jordan

10/11/2021, 3:24 PM
yeah - the cutover from
docker-compose
to
docker compose
seems very confusing to me. https://docs.docker.com/cloud/ecs-integration/#prerequisites try these - either by installing docker desktop or using the standalone script that installs
docker compose
k

Koby Kilimnik

10/11/2021, 3:25 PM
i use linux, dont have that privilage
looks like the installation didnt work
meh 😕
j

jordan

10/11/2021, 3:26 PM
https://docs.docker.com/cloud/ecs-integration/#install-script this script in particular didn’t work?
k

Koby Kilimnik

10/11/2021, 3:28 PM
You already have the Docker Compose CLI installed, overriding with latest version
i bet its a docker plugin path definition thing
j

jordan

10/11/2021, 3:29 PM
this is all vaguely familiar from when i needed to get this installed on our buildkite boxes - let me see if i can dig up what i did to make that happen
k

Koby Kilimnik

10/11/2021, 3:29 PM
docker plugin enable compose
Error response from daemon: plugin "compose" not found
j

jordan

10/11/2021, 3:33 PM
https://github.com/dagster-io/dagster/commit/a759b176e6babfc2aca0e407a25b7362e8b5b24c hm 🤔 I remember running into these exact issues with the install script but it looks like i didn’t leave myself any breadcrumbs for how i got around it. i’ll keep digging.
k

Koby Kilimnik

10/11/2021, 3:33 PM
10x