https://dagster.io/ logo
Powered by
Title
f

Félix Tremblay

02/20/2023, 6:57 PM
Hello 👋 As shown in dagster-cloud-environment-variables-and-secrets, the Local variables can be downloaded to a .env file. Is there a way to download this file programmatically (e.g. using the API)? For users developing in ephemeral development environment (e.g. Codespaces or GitPod), having to get the .env file manually each time (multiple times per week) is affecting productivity. A workaround is to define the needed env vars in GitHub Codespaces settings, but that feels like unnecessary duplications and a door for inconsistencies and errors. The ideal situation is one where all secrets are managed centrally in Dagster Cloud UI, and the .env file is fetched automatically each time we launch the Codespaces/GitPod instance.
d

daniel

02/21/2023, 1:55 PM
Hi Félix - this is available over the GraphQL API in the field 
localSecretsFileContents
: 
query LocalSecretsFileQuery {
    localSecretsFileContents
}
so if you made a graphql query using a Dagster Cloud User token in the "Dagster-Cloud-Api-Token" header, I would expect it to return the contents that could be placed in that file
👍 1
f

Félix Tremblay

02/21/2023, 7:59 PM
Thank you @daniel
Hello @daniel, I am not able to make it work and need some help
image.png
{
  "error": {
    "data": null,
    "errors": [
      {
        "message": "Cannot query field 'localSecretsFileContents' on type 'DagitQuery'.",
        "locations": [
          {
            "line": 2,
            "column": 3
          }
        ]
      }
    ]
  }
}
i don't know what to put in the HTTP HEADERS box
d

daniel

02/25/2023, 8:21 PM
That looks like you're hitting open source dagit? (I see 127.0.0.1 in the URL bar) localSecretsFileContents only works in cloud
i.e. that url should be something like <your org>.dagster.cloud/prod/graphql, not 127.0.0.1
f

Félix Tremblay

02/25/2023, 8:38 PM
Thanks for the clarification and thank you for creating such a great open source program
:condagster: 1
I noticed some relevant information on how to use GraphQL with the Cloud is missing from the documentation. I think it would be really helpful for other users if the documentation could include more information on this topic. Is this something you might consider adding in a future update?
by searching in Slack I found this link which is exactly what I need to programmatically fetch the secrets
In case someone else wants to do the same thing. Here's the script: 
import os
import warnings

from dagster import ExperimentalWarning
from dagster_graphql import DagsterGraphQLClient
from gql.transport.requests import RequestsHTTPTransport

# Ignore experimental warnings
warnings.filterwarnings("ignore", category=ExperimentalWarning)

# Create GraphQL client for Dagster Cloud
token = os.environ.get("DAGSTER_CLOUD_API_TOKEN")
org = os.environ.get("DAGSTER_CLOUD_ORGANIZATION")
url = f"https://{org}.dagster.cloud/prod"
cloud_client = DagsterGraphQLClient(
    url,
    transport=RequestsHTTPTransport(
        url=url + "/graphql", headers={"Dagster-Cloud-Api-Token": token}
    ),
    use_https=True,
)

# Define query to fetch secrets for "local" deployment scope
# <https://docs.dagster.io/dagster-cloud/developing-testing/environment-variables-and-secrets#exporting-variables-to-a-env-file>
query = "query LocalSecretsFileQuery { localSecretsFileContents }"

# Get secrets
result = cloud_client._execute(query)
secrets = result["localSecretsFileContents"]

# Write secrets to file
path = ".env"
with open(path, "w") as f:
    f.write(secrets)
:blob_fistbumpr: 1
2 Views
#dagster-cloudJoin Slack
Powered by