https://dagster.io/ logo
Title
d

Dylan Bienstock

06/02/2021, 5:03 PM
Hi! I have been using the DagsterGraphQLClient to submit pipeline runs and get a pipelines run status. I create the client like so
client = DagsterGraphQLClient("localhost", port_number=3000)
and it is working when I run dagit locally. However, when i deploy it to my dagster k8 instance it is not able to connect. Is there a different port I should be using? Or do I need to open something up to allow it to communicate?
r

rex

06/02/2021, 5:16 PM
Are you using the helm chart to deploy? The port might be different, depending on your
values.yaml
. Currently, the default is 80 https://sourcegraph.com/github.com/dagster-io/dagster/-/blob/helm/dagster/templates/helpers/_helpers.tpl#L34:30
d

Dylan Bienstock

06/02/2021, 5:25 PM
Yes I am using the helm chart to deploy and the dagit port is set to 80. However, when i update the client to
client = DagsterGraphQLClient("localhost", port_number=80)
I still get the following connection refused error. I am also following the deployment architecture example where my company repo and dagit server are on separate pods so this instantiation exist on a seperate pod than what dagit is running on
r

rex

06/02/2021, 5:32 PM
Ah I think you’ll need to portforward from your dagit pod if you want to try this out locally.
Since the client is external to your kubernetes cluster, you’ll need to allow access into the cluster to use the client. A more permanent solution would be to use an Ingress, but for your purposes, you could just portforward from the dagit pod to test this out
kubectl --namespace <NAMESPACE> port-forward <DAGIT_POD_NAME> 8080:80
then, we can instantiate the client using the port forwarding to 80
client = DagsterGraphQLClient("localhost", port_number=8080)
k

Kirk Stennett

06/02/2021, 5:46 PM
What is the best way to set up the client in the user code deployment in that same cluster? As in: a solid is running the graphql client and needs to connect
r

rex

06/02/2021, 6:01 PM
Do you have an external url that allows you to access the Dagit service? You could make a Dagster
resource
for the client, using that url as the hostname
k

Kirk Stennett

06/02/2021, 6:10 PM
Unfortunately not, we haven't set up an ingress for the service. We got it working by using the internal service IP, but I'm not sure if there's an easy way to get that each run. Our cluster is on spot instances so it could potentially change that internal IP frequently. Any thoughts?
r

rex

06/02/2021, 6:20 PM
I think you’ll have to rely on the kubernetes api to dynamically fetch the endpoint for the dagit service if that’s the case
👍 1
k

Kirk Stennett

06/08/2021, 9:01 PM
For anyone else wondering about this: we got it working by using the internal service address.
dagster-dagit.<namespace>.svc.cluster.local:80