https://dagster.io/ logo
Title
r

Rohan Kshirsagar

02/04/2022, 5:59 PM
Hey all! Im just getting started with dagster and dagster cloud. I was able to get my initial job running locally. When I tried to move to Dagster Cloud, I ran into a hiccup. I ran the agent successfully and set up the cli
```❯ dagster-cloud agent run
2022-02-03 10:20:54 -0800 - dagster_cloud - INFO - Starting Dagster Cloud agent...
2022-02-03 10:20:54 -0800 - dagster_cloud - INFO - Waiting for workspace to be ready...
2022-02-03 10:20:59 -0800 - dagster_cloud - INFO - Started polling for requests from https://rohan.agent.dagster.cloud```
 I kept this ^ running in a terminal
```❯ dagster-cloud config setup
? How would you like to authenticate the CLI? Authenticate using token or keep current settings
? Dagster Cloud organization: rohan
? Dagster Cloud user token: *************************************
? Default deployment: prod```
However, when I try to add a location to my workspace I run into this issue
❯ dagster-cloud workspace add-location unify_news_dag --package-name unify_news_dag --deployment prod
Error: ('Unable to add/update code location: ', 'dagster.core.errors.DagsterUserCodeUnreachableError: Could not send request to agent since no agents have recently heartbeated\n')
a

alex

02/04/2022, 6:35 PM
hmm - do you get that error consistently?
r

Rohan Kshirsagar

02/04/2022, 8:08 PM
yes
a

alex

02/04/2022, 8:11 PM
when i look at https://rohan.dagster.cloud/prod/instance/health it seems there are no agents running, did those terminals with
agent run
going in them stop?
r

Rohan Kshirsagar

02/04/2022, 8:32 PM
oh I may have exited out since then
let me try again
status is not changed
❯ dagster-cloud agent run ~/dagster_home/
2022-02-04 12:32:52 -0800 - dagster_cloud - INFO - Starting Dagster Cloud agent...
2022-02-04 12:32:52 -0800 - dagster_cloud - INFO - Waiting for workspace to be ready...
2022-02-04 12:32:57 -0800 - dagster_cloud - INFO - Started polling for requests from <https://rohan.agent.dagster.cloud>
is this important?
a

alex

02/04/2022, 8:43 PM
hmm strange
so the agent has to be running continuously - are you stopping the process or closing the terminal?
from our end it appears the agents have stopped
r

Rohan Kshirsagar

02/04/2022, 8:57 PM
no i leave it open in a tmux pane
is there a debug verbose mode
when I interrupt it i see three Aborted!s
❯ dagster-cloud agent run ~/dagster_home/
2022-02-04 12:32:52 -0800 - dagster_cloud - INFO - Starting Dagster Cloud agent...
2022-02-04 12:32:52 -0800 - dagster_cloud - INFO - Waiting for workspace to be ready...
2022-02-04 12:32:57 -0800 - dagster_cloud - INFO - Started polling for requests from <https://rohan.agent.dagster.cloud>
^C
Aborted!

Aborted!

Aborted!
a

alex

02/04/2022, 9:01 PM
I can confirm that we were seeing your agent making requests and that it stopped just now when you interrupted it, investigating why the app thinks theres no heartbeats
r

Rohan Kshirsagar

02/04/2022, 9:02 PM
i re- installed and reran in a new tab in Iterm and got a different output
❯ dagster-cloud agent run
2022-02-04 12:56:43 -0800 - dagster_cloud - INFO - Starting Dagster Cloud agent...
2022-02-04 12:56:46 -0800 - dagster_cloud - INFO - Waiting for workspace to be ready...
2022-02-04 12:56:47 -0800 - dagster_cloud - INFO - Updating server for location unify_news_dag
2022-02-04 12:56:48 -0800 - dagster_cloud - INFO - Created a new server for location unify_news_dag
2022-02-04 12:56:48 -0800 - dagster_cloud - INFO - Updating server for location .
2022-02-04 12:56:49 -0800 - dagster_cloud - INFO - Created a new server for location .
2022-02-04 12:56:51 -0800 - dagster_cloud - INFO - Started polling for requests from <https://rohan.agent.dagster.cloud>
but still shows no active agent
a

alex

02/04/2022, 9:02 PM
thanks - just keep it running and we will investigate
👍 1
alright definitely some stuff to fix on our side - but can you re-sync your local machines clock?
were using a timestamp from the agent (that should be replaced with one from the server) and it seems your machine is ~5 minutes behind which causes us to think the agent hasn’t heartbeat recently enough to think we are in a healthy state
r

Rohan Kshirsagar

02/04/2022, 11:06 PM
yeah i will try that in a bit
okay we’re good!
for some reason there are two successfully running agents
and now theres one