https://dagster.io/ logo
Join Slack
Powered by
Timezone for start and end times from GraphQL The...
# ask-community
s
Timezone for start and end times from GraphQL The response I get back from a GraphQL query like
Copy code
{
  repositoriesOrError {
    ... on RepositoryConnection {
      nodes {
        jobs {
          runs {
            id
            startTime
            endTime
          }
        }
      }
    }
  }
}
looks like
Copy code
{
  "data": {
    "repositoriesOrError": {
      "nodes": [
        {
          "jobs": [
            {
              "runs": [
                {
                  "id": "25718726-4265-45fb-b92f-ef656ec0703a",
                  "startTime": 1675048818.020318,
                  "endTime": 1675048823.704493
                }
              ]
            }
          ]
        }
      ]
    }
  }
}
The 
startTime
and 
endTime
have documentation like 
startTime: Float
TYPE DETAILS
The 
Float
scalar type represents signed double-precision fractional values as specified by IEEE 754.
What can we assume about the timezone or locale of those times?
o
hi @Stefan Adelbert ! all such timestamps will be in UTC
s
Thanks for the response @owen. This is what I was hoping for and (in my humble opinion) is by far the best way to handle timestamps. From what I can tell though, the timestamps represent local time, not UTC. But now that I know that they should be UTC timestamps, I will do some more testing.
@owen I've just checked this again and what I'm seeing is timestamps which are not UTC. For example, I've just run a job on my local machine and my local time (
Australia/Sydney
) is 
Fri 17 Feb 2023 15:38:56 AEDT
. I'm running the dagster stack in docker compose. When I query the runs using GraphQL with
Copy code
{
  repositoriesOrError {
	  ... on RepositoryConnection {
      nodes {
        jobs {
          runs {
            id
            startTime
            endTime
          }
        }
      }
    }
  }
}
I get
Copy code
...
        {
          "jobs": [
            {
              "runs": [
                {
                  "id": "bdf713fc-adbd-4514-b4fa-1b35f7a55531",
                  "startTime": 1676608531.281406,
                  "endTime": 1676608531.53234
                }
              ]
            },
            {
              "runs": []
            },
            {
              "runs": []
            }
          ]
        },
...
Converting 
startTime
into a datetime, I get a naive datetime (of course), but apparently at my current location.
Copy code
>>> import datetime
>>> datetime.datetime.fromtimestamp(1676608531.281406)
datetime.datetime(2023, 2, 17, 15, 35, 31, 281406)
>>> datetime.datetime.utcnow()
datetime.datetime(2023, 2, 17, 4, 43, 26, 707856)
>>> datetime.datetime.now()
datetime.datetime(2023, 2, 17, 15, 43, 30, 499757)
I suspect that it has to do with the locale where 
dagit
is running, i.e. the start time is being localised and the converted to a timestamp, kinda like the difference between these
Copy code
>>> datetime.datetime.timestamp(datetime.datetime.utcnow())
1676569595.505416
>>> datetime.datetime.timestamp(datetime.datetime.now())
1676609192.641514
I've had a quick look at the dagster source code and I can see calls like 
pendulum.now("UTC")
peppered around. And generally there aren't calls to plain old 
now()
other than in tests.
o
just looked into this a bit more, and I think this is potentially just good-old fashioned tz-naive confusing behavior. we do just return the timestamp as a float directly from the database in our graphql resolver, so it never gets converted to a timestamp (and so is basically guaranteed to be in UTC). datetime.datetime.fromtimestamp returns a timezone-naive datetime in the local timezone: https://docs.python.org/3/library/datetime.html#datetime.date.fromtimestamp, so I think what you're experiencing is all expected
👏 1
s
Grrrrr... 😠 That is so frustrating. How many times will timezones bite me in the a...? I think it is time for me to move to 
pendulum
for good. Thank you @owen and my apologies for using up some of your time.
40 Views
Open in Slack
PreviousNext
Powered by