https://dagster.io/ logo
#ask-community
Title
# ask-community
s

Stefan Adelbert

02/16/2023, 4:27 AM
Timezone for start and end times from GraphQL The response I get back from a GraphQL query like
Copy code
{
  repositoriesOrError {
    ... on RepositoryConnection {
      nodes {
        jobs {
          runs {
            id
            startTime
            endTime
          }
        }
      }
    }
  }
}
looks like
Copy code
{
  "data": {
    "repositoriesOrError": {
      "nodes": [
        {
          "jobs": [
            {
              "runs": [
                {
                  "id": "25718726-4265-45fb-b92f-ef656ec0703a",
                  "startTime": 1675048818.020318,
                  "endTime": 1675048823.704493
                }
              ]
            }
          ]
        }
      ]
    }
  }
}
The
startTime
and
endTime
have documentation like
startTime: Float
TYPE DETAILS
The
Float
scalar type represents signed double-precision fractional values as specified by IEEE 754.
What can we assume about the timezone or locale of those times?
o

owen

02/16/2023, 5:49 PM
hi @Stefan Adelbert ! all such timestamps will be in UTC
s

Stefan Adelbert

02/16/2023, 10:51 PM
Thanks for the response @owen. This is what I was hoping for and (in my humble opinion) is by far the best way to handle timestamps. From what I can tell though, the timestamps represent local time, not UTC. But now that I know that they should be UTC timestamps, I will do some more testing.
@owen I've just checked this again and what I'm seeing is timestamps which are not UTC. For example, I've just run a job on my local machine and my local time (
Australia/Sydney
) is
Fri 17 Feb 2023 15:38:56 AEDT
. I'm running the dagster stack in docker compose. When I query the runs using GraphQL with
Copy code
{
  repositoriesOrError {
	  ... on RepositoryConnection {
      nodes {
        jobs {
          runs {
            id
            startTime
            endTime
          }
        }
      }
    }
  }
}
I get
Copy code
...
        {
          "jobs": [
            {
              "runs": [
                {
                  "id": "bdf713fc-adbd-4514-b4fa-1b35f7a55531",
                  "startTime": 1676608531.281406,
                  "endTime": 1676608531.53234
                }
              ]
            },
            {
              "runs": []
            },
            {
              "runs": []
            }
          ]
        },
...
Converting
startTime
into a datetime, I get a naive datetime (of course), but apparently at my current location.
Copy code
>>> import datetime
>>> datetime.datetime.fromtimestamp(1676608531.281406)
datetime.datetime(2023, 2, 17, 15, 35, 31, 281406)
>>> datetime.datetime.utcnow()
datetime.datetime(2023, 2, 17, 4, 43, 26, 707856)
>>> datetime.datetime.now()
datetime.datetime(2023, 2, 17, 15, 43, 30, 499757)
I suspect that it has to do with the locale where
dagit
is running, i.e. the start time is being localised and the converted to a timestamp, kinda like the difference between these
Copy code
>>> datetime.datetime.timestamp(datetime.datetime.utcnow())
1676569595.505416
>>> datetime.datetime.timestamp(datetime.datetime.now())
1676609192.641514
I've had a quick look at the dagster source code and I can see calls like
pendulum.now("UTC")
peppered around. And generally there aren't calls to plain old
now()
other than in tests.
o

owen

02/17/2023, 6:46 PM
just looked into this a bit more, and I think this is potentially just good-old fashioned tz-naive confusing behavior. we do just return the timestamp as a float directly from the database in our graphql resolver, so it never gets converted to a timestamp (and so is basically guaranteed to be in UTC). datetime.datetime.fromtimestamp returns a timezone-naive datetime in the local timezone: https://docs.python.org/3/library/datetime.html#datetime.date.fromtimestamp, so I think what you're experiencing is all expected
👏 1
s

Stefan Adelbert

02/19/2023, 11:43 PM
Grrrrr... 😠 That is so frustrating. How many times will timezones bite me in the a...? I think it is time for me to move to
pendulum
for good. Thank you @owen and my apologies for using up some of your time.
7 Views