https://dagster.io/ logo
Join Slack
Powered by
when attempting to do a graphql query for asset ma...
# ask-community
d
when attempting to do a graphql query for asset materializations, I'm getting the following timeout:
Copy code
(psycopg2.errors.QueryCanceled) canceling statement due to statement timeout

[SQL: SELECT event_logs.id, event_logs.event
FROM event_logs
WHERE event_logs.dagster_event_type = %(dagster_event_type_1)s AND event_logs.asset_key = %(asset_key_1)s ORDER BY event_logs.id DESC]
[parameters: {'dagster_event_type_1': 'ASSET_MATERIALIZATION', 'asset_key_1': '["haruko", "fund_open_positions_by_balances"]'}]
(Background on this error at: <https://sqlalche.me/e/20/e3q8>)
🤖 1
Here's the gql query:
Copy code
query RepositoriesQuery {
      repositoriesOrError {
        ... on RepositoryConnection {
          nodes {
            name
            assetNodes{
                assetKey {
                  path
                }
                assetMaterializations {
                    timestamp
                    metadataEntries {
                      label
                      description
                      ... on PathMetadataEntry {
                        path
                      }
                      ... on NotebookMetadataEntry {
                        path
                      }
                      ... on JsonMetadataEntry {
                        jsonString
                      }
                      ... on UrlMetadataEntry {
                        url
                      }
                      ... on TextMetadataEntry {
                        text
                      }
                      ... on MarkdownMetadataEntry {
                        mdStr
                      }
                      ... on PythonArtifactMetadataEntry {
                        module
                        name
                      }
                      ... on FloatMetadataEntry {
                        floatValue
                      }
                      ... on IntMetadataEntry {
                        intValue
                        intRepr
                      }
                      ... on BoolMetadataEntry {
                        boolValue
                      }
                      ... on PipelineRunMetadataEntry {
                        runId
                      }
                      ... on AssetMetadataEntry {
                        assetKey {
                          path
                        }
                      }
                    }
                    partition
                }
                groupName
                description
                partitionDefinition {
                  description
                  type
                  dimensionTypes {
                    name
                    description
                    type
                    dynamicPartitionsDefinitionName
                  }
                  name
                }
                id
              }
          }
        }
      }
    }
z
You might need to break up your queries into queries for materializations for an individual asset key and then merge the results at the end. If you have a lot of materializations the query will never complete before the timeout. You could first query for all the asset keys, then iterate over them and query the materializations for each asset key individually, then combine at the end.
d
So, I'm now getting the same error in dagit when Ijust attempt to view the asset. And I'm also getting it when I attempt to query the gql endpoint directly.
Copy code
Operation name: AssetEventsQuery

Message: (psycopg2.errors.QueryCanceled) canceling statement due to statement timeout

[SQL: SELECT event_logs.id, event_logs.event 
FROM event_logs 
WHERE event_logs.dagster_event_type = %(dagster_event_type_1)s AND event_logs.asset_key = %(asset_key_1)s AND event_logs.partition IN (%(partition_1_1)s) ORDER BY event_logs.id DESC]
[parameters: {'dagster_event_type_1': 'ASSET_MATERIALIZATION', 'asset_key_1': '["haruko", "fund_perp_positions"]', 'partition_1_1': 'Flagship Fund'}]
(Background on this error at: <https://sqlalche.me/e/20/e3q8>)

Path: ["assetOrError","assetMaterializations"]

Locations: [{"line":17,"column":7}]

Stack Trace:
  File "/usr/local/lib/python3.11/site-packages/graphql/execution/execute.py", line 521, in execute_field
    result = resolve_fn(source, info, **args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster_graphql/schema/pipelines/pipeline.py", line 232, in resolve_assetMaterializations
    events = get_asset_materializations(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster_graphql/implementation/fetch_assets.py", line 256, in get_asset_materializations
    event_records = instance.get_event_records(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster/_utils/__init__.py", line 649, in inner
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/instance/__init__.py", line 1779, in get_event_records
    return self._event_storage.get_event_records(event_records_filter, limit, ascending)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster/_core/storage/event_log/sql_event_log.py", line 952, in get_event_records
    results = conn.execute(query).fetchall()
              ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1412, in execute
    return meth(
           ^^^^^
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 483, in _execute_on_connection
    return connection._execute_clauseelement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1635, in _execute_clauseelement
    ret = self._execute_context(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1844, in _execute_context
    return self._exec_single_context(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1984, in _exec_single_context
    self._handle_dbapi_exception(
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2339, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
    cursor.execute(statement, parameters)

The above exception was the direct cause of the following exception:

Message: psycopg2.errors.QueryCanceled: canceling statement due to statement timeout



Stack Trace:
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
    self.dialect.do_execute(
  File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
    cursor.execute(statement, parameters)
Asset runs on 5 min interval. So, there have been a fair amount of materializations but dagster seems completely unable to handle it.
Just a bump here.
2 Views
Open in Slack
PreviousNext
Powered by