@Scout Hi I currently defined a sensor for my dbt jobs but upon executing the job I see this unexpected error :

dagster._core.errors.DagsterSubprocessError: During multiprocess execution errors occurred in child processes:
In process 21: dagster._check.CheckError: Invariant failed.

Stack Trace:
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/executor/child_process_executor.py", line 79, in _execute_command_in_child_process
    for step_event in command.execute():
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/executor/multiprocess.py", line 97, in execute
    yield from execute_plan_iterator(
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/api.py", line 875, in __iter__
    yield from self.iterator(
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/plan/execute_plan.py", line 39, in inner_plan_execution_iterator
    compute_log_manager = job_context.instance.compute_log_manager
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/instance/__init__.py", line 799, in compute_log_manager
    compute_log_manager = cast(InstanceRef, self._ref).compute_log_manager
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/instance/ref.py", line 533, in compute_log_manager
    return self.compute_logs_data.rehydrate(as_type=ComputeLogManager)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster/_serdes/config_class.py", line 132, in rehydrate
    return klass.from_config_value(self, check.not_none(result.value))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster_gcp/gcs/compute_log_manager.py", line 119, in from_config_value
    return cls(inst_data=inst_data, **config_value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dagster_gcp/gcs/compute_log_manager.py", line 88, in __init__
    check.invariant(self._bucket.exists())
  File "/usr/local/lib/python3.11/dist-packages/dagster/_check/__init__.py", line 1614, in invariant
    raise CheckError("Invariant failed.")


  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/execution/api.py", line 764, in job_execution_iterator
    for event in job_context.executor.execute(job_context, execution_plan):
  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/executor/multiprocess.py", line 311, in execute
    raise DagsterSubprocessError(

Do you have any idea ?

The error you're encountering,

dagster._core.errors.DagsterSubprocessError

, indicates that there were issues during the multiprocess execution of your job. Specifically, the error

dagster._check.CheckError: Invariant failed.

suggests that an internal assertion within Dagster has failed. From the stack trace, it appears that the error is related to the

compute_log_manager

within the Dagster instance. The

compute_log_manager

is responsible for handling logs produced by the execution of steps within a job. The error is raised when the code checks whether the GCS bucket, which is supposed to store the compute logs, exists. Here's the relevant part of the stack trace:

python File "/usr/local/lib/python3.11/dist-packages/dagster_gcp/gcs/compute_log_manager.py", line 88, in __init__     check.invariant(self._bucket.exists())

This line is checking that the GCS bucket configured for storing compute logs actually exists. The

CheckError

is thrown because this invariant check fails, meaning the bucket does not exist or is not accessible. To resolve this issue, you should: 1. Verify that the GCS bucket specified for your compute logs is correctly configured and exists. 2. Ensure that the Dagster instance has the necessary permissions to access and write to the GCS bucket. 3. Check that the GCS bucket name and any related configuration settings are correct in your Dagster instance configuration. If after checking these points the issue persists, you may want to look into the following: - Review the configuration of your

compute_log_manager

to ensure it is set up correctly. - Check for any network issues that might be preventing access to the GCS bucket. - Review the IAM permissions for the service account used by Dagster to interact with GCS to ensure it has the necessary roles and permissions. If you need further assistance, you might want to provide more context or configuration details, which could help in diagnosing the problem more accurately.

@Scout When I look at my sensors I can see this error message :

dagster._core.errors.SensorExecutionError: Error occurred during the execution of evaluation_fn for sensor dbt_job_fail  File "/usr/local/lib/python3.11/dist-packages/dagster/_grpc/impl.py", line 384, in get_external_sensor_execution    with user_code_error_boundary(  File "/usr/lib/python3.11/contextlib.py", line 155, in __exit__    self.gen.throw(typ, value, traceback)  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/errors.py", line 297, in user_code_error_boundary    raise new_error from eThe above exception was caused by the following exception:dagster._check.CheckError: Invariant failed.  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/errors.py", line 287, in user_code_error_boundary    yield  File "/usr/local/lib/python3.11/dist-packages/dagster/_grpc/impl.py", line 390, in get_external_sensor_execution    return sensor_def.evaluate_tick(sensor_context)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/definitions/sensor_definition.py", line 780, in evaluate_tick    result = self._evaluation_fn(context)             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/definitions/sensor_definition.py", line 1097, in _wrapped_fn    result.append(next(raw_evaluation_result))                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/definitions/run_status_sensor_definition.py", line 835, in _wrapped_fn    logger=context.log,           ^^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/definitions/sensor_definition.py", line 418, in log    self._logger = self._exit_stack.enter_context(                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/lib/python3.11/contextlib.py", line 505, in enter_context    result = _enter(cm)             ^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/definitions/instigation_logger.py", line 105, in __enter__    and isinstance(self._instance.compute_log_manager, CapturedLogManager)                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/instance/__init__.py", line 799, in compute_log_manager    compute_log_manager = cast(InstanceRef, self._ref).compute_log_manager                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster/_core/instance/ref.py", line 533, in compute_log_manager    return self.compute_logs_data.rehydrate(as_type=ComputeLogManager)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster/_serdes/config_class.py", line 132, in rehydrate    return klass.from_config_value(self, check.not_none(result.value))           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster_gcp/gcs/compute_log_manager.py", line 119, in from_config_value    return cls(inst_data=inst_data, **config_value)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "/usr/local/lib/python3.11/dist-packages/dagster_gcp/gcs/compute_log_manager.py", line 88, in __init__    check.invariant(self._bucket.exists())  File "/usr/local/lib/python3.11/dist-packages/dagster/_check/__init__.py", line 1614, in invariant    raise CheckError("Invariant failed.")

The error message you're seeing,

dagster._core.errors.SensorExecutionError

, indicates that there was an error during the execution of the evaluation function for your sensor named

dbt_job_fail

. The

CheckError: Invariant failed.

suggests that an internal assertion within Dagster has failed, similar to the previous error you encountered. The stack trace points to an issue with the

compute_log_manager

again, specifically related to the Google Cloud Storage (GCS) bucket that is supposed to store the compute logs. The error is raised when the code checks whether the GCS bucket exists and is accessible. Here's the relevant part of the stack trace:

python File "/usr/local/lib/python3.11/dist-packages/dagster_gcp/gcs/compute_log_manager.py", line 88, in __init__     check.invariant(self._bucket.exists())

This line is checking that the GCS bucket configured for storing compute logs actually exists. The

CheckError

is thrown because this invariant check fails, meaning the bucket does not exist or is not accessible. To resolve this issue, you should: 1. Verify that the GCS bucket specified for your compute logs is correctly configured and exists. 2. Ensure that the Dagster instance has the necessary permissions to access and write to the GCS bucket. 3. Check that the GCS bucket name and any related configuration settings are correct in your Dagster instance configuration. If after checking these points the issue persists, you may want to look into the following: - Review the configuration of your

compute_log_manager

to ensure it is set up correctly. - Check for any network issues that might be preventing access to the GCS bucket. - Review the IAM permissions for the service account used by Dagster to interact with GCS to ensure it has the necessary roles and permissions. Since the error is occurring within the sensor execution, it's also possible that there's an issue with how the sensor is configured or how it's interacting with the

compute_log_manager

. If the sensor is supposed to trigger based on logs or events in GCS, make sure that the sensor's logic is correctly implemented and that it's using the correct GCS bucket and path. If you need further assistance, you might want to provide more context or configuration details, which could help in diagnosing the problem more accurately.

@Scout How can I verify the configuration of my compute_log_manager ?

To verify the configuration of your

compute_log_manager

, you should review your Dagster instance's

dagster.yaml

configuration file. This file contains the settings for your Dagster instance, including the compute log manager configuration. Here's what you should do: 1. Locate your

dagster.yaml

file. By default, this file is located in the

DAGSTER_HOME

directory, which is an environment variable that points to the directory where Dagster stores instance-level configuration and data. 2. Open the

dagster.yaml

file and look for the

compute_logs

section. This section defines the configuration for the compute log manager. 3. Verify that the

compute_logs

section is correctly configured for the

GCSComputeLogManager

. The configuration should specify the GCS bucket and any other necessary settings. Here's an example of what the

compute_logs

section might look like for a GCS compute log manager:

yaml compute_logs:   module: dagster_gcp.gcs.compute_log_manager   class: GCSComputeLogManager   config:     bucket: "my-gcs-bucket-name"     local_dir: "/path/to/local/dir"     prefix: "my-log-prefix"

In this example, replace

"my-gcs-bucket-name"

with the actual name of your GCS bucket, and adjust the

local_dir

and

prefix

as needed. 4. If you're using environment variables to configure the

compute_log_manager

, ensure that these environment variables are set correctly in the environment where your Dagster instance is running. 5. After verifying or updating the configuration, save the

dagster.yaml

file and restart any Dagster processes (e.g., Dagit, Dagster daemon) to ensure that the changes take effect. 6. Ensure that the GCS bucket specified in the configuration exists and that the Dagster instance has the necessary permissions to access it. If you've verified the configuration and the issue persists, you may want to check the logs for any additional error messages that could provide more insight into the problem. Additionally, you can check the Dagster documentation for more details on configuring the

GCSComputeLogManager

and ensure that all required settings are correctly specified.

@Scout

####################################################################################################
# Compute Log Manager: Configuration for the compute log manager
####################################################################################################
computeLogManager:
  # Type can be one of [
  #   NoOpComputeLogManager,
  #   AzureBlobComputeLogManager,
  #   GCSComputeLogManager,
  #   S3ComputeLogManager,
  #   CustomComputeLogManager,
  # ]
  type: GCSComputeLogManager
  config:
    gcsComputeLogManager:
      bucket: automotive-dagster-compute-logs
      prefix: dagster-compute-logs

I see this in my yaml file

Based on the snippet you've provided from your

dagster.yaml

file, it looks like you have configured the

GCSComputeLogManager

to use a GCS bucket named

automotive-dagster-compute-logs

with a prefix of

dagster-compute-logs

for your compute logs. Here are a few steps to ensure that your configuration is correct and to troubleshoot any issues: 1. Bucket Name: Confirm that the bucket name

automotive-dagster-compute-logs

is correct and that the bucket exists in your Google Cloud Storage. 2. Permissions: Make sure that the service account or credentials used by Dagster have the necessary permissions to read from and write to the specified GCS bucket. 3. Prefix: The prefix

dagster-compute-logs

is used to organize logs within the bucket. Verify that this is the desired prefix and that it doesn't conflict with any other resources in the bucket. 4. Environment Variables: If your configuration relies on environment variables for credentials (not shown in the snippet), ensure that these are set correctly in the environment where Dagster is running. 5. Syntax: The configuration block should be correctly indented and formatted as YAML. The snippet you provided seems to have an extra level of nesting under

gcsComputeLogManager

, which might not be necessary. The documentation suggests a slightly different format:

yaml compute_logs:   module: dagster_gcp.gcs.compute_log_manager   class: GCSComputeLogManager   config:     bucket: "automotive-dagster-compute-logs"     prefix: "dagster-compute-logs"

6. Restart Dagster: After making any changes to the

dagster.yaml

file, you need to restart your Dagster processes to apply the new configuration. 7. Check for Errors: If you continue to encounter errors, check the Dagster logs for any error messages that could provide more insight into the problem. If you've gone through these steps and the issue persists, you may want to consult the Dagster documentation for more detailed information on configuring the

GCSComputeLogManager

or reach out to the Dagster community for further assistance.