Joshua Greenhalgh
02/23/2024, 4:58 PMfrom dagster import run_status_sensor, RunStatusSensorContext, DagsterRunStatus, EnvVar
from slack_sdk import WebClient
@run_status_sensor(run_status=DagsterRunStatus.SUCCESS,monitor_all_code_locations=True)
def slack_on_run_success(context: RunStatusSensorContext):
slack_client = WebClient(token=EnvVar("SLACK_TOKEN"))
slack_client.chat_postMessage(
channel=EnvVar("SLACK_CHANNEL"),
text=f'Job "{context.dagster_run.job_name}" succeeded.',
)
@run_status_sensor(run_status=DagsterRunStatus.FAILURE,monitor_all_code_locations=True)
def slack_on_run_failure(context: RunStatusSensorContext):
slack_client = WebClient(token=EnvVar("SLACK_TOKEN"))
slack_client.chat_postMessage(
channel=EnvVar("SLACK_CHANNEL"),
text=f'Job "{context.dagster_run.job_name}" failed.',
)
The EnvVar("foo") doesn't seem to be working - whats the way to acces env vars loaded into the config?Scout
02/23/2024, 4:58 PMEnvVar("SLACK_TOKEN")
and EnvVar("SLACK_CHANNEL")
to access environment variables. However, the EnvVar
class is not the correct way to access environment variables in this context. Instead, you should use the os.environ
dictionary provided by Python's standard library to access environment variables. Here's how you can modify your sensors to correctly access the environment variables for the Slack token and channel: python import os from dagster import run_status_sensor, RunStatusSensorContext, DagsterRunStatus from slack_sdk import WebClient @run_status_sensor(run_status=DagsterRunStatus.SUCCESS, monitor_all_code_locations=True) def slack_on_run_success(context: RunStatusSensorContext): slack_client = WebClient(token=os.environ["SLACK_TOKEN"]) slack_client.chat_postMessage( channel=os.environ["SLACK_CHANNEL"], text=f'Job "{context.dagster_run.job_name}" succeeded.', ) @run_status_sensor(run_status=DagsterRunStatus.FAILURE, monitor_all_code_locations=True) def slack_on_run_failure(context: RunStatusSensorContext): slack_client = WebClient(token=os.environ["SLACK_TOKEN"]) slack_client.chat_postMessage( channel=os.environ["SLACK_CHANNEL"], text=f'Job "{context.dagster_run.job_name}" failed.', )
In this corrected version, os.environ["SLACK_TOKEN"]
and os.environ["SLACK_CHANNEL"]
are used to retrieve the values of the SLACK_TOKEN
and SLACK_CHANNEL
environment variables, respectively. Make sure that these environment variables are set in the environment where your Dagster instance is running. The documentation provides examples of using os.environ
to access environment variables for similar use cases, which is the standard way to access environment variables in Python.Joshua Greenhalgh
02/23/2024, 4:59 PMScout
02/23/2024, 4:59 PMEnvVar
class in Dagster is specifically designed for use within Dagster's configuration system. It allows you to reference environment variables in your Dagster configuration YAML files or dictionaries, which are then resolved at runtime. This is particularly useful when you want to keep sensitive information, such as secrets or access tokens, out of your configuration files and instead reference them via environment variables. However, when writing Python code outside of the configuration system, such as in your sensor definitions, you should use the standard Python method for accessing environment variables, which is through the os.environ
dictionary. This is because EnvVar
is not meant to be used as a direct replacement for os.environ
in Python code; it is a Dagster-specific construct for configuration purposes. The documentation highlights the difference between using os.getenv
and EnvVar
within Dagster. When os.getenv
is used, the variable's value is retrieved when the code location is loaded and will be visible in the UI. In contrast, when EnvVar
is used within the configuration system, the variable's value is retrieved at runtime and won't be visible in the UI, providing additional security for sensitive values. In your sensor code, you are trying to access environment variables directly in Python code, not within a configuration file or dictionary. Therefore, you should use os.environ
to access the environment variables, as shown in the corrected code example provided earlier.