19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
dagster._core.errors.DagsterInvalidConfigError: Er...
# ask-communitya
Andres Crucetta
04/12/2023, 3:12 PMdagster._core.errors.DagsterInvalidConfigError: Error in config for resources
E Error 1: Post processing at path rootdb connconfig:user of original value {'env': 'SNOWFLAKE_USER'} failed:
E dagster._config.errors.PostProcessingError: You have attempted to fetch the environment variable "SNOWFLAKE_USER" which is not set. In order for this execution to succeed it must be set in this environment.
What's the best way to access resources? I'm conflicted on whether to use EnvVar or os.getenv(). I have .env set-up in my root directory but somehow it's unable to access it using the Pythonic config/resources.
Code in thread.
Andres Crucetta
04/12/2023, 3:12 PM Copy code
import pytest
from dagster import EnvVar, build_op_context
from orchestrator import DatabaseResource
from orchestrator.ops import load_sftp_to_snowflake, DataTransferConfig
@pytest.fixture
def data_config():
"""
Op configuration parameters.
"""
return DataTransferConfig(
raw_table="RAW_CERPASS_MAC",
azure_stage="STAGE_CERPASS_MAC",
file_format="PLAIN_TEXT"
)
@pytest.fixture
def db_resource():
"""
Snowflake database resource.
"""
return DatabaseResource(
user=EnvVar("SNOWFLAKE_USER"),
account=EnvVar("SNOWFLAKE_ACCOUNT"),
password=EnvVar("SNOWFLAKE_PASSWORD"),
warehouse=EnvVar("SNOWFLAKE_WAREHOUSE_NAME"),
)
def test_load_sftp_to_snowflake(data_config: DataTransferConfig,
db_resource: DatabaseResource):
"""
Test the copy into op.
"""
context = build_op_context()
result = load_sftp_to_snowflake(context, config=data_config, db_conn=db_resource)
assert result is TrueAndres Crucetta
04/12/2023, 3:12 PM Copy code
defs = Definitions(
sensors=all_sensors,
schedules=[upload_data_cerpass_mac],
jobs=BindResourcesToJobs([upload_data]),
resources={
"db_conn": DatabaseResource(
account=EnvVar("SNOWFLAKE_ACCOUNT"),
user=EnvVar("SNOWFLAKE_USER"),
password=EnvVar("SNOWFLAKE_PASSWORD"),
warehouse=EnvVar("SNOWFLAKE_WAREHOUSE_NAME"),
)
},
)c
Charlie Bini
04/12/2023, 8:10 PMyeah EnvVar isn't working for me either. It just literally plugs in the variable name. I think it's not implemented yet?
f
Frederik Löw
04/25/2023, 1:02 PMIt is the same for me.
25 Views