Hi there. I see in the API reference for the

dagster_aws.secretsmanager.secretsmanager_secrets_resource

definition that a

context

ref is passed to a job function in the example, however this doesn’t seem to be allowed at runtime, as I get the “reserved keyword” error. Are the docs incorrect, or am I perhaps missing something? I’m referring to this block in particular;

@job(resource_defs={'secrets': secretsmanager_secrets_resource})
def example_job(context):
    example_secretsmanager_secrets_op()
    example_secretsmanager_secrets_op_2()

Hi André. Is there a stacktrace that you can share?

Hi Phil, sure, here it is;

docker_example_user_code     | Traceback (most recent call last):
docker_example_user_code     |   File "/usr/local/bin/dagster", line 8, in <module>
docker_example_user_code     |     sys.exit(main())
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/cli/__init__.py", line 50, in main
docker_example_user_code     |     cli(auto_envvar_prefix=ENV_PREFIX)  # pylint:disable=E1123
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1130, in __call__
docker_example_user_code     |     return self.main(*args, **kwargs)
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1055, in main
docker_example_user_code     |     rv = self.invoke(ctx)
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
docker_example_user_code     |     return _process_result(sub_ctx.command.invoke(sub_ctx))
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1657, in invoke
docker_example_user_code     |     return _process_result(sub_ctx.command.invoke(sub_ctx))
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1404, in invoke
docker_example_user_code     |     return ctx.invoke(self.callback, **ctx.params)
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/click/core.py", line 760, in invoke
docker_example_user_code     |     return __callback(*args, **kwargs)
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/cli/api.py", line 571, in grpc_command
docker_example_user_code     |     container_context=json.loads(container_context) if container_context != None else None,
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/grpc/server.py", line 835, in __init__
docker_example_user_code     |     container_context=container_context,
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/grpc/server.py", line 213, in __init__
docker_example_user_code     |     loadable_target_origin, self._entry_point
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/grpc/server.py", line 99, in __init__
docker_example_user_code     |     loadable_target_origin.attribute,
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/grpc/utils.py", line 33, in get_loadable_targets
docker_example_user_code     |     else loadable_targets_from_python_file(python_file, working_directory)
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/core/workspace/autodiscovery.py", line 26, in loadable_targets_from_python_file
docker_example_user_code     |     loaded_module = load_python_file(python_file, working_directory)
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/core/code_pointer.py", line 86, in load_python_file
docker_example_user_code     |     return import_module_from_path(module_name, python_file)
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/seven/__init__.py", line 47, in import_module_from_path
docker_example_user_code     |     spec.loader.exec_module(module)
docker_example_user_code     |   File "<frozen importlib._bootstrap_external>", line 728, in exec_module
docker_example_user_code     |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
docker_example_user_code     |   File "repo.py", line 72, in <module>
docker_example_user_code     |     @job(resource_defs={'secrets': secretsmanager_secrets_resource})
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/core/definitions/decorators/job_decorator.py", line 70, in __call__
docker_example_user_code     |     config_mapping=None,
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/core/definitions/composition.py", line 1003, in do_composition
docker_example_user_code     |     exclude_nothing=False,
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/core/definitions/decorators/solid_decorator.py", line 404, in resolve_checked_solid_fn_inputs
docker_example_user_code     |     for inferred in inferred_props.values()
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/core/definitions/decorators/solid_decorator.py", line 405, in <genexpr>
docker_example_user_code     |     if inferred.name in inputs_to_infer
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/core/definitions/input.py", line 237, in create_from_inferred
docker_example_user_code     |     default_value=inferred.default_value,
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/core/definitions/input.py", line 88, in __init__
docker_example_user_code     |     self._name = check_valid_name(name) if name else None
docker_example_user_code     |   File "/usr/local/lib/python3.7/site-packages/dagster/core/definitions/utils.py", line 54, in check_valid_name
docker_example_user_code     |     f'"{name}" is not a valid name in Dagster. It conflicts with a Dagster or python reserved keyword.'
docker_example_user_code     | dagster.core.errors.DagsterInvalidDefinitionError: "context" is not a valid name in Dagster. It conflicts with a Dagster or python reserved keyword.
docker_example_user_code exited with code 1

Hi André. This is a bug in our example code… I think you should omit the

context

arg from the job and everything should work correctly:

@job(resource_defs={'secrets': secretsmanager_secrets_resource})
def example_job():
    example_secretsmanager_secrets_op()
    example_secretsmanager_secrets_op_2()

Thanks Phil, I appreciate your time and feedback. Just a last question related to your implementation of the

secretsmanager_secrets_resource

; I am able to pull the secrets from AWS’ Secrets Manager and can see via the logs that it’s being stored in the

/opt/dagster/dagster_home/storage/XX-XX-XX-XX-XX/OP_NAME/result

directory, however it’s not being added as an ENV var, even though I have

add_to_environment

set to

True

. The documentation does not give an indication of where to name the ENV var you want it to be saved to and it does not seem like it’s assigning a default name either?

cc @ben who might know this better: I think the secrets stored within secretsmanager are stored as environment variables (name/value) individually, within the scope of the resource. So if an

op

includes the secrets manager resource, then the op should have access to those individual secrets.

Yes, the environment variables should be made available in the scope of the resource, so the body of the op which requires the resource. You can also access them directly from the resource object.

Thanks for the feedback, guys. Much appreciated. I’ve been able to access the vars within the scope of the op.