Hi Everyone! I would like to ask about dagster-snowflake library. Here is the documentation for this topic: https://docs.dagster.io/_apidocs/libraries/dagster-snowflake Here is the example code what I used for snowflake resource connection:

from dagster import job, op
from dagster_snowflake import snowflake_resource

@op(required_resource_keys={'snowflake'})
def get_one(context):
    context.resources.snowflake.execute_query('SELECT 1')

@job(resource_defs={'snowflake': snowflake_resource})
def my_snowflake_job():
    get_one()

my_snowflake_job.execute_in_process(
    run_config={
        'resources': {
            'snowflake': {
                'config': {
                    'account': {'env': 'SNOWFLAKE_ACCOUNT'},
                    'user': {'env': 'SNOWFLAKE_USER'},
                    'password': {'env': 'SNOWFLAKE_PASSWORD'},
                    'database': {'env': 'SNOWFLAKE_DATABASE'},
                    'schema': {'env': 'SNOWFLAKE_SCHEMA'},
                    'warehouse': {'env': 'SNOWFLAKE_WAREHOUSE'},
                }
            }
        }
    }
)

and I’d like to use private key as in this Snowflake documentation: https://docs.snowflake.com/en/user-guide/python-connector-example.html#label-python-key-pair-authn-rotation

import snowflake.connector
import os
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives.asymmetric import dsa
from cryptography.hazmat.primitives import serialization
with open("<path>/rsa_key.p8", "rb") as key:
    p_key= serialization.load_pem_private_key(
        key.read(),
        password=os.environ['PRIVATE_KEY_PASSPHRASE'].encode(),
        backend=default_backend()
    )

pkb = p_key.private_bytes(
    encoding=serialization.Encoding.DER,
    format=serialization.PrivateFormat.PKCS8,
    encryption_algorithm=serialization.NoEncryption())

ctx = snowflake.connector.connect(
    user='<user>',
    account='<account_identifier>',
    private_key=pkb,
    warehouse=WAREHOUSE,
    database=DATABASE,
    schema=SCHEMA
    )

cs = ctx.cursor()

but when I have tried to use in the above dagster code the private key parameter, I have got this error message:

dagster._core.errors.DagsterInvalidConfigError: Error in config for resource snowflake

Error 1: Received unexpected config entry "private_key" at path root:config.

Error 2: Missing required config entry "password" at path root:

Does the dagster-snowflake library ready for private key authentication? Thank your in advance for your response. @Laszlo Bencze

Hey @jamie - is this a topic you're knowledgeable on?

hey! we had a user contribute PR to add this, but it looks like it hasn’t been touched since I reviewed in early December. I think I can take over the PR and get it submitted. I’ll plan to do that this week, so it’ll likely be available in next week’s release

Hi @jamie, Thank you for your support. We are looking forward to seeing it.

ah that could be a mistake. We do a manual check that the password is not none and when i tested locally i did it with an encrypted private key so i wouldn’t have run into the issue. I can put up a fix that will remove the check. it’ll be out this week! thanks for your patience!

thanks @jamie

hey @jamie May I know that that fix has been released this week?

yes, it should be in the release that went out yesterday

okey, we will check it. thank you for quick fix.

Hi @jamie, I have wanted to update dagster-snowflake package (to 0.17.12) and I have got this error message:

ERROR: No matching distribution found for dagster==1.1.12

As I see on the https://pypi.org/project/dagster/ that there isn’t 1.1.12 version. Could you help on this?

looks like pypi is in a bad cache state right now. but dagster

1.1.11

and dagster-snowflake

0.17.11

should have the change. I think we are releasing .12 to fix an unrelated regression, and you can update to that once the release is done and pypi is back to normal. until then, the .11 versions should work for you

Hi @jamie, I would like to ask for your help. Because my code isn’t working still. I don’t have encrypted key-pair. I have private- and public key without any password or encryption. It is in my local machine. So when I use dagster-snowflake (0.17.12 or 0.17.13) and dagster (1.1.12 or 1.1.13) I am getting these errors: 1. When I don’t use `private_key_password`I get this:

TypeError: load_pem_private_key() missing 1 required positional argument: 'password'

2. When I use

private_key_password

with `None`value:

Error 1: Value at path root:config:private_key_password must not be None. Expected "(String | { env: String })"

3. When I use

private_key_password

with

'None'

value (as String):

TypeError: Password was given but private key is not encrypted.

Please help me how I can configure dagster-snowflake resource when there isn’t any private-key encryption or password. Thank you in advance.

i’m on a plane today but i have an idea for what’s going on. i may not be able to get to it right away, but i’ll keep you updated. in the meantime, what happens if you set the password to “” (empty string)?

Hi @jamie, If I set the password to empty string, I will get

The above exception was caused by the following exception:

TypeError: Password was given but private key is not encrypted.

error message.

ok thank you for the info! we’re on a company offsite this week, so ill be a bit slower in debugging this. i’ll keep you updated on my progress though

Hi @jamie, Do you have any new information about key-pair auth without password?

Hi! I’ve been looking at it a bit - basically deep in the call stack for one of the authentication calls, the

password

parameter is non-optional, so even if we pass a None from our end, it fails within third-party code. However, Snowflake claims that non-password key pairs should work, I just haven’t been able to find any documentation showing how to do key pair auth without the function that fails without a password

Hi Jamie, is there any news in this case?

hey - not much news unfortunately. Last i tried to figure this out i got pretty stuck. iirc the third party libraries used for authentication have a requirement that the password field be non empty and i struggled to find any documentation for how to use the python api to authenticate with non encrypted passwords.

@jamie hi Jamie, so when can we expect any solution here? or what can be done as a next step? thanks a lot, Edina