dagster #announcements

announcements

Noah K

12/31/2020, 2:15 PM

grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
 status = StatusCode.UNKNOWN
 details = "Exception iterating responses: generator didn't yield"
 debug_error_string = "{"created":"@1609424052.213181825","description":"Error received from peer ipv4:10.0.6.6:9000","file":"src/core/lib/surface/call.cc","file_line":1062,"grpc_message":"Exception iterating responses: generator didn't yield","grpc_status":2}"

Noah K

12/31/2020, 2:15 PM

I had thought it was related to my code failing to load but now I'm not so sure?

Noah K

12/31/2020, 2:15 PM

Is there any hidden way to get logs out of the api daemon?

Noah K

12/31/2020, 2:39 PM

There we go, finally found it, syntax error in a library file 😄

Noah K

12/31/2020, 2:39 PM

(not a dagster library, one of mine)

Noah K

12/31/2020, 2:51 PM

Er, nope, same error from dagit.

Noah K

12/31/2020, 2:52 PM

gRPC server can definitely load the definitions code now

Noah K

12/31/2020, 3:02 PM

Aha, the Python grpc library is instrumented with logging

Noah K

12/31/2020, 3:03 PM

I should be able to convince that to activate I think?

Noah K

12/31/2020, 3:04 PM

Oh but it doesn't log any more than is sent back

Jai Kumaran

01/02/2021, 6:56 AM

Hey all, I have been pleasantly surprised at the level of abstractions available in dagster. I really like it, the main reason being that it embraces declarative and compositional patterns. Btw, I have been an early user of graphql since 2015 and that could be a reason why I like dagster too. I was wondering if there is an architectural view of dagster somewhere? What I am looking for is how the core packages tie together. I am pretty familiar with the JS ecosystem. I took a quick look at the repo and found dagit to be the react front end, made with create-react-app/apollo/etc. Dagster is the python module (?) what’s scala used for ? The graphQL server is within dagster or is there a separate server component? I am planning to use dagster for my startup, my use case would be different from the core (ML & ETL) use cases dagster is designed for. So looking at the architecture, I could probably customize my deployments.

Jai Kumaran

01/02/2021, 5:36 PM

Is there a way we can construct pipelines using the graphql API? Lets say I have written some solids and want to construct the pipeline using a yaml file or the graphql api?

Jai Kumaran

01/04/2021, 10:36 AM

I am getting an error, while trying the airline-demo example. Here’s the stack trace. I am using venv. My local postgres db is ok. The

pip list

is:

(env) Jais-MacBook-Pro:airline_demo jai$ pip list
Package                Version      Location
---------------------- ------------ ------------------------------------------------------
airline-demo           dev          /Users/jai/work/test-env/dagster/examples/airline_demo
alembic                1.4.3
certifi                2020.12.5
chardet                4.0.0
click                  7.1.2
coloredlogs            14.0
contextlib2            0.6.0.post1
croniter               0.3.37
dagster                0.9.22.post0
docstring-parser       0.7.1
funcsigs               1.0.2
future                 0.18.2
grpcio                 1.34.0
grpcio-health-checking 1.34.0
humanfriendly          9.1
idna                   2.10
Mako                   1.1.3
MarkupSafe             1.1.1
natsort                7.1.0
pendulum               1.4.4
pip                    20.1.1
protobuf               3.14.0
pyrsistent             0.17.3
python-dateutil        2.8.1
python-editor          1.0.4
pytz                   2020.5
pytzdata               2020.1
PyYAML                 5.3.1
requests               2.25.1
Rx                     1.6.1
setuptools             47.1.0
six                    1.15.0
SQLAlchemy             1.3.22
tabulate               0.8.7
toposort               1.6
tqdm                   4.55.1
tzlocal                1.5.1
urllib3                1.26.2
watchdog               1.0.2

Jai Kumaran

01/04/2021, 10:36 AM

(airline) Jais-MacBook-Pro:airline_demo jai$ dagit
/usr/local/lib/python3.8/site-packages/dagster/core/instance/config.py:22: UserWarning: The dagster instance configuration file (dagster.yaml) is not present at /Users/jai/work/dagstertut. Dagster uses this file to know where and how to store local artifacts, information about past runs, and structured events.
If nothing is specified, Dagster will store this information in the local filesystem in the /Users/jai/work/dagstertut directory.
  warnings.warn(
/usr/local/lib/python3.8/site-packages/dagster/cli/workspace/cli_target.py:101: UserWarning: You are automatically loading a "repository.yaml", a deprecated capability. This capability will be eliminated in 0.9.0.
  warnings.warn(
/usr/local/lib/python3.8/site-packages/dagster/cli/workspace/workspace.py:48: UserWarning: Error loading repository location airline_demo.repository:airline_demo_repo:(DagsterUserCodeProcessError) - dagster.core.errors.DagsterUserCodeProcessError: (ModuleNotFoundError) - ModuleNotFoundError: No module named 'airline_demo'

Stack Trace: 
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 343, in ListRepositories
    self._repository_symbols_and_code_pointers.loadable_repository_symbols,
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 105, in loadable_repository_symbols
    self.load()
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 95, in load
    self._loadable_repository_symbols = load_loadable_repository_symbols(
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 119, in load_loadable_repository_symbols
    loadable_targets = get_loadable_targets(
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/utils.py", line 29, in get_loadable_targets
    [LoadableTarget(attribute, load_def_in_module(module_name, attribute))]
  File "/usr/local/lib/python3.8/site-packages/dagster/core/definitions/reconstructable.py", line 420, in load_def_in_module
    return def_from_pointer(CodePointer.from_module(module_name, attribute))
  File "/usr/local/lib/python3.8/site-packages/dagster/core/definitions/reconstructable.py", line 432, in def_from_pointer
    target = pointer.load_target()
  File "/usr/local/lib/python3.8/site-packages/dagster/core/code_pointer.py", line 307, in load_target
    module = load_python_module(self.module, warn_only=True)
  File "/usr/local/lib/python3.8/site-packages/dagster/core/code_pointer.py", line 240, in load_python_module
    raise error
  File "/usr/local/lib/python3.8/site-packages/dagster/core/code_pointer.py", line 205, in load_python_module
    module = importlib.import_module(module_name)
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked


Stack Trace: 
  File "/usr/local/lib/python3.8/site-packages/dagster/cli/workspace/workspace.py", line 43, in _load_handle
    handle = RepositoryLocationHandle.create_from_repository_location_origin(origin)
  File "/usr/local/lib/python3.8/site-packages/dagster/core/host_representation/handle.py", line 47, in create_from_repository_location_origin
    return ManagedGrpcPythonEnvRepositoryLocationHandle(repo_location_origin)
  File "/usr/local/lib/python3.8/site-packages/dagster/core/host_representation/handle.py", line 188, in __init__
    list_repositories_response = sync_list_repositories_grpc(self.client)
  File "/usr/local/lib/python3.8/site-packages/dagster/api/list_repositories.py", line 16, in sync_list_repositories_grpc
    raise DagsterUserCodeProcessError(

  warnings.warn(
Loading repository...
Serving on <http://127.0.0.1:3000> in process 38959

aakash indurkhya

01/04/2021, 9:59 PM

hi guys - new to Dagster and just running the tutorials right now. I'm still evaluating whether Dagster is the right fit for a project i'm working on. My main question is the following: why does it take roughly 10 seconds between launching execution of a given pipeline and the actual start of pipeline running. AKA there are about 10 seconds where the state of the pipeline is "NOT STARTED"

aakash indurkhya

01/04/2021, 10:00 PM

happy to provide additional context as needed 🙂

Noah K

01/04/2021, 11:00 PM

Do I have to do anything special on https://github.com/dagster-io/dagster/pull/3489 to get the review site to notice it?

Noah K

01/04/2021, 11:01 PM

I didn't see anything in the contributions guide on actually opening the PR in a special way 🙂

Noah K

01/05/2021, 10:29 PM

@Ben Gotow Do most people not run Dagit as a permanent service?

Noah K

01/05/2021, 10:30 PM

(I'm happy to write the code for disabling raw HTML events if that is the only concern)

Adrian

01/07/2021, 10:03 PM

hi guys - I'm setting up Dagster using the helm_chart (using Terraforms helm_release provider) in the values.yml. I enabled ingress for Dagit and it generated the following the ingress in posted below in the thread. I've setup Nginx as the load balancer but it doesn't seem to be picking it up. What could I be missing?

Josh Karlin

01/08/2021, 6:01 PM

hi, is there a success/failure hook for a whole pipeline? i can only see them at the solid level

➕ 1

rex

01/09/2021, 1:35 AM

:1000: 🥳

🙌 1

mrdavidlaing

01/09/2021, 10:04 AM

Is there an ETA for the 0.10 release?

Alexis M

01/11/2021, 7:45 AM

Hi everyone, we are having this error regularly on the Dagit UI when trying to launch a pipeline run. To temporarly fix it, we trigger a fresh deployment of our dagster container but it re-appears after a certain undetermined time. We use the DefaultRunLauncher and we deploy Dagster in its version 0.9.19 in a Docker container hosted on a EC2 machine. What can we do to further investigate this ?

grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses"
debug_error_string = "{"created":"@1610350697.550486745","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":4142,"referenced_errors":[{"created":"@1610350697.550481483","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":397,"grpc_status":14}]}"
>
  File "/usr/local/lib/python3.7/site-packages/dagster_graphql/implementation/utils.py", line 14, in _fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/dagster_graphql/implementation/execution/launch_execution.py", line 13, in launch_pipeline_reexecution
    return _launch_pipeline_execution(graphene_info, execution_params, is_reexecuted=True)
  File "/usr/local/lib/python3.7/site-packages/dagster_graphql/implementation/execution/launch_execution.py", line 48, in _launch_pipeline_execution
    run = do_launch(graphene_info, execution_params, is_reexecuted)
  File "/usr/local/lib/python3.7/site-packages/dagster_graphql/implementation/execution/launch_execution.py", line 36, in do_launch
    pipeline_run = create_valid_pipeline_run(graphene_info, external_pipeline, execution_params)
  File "/usr/local/lib/python3.7/site-packages/dagster_graphql/implementation/execution/run_lifecycle.py", line 21, in create_valid_pipeline_run
    step_keys_to_execute=step_keys_to_execute,
  File "/usr/local/lib/python3.7/site-packages/dagster_graphql/implementation/external.py", line 97, in get_external_execution_plan_or_raise
    step_keys_to_execute=None,
  File "/usr/local/lib/python3.7/site-packages/dagster_graphql/implementation/context.py", line 121, in get_external_execution_plan
    step_keys_to_execute=step_keys_to_execute,
  File "/usr/local/lib/python3.7/site-packages/dagster/core/host_representation/repository_location.py", line 372, in get_external_execution_plan
    step_keys_to_execute=step_keys_to_execute,
  File "/usr/local/lib/python3.7/site-packages/dagster/api/snapshot_execution_plan.py", line 38, in sync_get_external_execution_plan_grpc
    pipeline_snapshot_id=pipeline_snapshot_id,
  File "/usr/local/lib/python3.7/site-packages/dagster/grpc/client.py", line 117, in execution_plan_snapshot
    execution_plan_snapshot_args
  File "/usr/local/lib/python3.7/site-packages/dagster/grpc/client.py", line 73, in _query
    response = getattr(stub, method)(request_type(**kwargs), timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 923, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/usr/local/lib/python3.7/site-packages/grpc/_channel.py", line 826, in _end_unary_response_blocking
    raise _InactiveRpcError(state)

mrdavidlaing

01/11/2021, 9:43 AM

Q: Do contributors need to sign a CLA?

Jarek Piotrowski

01/11/2021, 12:38 PM

Hello Dagster Community! I’m working on a new tool that solves the main problem we currently see in documentation: it is very far from the codebase! That is why we develop a tool that enables linking code with documentation, so that: • developers or technical writers responsible for Dagster documentation can get notifications on Github when a code that is linked to docs changes, • new contributors, while reading documentation, can quickly view source code connected to a given functionality so that they can quickly understand the codebase. Do you think that such a tool would be useful to you? More information and preview is on our website: https://www.hastydocs.com All feedback is very appreciated! Cheers, Jarek

❤️ 3

Ryan

01/11/2021, 4:30 PM

Hi all. Is there any way of applying the

@solid

decorator multiple times and having it "just work"? From what I can see it seems like subsequent applications override the first, or something else prevents them from working together. Use-case: we have certain config we want to apply to many solids which is standardised, but a bit verbose to type out every time (e.g. OutputDefinitions). We'd like to wrap it all into our own, standard decorator, which would apply some

@solid

config, but then we should also be able to add to that on a solid-by-solid basis when needed with an additional

@solid

decoration.

Andrew Herbst

01/11/2021, 5:00 PM

Hi all, does anyone know if Dagster supports executing Google Cloud Dataflow jobs, specifically those that are implemented in Scala? We have a substantial amount of code in that world and are interested in potentially using dagster as an orchestrator for those jobs.

Andy H

01/11/2021, 5:27 PM

Hello everyone, I can’t seem to find any documentation on this, so I am wondering if any of you know whether it is possible to specify the

run_id

in advance of executing a pipeline run launch?

Title

Andy H

01/11/2021, 5:27 PM

Hello everyone, I can’t seem to find any documentation on this, so I am wondering if any of you know whether it is possible to specify the

run_id

in advance of executing a pipeline run launch?

alex

01/11/2021, 5:30 PM

How are you doing the launch (dagit ui, graphql api, cli, …) ? I believe this is theoretically possible but the run_id argument may not be threaded everywhere correctly. This is a use case that doesn’t get much attention. Out of curiosity, why are you manually setting run id?

Andy H

01/11/2021, 5:32 PM

Hi Alex, we execute a pipeline launch by way of a scheduled image run. The pipeline runner grabs a task from a queue which already has an id. I’d love to be able to use this id as the run_id for the pipeline, because all of the logging calls will line up between the task id and the pipeline logging. It would be nice to not have to track a pipeline run id against a task id.

The literal execution of the pipeline is with

execute_launch_command

alex

01/11/2021, 5:41 PM

Ok cool, thanks for the context. That makes a lot of sense.

Andy H

01/11/2021, 5:43 PM

For sure, let me know if I can provide any more details, if that would be helpful

alex

01/11/2021, 5:48 PM

https://dagster.phacility.com/D5910 should go out Thursday in

0.10.0

👍 1

💯 1

Andy H

01/11/2021, 5:56 PM

Wow, thanks alex!!

sandy

01/11/2021, 6:44 PM

This maybe doesn't get you all the way there, but something you could do that might be helpful is set the task_id as a run tag on your sun

Andy H

01/11/2021, 7:23 PM

@sandy Thanks for the possible approach, I hadn't thought of that!

View count: 1