dagster #announcements

Join the conversation Join Slack

announcements

Chris Roth

04/14/2020, 8:45 PM

also, how do i get the raw log output to work? i'm currently using sqs/celery and seeing nothing when i view raw logs

Muthu

04/14/2020, 9:39 PM

#general getting this error

ModuleNotFoundError: No module named 'dagster.serdes'

Muthu

04/14/2020, 9:40 PM

dagster --version dagster, version 0.7.4

Muthu

04/14/2020, 9:45 PM

DAGSTER_HOME=/opt/dc/workflow_engine/app /Users/muthu/miniconda3/envs/workflow/bin/dagit -f <http://dcpipes.py|dcpipes.py> -n define_repo -h 0.0.0.0 -p 3000
Traceback (most recent call last):
  File "/Users/muthu/miniconda3/envs/workflow/lib/python3.6/site-packages/dagster/core/serdes/__init__.py", line 185, in rehydrate
    module = importlib.import_module(self.module_name)
  File "/Users/muthu/miniconda3/envs/workflow/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/muthu/miniconda3/envs/workflow/lib/python3.6/site-packages/dagster_postgres/__init__.py", line 1, in <module>
    from .event_log import PostgresEventLogStorage
  File "/Users/muthu/miniconda3/envs/workflow/lib/python3.6/site-packages/dagster_postgres/event_log/__init__.py", line 1, in <module>
    from .event_log import PostgresEventLogStorage
  File "/Users/muthu/miniconda3/envs/workflow/lib/python3.6/site-packages/dagster_postgres/event_log/event_<http://log.py|log.py>", line 17, in <module>
    from dagster.serdes import (
ModuleNotFoundError: No module named 'dagster.serdes'

During handling of the above exception, another exception occurred:

Chris Roth

04/14/2020, 9:47 PM

Has anyone been able to get celery and SQS to work? It is creating a new queue for every run for me. I've tried changing backend between ampq and rpc but same thing happens either way. It seems like this is causing it to run on my AWS workers no matter what I do instead of my local celery worker

Muthu

04/14/2020, 10:21 PM

hi… thanks for your help… i’ve updated to 0.7.6… and connected event logs to Postgres… after triggering the pipeline, its not update the logs on the dagit immediately… it needs browser refresh

John Mav

04/15/2020, 4:09 PM

I’ve been following the examples here: https://dagster.readthedocs.io/en/stable/sections/deploying/local.html#dagit-in-docker to get Dagit running within a docker container. I have a simple pipeline that I am using to test the cron service that gets setup and everything seems like it’s running fine. However when I turn on the pipeline schedule from the schedules tab in the UI the pipeline never runs? The cron service is running and when I list the jobs from the crontab I do see the specific test job e.g.

root:/opt/dagster/app# crontab -l

* * * * * /var/shared/dagster/schedules/scripts/Repo.test_run.sh # dagster-schedule: Repo.test_run

I’m wondering what I am missing here so that turning “on” the schedule actually runs the job?

Chris Roth

04/16/2020, 5:52 PM

i noticed that the runs page in dagit takes a long time to load when i have a lot of runs - anyone else encountering this?

Chris Roth

04/16/2020, 6:00 PM

also, feature request - it would be nice if i could click on the tags in the runs page to add the tag to the filter search bar

Chris Roth

04/16/2020, 6:00 PM

having a hard time copying and pasting tags, search bar doesn't like pasting

Chris Roth

04/16/2020, 6:03 PM

also! it would be cool if i could see which celery worker each solid is running on

Chris Roth

04/16/2020, 7:17 PM

I'm noticing something mysterious. I have 4 celery workers set up with Redis and I'm running a pipeline with about 6 serial solids in it. I wrote the solids before setting up celery so they read and write a json file to and from disk instead of passing as an intermediate. But somehow the downstream solids are succeeding even on celery. How is this possible? It seems kind of like all of the solids in a pipeline are always getting run on the same worker

Chris Roth

04/16/2020, 8:18 PM

haha so many questions for you guys today! it's deploy day. i'm also setting a bunch of "This pipeline run has been marked as failed from outside the execution context"

Chris Roth

04/16/2020, 10:03 PM

every time i deploy dagit, the schedule gets set to "off". can this be setting get stored in postgres?

user

04/17/2020, 5:44 AM

Max Gasner just published a new version: 0.7.7.

max

04/17/2020, 5:49 AM

0_7_7_Release_Notes

🤘 3

😛artydagster: 3

:congadagster: 2

😛artywizard: 2

❤️ 1

🔥 7

Tobias Macey

04/17/2020, 5:36 PM

Working with types in Dagster, is there a defined type for the Dagster context object, or would that just be a regular

dict

type?

Chris Roth

04/17/2020, 11:19 PM

is there a way to pass config options from dagster-celery to celery? i'm getting

You're running the worker with superuser privileges

in docker. celery accepts

--uid

option but not

docker-celery

Chris Roth

04/18/2020, 12:31 AM

getting

dagster.core.errors.DagsterLaunchFailedError: Host <http://localhost:3000> failed sanity check. It is not a dagit server.

after upgrading to 0.7.7

Binh Pham

04/20/2020, 5:32 PM

is there any built-in watermarking in dagster? to do incremental runs

Chris Roth

04/21/2020, 3:16 PM

my scheduler isn't running 😞 it is scheduled for 3am but i don't see anything in the logs

Chris Roth

04/21/2020, 3:17 PM

it is enabled, and the entrypoint script is:

#!/bin/sh

# See: <https://unix.stackexchange.com/a/453053> - fixes inflated hard link count
touch /etc/crontab /etc/cron.*/*

service cron start

# Migrate dagster Postgres database
dagster instance migrate

# Add all schedules
dagster schedule up

# Restart previously running schedules
dagster schedule restart --restart-all-running 

dagit -h 0.0.0.0 -p 3000

Chris Roth

04/21/2020, 3:18 PM

Tobias Macey

04/21/2020, 4:34 PM

Working on the first stage of a pipeline and I believe I recall that Dagster doesn't currently have a means for running a fan-out/fan-in topology. The context being that I have a solid that generates a list of values and I want to execute the subsequent stages in sequence for each of those outputs. e.g. generate_list_of_strings fans out to do_something_with_string, do_something_else_with_string, etc. so that I have 5 solids running in parallel for item 1 in that list, and then repeat for each of the other strings.

Chris Roth

04/21/2020, 6:13 PM

i'm having an issue where one of my solids is getting stuck (not sure why, but i assume it's a hangup with my solid code and not a dagster bug), but it would be cool if there was a way to force terminate it so i could start it over

Chris Roth

04/21/2020, 7:16 PM

another question: sometimes i want to run solids in a certain order, but i don't want the input from the previous ones. is that possible, or is the only way to force them to run in series to have them use the input from the previous one?

Res Dev

04/21/2020, 8:21 PM

Anyone implemented reusable email alert plugin whenever a pipeline fails?

Res Dev

04/21/2020, 8:37 PM

I have SMTP env variables available, could write code to send email in case of exception in solid. But that means repeating that everywhere in every solid.

Res Dev

04/21/2020, 8:38 PM

I’m curious if we can detect “Failed” in any solid context in a pipeline and call a on_failed_callback with context error message.

Res Dev

04/21/2020, 8:38 PM

Similar to airflow