As per the documentation, I'm supposed to find conf files at /opt/dagster, but there is nothing there.

Hello, I'm on a read-only filesystem for deployment. Is there a means of completely shutting off telemetry so it doesn't try to reach for the event.log in logs? Or is there a means to put that directory in /tmp?

Hi, I have a strange problem. We are trying to run a pipeline from a jenkins job(

pipenv run dagster pipeline execute -n ...

), running the same command locally works fine but in the jenkins job all the INFO messages are missing, and so is the rest of stdout/stderr from other printouts etc.

Checking back in after being sidetracked from Dagster for a while. Great to see all these new faces and just wanted to say outstanding work on the recent releases ! That new UI looks killer ! Keep up the great work blob smile

I built a simple pipeline to apply an xgboost model to a new dataset and I'm getting issues with loading the pickled model object

_pickle.UnpicklingError: NEWOBJ class argument isn't a type object

Does anyone know why dagster isn't recognizing the object?

Is there documentation on a full list of options for config and what they mean? Like for resources (aws, spark), pipelines, solids etc?

prha just published a new version: 0.7.8.

@ramshackle-jamathon airflow dag autoingestion ready for use :-)

In one of the 0.7.x releases I noticed that the Great Expectations integration was removed due to drift in their API. Does anyone know what the plans are for reimplementing that integration, or just relying on their library directly?

👋🏻

feature request: it would be cool if there was a button in the runs page to terminate all started pipelines

because i have a bunch of runs that have gotten stuck in the started status from days ago and never finished. i think many of them essentially just timed out. i also forgot if i mentioned it before but it would be sweet to have a solid config param for a timeout, too

Hello! Our org has decided to build around dagster for our data pipeline needs and we are excited to be a part of the community. While creating some example pipelines for our users, we were a bit confused on best practice for structuring our

ExpectationResult

objects. As an example, we have a composite solid that verifies a number of values are within acceptable thresholds. Would it be preferred to do: 1. many

ExpectationResult

objects per solid, with one

JsonMetadataEntryData

metadata entry 2. one

ExpectationResult

object per solid, with many

JsonMetadataEntryData

metadata entries While trying to decide, we were also unsure on future uses of the different children of

EventMetadataEntry

(path, url. json, python attribute, markdown, text). Are there any set plans for using these? I think knowing that could perhaps help us better understand

i'm experimenting with a dynamically generated composite solid:

dagit is able to generate the pipeline successfully, but the "Start Execution" button in the playground is greyed out

nvm, i think i was just missing the inputs - but wasn't getting any errors

@Chris Roth the visual indicators for missing config have gotten subtler recently -- maybe too subtle

i have an open issue: https://github.com/dagster-io/dagster/issues/2410

question. a lot of my solids use the same set of configuration params. is it possible to have pipeline-level config params instead of having to re-define these configs for each solid or pass them from one solid to the next?

Hi there, bumping into an issue setting up a dagster instance with the cron scheduler. I'm seeing a path issue when running

dagster schedule up

I believe the import just needs to be switched to

from dagster.core.serdes import ConfigurableClass

in https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-cron/dagster_cron/cron_scheduler.py

@sashank @prha ^^

Happy to make a PR if its helpful, realize its a bit late in the day 😄

Sashank Thupukari just published a new version: 0.7.9.

@dwall ^--- note the run launcher changes

hi everyone, i'm looking at using dagster to clean up some of the data pipes (aka shell scripts) i use for instant domain search.