Sweet, I just signed up to literally ask the same question that has just been answered. Awesome guys, keep up the great work!

Two things I faced while setting up the dev environment:

1.

pytest python_modules/dagster/dagster_tests

was giving a lot of import module errors. Using

python -m pytest python_modules/dagster/dagster_tests

made it run successfully.

2. I had to separately install

sphinx-autobuild

to run

make livehtml

successfully

Hello folks ... Johnny from Colombia ... NO coke, No Narcos just ... Data

Hi folks, Greg from BrightHive here!

👋 hey Greg!

Hi All, Amanda from BrightHive!

Hi! Tom from BrightHive here.

I just looked up BrightHive, looks like a cool platform! Any interest in talking about it on the Data Engineering Podcast? (www.dataengineeringpodcast.com)

Hi there ! Amazing work you’ve done with Dagster. I’ll definitively try this out and look forward for further developments (and maybe contribute one day). Thanks guys ! I’ve seen that the Redshift integration is coming soon, is that correct ?

Hi Dagsters! I am very happy to see this project appear. There has long been a need for a sane development platform for data pipelines and I think Dagster looks like a very promising solution! I am currently trying to re-implement a pipeline, currently written in Snakemake, in Dagster. There are some tasks in this pipeline that are very computationally heavy and the DAG will be highly parallel, so I was thinking that Dask would be the ideal execution substrate for this pipeline. However, some of these tasks are extremely memory hungry and so we would like to put some resource constraints on the tasks, which should allow Dask to not schedule too many of these tasks at once. As far as I can tell, this is how one would do that in Dask: https://distributed.dask.org/en/latest/resources.html So my question is, if there is any way to define these constraints in Dagster and make sure they are propagated down to Dask? Or am I going about this in the wrong way?

👋 Hey there folks, John from HelloSign, we’re actually doing a full overhaul of our data infrastructure (migrating from cron jobs to Airflow) and we’re planning on utilizing Dagster for our data pipeline development. Very excited about this project and very excited to see how it evolves. I’m sure I’ll be popping into here with plenty of questions 😄

Hey guys. Got this error during

pip install dagster

to my current project

ERROR: dagster 0.5.5 has requirement pyyaml==4.2b1, but you'll have pyyaml 5.1.1 which is incompatible.

Hi everyone, I'm new to dagster and looking for an example for running independent solids in parallel. I appreciate your help in advance!

Hello! I am following the tutorial, it seems like it doesn't print the results out when I execute such a command "dagster pipeline execute -f actual_dag.py -n actual_dag_pipeline"

should it print the output on my terminal? or I can only access from UI?

@Selin apologies for the confusion. aware of this issue and we’ll actually be pushing a minor release during the day to address it and other concerns. In the meantime you can print or log manually to confirm that execution is proceeding as expected.

Thanks @schrockn. Clear!

Max Gasner just published a new version: 0.5.6.

Hello, all. I'm having trouble getting the dagit Hello World to work on my Windows conda installation. I've made a conda environment called dagster, then conda installed pip in the environment and pip installed dagster and dagit in the environment. I am able to successfully run

dagster pipeline execute -f hello_world.py -n hello_world_pipeline

in the environment, but running

dagit -f hello_world.py -n hello_world_pipeline

yields a missing command error:

'dagit' is not recognized as an internal or external command, operable program or batch file.

I've tried doing this in an Anaconda prompt as an administrator, and I've tried adding the

[conda path]/envs/dagster/Scripts

folder (which contains the

dagit-cli

windows executable) to PATH but this still yields the same error. I have also tried creating the env and pip-installing packages while in an administrator prompt, but get the same error. Thank you for your time.

Hello! Just getting started with dagster. If a pipeline fails midway, is there support for rolling back the changes? For example, if we create a hive external table in step 1 and the query fails on step 2 due to some data error in that table, do we have to build routines to drop the table from step 1? And if so, what would be the best way to set that up in dagster?

Hello from London!

<!channel> What's the difference between Context and Modes in Dagster?

Hi all! I’m new here and using Dagster for work. I opened an issue w/ composite solids, and I’m wondering if anyone saw it. I ended up redesigning the pipeline not to use composites (probably better that way since it was simpler than I thought) but I’m pretty sure there’s an issue with Input/Output mapping https://github.com/dagster-io/dagster/issues/1674

P.S. I love Dagster

Hey guys, I got this error when running

dagit

while following the airline demo tutorial. I did install the dagster_examples using the

pip install -e .

command.

ModuleNotFoundError: No module named 'dagster_examples'

Hey all, I’ve been reading through the docs and two strange things happened. First, that some functions as

Selector

and

input_configuration_schemas

aren’t available to be imported through dagster’s top module. I’ve installed dagster through

pip

with

v.0.5.6

. I was also following the respective tutorial for this version. The other thing is that, the master branch doesn’t reflect the latest version. I’ve noticed it while trying checking the code for

solid

decorator the API docs shows that there’s a field for passing configuration values as a dictionary. This available in the master’s branch codebase.