I think there may be some issues with the dagster-bash solids. I am on 0.7.5. When I run a basic pipeline, using the bash_script_solid, I am seeing the solid executed on when loading the repository into dagit, as startup. My solid definition looks something like the following:

@pipleine
def my_pipeline():
    run_cmd_in_venv = bash_script_solid(
        '.../dagster_pipelines/hello_world.sh',
        name = 'hello'
    )

    execute_solid(
        run_cmd_in_venv
    )

the output on starting dagit is as follows:

Loading repository...
2020-04-03 13:29:16 - dagster - DEBUG - ephemeral_hello_solid_pipeline - ed74cc0e-4ab8-4f0d-ad3f-3a0839c2ab3c - PIPELINE_START - Started execution of pipeline "ephemeral_hello_solid_pipeline".
2020-04-03 13:29:16 - dagster - DEBUG - ephemeral_hello_solid_pipeline - ed74cc0e-4ab8-4f0d-ad3f-3a0839c2ab3c - ENGINE_EVENT - Executing steps in process (pid: 29701)
 event_specific_data = {"error": null, "marker_end": null, "marker_start": null, "metadata_entries": [["pid", null, ["29701"]], ["step_keys", null, ["['hello.compute']"]]]}
            step_key = "hello.compute"
2020-04-03 13:29:16 - dagster - DEBUG - ephemeral_hello_solid_pipeline - ed74cc0e-4ab8-4f0d-ad3f-3a0839c2ab3c - STEP_START - Started execution of step "hello.compute".
               solid = "hello"
    solid_definition = "hello"
            step_key = "hello.compute"
2020-04-03 13:29:16 - dagster - INFO - system - ed74cc0e-4ab8-4f0d-ad3f-3a0839c2ab3c - using temporary directory: /tmp
               solid = "hello"
    solid_definition = "hello"
            step_key = "hello.compute"
2020-04-03 13:29:16 - dagster - INFO - system - ed74cc0e-4ab8-4f0d-ad3f-3a0839c2ab3c - Temporary script location: /tmp/hellop5yj64ld
               solid = "hello"
    solid_definition = "hello"
            step_key = "hello.compute"
2020-04-03 13:29:16 - dagster - INFO - system - ed74cc0e-4ab8-4f0d-ad3f-3a0839c2ab3c - Running command: 
echo "Hello"
               solid = "hello"
    solid_definition = "hello"
            step_key = "hello.compute"
2020-04-03 13:29:16 - dagster - INFO - system - ed74cc0e-4ab8-4f0d-ad3f-3a0839c2ab3c - Hello

               solid = "hello"
    solid_definition = "hello"
            step_key = "hello.compute"
2020-04-03 13:29:16 - dagster - INFO - system - ed74cc0e-4ab8-4f0d-ad3f-3a0839c2ab3c - Command exited with return code 0

... you can see that it executes the solid, which is pretty undesirable. Am I doing something wrong?

hey Owen! I think you just need to change your code to something like:

@pipeline
def my_pipeline():
    run_cmd_in_venv = bash_script_solid(
        '.../dagster_pipelines/hello_world.sh',
        name = 'hello'
    )
    run_cmd_in_venv()

ah, okay. My bad. I'll try that. Thanks.

np! glad you flagged, we need to document this better; I filed https://github.com/dagster-io/dagster/issues/2345

I'm looking for a way to dynamically pass variables through to a bash solid at runtime. I can see how to do it statically via the environment. However, I wasn't able to work out how to do it dynamically. I tried defining an input definition, but that didn't seem to work with the bash solids. Could be that I am doing something wrong, should the input defs work with bash solids?

@Owen yeah you may need to customize the solid impl, right now the default bash solids are generated w/ a single

Nothing

input: https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-bash/dagster_bash/solids.py However, I just landed on master a change to extract the core bash solid compute functions to a separate utils module here: https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-bash/dagster_bash/utils.py so I think you could define your own solid with something like:

from dagster import solid, InputDefinition
from dagster_bash.utils import execute


@solid(input_defs=[InputDefinition('my_input', str)])
def my_solid(context, my_input):
    return execute('echo "{my_input}"'.format(my_input=my_input), output_logging='BUFFER', log=context.log)

(not tested, but that should work)

Thanks for the feedback Nate, I'll give that a whirl.

Sure thing! Keep us posted