https://dagster.io/ logo
#announcements
Title
# announcements
s

sephi

06/09/2020, 11:43 AM
Hi, We want to run a
bash_command_solid
inside a
composit_solid
- and need to set a dependancy between the output of the bash_command and another solid in the
pipeline
. We are not able to get an output from the bash_command_solid. What would be the correct approach for such task? While researching this issue - I'm trying to understand the bash_script_solid test example - can you get the return from the bash script? what does the following line do in the pipeline ?
a()
This is gist of the pipeline that we want to try and run:
Copy code
from dagster_bash import bash_script_solid
from dagster import file_relative_path, pipeline, composite_solid

@composite_solid()
def composite_bash():
    a = bash_script_solid(file_relative_path(__file__, 'hello_world.sh'), name='my_script')
    return a

@solid()
def print_string(str_print):
    print(str_print)

@composite_solid()
def composite_print(str_to_print):
    print_string(str_to_print)

@pipeline
def pip():
    a = composite_bash()
    composite_print(a)
But this is a non starter since there is an issue with the input /output of the
composit_solid
composite_solid composite_bash returned problematic value of type <class 'dagster.core.definitions.solid.SolidDefinition'>. Expected return value from invoked solid or dict mapping output name to return values from invoked solids
a

alex

06/09/2020, 1:30 PM
bash_script_solid
is actually a solid factory, so what may fix it for you is just adding
()
to invoke the solid
Copy code
a = bash_script_solid(file_relative_path(__file__, 'hello_world.sh'), name='my_script')()
s

sephi

06/09/2020, 1:46 PM
Thx @alex Now the sample code is running without any errors, but nothing is printed out. If I understand the code correctly there is no option to receive an output from
bash_script_solid
. Here is the updated code:
Copy code
from dagster_bash import bash_script_solid
from dagster import file_relative_path, pipeline, composite_solid, solid

@composite_solid()
def composite_bash()->str:
    a = bash_script_solid(file_relative_path(__file__, 'hello_world.sh'), name='my_script')()
    return a

@solid()
def print_string(context, str_print):
    print(str_print)

@composite_solid()
def composite_print(str_to_print):
    print_string(str_to_print)

@pipeline
def pip():
    a = composite_bash()
    composite_print(a)
If we still would need to set a dependency between composite_bash and composite_print - would you use
Nothing
or is there another option?
a

alex

06/09/2020, 2:16 PM
hmm the solid that the
bash_script_solid
returns returns its output, so what you have should work
how are you executing this?
you may want to switch from
print
to
<http://context.log.info|context.log.info>
if you want to see the entry in the event stream
s

sephi

06/09/2020, 3:04 PM
Thx! Now it works - I forgot to run
execute_pipeline
👍 1
Assuming we would want to log / materialize some of the processes . 1. how would you suggest to do so if we are using
bash_coommand_solid
? 2. should we go down the route that @Tobias Macey did and work directly with
execute
?
a

alex

06/10/2020, 1:12 PM
log / materialize some of the processes
I dont quite understand what you mean here, but I am guessing yes to use an approach of building the solid you need from the building blocks in the dagster-bash library
s

sephi

06/11/2020, 5:33 AM
thx
h

Harry Nguyen

10/10/2020, 4:13 AM
@alex is there anyway that we get the output from
print
I'm debugging code in another module which does not have access to context
a

alex

10/12/2020, 2:46 PM
you should be able to see it in stdout/stderr of where ever the process is running