https://dagster.io/ logo
Join Slack
Powered by
Hi, We want to run a `bash_command_solid` inside ...
# announcements
s
Hi, We want to run a 
bash_command_solid
inside a 
composit_solid
- and need to set a dependancy between the output of the bash_command and another solid in the 
pipeline
. We are not able to get an output from the bash_command_solid. What would be the correct approach for such task? While researching this issue - I'm trying to understand the bash_script_solid test example - can you get the return from the bash script? what does the following line do in the pipeline ?
a()
This is gist of the pipeline that we want to try and run:
Copy code
from dagster_bash import bash_script_solid
from dagster import file_relative_path, pipeline, composite_solid

@composite_solid()
def composite_bash():
    a = bash_script_solid(file_relative_path(__file__, 'hello_world.sh'), name='my_script')
    return a

@solid()
def print_string(str_print):
    print(str_print)

@composite_solid()
def composite_print(str_to_print):
    print_string(str_to_print)

@pipeline
def pip():
    a = composite_bash()
    composite_print(a)
But this is a non starter since there is an issue with the input /output of the 
composit_solid
composite_solid composite_bash returned problematic value of type <class 'dagster.core.definitions.solid.SolidDefinition'>. Expected return value from invoked solid or dict mapping output name to return values from invoked solids
a
bash_script_solid
is actually a solid factory, so what may fix it for you is just adding 
()
to invoke the solid
Copy code
a = bash_script_solid(file_relative_path(__file__, 'hello_world.sh'), name='my_script')()
s
Thx @alex Now the sample code is running without any errors, but nothing is printed out. If I understand the code correctly there is no option to receive an output from 
bash_script_solid
. Here is the updated code:
Copy code
from dagster_bash import bash_script_solid
from dagster import file_relative_path, pipeline, composite_solid, solid

@composite_solid()
def composite_bash()->str:
    a = bash_script_solid(file_relative_path(__file__, 'hello_world.sh'), name='my_script')()
    return a

@solid()
def print_string(context, str_print):
    print(str_print)

@composite_solid()
def composite_print(str_to_print):
    print_string(str_to_print)

@pipeline
def pip():
    a = composite_bash()
    composite_print(a)
If we still would need to set a dependency between composite_bash and composite_print - would you use 
Nothing
or is there another option?
a
hmm the solid that the 
bash_script_solid
returns returns its output, so what you have should work
how are you executing this?
you may want to switch from 
print
to 
<http://context.log.info|context.log.info>
if you want to see the entry in the event stream
s
Thx! Now it works - I forgot to run 
execute_pipeline
👍 1
Assuming we would want to log / materialize some of the processes . 1. how would you suggest to do so if we are using 
bash_coommand_solid
? 2. should we go down the route that @Tobias Macey did and work directly with 
execute
?
a
log / materialize some of the processes
I dont quite understand what you mean here, but I am guessing yes to use an approach of building the solid you need from the building blocks in the dagster-bash library
s
thx
h
@alex is there anyway that we get the output from 
print
I'm debugging code in another module which does not have access to context
a
you should be able to see it in stdout/stderr of where ever the process is running
Open in Slack
PreviousNext
Powered by