https://dagster.io/ logo
#ask-community
Title
# ask-community
d

Daniel Galea

12/12/2022, 3:18 PM
Hi, I am writing unit tests to test a few Ops and I am getting this exception, but only when I am running two separate tests. If I comment out one test then I don't get this error. The exception is being ignored, so technically my tests pass but still I am assuming that something is wrong. Have you experiences this before?
Exception ignored in: <generator object build_resources at 0x7f5bfcd93df0>
Traceback (most recent call last):
File ".venv/lib/python3.10/site-packages/dagster/_core/execution/build_resources.py", line 114, in build_resources
list(resources_manager.generate_teardown_events())
File ".venv/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 533, in generate_teardown_events
yield from self.generator
File ".venv/lib/python3.10/site-packages/dagster/_core/execution/resources_init.py", line 279, in resource_initialization_event_generator
yield from manager.generate_teardown_events()
File ".venv/lib/python3.10/site-packages/dagster/_utils/__init__.py", line 533, in generate_teardown_events
yield from self.generator
File ".venv/lib/python3.10/site-packages/dagster/_core/execution/resources_init.py", line 351, in single_resource_event_generator
with user_code_error_boundary(DagsterResourceFunctionError, msg_fn, log_manager=context.log):
File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__
next(self.gen)
File ".venv/lib/python3.10/site-packages/dagster/_core/errors.py", line 184, in user_code_error_boundary
with raise_execution_interrupts():
File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__
next(self.gen)
File ".venv/lib/python3.10/site-packages/dagster/_core/errors.py", line 154, in raise_execution_interrupts
with raise_interrupts_as(DagsterExecutionInterruptedError):
File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__
next(self.gen)
File ".venv/lib/python3.10/site-packages/dagster/_utils/interrupts.py", line 90, in raise_interrupts_as
_replace_interrupt_signal(original_signal_handler)
File ".venv/lib/python3.10/site-packages/dagster/_utils/interrupts.py", line 19, in _replace_interrupt_signal
signal.signal(signal.SIGINT, new_signal_handler)
File "/usr/lib/python3.10/signal.py", line 56, in signal
handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object
For reference, these are my tests: I am using pytest and pytest-mock to write my tests. Test 1:
def test_launch_cluster_cluster_exists(mocker: MockerFixture):
emr_mock = mocker.patch("dagster_aws.emr.emr.EmrJobRunner")
emr_mock.run_job_flow.return_value = "CLUSTER-ID"
emr_mock.cluster_id_from_name.return_value = ""
with mocker.patch('dagster_aws.emr.emr.EmrJobRunner.cluster_id_from_name', side_effect=EmrError()):
context = build_op_context(
resources={
"emr_job_runner": emr_mock
},
config={
"emr_release_label": "emr-6.8.0",
"cluster_name": "TestCluster",
"master_node_instance_type": "m5.xlarge",
"worker_node_instance_type": "m5.xlarge",
"worker_node_instance_count": 2,
"ec2_subnet_id": "subnet-1293819381907",
"worker_node_spot_bid_price": "0.9"
}
)
cluster_id = launch_emr_cluster(context=context)
assert cluster_id == "CLUSTER-ID"
Test 2:
def test_launch_cluster_not_cluster_exists(mocker: MockerFixture):
emr_mock = mocker.patch("dagster_aws.emr.emr.EmrJobRunner")
emr_mock.run_job_flow.return_value = ""
emr_mock.cluster_id_from_name.return_value = "EXISTING-CLUSTER-ID"
context = build_op_context(
resources={
"emr_job_runner": emr_mock
},
config={
"emr_release_label": "emr-6.8.0",
"cluster_name": "TestCluster",
"master_node_instance_type": "m5.xlarge",
"worker_node_instance_type": "m5.xlarge",
"worker_node_instance_count": 2,
"ec2_subnet_id": "subnet-1293819381907",
"worker_node_spot_bid_price": "0.9"
}
)
cluster_id = launch_emr_cluster(context=context)
assert cluster_id == "EXISTING-CLUSTER-ID"