Local Executor test_execution[unlimited] seems to randomly fail

[potiuk]

Recently test_execution started to fail with:

FAILED tests/executors/test_local_executor.py::TestLocalExecutor::test_execution[unlimited] - Failed: Timeout >60.0s

Stacktrace:

_________________ TestLocalExecutor.test_execution[unlimited] __________________
tests/executors/test_local_executor.py:129: in test_execution
    self._test_execute(parallelism=parallelism)
/usr/local/lib/python3.10/unittest/mock.py:1379: in patched
    return func(*newargs, **newkeywargs)
tests/executors/test_local_executor.py:106: in _test_execute
    executor.end()
airflow/executors/local_executor.py:234: in end
    proc.join()
/usr/local/lib/python3.10/multiprocessing/process.py:149: in join
    res = self._popen.wait(timeout)
/usr/local/lib/python3.10/multiprocessing/popen_fork.py:43: in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
/usr/local/lib/python3.10/multiprocessing/popen_fork.py:27: in poll
    pid, sts = os.waitpid(self.pid, flag)
E   Failed: Timeout >60.0s
----------------------------- Captured stdout call -----------------------------
[2025-01-15T02:12:46.747+0000] {local_executor.py:63} INFO - Worker starting up pid=91
[2025-01-15T02:12:46.761+0000] {local_executor.py:63} INFO - Worker starting up pid=92
[2025-01-15T02:12:46.769+0000] {local_executor.py:99} ERROR - uhoh
Traceback (most recent call last):
  File "/opt/airflow/airflow/executors/local_executor.py", line 95, in _run_worker
    _execute_work(log, workload)
  File "/opt/airflow/airflow/executors/local_executor.py", line 116, in _execute_work
    supervise(
  File "/usr/local/lib/python3.10/unittest/mock.py", line 1114, in __call__
    return self._mock_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/unittest/mock.py", line 1118, in _mock_call
    return self._execute_mock_call(*args, **kwargs)
  File "/usr/local/lib/python3.10/unittest/mock.py", line 1179, in _execute_mock_call
    result = effect(*args, **kwargs)
  File "/opt/airflow/tests/executors/test_local_executor.py", line 74, in fake_supervise
    raise RuntimeError("fake failure")
RuntimeError: fake failure
[2025-01-15T02:12:46.769+0000] {local_executor.py:219} INFO - Shutting down LocalExecutor; waiting for running tasks to finish.  Signal again if you don't want to wait.
[2025-01-15T02:12:46.778+0000] {local_executor.py:63} INFO - Worker starting up pid=93

Example failure: https://github.com/apache/airflow/actions/runs/12779948905/job/35625787707#step:6:7344

[ajitg25]

@potiuk Tested this locally, looks like its working now

Ran breeze testing core-tests --run-in-parallel --run-db-tests-only

[potiuk]

Yeah. It works "in general" - just fails randomly (flaky test)