gh-144763: Don't detach the GIL in tracemalloc

[vstinner]

tracemalloc no longer detachs the GIL to acquire its internal lock.

• Issue: Crash in _io_TextIOWrapper_write_impl when running tracemalloc.clear_traces in threads #144763

[vstinner]

This change basically reverts commit 01157e0 (PR gh-139449) which fixed another deadlock on tracemalloc.stop() :-) See issue gh-139116.

cc @ZeroIntensity @encukou

[vstinner]

This change basically reverts commit 01157e0 (PR #139449) which fixed another deadlock on tracemalloc.stop() :-) See issue #139116.

I pushed a change to avoid the deadlock on tracemalloc.stop(): don't call PyRefTracer_SetTracer() while holding TABLES_LOCK().

[kumaraditya303]

I think the problem is that critical sections requires an active thread state and this code can be called with detached thread state iirc.

[vstinner]

tracemalloc_get_frame() is called with an attached thread state, see the caller traceback_get_frames() which has the code:

    PyThreadState *tstate = _PyThreadState_GET();
    assert(tstate != NULL);

Example of deadlock when running #144763 (comment) reproducer on a free-threaded build:

• Thread A: _PyCode_GetTLBC() => ... => tracemalloc_alloc(): TABLES_LOCK()
• Thread B: _PyTraceMalloc_TraceRef() => ... => tracemalloc_get_frame() => PyUnstable_InterpreterFrame_GetLine() => PyCode_Addr2Line(): Py_BEGIN_CRITICAL_SECTION(co)

Locks:

• Thread A is in a Py_BEGIN_CRITICAL_SECTION(co) critical section (_PyCode_GetTLBC()) and waits for TABLES_LOCK().
• Thread B has TABLES_LOCK() lock and waits for Py_BEGIN_CRITICAL_SECTION(co) critical section.

Thread A and thread B want to use Py_BEGIN_CRITICAL_SECTION(co) on the same code object (0x20000844810).

=> deadlock :-(

[ZeroIntensity]

Is it necessary to keep the attached thread state in TABLES_LOCK? It seems the bulk of the fix is just calling PyRefTracer_SetTracer without holding the lock.

Alternatively, we could just use Py_BEGIN_CRITICAL_SECTION_MUTEX instead of PyMutex_Lock.

[ZeroIntensity]

We should probably use "attached thread state" terminology here, since this only affects free-threaded builds.