About

Right now psutil's C extensions only load in the main interpreter. Importing psutil into an isolated sub-interpreter fails:

>>> from concurrent import interpreters
>>> interp = interpreters.create()          # own GIL, isolated
>>> interp.exec("import psutil")          
...
ImportError: module psutil._psutil_linux does not support loading in subinterpreters

#2855 converted all C extension modules to PEP 489 multi-phase init (superseding the earlier attempt in #2577), but only the mechanical part. 2 things are still missing:

1. The custom C exceptions (ZombieProcessError, and on Windows TimeoutExpired / TimeoutAbandoned) are process-global and raised directly from arch/posix/proc.c, arch/osx/proc_utils.c and arch/windows/proc.c. For isolation each interpreter needs its own copy, so they should move into per-module state.

2. The modules don't declare Py_mod_multiple_interpreters, so they default to SUPPORTED Flipping this to PER_INTERPRETER_GIL_SUPPORTED is only safe once step 1 is done.

The ABI / wheel problem

Right now we ship a single cp38-abi3 wheel. Under that limited API the newer module slots (Py_mod_multiple_interpreters, Py_mod_gil, ...) aren't even defined, so:

• adding the slot unguarded doesn't compile (this already bit Adopt multi-phase init (PEP 489) #2577)
• we can't use #ifdef since it's evaluated against Python 3.8

So we either provide different wheels (for 3.8 and 3.12) or we do some magic at runtime instead of compile time.
I would prefer the "magic way" and provide only 3.8 wheels that work everywhere.

Idea: pick the slots at import time

The host sets m_slots after PyInit_* runs, after we detect the Python version at runtime. More or less (not tested):

// Added after 3.8; the limited API hides the macros, so hard-code them.
#ifndef Py_mod_multiple_interpreters
#  define Py_mod_multiple_interpreters 3
#  define Py_MOD_PER_INTERPRETER_GIL_SUPPORTED ((void *)2)
#endif

static PyModuleDef_Slot slots_main[] = {
    {Py_mod_exec, psutil_linux_exec},
#ifdef Py_mod_gil
    {Py_mod_gil, Py_MOD_GIL_NOT_USED},
#endif
    {0, NULL},
};

static PyModuleDef_Slot slots_subinterp[] = {
    {Py_mod_exec, psutil_linux_exec},
    {Py_mod_multiple_interpreters, Py_MOD_PER_INTERPRETER_GIL_SUPPORTED},
#ifdef Py_mod_gil
    {Py_mod_gil, Py_MOD_GIL_NOT_USED},
#endif
    {0, NULL},
};