Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/serious_python/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
## 4.3.6

* **Android:** PEP 734 subinterpreters (Python 3.14's `concurrent.interpreters` / `InterpreterPoolExecutor`) now work in built apps. Previously the main interpreter could import them, but every *subinterpreter* failed to import any relocated C extension (`ModuleNotFoundError: _struct` / `_interpqueues` / ...) — which broke the whole feature (its cross-interpreter transport pickles → `_struct`, and its queues need `_interpqueues`). The native-module finder lives on `sys.meta_path`, which is per-interpreter, and was installed only in the main interpreter. See `serious_python_android` 4.3.6.
* **Windows:** fix startup with non-ASCII app paths or environment values — Dart FFI strings (UTF-8) are now converted to UTF-16 before being passed to the Windows CRT, so paths/env values are no longer corrupted through the process ANSI code page, and Python UTF-8 mode is enabled before `Py_Initialize()`. See `serious_python_windows` 4.3.6 and flet-dev/flet#6641.
* Bundled python-build snapshot re-pinned to **20260720** (`dart_bridge` **1.5.0 → 1.5.1**, which carries the Windows fix above). Python versions (**3.12.13 / 3.13.14 / 3.14.6**) are unchanged.

Expand Down
1 change: 1 addition & 0 deletions src/serious_python_android/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
## 4.3.6

* **PEP 734 subinterpreters now work** (Python 3.14 `concurrent.interpreters` / `InterpreterPoolExecutor`) — enabling true multi-core CPU parallelism inside a single process, which `multiprocessing` cannot provide on Android. Previously the main interpreter imported the machinery fine, but every subinterpreter raised `ModuleNotFoundError` for `_struct` / `_interpqueues` / any other C extension, so `InterpreterPoolExecutor` and the low-level API were unusable. Cause: Android relocates C extensions (native-mmap packaging) and resolves them through `_SorefFinder` on `sys.meta_path`; `sys.meta_path` is per-interpreter, and `install()` only ran in the main interpreter, so a freshly created subinterpreter had a `meta_path` without the finder. Fix: `_sp_bootstrap.install()` now also wraps `concurrent.interpreters.create()` so every new interpreter installs the finder before use. The install runs via `Interpreter.exec()` — a source string, which is pickle-free (unlike `Interpreter.call()`), so it works *before* `_struct` is importable in the child. Because `InterpreterPoolExecutor` builds its workers via `concurrent.interpreters.create()`, the pool is fixed transparently with no user-code change. A no-op before 3.14; iOS/desktop were unaffected.
* Re-pins the bundled python-build snapshot to **20260720** (`dart_bridge` **1.5.0 → 1.5.1**). 1.5.1 is a Windows-only UTF-8 startup fix (see `serious_python_windows` 4.3.6); the Android runtime is functionally unchanged from 20260719.

## 4.3.4
Expand Down
63 changes: 62 additions & 1 deletion src/serious_python_android/python/_sp_bootstrap.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ def find_spec(self, fullname, path=None, target=None):
return spec


def install():
def _install_finder():
"""Insert the finder at the front of `sys.meta_path` (idempotent)."""
global _installed
if _installed:
Expand All @@ -158,3 +158,64 @@ def install():
return
sys.meta_path.insert(0, _SorefFinder())
_installed = True


def _patch_subinterpreters():
"""Make PEP 734 subinterpreters (Python 3.14+) able to import relocated
native modules.

`sys.meta_path` is *per-interpreter*: a subinterpreter created by
`concurrent.interpreters` / `InterpreterPoolExecutor` starts with a fresh
meta_path that does NOT contain the `_SorefFinder` installed by `_install_finder`,
so it can import no relocated C extension (`_struct`, `_interpqueues`, ...). Since
that machinery ships work between interpreters via pickle (`_struct`) and
cross-interpreter queues (`_interpqueues`), the whole feature is unusable.

Wrap `concurrent.interpreters.create()` so every new interpreter installs
the finder before it is used. The install runs via `Interpreter.exec()` — a
source string, which is pickle-free (unlike `Interpreter.call()`), so it
works *before* `_struct` is importable in the child. Idempotent; a no-op
before 3.14 (module absent) and cannot fix
`InterpreterPoolExecutor(initializer=...)` because the initializer is
delivered over the same broken pickle path — so it must run here, at
creation time.

NOTE: this relies on callers resolving `interpreters.create` as a module
attribute at call time (as `InterpreterPoolExecutor` does on 3.14) rather
than binding it via `from concurrent.interpreters import create`. If a
future CPython changes that, the patch silently stops applying to the pool
and subinterpreter imports regress — verified working on 3.14.6.
"""
try:
from concurrent import interpreters
except Exception:
return # < 3.14, or subinterpreters unavailable
if getattr(interpreters.create, "_sp_patched", False):
return
_orig_create = interpreters.create

def create(*args, **kwargs):
interp = _orig_create(*args, **kwargs)
try:
interp.exec("import _sp_bootstrap\n_sp_bootstrap.install()\n")
except Exception as e:
# Finder not installed in the child: it will hit the original
# ModuleNotFoundError on its first relocated import. Leave a
# breadcrumb (cf. the SP_BOOTSTRAP audit in `install`) instead of
# failing `create` outright.
sys.stderr.write("SP_BOOTSTRAP subinterpreter finder install failed: %r\n" % (e,))
return interp

create._sp_patched = True
interpreters.create = create


def install():
"""Entry point called from the dart-bridge Android bootstrap.

Installs the native-module finder in the current interpreter and — on
3.14+ — teaches every future subinterpreter to install it too.
Safe to call in any interpreter; idempotent.
"""
_install_finder()
_patch_subinterpreters()
Loading