Skip to content

[GR-60660] Multi context C extensions with copying #463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 61 commits into from
Dec 20, 2024
Merged

[GR-60660] Multi context C extensions with copying #463

merged 61 commits into from
Dec 20, 2024

Conversation

@graalvmbot
Copy link
Collaborator

@graalvmbot graalvmbot commented Dec 20, 2024

No description provided.

timfel added 30 commits December 19, 2024 08:35
@timfel
Allow C API to be loaded twice
172c1c7
@timfel
Make PythonGC and TraceNativeMemory engine options so statically shar…
eaf2029 
…ing them in the c api across contexts is safe
@timfel
Document why graalpy_finalizing is safe to share across contexts
71838fa
@timfel
Document and implement caveats around TRUFFLE_CONTEXT
ffaf04a
@timfel
Never store the tstate_current thread-local on the native side when m…
feea9ae 
…ultiple contexts are using the C API
@timfel
Make _Py_Identifier subinterpreter safe by caching the TruffleStrings…
d701477 
… globally
@timfel
Make testSharingWithCpythonSre a bit harder on the racyness
be6ea11
@timfel
allow only one context to initialise the C API at a time
456bd3c
@timfel
Document why initialize_graal_capi is fine to store into globals from…
5ba69aa 
… different contexts
@timfel
finer grained locks around c api initialisation
fb3d58a
@timfel
ensure our MultiContextTest also tests executing different context C …
f5ee2b7 
…API on the same thread later
@timfel
Undo changes to allow subinterpreters sharing libraries, but keep com…
0171f2a 
…ments to explain which parts could work
@timfel
Base implementation of library-copying strategy
24b11ec
@timfel
HACK: run patchelf to see if library isolation with copying works
04ce631
@timfel
Update documentation about native extensions
5cd4609
@timfel
WIP: Begin adapting Sulong's Elf reader to implement our shared objec…
5f3485f 
…t patching scheme
@timfel
Adapt Sulong's MachO parser code to modify MachO files
95f50b1
@timfel
Move some Python C API specific code from CExtContext to CApiContext
39ea1d4
@timfel
Delete code for ELF patching and rely on patchelf, the format has too…
f8031f0 
… many extensions for the maintenance headache
@timfel
Implement code to load relocated C API and C extension shared objects
119b8e8
@timfel
Add embedding tools API to prepare copied native libraries
f2eb326
@timfel
Add __graalpython__ APIs to get the soabi and capi library name
fee9d38
@timfel
Modify capi context to search relocated libraries from graalpy home f…
7a2ea78 
…rom the venv root
@timfel
Create junit test for copying + patching C extension files
fb2840f
@timfel
Use a shutdown hook to delete the temporary directories after patchin…
d29c088 
…g C ext files
@timfel
set a java system property for junit when mx is verbose
42e0dd6
@timfel
Expand test for native c ext relocation
45098e9
@timfel
Flip the default for IsolateNativeModules
c53192e
@timfel
Get process environment via Truffle API
c75a34c
@timfel
Move library relocation back into graalpython to be dynamic
c083f06
timfel added 26 commits December 19, 2024 08:39
@timfel
remove redundant test
05f8642
@timfel
Tweak error messages around sys.prefix
30f76fc
@timfel
Send output from patchelf to logger
cd92b6e
@timfel
Install required dependencies to patch cexts directly into testing venv
2bdd376
@timfel
reapply throwing an exception when llvm mode is needed
e0660dd
@timfel
Fix tests on Windows
4d039f8
@timfel
Use delvewheel to patch dlls
625e8f8
@timfel
Fix removal of temporary venv from test
8c2edbc
@timfel
Support defining other values for optional flags than 0
25feb80
@timfel
Formatting and style cleanup
f490f69
@timfel
Make MultiContextCExtTest work when run purely with maven surefire
a2d9b2e
@timfel
Move MultiContextCExtTest to run in its own process
486b54d
@timfel
Add changelog entry
9497899
@timfel
Fix copying c extensions on darwin
72ab5b6
@timfel
Give a report tag based on the type of junit tests we run
a23366e
@timfel
Make it clear that multi context works not only in jvm mode
09c454d
@timfel
More unique name for copied C libraries
376c67d
@timfel
Do not allow reuse of C api libraries because dlclose is not reliable
fb7b0a7
@timfel
Make SharedObjects AutoCloseable
77e9264
@timfel
Make maximum number of C API copies configurable
3cf7f35
@timfel
Fix rebase
ed9456e
@timfel
Fix style
d20468a
@timfel
Fix codesigning our modified dylibs on macos
9a1b2c9
@timfel
Support C ext copying on linux only for now
34584e4
@timfel
Clean up MultiContextCExtTest
11986fc
@timfel
Move details about C extension copying to IMPLEMENTATION_DETAILS
269c596
@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Dec 20, 2024
@graalvmbot graalvmbot closed this Dec 20, 2024
@graalvmbot graalvmbot deleted the tim/multi-context-cext-copying branch December 20, 2024 08:52
@graalvmbot graalvmbot merged commit fe73dc7 into master Dec 20, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@graalvmbot @timfel