From cf958edee23b4a753f75f2a1bb720678b736cc7e Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 12:28:04 -0700 Subject: [PATCH 01/41] Add an explanation about refcounts and GC. --- pep-0683.rst | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+) diff --git a/pep-0683.rst b/pep-0683.rst index 8afa1ac9faf..e91872b0943 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -349,11 +349,77 @@ Open Issues References ========== +Discussions +----------- + This was discussed in December 2021 on python-dev: * https://mail.python.org/archives/list/python-dev@python.org/thread/7O3FUA52QGTVDC6MDAV5WXKNFEDRK5D6/#TBTHSOI2XRWRO6WQOLUW3X7S5DUXFAOV * https://mail.python.org/archives/list/python-dev@python.org/thread/PNLBJBNIQDMG2YYGPBCTGOKOAVXRBJWY +Runtime Object State +-------------------- + +Here is the internal state that the CPython runtime keeps +for each Python object: + +* `PyObject.ob_refcnt`_: the object's `refcount `_ +* `_PyGC_Head`_: (optional) the object's node in a list of `"GC" objects `_ +* `_PyObject_HEAD_EXTRA`_: (optional) the object's node in the list of heap objects + +``ob_refcnt`` is part of the memory allocated for every object. +However, ``_PyObject_HEAD_EXTRA`` is allocated only if CPython was built +with ``Py_TRACE_REFS`` defined. ``PyGC_Head`` is allocated only if the +object's type has ``Py_TPFLAGS_HAVE_GC`` set. Typically this is only +container types (e.g. ``list``). Also note that ``PyObject.ob_refcnt`` +and ``_PyObject_HEAD_EXTRA`` are part of ``PyObject_HEAD``. + +.. _PyObject.ob_refcnt: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L107 +.. _PyGC_Head: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/internal/pycore_gc.h#L11-L20 +.. __PyObject_HEAD_EXTRA: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L68-L72 + +.. _refcounting: + +Reference Counting, with Cyclic Garbage Collection +-------------------------------------------------- + +Garbage collection is a memory management feature of some programming +languages. It means objects are cleaned up (e.g. memory freed) +once they are no longer used. + +Refcounting is one approach to garbage collection. The language runtime +tracks how many references are held to an object. When code takes +ownership of a reference to an object or releases it, the runtime +is notified and it increments or decrements the refcount accordingly. +When the refcount reaches 0, the runtime cleans up the object. + +With CPython, code must explicitly take or release references using +the C-API's ``Py_INCREF()`` and ``Py_DECREF()``. These macros happen +to directly modify the object's refcount (unfortunately, since that +causes ABI compatibility issues if we want to change our garbage +collection scheme). Also, when an object is cleaned up in CPython, +it also releases any references (and resources) it owns +(before it's memory is freed). + +Sometimes objects may be involved in reference cycles, e.g. where +object A holds a reference to object B and object B holds a reference +to object A. Consequently, neither object would ever be cleaned up +even if no other references were held (i.e. a memory leak). The +most common objects involved in cycles are containers. + +CPython has dedicated machinery to deal with reference cycles, which +we call the "cyclic garbage collector", or often just +"garbage collector" or "GC". Don't let the name confuse you. +It only deals with breaking reference cycles. + +See the docs for a more detailed explanation of refcounting +and cyclic garbage collection: + +* https://docs.python.org/3.11/c-api/intro.html#reference-counts +* https://docs.python.org/3.11/c-api/refcounting.html +* https://docs.python.org/3.11/c-api/typeobj.html#c.PyObject.ob_refcnt +* https://docs.python.org/3.11/c-api/gcsupport.html + Copyright ========= From 2d8f2f840094011a9860dfa5258cd9a997d088f8 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 13:08:02 -0700 Subject: [PATCH 02/41] Rewrite the abstract. --- pep-0683.rst | 41 +++++++++++++++++++++++------------------ 1 file changed, 23 insertions(+), 18 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index e91872b0943..8199dd6cf19 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -14,24 +14,29 @@ Resolution: Abstract ======== -Under this proposal, any object may be marked as immortal. -"Immortal" means the object will never be cleaned up (at least until -runtime finalization). Specifically, the `refcount`_ for an immortal -object is set to a sentinel value, and that refcount is never changed -by ``Py_INCREF()``, ``Py_DECREF()``, or ``Py_SET_REFCNT()``. -For immortal containers, the ``PyGC_Head`` is never -changed by the garbage collector. - -Avoiding changes to the refcount is an essential part of this -proposal. For what we call "immutable" objects, it makes them -truly immutable. As described further below, this allows us -to avoid performance penalties in scenarios that -would otherwise be prohibitive. - -This proposal is CPython-specific and, effectively, describes -internal implementation details. - -.. _refcount: https://docs.python.org/3.11/c-api/intro.html#reference-counts +Currently the CPython runtime maintains a +`small amount of mutable state `_ on each object. +Because of this, otherwise immutable objects are actually mutable. This +can have a large negative impact on CPU and memory performance, +especially for approaches to increasing Python's scalability. +The solution proposed here provides a way to mark an object +as one for which that per-object runtime state +should not change. + +Specifically, if an object's refcount matches a specific value then +that object is treated as "immortal". If an object is immortal then +its refcount will never be modified by ``Py_INCREF()``, etc. +Consequently, the refcount will never reach 0, so that object will +never be cleaned up (unless explicitly done, e.g. during runtime +finalization). Additionally, all other per-object runtime state +for an immortal object will be considered immutable. This approach +has some possible negative impact, which is explained below, +along with mitigations. The fundamental improvement is +that now an object can be truly immutable. + +This proposal is meant to be CPython-specific and to affect only +internal implementation details. There are some slight exceptions +to that which are explained below. Motivation From 1117b6b6ddf7e4b4babf46c93e12b4ddc13283a9 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 13:14:28 -0700 Subject: [PATCH 03/41] Tweak the abstract. --- pep-0683.rst | 37 +++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 8199dd6cf19..93272844bd2 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -15,28 +15,29 @@ Abstract ======== Currently the CPython runtime maintains a -`small amount of mutable state `_ on each object. -Because of this, otherwise immutable objects are actually mutable. This -can have a large negative impact on CPU and memory performance, -especially for approaches to increasing Python's scalability. -The solution proposed here provides a way to mark an object -as one for which that per-object runtime state -should not change. - -Specifically, if an object's refcount matches a specific value then -that object is treated as "immortal". If an object is immortal then -its refcount will never be modified by ``Py_INCREF()``, etc. -Consequently, the refcount will never reach 0, so that object will +`small amount of mutable state `_ in the +allocated memory of each object. Because of this, otherwise immutable +objects are actually mutable. This can have a large negative impact +on CPU and memory performance, especially for approaches to increasing +Python's scalability. The solution proposed here provides a way +to mark an object as one for which that per-object +runtime state should not change. + +Specifically, if an object's refcount matches a very specific value +(defined below) then that object is treated as "immortal". If an object +is immortal then its refcount will never be modified by ``Py_INCREF()``, +etc. Consequently, the refcount will never reach 0, so that object will never be cleaned up (unless explicitly done, e.g. during runtime finalization). Additionally, all other per-object runtime state -for an immortal object will be considered immutable. This approach -has some possible negative impact, which is explained below, -along with mitigations. The fundamental improvement is -that now an object can be truly immutable. +for an immortal object will be considered immutable. -This proposal is meant to be CPython-specific and to affect only +This approach has some possible negative impact, which is explained +below, along with mitigations. The fundamental improvement is that +now an object can be truly immutable. + +(This proposal is meant to be CPython-specific and to affect only internal implementation details. There are some slight exceptions -to that which are explained below. +to that which are explained below.) Motivation From 827c9a52483c19d8f2d461a9e5932d5415fb4820 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 14:04:39 -0700 Subject: [PATCH 04/41] Update the Motivation section. --- pep-0683.rst | 39 ++++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 93272844bd2..e4b37d024ff 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -43,18 +43,20 @@ to that which are explained below.) Motivation ========== -Without immortal objects, all objects are effectively mutable. That -includes "immutable" objects like ``None`` and ``str`` instances. -This is because every object's refcount is frequently modified -as it is used during execution. In addition, for containers -the runtime may modify the object's ``PyGC_Head``. These -runtime-internal state currently prevent -full immutability. - -This has a concrete impact on active projects in the Python community. -Below we describe several ways in which refcount modification has -a real negative effect on those projects. None of that would -happen for objects that are truly immutable. +As noted above, currently all objects are effectively mutable. That +includes "immutable" objects like ``str`` instances. This is because +every object's refcount is frequently modified as the object is used +during execution. This is especially significant for a number of +commonly used global (builtin) objects, e.g. ``None``. Such objects +are used a lot, both in Python code and internally. That adds up to +a consistent high volume of refcount changes. + +The effective mutability of all Python objects has a concrete impact +on parts of the Python community, e.g. projects that aim for +scalability like Instragram or the effort to make the GIL +per-interpreter. Below we describe several ways in which refcount +modification has a real negative effect on such projects. +None of that would happen for objects that are truly immutable. Reducing Cache Invalidation --------------------------- @@ -79,15 +81,14 @@ Avoiding Data Races Speaking of multi-core, we are considering making the GIL a per-interpreter lock, which would enable true multi-core parallelism. Among other things, the GIL currently protects against races between -multiple threads that concurrently incref or decref. Without a shared -GIL, two running interpreters could not safely share any objects, -even otherwise immutable ones like ``None``. +multiple concurrent threads that may incref or decref the same object. +Without a shared GIL, two running interpreters could not safely share +any objects, even otherwise immutable ones like ``None``. This means that, to have a per-interpreter GIL, each interpreter must -have its own copy of *every* object, including the singletons and -static types. We have a viable strategy for that but it will -require a meaningful amount of extra effort and extra -complexity. +have its own copy of *every* object. That includes the singletons and +static types. We have a viable strategy for that but it will require +a meaningful amount of extra effort and extra complexity. The alternative is to ensure that all shared objects are truly immutable. There would be no races because there would be no modification. This From ac2e3fbb3822e1afc5489c6b8222a7040a0f6ebc Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 14:29:15 -0700 Subject: [PATCH 05/41] Mention mmap. --- pep-0683.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/pep-0683.rst b/pep-0683.rst index e4b37d024ff..f5ae30e5d39 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -109,7 +109,9 @@ refcount semantics drastically reduce the benefits and has led to some sub-optimal workarounds. Also note that "fork" isn't the only operating system mechanism -that uses copy-on-write semantics. +that uses copy-on-write semantics. Anything that uses ``mmap`` +relies on copy-on-write, including sharing data from shared objects +files between processes. Rationale From 525717ea14667096ed46e1a1a6fdebad5c4d7c64 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 14:29:46 -0700 Subject: [PATCH 06/41] Note possible changes. --- pep-0683.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/pep-0683.rst b/pep-0683.rst index f5ae30e5d39..16d61c7f14d 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -310,6 +310,12 @@ accessible on the runtime state, e.g. in a ``_PyRuntimeState`` or ``PyInterpreterState`` field. We may need to add a tracking mechanism to the runtime state for a small number of objects. +Possible Changes +---------------- + +* mark every interned string as immortal +* mark the "interned" dict as immortal if shared else share all interned strings + Documentation ------------- From c643a65203b15a6aba58fcec56f874fdca095047 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 14:39:41 -0700 Subject: [PATCH 07/41] Clarify what "containers" means. --- pep-0683.rst | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 16d61c7f14d..2bcdfe056d8 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -217,7 +217,8 @@ objects post-init and that code will be in one place. Non-Obvious Consequences ------------------------ -* immortal containers effectively immortalize each contained item +* immortal objects that hold references to other objects + ("containers") effectively immortalize each contained item * the same is true for objects held internally by other objects (e.g. ``PyTypeObject.tp_subclasses``) * an immortal object's type is effectively immortal @@ -235,7 +236,7 @@ The approach involves these fundamental changes: * update ``Py_INCREF()`` and ``Py_DECREF()`` to no-op for objects with the magic refcount (or its most significant bit) * do the same for any other API that modifies the refcount -* stop modifying ``PyGC_Head`` for immortal containers +* stop modifying ``PyGC_Head`` for immortal GC objects ("containers") * ensure that all immortal objects are cleaned up during runtime finalization @@ -291,18 +292,18 @@ Object Cleanup In order to clean up all immortal objects during runtime finalization, we must keep track of them. -For container objects we'll leverage the GC's permanent generation by -pushing all immortalized containers there. During runtime shutdown, the -strategy will be to first let the runtime try to do its best effort of -deallocating these instances normally. Most of the module deallocation -will now be handled by pylifecycle.c:finalize_modules which cleans up -the remaining modules as best as we can. It will change which modules -are available during __del__ but that's already defined as undefined -behavior by the docs. Optionally, we could do some topological disorder -to guarantee that user modules will be deallocated first before the -stdlib modules. Finally, anything leftover (if any) can be found -through the permanent generation gc list which we can clear after -finalize_modules. +For GC objects ("containers") we'll leverage the GC's permanent +generation by pushing all immortalized containers there. During +runtime shutdown, the strategy will be to first let the runtime try +to do its best effort of deallocating these instances normally. Most +of the module deallocation will now be handled by +``pylifecycle.c:finalize_modules()`` which cleans up the remaining +modules as best as we can. It will change which modules are available +during __del__ but that's already defined as undefined behavior by the +docs. Optionally, we could do some topological disorder to guarantee +that user modules will be deallocated first before the stdlib modules. +Finally, anything leftover (if any) can be found through the permanent +generation gc list which we can clear after finalize_modules(). For non-container objects, the tracking approach will vary on a case-by-case basis. In nearly every case, each such object is directly From 723bd676ac1f9b0c8eebddd14af4785e28d7e811 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 14:41:18 -0700 Subject: [PATCH 08/41] cache -> CPU cache. --- pep-0683.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 2bcdfe056d8..1a08a30058b 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -58,10 +58,10 @@ per-interpreter. Below we describe several ways in which refcount modification has a real negative effect on such projects. None of that would happen for objects that are truly immutable. -Reducing Cache Invalidation ---------------------------- +Reducing CPU Cache Invalidation +------------------------------- -Every modification of a refcount causes the corresponding cache +Every modification of a refcount causes the corresponding CPU cache line to be invalidated. This has a number of effects. For one, the write must be propagated to other cache levels From 1fe6725506480475844c92b684bbce7172c9a201 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 14:43:45 -0700 Subject: [PATCH 09/41] Indicate who reached conclusions. --- pep-0683.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 1a08a30058b..01277cca32d 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -117,10 +117,11 @@ files between processes. Rationale ========= -The proposed solution is obvious enough that two people came to the -same conclusion (and implementation, more or less) independently. -Other designs were also considered. Several possibilities -have also been discussed on python-dev in past years. +The proposed solution is obvious enough that both of this proposal's +authors came to the same conclusion (and implementation, more or less) +independently. Other designs were also considered. Several +possibilities have also been discussed on python-dev +in past years. Alternatives include: From 79697e47ef6eddf2d3ccbfeb577692c333598f56 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 14:58:19 -0700 Subject: [PATCH 10/41] Note more possible changes. --- pep-0683.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pep-0683.rst b/pep-0683.rst index 01277cca32d..9842d545ef5 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -317,6 +317,8 @@ Possible Changes * mark every interned string as immortal * mark the "interned" dict as immortal if shared else share all interned strings +* (Larry,MvL) mark all constants unmarshalled for a module as immortal +* (Larry,MvL) allocate (immutable) immortal objects in their own memory page(s) Documentation ------------- From 4534dcc63198187d8c8d7a7d5374e9e7480e33b2 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 15:01:09 -0700 Subject: [PATCH 11/41] Mention Pyston immortal objects. --- pep-0683.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 9842d545ef5..65dddde13e5 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -119,9 +119,9 @@ Rationale The proposed solution is obvious enough that both of this proposal's authors came to the same conclusion (and implementation, more or less) -independently. Other designs were also considered. Several -possibilities have also been discussed on python-dev -in past years. +independently. The Pyston project `uses a similar approach `_. +Other designs were also considered. Several possibilities have also +been discussed on python-dev in past years. Alternatives include: @@ -141,6 +141,8 @@ add some hacks to the C-API to preserve compatibility for the may objects exposed there. The story is much, much simpler with immortal objects +.. _pyston: https://mail.python.org/archives/list/python-dev@python.org/message/TPLEYDCXFQ4AMTW6F6OQFINSIFYBRFCR/ + Impact ====== From 34b3e884f8ba1594553126882f0ee16dd89a2906 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 15:06:52 -0700 Subject: [PATCH 12/41] Note more possible changes. --- pep-0683.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 65dddde13e5..163ba85eb4b 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -141,8 +141,6 @@ add some hacks to the C-API to preserve compatibility for the may objects exposed there. The story is much, much simpler with immortal objects -.. _pyston: https://mail.python.org/archives/list/python-dev@python.org/message/TPLEYDCXFQ4AMTW6F6OQFINSIFYBRFCR/ - Impact ====== @@ -321,6 +319,9 @@ Possible Changes * mark the "interned" dict as immortal if shared else share all interned strings * (Larry,MvL) mark all constants unmarshalled for a module as immortal * (Larry,MvL) allocate (immutable) immortal objects in their own memory page(s) +* drop refcount operations in code where we know the object is immortal + (e.g. ``Py_RETURN_NONE``) +* specialize for immortal objects in the eval loop (`pyston`_) Documentation ------------- @@ -370,6 +371,8 @@ Open Issues References ========== +.. _pyston: https://mail.python.org/archives/list/python-dev@python.org/message/TPLEYDCXFQ4AMTW6F6OQFINSIFYBRFCR/ + Discussions ----------- From 70dabb15b07860e72c38a5ac1ca5b161e8f1ad71 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 15:19:43 -0700 Subject: [PATCH 13/41] Start a Performance Regression Mitigation section. --- pep-0683.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/pep-0683.rst b/pep-0683.rst index 163ba85eb4b..ff536e18ed0 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -312,6 +312,17 @@ accessible on the runtime state, e.g. in a ``_PyRuntimeState`` or ``PyInterpreterState`` field. We may need to add a tracking mechanism to the runtime state for a small number of objects. +Performance Regression Mitigation +--------------------------------- + +In the interest of clarify, here are some of the ways we are going +to try to recover some of the lost performance: + +* ... + +Note that these are not part of the proposal. They are included here +for clarity. + Possible Changes ---------------- From 4a9398e98782989bc114a2b214137f6bd0686e84 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 15:46:57 -0700 Subject: [PATCH 14/41] Add notes from python-dev thread. --- pep-0683.rst | 309 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 309 insertions(+) diff --git a/pep-0683.rst b/pep-0683.rst index ff536e18ed0..436b734585d 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -39,6 +39,60 @@ now an object can be truly immutable. internal implementation details. There are some slight exceptions to that which are explained below.) +.. + > I think that is a naïve statement. Refcounting is + > implementation-specific, but it's hardly an *internal* detail. + + Sorry for any confusion. I didn't mean to say that refcounting is an + internal detail. Rather, I was talking about how the proposed change + in refcounting behavior doesn't affect any guaranteed/documented + behavior, hence "internal". + + >> There is + >> code that targets CPython specifically, and relies on the details. + > + > Could you elaborate? Do you mean such code relies on specific refcount values? + > + >> The refcount has public getters and setters, + > + > Agreed. However, what behavior do users expect and what guarantees do + > we make? Do we indicate how to interpret the refcount value they + > receive? What are the use cases under which a user would set an + > object's refcount to a specific value? Are users setting the refcount + > of objects they did not create? + + That's what I hoped the PEP would tell me. Instead of simply claiming + that there won't be issues, it should explain why we won't have any issues. + + >> and you need a pretty good + >> grasp of the concept to write a C extension. + > + > I would not expect this to be affected by this PEP, except in cases + > where users are checking/modifying refcounts for objects they did not + > create (since none of their objects will be immortal). + > + >> I think that it's safe to assume that this will break people's code, + > + > Do you have some use case in mind, or an example? From my perspective + > I'm having a hard time seeing what this proposed change would break. + + IMO, the reasoning should start from the assumption that things will + break, and explain why they won't (or why the breakage is acceptable). + If the PEP simply tells me upfront that things will be OK, I have a hard + time trusting it. + + IOW, it's clear you've thought about this a lot (especially after + reading your replies here), but it's not clear from the PEP. + That might be editorial nitpicking, if it wasn't for the fact that I + want find any gaps in your research and reasoning, and invite everyone + else to look for them as well. + + >> and this PEP should convince us that the breakage is worth it rather than + >> dismiss the issue. + > + > Sorry, I didn't mean to be dismissive. I agree that if there is + > breakage this PEP must address it. + Motivation ========== @@ -75,6 +129,25 @@ This is true even for otherwise immutable objects like ``True``, ``0``, and ``str`` instances. This is also true even with the GIL, though the impact is smaller. +.. + > This looks out of context. Python has a per-process GIL. It should it go + > after the next section. + + This isn't about a data race. I'm talking about how if an object is + active in two different threads (on distinct cores) then incref/decref + in one thread will invalidate the cache (line) in the other thread. + The only impact of the GIL in this case is that the two threads aren't + running simultaneously and the cache invalidation on the idle thread + has less impact. + +.. + > This is also true even with the GIL, though the impact is smaller. + + Smaller than what? The baseline for that comparison is a hypothetical + GIL-less interpreter, which is only introduced in the next section. + Perhaps say something like "Python's GIL helps avoid this effect, but + doesn't eliminate it." + Avoiding Data Races ------------------- @@ -97,6 +170,28 @@ otherwise immutable objects. With immortal objects, support for a per-interpreter GIL becomes much simpler. +.. + >>> Weren't you planning a PEP on subinterpreter GIL as well? Do you want to + >>> submit them together? + >> + >> IMO, as it is, the PEP's motivation doesn't really stand on its own. + >> It's only worth it as a step towards per-interpreter GIL. + >> + >> I'd have to think about that. The other PEP I'm writing for + >> per-interpreter GIL doesn't require immortal objects. They just + >> simplify a number of things. That's my motivation for writing this + >> PEP, in fact. :) + > + > Please think about it. + > If you removed the benefits for per-interpreter GIL, the motivation + > section would be reduced to is memory savings for fork/CoW. (And lots of + > performance improvements that are great in theory but sum up to a 4% loss.) + + Sounds good. Would this involve more than a note at the top of the PEP? + + And just to be clear, I don't think the fate of a per-interpreter GIL + PEP should not depend on this one. + Avoiding Copy-on-Write ---------------------- @@ -113,6 +208,35 @@ that uses copy-on-write semantics. Anything that uses ``mmap`` relies on copy-on-write, including sharing data from shared objects files between processes. +.. + > Anyway, I don't believe stopping refcounting will fix the CoW issue + > yet. See this article [1] again. + > + > [1] https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172 + + That's definitely an important point, given that the main objective of + the proposal is to allow disabling mutation of runtime-internal object + state so that some objects can be made truly immutable. + + I'm sure Eddie has some good insight on the matter (and may have even + been involved in writing that article). Eddie? + + > Note that they failed to fix CoW by stopping refcounting code objects! (*) + > Most CoW was caused by cyclic GC and finalization caused most CoW. + + That's a good observation! + + > (*) It is not surprising to me because eval loop don't incre/decref + > most code attributes. They borrow reference from the code object. + + +1 + + > So we need a sample application and profile it, before saying it fixes CoW. + > Could you provide some data, or drop the CoW issue from this PEP until + > it is proved? + + We'll look into that. + Rationale ========= @@ -186,6 +310,46 @@ the performance benefits of immortal objects. However, the high bit of the refcount will still match ``_Py_IMMORTAL_REFCNT`` so we can still identify such objects as immortal. +.. + >> So, any extension that uses the stable ABI will break an invariant. + >> What'll be the impact? The total refcount will probably go out of sync, + >> anything else? + > + > The impact would be: objects incref/decref'ed by such a module would + > be exposed to some of the performance penalties described earlier in + > the PEP. I expect the potential aggregate cost would be relatively + > small. + > + >> If an extension DECREFs an immortal object, will it still match + >> _Py_IMMORTAL_REFCNT? How is that guaranteed? + > + > It wouldn't match _Py_IMMORTAL_REFCNT, but the high bit of + > _Py_IMMORTAL_REFCNT would still match. That bit is what we would + > actually be checking, rather than the full value. + + It makes sense once you know _Py_IMMORTAL_REFCNT has two bits set. Maybe + it'd be good to note that detail -- it's an internal detail, but crucial + for making things safe. + + >> What about extensions compiled with Python 3.11 (with this PEP) that use + >> an older version of the stable ABI, and thus should be compatible with + >> 3.2+? Will they use the old versions of the macros? How will that be tested? + > + > It wouldn't matter unless an object's refcount reached + > _Py_IMMORTAL_REFCNT, at which point incref/decref would start + > noop'ing. What is the likelihood (in real code) that an object's + > refcount would grow that far? Even then, would such an object ever be + > expected to go back to 0 (and be dealloc'ed)? Otherwise the point is + > moot. + + That's exactly the questions I'd hope the PEP to answer. I could + estimate that likelihood myself, but I'd really rather just check your + work ;) + + (Hm, maybe I couldn't even estimate this myself. The PEP doesn't say + what the value of _Py_IMMORTAL_REFCNT is, and in the ref implementation + a comment says "This can be safely changed to a smaller value".) + No user-facing behavior changes, with the following exceptions: * code that inspects the refcount (e.g. ``sys.getrefcount()`` @@ -195,11 +359,34 @@ No user-facing behavior changes, with the following exceptions: Neither should cause a problem. +.. + > Importantly, our system allows for the reference count of immortal objects to change, as long as it doesn't go below half of the original very-high value. So extension code with no concept of immortality will still update the reference counts of immortal objects, but this is fine. Because of this we haven't seen any issues with extension modules. + + As Guido noted, we are taking a similar approach for the sake of older + extensions built with the limited API. As a precaution, we start the + refcount for immortal objects basically at _Py_IMMORTAL_REFCNT * 1.5. + Then we only need to check the high bit of _Py_IMMORTAL_REFCNT to see + if an object is immortal. + + > + > The small amount of compatibility challenges we've run into have been in testing code that checks for memory leaks. For example this code breaks on Pyston: + > + > [snip] + > + > This might work with this PEP, but we've also seen code that asserts that the refcount increases by a specific value, which I believe wouldn't. + > + > For Pyston we've simply disabled these tests, figuring that our users still have CPython to test on. Personally I consider this breakage to be small, but I hadn't seen anyone mention the potential usage of sys.getrefcount() so I thought I'd bring it up. + Alternate Python Implementations -------------------------------- This proposal is CPython-specific. +.. + IMO it's specific to the C API, which is wider than just CPython. I + don't think we can just assume it'll have no impact on other + implementations. + Security Implications --------------------- @@ -222,11 +409,57 @@ Non-Obvious Consequences ("containers") effectively immortalize each contained item * the same is true for objects held internally by other objects (e.g. ``PyTypeObject.tp_subclasses``) + +.. + > So, do immortal lists immortalize values append()ed to them? (Can you + > even have an immortal list? Are there limits on what can be immortal?) + + We have no plans to do more than ever explicitly immortalize objects. + So an immortal list is fine but it would have no effect on the + immortality of items it contains, other than implicitly (since the + list holds a reference to each item). + + In general, it would be best to only immortalize immutable objects. + If we want to share any objects shared between threads without + protection (e.g. per-interpreter GIL) then such objects must be + immortal and immutable. So lists and dicts, etc. couldn't be shared + (assuming we can't prevent further mutation). + + However, for objects that will never be shared, it can be practical to + make some of them immortal too. For example, sys.modules is a + per-interpreter dict that we do not expect to ever get freed until the + corresponding interpreter is finalized. By making it immortal, we no + longer incur the extra overhead during incref/decref. + + We can apply this idea in the pursuit of getting back some of that 4% + performance we lost. At the end of runtime init we can mark *all* + objects as immortal and avoid the extra cost in incref/decref. We + only need to worry about immutability with objects that we plan on + sharing between threads without a GIL. + + (FYI, we still need to look closely at the impact of this approach on GC.) + * an immortal object's type is effectively immortal + +.. + > Should this be enforced? + + There is nothing to enforce. The object holds a reference to its type + so the type will never be cleaned up as long as the immortal object + isn't. Hence the type of an immortal object is effectively immortal. + We don't need the type to actually be marked as immortal. + * though extremely unlikely (and technically hard), any object could be incref'ed enough to reach ``_Py_IMMORTAL_REFCNT`` and then be treated as immortal +.. + > What would it take? + + Basically, you;d have to do it deliberately (e.g. incref the object in + a tight loop). Even with a tight loop it would take a long time to + count up to 2^60 or whatever the chosen value is. + Specification ============= @@ -257,6 +490,32 @@ adding (publicly accessible) private API to do things like immortalize an object or tell if one is immortal. +.. + > This is a public change. + + I agree that the change to the implementation of some public API is + certainly public, as is the change in behavior for immortal objects, + as is the potential <4% performance regression. By "public feature" I + was referring to immortal objects. We are not exposing that to users, + other than that they might notice some objects now have a really high + refcount that does not change. + + > Py_INCREF increments the reference count. + > Py_REFCNT gets the reference count. + > For immortal objects, Py_INCREF will no longer function as documented in + > 3.10, and Py_REFCNT can be used to witness it. Both are public API. + + You are right that "Increment the reference count for object o." (as + documented) will not be true for an immortal object. Instead it would + be something like "indicate that there is an additional reference for + object o". I'll be sure to update the PEP, to add that change to the + docs wording. + + Regardless, how important is that distinction? If it matters then + clearly this proposal needs to change. As an exercise, we can + consider one of the most used objects, None, and that we would make it + immortal. How would that impact users of Py_INCREF() and Py_REFCNT()? + Affected API ------------ @@ -287,6 +546,27 @@ The following objects will be made immortal: There will likely be others we have not enumerated here. +.. + > How will the candidates be chosen? + + Any objects that we would expect to share globally (ergo otherwise + immutable) will be made immortal. That means the static types, the + builtin singletons, the objects in _PyRuntimeState.global_objects, + etc. + +.. + > Should the intern dict be belonging to runtime, or (sub)interpreter? + > + > If the interned dict is belonging to runtime, all interned dict should + > be immortal to be shared between subinterpreters. + + Excellent questions. Making immutable objects immortal is relatively + simple. For the most part, mutable objects should not be shared + between interpreters without protection (e.g. the GIL). The interned + dict isn't exposed to Python code or the C-API, so there's less risk, + but it still wouldn't work without cleverness. So it should be + per-interpreter. It would be nice if it were global though. :) + Object Cleanup -------------- @@ -334,6 +614,18 @@ Possible Changes (e.g. ``Py_RETURN_NONE``) * specialize for immortal objects in the eval loop (`pyston`_) +.. + I suggest being a little more explicit (even blatant) that the particular details of: + + (1) which subset of functionally immortal objects are marked as immortal + (2) how to mark something as immortal + (3) how to recognize something as immortal + (4) which memory-management activities are skipped or modified for immortal objects + + are not only Cpython-specific, but are also private implementation details that are expected to change in subsequent versions. + + Ideally, things like the interned string dictionary or the constants from a pyc file will be not merely immortal, but stored in an immortal-only memory page, so that they won't be flushed or CoW-ed when a nearby non-immortal object is modified. Getting those details right will make a difference to performance, and you don't want to be locked in to the first draft. + Documentation ------------- @@ -364,6 +656,23 @@ modified again. Since we aren't enforcing any immutability for immortal objects it didn't make sense to emphasis that relationship. +.. + > Is it just not helpful, or is it disallowed? + + It is not disallowed. + + Also, I need to clarify that section since there are cases where + making a mutable object immortal can provide performance benefits, as + described earlier. + + > What about __subclasses__/tp_subclasses? + + That's one we'll have to deal with specially, e.g. for core static + types we'd store the object on PyInterpreterState. Then the + __subclasses__ getter would do a lookup on the current interpreter, + instead of using tp_subclasses. We could get rid of tp_subclasses or + perhaps use it only for the main interpreter. + Reference Implementation ======================== From 9d1483f82d0cdee04a61ec2b35d9ae4777efbd18 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 15:50:04 -0700 Subject: [PATCH 15/41] Add a Constraints section. --- pep-0683.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/pep-0683.rst b/pep-0683.rst index 436b734585d..91e37e64316 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -592,6 +592,15 @@ accessible on the runtime state, e.g. in a ``_PyRuntimeState`` or ``PyInterpreterState`` field. We may need to add a tracking mechanism to the runtime state for a small number of objects. +Constraints +----------- + +* ensure that otherwise immutable objects can be truly immutable +* do not hurt performance for normal Python use cases +* be careful when immortalizing objects that we don't actually expect + to persist until runtime finalization. +* be careful when immortalizing objects that are not otherwise immutable + Performance Regression Mitigation --------------------------------- From 307251d5d89f9ccb466538f2b89d0592ee0e960f Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 16:34:49 -0700 Subject: [PATCH 16/41] Drop a blank line. --- pep-0683.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/pep-0683.rst b/pep-0683.rst index 91e37e64316..d0481aefb32 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -283,7 +283,6 @@ applications to scale like never before. This is because they can then leverage multi-core parallelism without a tradeoff in memory usage. This is reflected in most of the above cases. - Performance ----------- From e416748225e149692e534202b07a96ea7eb38c94 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 16:37:03 -0700 Subject: [PATCH 17/41] Refer to the mitigation section. --- pep-0683.rst | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/pep-0683.rst b/pep-0683.rst index d0481aefb32..d3f6a4a56e7 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -288,7 +288,8 @@ Performance A naive implementation shows `a 4% slowdown`_. Several promising mitigation strategies will be pursued in the effort -to bring it closer to performance-neutral. +to bring it closer to performance-neutral. See the `mitigation`_ +section below. On the positive side, immortal objects save a significant amount of memory when used with a pre-fork model. Also, immortal objects provide @@ -600,6 +601,8 @@ Constraints to persist until runtime finalization. * be careful when immortalizing objects that are not otherwise immutable +.. mitigation: + Performance Regression Mitigation --------------------------------- From 08fd2a6fe25dc7a4a8f1d0e3a9aea69da10c7242 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 17:07:01 -0700 Subject: [PATCH 18/41] Be specific about _Py_IMMORTAL_REFCNT. --- pep-0683.rst | 53 +++++++++++++++++++++++++--------------------------- 1 file changed, 25 insertions(+), 28 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index d3f6a4a56e7..6554bd22020 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -307,30 +307,12 @@ is changing. The approach is also compatible with extensions compiled to the stable ABI. Unfortunately, they will modify the refcount and invalidate all the performance benefits of immortal objects. However, the high bit -of the refcount will still match ``_Py_IMMORTAL_REFCNT`` so we can -still identify such objects as immortal. +of the refcount `will still match _Py_IMMORTAL_REFCNT <_Py_IMMORTAL_REFCNT_>`_ +so we can still identify such objects as immortal. At worst, objects +in that situation would feel the effects described in the `Motivation`_ +section. Even then the overall impact is unlikely to be significant. .. - >> So, any extension that uses the stable ABI will break an invariant. - >> What'll be the impact? The total refcount will probably go out of sync, - >> anything else? - > - > The impact would be: objects incref/decref'ed by such a module would - > be exposed to some of the performance penalties described earlier in - > the PEP. I expect the potential aggregate cost would be relatively - > small. - > - >> If an extension DECREFs an immortal object, will it still match - >> _Py_IMMORTAL_REFCNT? How is that guaranteed? - > - > It wouldn't match _Py_IMMORTAL_REFCNT, but the high bit of - > _Py_IMMORTAL_REFCNT would still match. That bit is what we would - > actually be checking, rather than the full value. - - It makes sense once you know _Py_IMMORTAL_REFCNT has two bits set. Maybe - it'd be good to note that detail -- it's an internal detail, but crucial - for making things safe. - >> What about extensions compiled with Python 3.11 (with this PEP) that use >> an older version of the stable ABI, and thus should be compatible with >> 3.2+? Will they use the old versions of the macros? How will that be tested? @@ -458,7 +440,7 @@ Non-Obvious Consequences Basically, you;d have to do it deliberately (e.g. incref the object in a tight loop). Even with a tight loop it would take a long time to - count up to 2^60 or whatever the chosen value is. + count up to 2^61 or whatever the chosen value is. Specification @@ -466,7 +448,7 @@ Specification The approach involves these fundamental changes: -* add ``_Py_IMMORTAL_REFCNT`` (the magic value) to the internal C-API +* add `_Py_IMMORTAL_REFCNT`_ (the magic value) to the internal C-API * update ``Py_INCREF()`` and ``Py_DECREF()`` to no-op for objects with the magic refcount (or its most significant bit) * do the same for any other API that modifies the refcount @@ -477,10 +459,6 @@ The approach involves these fundamental changes: Then setting any object's refcount to ``_Py_IMMORTAL_REFCNT`` makes it immortal. -To be clear, we will likely use the most-significant bit of -``_Py_IMMORTAL_REFCNT`` to tell if an object is immortal, rather -than comparing with ``_Py_IMMORTAL_REFCNT`` directly. - (There are other minor, internal changes which are not described here.) This is not meant to be a public feature but rather an internal one. @@ -516,6 +494,25 @@ is immortal. consider one of the most used objects, None, and that we would make it immortal. How would that impact users of Py_INCREF() and Py_REFCNT()? +_Py_IMMORTAL_REFCNT +------------------- + +We will add two internal constants:: + #define _Py_IMMORTAL_BIT (1LL << (8 * sizeof(Py_ssize_t) - 4)) + #define _Py_IMMORTAL_REFCNT (_Py_IMMORTAL_BIT + (_Py_IMMORTAL_BIT / 2)) + +The refcount for immortal objects will be set to ``_Py_IMMORTAL_REFCNT``. +However, to check if an object is immortal we will compare its refcount +against just the bit:: + (op->ob_refcnt & _Py_IMMORTAL_BIT) != 0 + +The difference means that an immortal object will still be considered +immortal, even if somehow its refcount were modified (e.g. by an older +stable ABI extension). + +Note that top two bits of the refcount are already reserved for other +uses. That's why we are using the third top-most bit. + Affected API ------------ From ea5ec46f919883d4068fb6a8c36e1abd7d42e6bd Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 17:08:40 -0700 Subject: [PATCH 19/41] Move the Constraints section up. --- pep-0683.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 6554bd22020..34a74969aeb 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -494,6 +494,15 @@ is immortal. consider one of the most used objects, None, and that we would make it immortal. How would that impact users of Py_INCREF() and Py_REFCNT()? +Constraints +----------- + +* ensure that otherwise immutable objects can be truly immutable +* do not hurt performance for normal Python use cases +* be careful when immortalizing objects that we don't actually expect + to persist until runtime finalization. +* be careful when immortalizing objects that are not otherwise immutable + _Py_IMMORTAL_REFCNT ------------------- @@ -589,15 +598,6 @@ accessible on the runtime state, e.g. in a ``_PyRuntimeState`` or ``PyInterpreterState`` field. We may need to add a tracking mechanism to the runtime state for a small number of objects. -Constraints ------------ - -* ensure that otherwise immutable objects can be truly immutable -* do not hurt performance for normal Python use cases -* be careful when immortalizing objects that we don't actually expect - to persist until runtime finalization. -* be careful when immortalizing objects that are not otherwise immutable - .. mitigation: Performance Regression Mitigation From 815b4be18545a204a241185c1a158275e6de5cac Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 17:55:03 -0700 Subject: [PATCH 20/41] Clarify about mutable immortal objects. --- pep-0683.rst | 56 ++++++++++++++++++++++++---------------------------- 1 file changed, 26 insertions(+), 30 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 34a74969aeb..dc906c98625 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -494,6 +494,15 @@ is immortal. consider one of the most used objects, None, and that we would make it immortal. How would that impact users of Py_INCREF() and Py_REFCNT()? +.. + > What about __subclasses__/tp_subclasses? + + That's one we'll have to deal with specially, e.g. for core static + types we'd store the object on PyInterpreterState. Then the + __subclasses__ getter would do a lookup on the current interpreter, + instead of using tp_subclasses. We could get rid of tp_subclasses or + perhaps use it only for the main interpreter. + Constraints ----------- @@ -503,6 +512,23 @@ Constraints to persist until runtime finalization. * be careful when immortalizing objects that are not otherwise immutable +Immortal Mutable Objects +------------------------ + +Many of the use cases for immortal objects center on immutability. +However, it may be appropriate for a mutable object (e.g. a dict) to be +marked as immortal. For example, there may be sufficient guarantee +that the object won't actually be modified. + +A particularly relevant case relates to mutable objects that we do +expect to be modified. We may know that such an object will never be +released (until runtime finalization). In that case we can mark the +object as immortal in order to avoid the extra overhead in +``Py_INCREF()`` and ``Py_DECREF()``. However, unlike immutable immortal +objects, such an object still can't be shared between two threads, at +least not without a lock to guard modification (e.g. the GIL or some +granular lock). + _Py_IMMORTAL_REFCNT ------------------- @@ -652,36 +678,6 @@ We wouldn't add a note anywhere else (including for ``Py_INCREF()`` and ``Py_DECREF()``) since the feature is otherwise transparent to users. -Rejected Ideas -============== - -Equate Immortal with Immutable ------------------------------- - -Making a mutable object immortal isn't particularly helpful. -The exception is if you can ensure the object isn't actually -modified again. Since we aren't enforcing any immutability -for immortal objects it didn't make sense to emphasis -that relationship. - -.. - > Is it just not helpful, or is it disallowed? - - It is not disallowed. - - Also, I need to clarify that section since there are cases where - making a mutable object immortal can provide performance benefits, as - described earlier. - - > What about __subclasses__/tp_subclasses? - - That's one we'll have to deal with specially, e.g. for core static - types we'd store the object on PyInterpreterState. Then the - __subclasses__ getter would do a lookup on the current interpreter, - instead of using tp_subclasses. We could get rid of tp_subclasses or - perhaps use it only for the main interpreter. - - Reference Implementation ======================== From 6179e813adb3eef4f3a9177e79aae466f7290547 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 17:58:32 -0700 Subject: [PATCH 21/41] Add the Scope section. --- pep-0683.rst | 56 ++++++++++++++++++++++++++++------------------------ 1 file changed, 30 insertions(+), 26 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index dc906c98625..77b2692907a 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -461,6 +461,18 @@ makes it immortal. (There are other minor, internal changes which are not described here.) +Constraints +----------- + +* ensure that otherwise immutable objects can be truly immutable +* do not hurt performance for normal Python use cases +* be careful when immortalizing objects that we don't actually expect + to persist until runtime finalization. +* be careful when immortalizing objects that are not otherwise immutable + +Scope +----- + This is not meant to be a public feature but rather an internal one. So the proposal does *not* including adding any new public C-API, nor any Python API. However, this does not prevent us from @@ -495,22 +507,17 @@ is immortal. immortal. How would that impact users of Py_INCREF() and Py_REFCNT()? .. - > What about __subclasses__/tp_subclasses? + I suggest being a little more explicit (even blatant) that the particular details of: - That's one we'll have to deal with specially, e.g. for core static - types we'd store the object on PyInterpreterState. Then the - __subclasses__ getter would do a lookup on the current interpreter, - instead of using tp_subclasses. We could get rid of tp_subclasses or - perhaps use it only for the main interpreter. - -Constraints ------------ + (1) which subset of functionally immortal objects are marked as immortal + (2) how to mark something as immortal + (3) how to recognize something as immortal + (4) which memory-management activities are skipped or modified for immortal objects + + are not only Cpython-specific, but are also private implementation details that are expected to change in subsequent versions. + + Ideally, things like the interned string dictionary or the constants from a pyc file will be not merely immortal, but stored in an immortal-only memory page, so that they won't be flushed or CoW-ed when a nearby non-immortal object is modified. Getting those details right will make a difference to performance, and you don't want to be locked in to the first draft. -* ensure that otherwise immutable objects can be truly immutable -* do not hurt performance for normal Python use cases -* be careful when immortalizing objects that we don't actually expect - to persist until runtime finalization. -* be careful when immortalizing objects that are not otherwise immutable Immortal Mutable Objects ------------------------ @@ -599,6 +606,15 @@ There will likely be others we have not enumerated here. but it still wouldn't work without cleverness. So it should be per-interpreter. It would be nice if it were global though. :) +.. + > What about __subclasses__/tp_subclasses? + + That's one we'll have to deal with specially, e.g. for core static + types we'd store the object on PyInterpreterState. Then the + __subclasses__ getter would do a lookup on the current interpreter, + instead of using tp_subclasses. We could get rid of tp_subclasses or + perhaps use it only for the main interpreter. + Object Cleanup -------------- @@ -648,18 +664,6 @@ Possible Changes (e.g. ``Py_RETURN_NONE``) * specialize for immortal objects in the eval loop (`pyston`_) -.. - I suggest being a little more explicit (even blatant) that the particular details of: - - (1) which subset of functionally immortal objects are marked as immortal - (2) how to mark something as immortal - (3) how to recognize something as immortal - (4) which memory-management activities are skipped or modified for immortal objects - - are not only Cpython-specific, but are also private implementation details that are expected to change in subsequent versions. - - Ideally, things like the interned string dictionary or the constants from a pyc file will be not merely immortal, but stored in an immortal-only memory page, so that they won't be flushed or CoW-ed when a nearby non-immortal object is modified. Getting those details right will make a difference to performance, and you don't want to be locked in to the first draft. - Documentation ------------- From 0fabda9e86dae5efd8acb2f6c49ee1350125997a Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 19:06:09 -0700 Subject: [PATCH 22/41] Step 1 toward clarifying the compatibility story. --- pep-0683.rst | 216 +++++++++++++++++++++++++++++---------------------- 1 file changed, 121 insertions(+), 95 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 77b2692907a..4eb231834c1 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -37,61 +37,7 @@ now an object can be truly immutable. (This proposal is meant to be CPython-specific and to affect only internal implementation details. There are some slight exceptions -to that which are explained below.) - -.. - > I think that is a naïve statement. Refcounting is - > implementation-specific, but it's hardly an *internal* detail. - - Sorry for any confusion. I didn't mean to say that refcounting is an - internal detail. Rather, I was talking about how the proposed change - in refcounting behavior doesn't affect any guaranteed/documented - behavior, hence "internal". - - >> There is - >> code that targets CPython specifically, and relies on the details. - > - > Could you elaborate? Do you mean such code relies on specific refcount values? - > - >> The refcount has public getters and setters, - > - > Agreed. However, what behavior do users expect and what guarantees do - > we make? Do we indicate how to interpret the refcount value they - > receive? What are the use cases under which a user would set an - > object's refcount to a specific value? Are users setting the refcount - > of objects they did not create? - - That's what I hoped the PEP would tell me. Instead of simply claiming - that there won't be issues, it should explain why we won't have any issues. - - >> and you need a pretty good - >> grasp of the concept to write a C extension. - > - > I would not expect this to be affected by this PEP, except in cases - > where users are checking/modifying refcounts for objects they did not - > create (since none of their objects will be immortal). - > - >> I think that it's safe to assume that this will break people's code, - > - > Do you have some use case in mind, or an example? From my perspective - > I'm having a hard time seeing what this proposed change would break. - - IMO, the reasoning should start from the assumption that things will - break, and explain why they won't (or why the breakage is acceptable). - If the PEP simply tells me upfront that things will be OK, I have a hard - time trusting it. - - IOW, it's clear you've thought about this a lot (especially after - reading your replies here), but it's not clear from the PEP. - That might be editorial nitpicking, if it wasn't for the fact that I - want find any gaps in your research and reasoning, and invite everyone - else to look for them as well. - - >> and this PEP should convince us that the breakage is worth it rather than - >> dismiss the issue. - > - > Sorry, I didn't mean to be dismissive. I agree that if there is - > breakage this PEP must address it. +to that which are explained below. See `scope`_.) Motivation @@ -243,7 +189,7 @@ Rationale The proposed solution is obvious enough that both of this proposal's authors came to the same conclusion (and implementation, more or less) -independently. The Pyston project `uses a similar approach `_. +independently. The Pyston project `uses a similar approach `_. Other designs were also considered. Several possibilities have also been discussed on python-dev in past years. @@ -301,8 +247,31 @@ performance. Backward Compatibility ----------------------- -This proposal is completely compatible. It is internal-only so no API -is changing. +This proposal is meant to be completely compatible. It focuses strictly +on internal implementation details. It does not involve changes to any +public API, other a few minor changes in behavior related to refcounts +(but only for immortal objects): + +* code that inspects the refcount will see a really, really large value +* the new noop behavior may break code that: + + * depends specifically on the refcount to always increment or decrement + (or have a specific value from ``Py_SET_REFCNT()``) + * relies on any specific refcount value, other than 0 + * directly manipulates the refcount to store extra information there + +Again, those changes in behavior only apply to immortal objects, not +most of the objects a user will access. Furthermore, users cannot mark +an object as immortal so no user-created objects will ever have that +changed behavior. Users that rely on any of the changing behavior for +global (builtin) objects are already in trouble. + +Also note that code which checks for refleaks should keep working fine, +unless it checks for hard-coded small values relative to some immortal +object. The problems noticed by `Pyston`_ shouldn't apply here since +we do not modify the refcount. + +See `scope`_ below for further discussion. The approach is also compatible with extensions compiled to the stable ABI. Unfortunately, they will modify the refcount and invalidate all @@ -332,42 +301,15 @@ section. Even then the overall impact is unlikely to be significant. what the value of _Py_IMMORTAL_REFCNT is, and in the ref implementation a comment says "This can be safely changed to a smaller value".) -No user-facing behavior changes, with the following exceptions: - -* code that inspects the refcount (e.g. ``sys.getrefcount()`` - or directly via ``ob_refcnt``) will see a really, really large - value -* ``Py_SET_REFCNT()`` will be a no-op for immortal objects - -Neither should cause a problem. - -.. - > Importantly, our system allows for the reference count of immortal objects to change, as long as it doesn't go below half of the original very-high value. So extension code with no concept of immortality will still update the reference counts of immortal objects, but this is fine. Because of this we haven't seen any issues with extension modules. - - As Guido noted, we are taking a similar approach for the sake of older - extensions built with the limited API. As a precaution, we start the - refcount for immortal objects basically at _Py_IMMORTAL_REFCNT * 1.5. - Then we only need to check the high bit of _Py_IMMORTAL_REFCNT to see - if an object is immortal. - - > - > The small amount of compatibility challenges we've run into have been in testing code that checks for memory leaks. For example this code breaks on Pyston: - > - > [snip] - > - > This might work with this PEP, but we've also seen code that asserts that the refcount increases by a specific value, which I believe wouldn't. - > - > For Pyston we've simply disabled these tests, figuring that our users still have CPython to test on. Personally I consider this breakage to be small, but I hadn't seen anyone mention the potential usage of sys.getrefcount() so I thought I'd bring it up. - Alternate Python Implementations -------------------------------- -This proposal is CPython-specific. - -.. - IMO it's specific to the C API, which is wider than just CPython. I - don't think we can just assume it'll have no impact on other - implementations. +This proposal is CPython-specific. However, it does relate to the +behavior of the C-API, which may affect other Python implementations. +Consequently, the effect of changed behavior described in +`Backward Compatibility`_ above also applies here (e.g. if another +implementation is tightly coupled to specific refcount values, other +than 0, or on exactly how refcounts change, then they may impacted). Security Implications --------------------- @@ -470,8 +412,10 @@ Constraints to persist until runtime finalization. * be careful when immortalizing objects that are not otherwise immutable -Scope ------ +.. scope: + +Scope of Changes +---------------- This is not meant to be a public feature but rather an internal one. So the proposal does *not* including adding any new public C-API, @@ -481,6 +425,70 @@ like immortalize an object or tell if one is immortal. .. + >> This proposal is CPython-specific and, effectively, describes + >> internal implementation details. + > + > I think that is a naïve statement. Refcounting is + > implementation-specific, but it's hardly an *internal* detail. + + Sorry for any confusion. I didn't mean to say that refcounting is an + internal detail. Rather, I was talking about how the proposed change + in refcounting behavior doesn't affect any guaranteed/documented + behavior, hence "internal". + + >> There is + >> code that targets CPython specifically, and relies on the details. + > + > Could you elaborate? Do you mean such code relies on specific refcount values? + > + >> The refcount has public getters and setters, + > + > Agreed. However, what behavior do users expect and what guarantees do + > we make? Do we indicate how to interpret the refcount value they + > receive? What are the use cases under which a user would set an + > object's refcount to a specific value? Are users setting the refcount + > of objects they did not create? + + That's what I hoped the PEP would tell me. Instead of simply claiming + that there won't be issues, it should explain why we won't have any issues. + + >> and you need a pretty good + >> grasp of the concept to write a C extension. + > + > I would not expect this to be affected by this PEP, except in cases + > where users are checking/modifying refcounts for objects they did not + > create (since none of their objects will be immortal). + > + >> I think that it's safe to assume that this will break people's code, + > + > Do you have some use case in mind, or an example? From my perspective + > I'm having a hard time seeing what this proposed change would break. + + IMO, the reasoning should start from the assumption that things will + break, and explain why they won't (or why the breakage is acceptable). + If the PEP simply tells me upfront that things will be OK, I have a hard + time trusting it. + + IOW, it's clear you've thought about this a lot (especially after + reading your replies here), but it's not clear from the PEP. + That might be editorial nitpicking, if it wasn't for the fact that I + want find any gaps in your research and reasoning, and invite everyone + else to look for them as well. + + >> and this PEP should convince us that the breakage is worth it rather than + >> dismiss the issue. + > + > Sorry, I didn't mean to be dismissive. I agree that if there is + > breakage this PEP must address it. + +.. + >> This is not meant to be a public feature but rather an internal one. + >> So the proposal does *not* including adding any new public C-API, + >> nor any Python API. However, this does not prevent us from + >> adding (publicly accessible) private API to do things + >> like immortalize an object or tell if one + >> is immortal. + > > This is a public change. I agree that the change to the implementation of some public API is @@ -506,6 +514,24 @@ is immortal. consider one of the most used objects, None, and that we would make it immortal. How would that impact users of Py_INCREF() and Py_REFCNT()? +.. + > Importantly, our system allows for the reference count of immortal objects to change, as long as it doesn't go below half of the original very-high value. So extension code with no concept of immortality will still update the reference counts of immortal objects, but this is fine. Because of this we haven't seen any issues with extension modules. + + As Guido noted, we are taking a similar approach for the sake of older + extensions built with the limited API. As a precaution, we start the + refcount for immortal objects basically at _Py_IMMORTAL_REFCNT * 1.5. + Then we only need to check the high bit of _Py_IMMORTAL_REFCNT to see + if an object is immortal. + + > + > The small amount of compatibility challenges we've run into have been in testing code that checks for memory leaks. For example this code breaks on Pyston: + > + > [snip] + > + > This might work with this PEP, but we've also seen code that asserts that the refcount increases by a specific value, which I believe wouldn't. + > + > For Pyston we've simply disabled these tests, figuring that our users still have CPython to test on. Personally I consider this breakage to be small, but I hadn't seen anyone mention the potential usage of sys.getrefcount() so I thought I'd bring it up. + .. I suggest being a little more explicit (even blatant) that the particular details of: @@ -662,7 +688,7 @@ Possible Changes * (Larry,MvL) allocate (immutable) immortal objects in their own memory page(s) * drop refcount operations in code where we know the object is immortal (e.g. ``Py_RETURN_NONE``) -* specialize for immortal objects in the eval loop (`pyston`_) +* specialize for immortal objects in the eval loop (see `Pyston`_) Documentation ------------- @@ -699,7 +725,7 @@ Open Issues References ========== -.. _pyston: https://mail.python.org/archives/list/python-dev@python.org/message/TPLEYDCXFQ4AMTW6F6OQFINSIFYBRFCR/ +.. _Pyston: https://mail.python.org/archives/list/python-dev@python.org/message/TPLEYDCXFQ4AMTW6F6OQFINSIFYBRFCR/ Discussions ----------- From b1daf6b2ed608f81efe64b5a7bc04ba5bfa1706b Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 19:07:17 -0700 Subject: [PATCH 23/41] Clarify about minimizing perf penalty. --- pep-0683.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0683.rst b/pep-0683.rst index 4eb231834c1..4d8c582cd64 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -407,7 +407,7 @@ Constraints ----------- * ensure that otherwise immutable objects can be truly immutable -* do not hurt performance for normal Python use cases +* minimize performance penalty for normal Python use cases * be careful when immortalizing objects that we don't actually expect to persist until runtime finalization. * be careful when immortalizing objects that are not otherwise immutable From 185ec1a7e46f2bb4d3925c8e84d9aaa8160dfeca Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 19:39:06 -0700 Subject: [PATCH 24/41] Clarify about public refcount details (and the docs). --- pep-0683.rst | 43 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 38 insertions(+), 5 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 4d8c582cd64..5dec1ba39d0 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -403,6 +403,29 @@ makes it immortal. (There are other minor, internal changes which are not described here.) +Public Refcount Details +----------------------- + +As part of this proposal, we must make sure that users can clearly +understand on which parts of the refcount behavior they can rely and +which are considered implementation details. Specifically, they should +use the existing public refcount-related API and the only refcount value +with any meaning is 0. All other values are considered "not 0". + +Arguably, the existing refcount-related API should be modified to reflect +what we want users to expect. Something like the following: + +* ``Py_INCREF()`` -> ``Py_ACQUIRE_REF()`` (or only support ``Py_NewRef()``) +* ``Py_DECREF()`` -> ``Py_RELEASE_REF()`` +* ``Py_REFCNT()`` -> ``Py_HAS_REFS()`` +* ``Py_SET_REFCNT()`` -> ``Py_RESET_REFS()`` and ``Py_SET_NO_REFS()`` + +However, such a change is not a part of this proposal. It is included +here to demonstrate the tighter focus for user expectations that would +benefit this change. + +See `Documentation`_ for more details. + Constraints ----------- @@ -693,9 +716,20 @@ Possible Changes Documentation ------------- -The feature itself is internal and will not be added to the documentation. +The immortal objects behavior and API are internal, implementation +details and will not be added to the documentation. + +However, we will update the documentation to make public guarantees +about refcount behavior more clear. That includes, specifically: -We *may* add a note about immortal objects to the following, +* ``Py_INCREF()`` - change "Increment the reference count for object o." + to "Acquire a new reference to object o." +* ``Py_DECREF()`` - change "Decrement the reference count for object o." + to "Release a reference to object o." +* similar for ``Py_XINCREF()``, ``Py_XDECREF()``, ``Py_NewRef()``, + ``Py_XNewRef()``, ``Py_Clear()``, ``Py_REFCNT()``, and ``Py_SET_REFCNT()`` + +We *may* also add a note about immortal objects to the following, to help reduce any surprise users may have with the change: * ``Py_SET_REFCNT()`` (a no-op for immortal objects) @@ -703,9 +737,8 @@ to help reduce any surprise users may have with the change: * ``sys.getrefcount()`` (value may be surprisingly large) Other API that might benefit from such notes are currently undocumented. - -We wouldn't add a note anywhere else (including for ``Py_INCREF()`` and -``Py_DECREF()``) since the feature is otherwise transparent to users. +We wouldn't add such a note anywhere else (including for ``Py_INCREF()`` +and ``Py_DECREF()``) since the feature is otherwise transparent to users. Reference Implementation From 7fa0f66e6564eb8ed43520c0798de8e4a9a6f02c Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 21:16:20 -0700 Subject: [PATCH 25/41] Clarify about the effect of the GIL. --- pep-0683.rst | 30 ++++++------------------------ 1 file changed, 6 insertions(+), 24 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 5dec1ba39d0..7a41d16e69a 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -69,30 +69,12 @@ and to main memory. This has small effect on all Python programs. Immortal objects would provide a slight relief in that regard. On top of that, multi-core applications pay a price. If two threads -are interacting with the same object (e.g. ``None``) then they will -end up invalidating each other's caches with each incref and decref. -This is true even for otherwise immutable objects like ``True``, -``0``, and ``str`` instances. This is also true even with -the GIL, though the impact is smaller. - -.. - > This looks out of context. Python has a per-process GIL. It should it go - > after the next section. - - This isn't about a data race. I'm talking about how if an object is - active in two different threads (on distinct cores) then incref/decref - in one thread will invalidate the cache (line) in the other thread. - The only impact of the GIL in this case is that the two threads aren't - running simultaneously and the cache invalidation on the idle thread - has less impact. - -.. - > This is also true even with the GIL, though the impact is smaller. - - Smaller than what? The baseline for that comparison is a hypothetical - GIL-less interpreter, which is only introduced in the next section. - Perhaps say something like "Python's GIL helps avoid this effect, but - doesn't eliminate it." +(running simultaneously on distinct cores) are interacting with the +same object (e.g. ``None``) then they will end up invalidating each +other's caches with each incref and decref. This is true even for +otherwise immutable objects like ``True``, ``0``, and ``str`` instances. +CPython's GIL helps reduce this effect, since only one thread runs at a +time, but it doesn't completely eliminate the penalty. Avoiding Data Races ------------------- From 67f701f13fab722fb90411c1030f2839befe8544 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 21:53:25 -0700 Subject: [PATCH 26/41] Explain about accidental immortality. --- pep-0683.rst | 63 ++++++++++++++++++++++++++-------------------------- 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 7a41d16e69a..ac4aafc83c3 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -227,7 +227,7 @@ performance. .. _a 4% slowdown: https://github.com/python/cpython/pull/19474#issuecomment-1032944709 Backward Compatibility ------------------------ +---------------------- This proposal is meant to be completely compatible. It focuses strictly on internal implementation details. It does not involve changes to any @@ -255,6 +255,9 @@ we do not modify the refcount. See `scope`_ below for further discussion. +Stable ABI +---------- + The approach is also compatible with extensions compiled to the stable ABI. Unfortunately, they will modify the refcount and invalidate all the performance benefits of immortal objects. However, the high bit @@ -263,25 +266,34 @@ so we can still identify such objects as immortal. At worst, objects in that situation would feel the effects described in the `Motivation`_ section. Even then the overall impact is unlikely to be significant. -.. - >> What about extensions compiled with Python 3.11 (with this PEP) that use - >> an older version of the stable ABI, and thus should be compatible with - >> 3.2+? Will they use the old versions of the macros? How will that be tested? - > - > It wouldn't matter unless an object's refcount reached - > _Py_IMMORTAL_REFCNT, at which point incref/decref would start - > noop'ing. What is the likelihood (in real code) that an object's - > refcount would grow that far? Even then, would such an object ever be - > expected to go back to 0 (and be dealloc'ed)? Otherwise the point is - > moot. - - That's exactly the questions I'd hope the PEP to answer. I could - estimate that likelihood myself, but I'd really rather just check your - work ;) - - (Hm, maybe I couldn't even estimate this myself. The PEP doesn't say - what the value of _Py_IMMORTAL_REFCNT is, and in the ref implementation - a comment says "This can be safely changed to a smaller value".) +Also see `_Py_IMMORTAL_REFCNT`_ below. + +Accidental Immortality +---------------------- + +Hypothetically, a regular object could be incref'ed so much that it +reaches the magic value needed to be considered immortal. That means +it would accidentally never be cleaned up (by going back to 0). + +While it isn't impossible, this accidental scenario is so unlikely +that we need not worry. Even if done deliberately by using +``Py_INCREF()`` in a tight loop and each iteration only took 1 CPU +cycle, it would take 2^61 cycles (on a 64-bit processor). At a fast +5 GHz that would still take nearly 500,000,000 seconds (over 5,000 days)! +If that CPU were 32-bit then it is (technically) more possible though +still highly unlikely. + +Also note that it is doubly unlikely to be a problem because it wouldn't +matter until the refcount got back to 0 and the object was cleaned up. +So any object that hit that magic "immortal" refcount value would have +to be decref'ed that many times again before the change in behavior +would be noticed. + +Again, the only realistic way that the magic refcount would be reached +(and then reversed) is if it were done deliberately. (Of course, the +same thing could be done efficiently using ``Py_SET_REFCNT()`` though +that would be even less of an accident.) At that point we don't +consider it a concern of this proposal. Alternate Python Implementations -------------------------------- @@ -355,17 +367,6 @@ Non-Obvious Consequences isn't. Hence the type of an immortal object is effectively immortal. We don't need the type to actually be marked as immortal. -* though extremely unlikely (and technically hard), any object could - be incref'ed enough to reach ``_Py_IMMORTAL_REFCNT`` and then - be treated as immortal - -.. - > What would it take? - - Basically, you;d have to do it deliberately (e.g. incref the object in - a tight loop). Even with a tight loop it would take a long time to - count up to 2^61 or whatever the chosen value is. - Specification ============= From 8a34e1a188b6db46b5f8cd273aa93061462e4a7f Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 22:39:19 -0700 Subject: [PATCH 27/41] Clarify about implicitly immortal objects, mutability, and restrictions. --- pep-0683.rst | 129 +++++++++++++++++++++++++++------------------------ 1 file changed, 69 insertions(+), 60 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index ac4aafc83c3..9c29bc47766 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -320,53 +320,6 @@ may be some extra complexity due to performance penalty mitigation. However, that should be limited to where we immortalize all objects post-init and that code will be in one place. -Non-Obvious Consequences ------------------------- - -* immortal objects that hold references to other objects - ("containers") effectively immortalize each contained item -* the same is true for objects held internally by other objects - (e.g. ``PyTypeObject.tp_subclasses``) - -.. - > So, do immortal lists immortalize values append()ed to them? (Can you - > even have an immortal list? Are there limits on what can be immortal?) - - We have no plans to do more than ever explicitly immortalize objects. - So an immortal list is fine but it would have no effect on the - immortality of items it contains, other than implicitly (since the - list holds a reference to each item). - - In general, it would be best to only immortalize immutable objects. - If we want to share any objects shared between threads without - protection (e.g. per-interpreter GIL) then such objects must be - immortal and immutable. So lists and dicts, etc. couldn't be shared - (assuming we can't prevent further mutation). - - However, for objects that will never be shared, it can be practical to - make some of them immortal too. For example, sys.modules is a - per-interpreter dict that we do not expect to ever get freed until the - corresponding interpreter is finalized. By making it immortal, we no - longer incur the extra overhead during incref/decref. - - We can apply this idea in the pursuit of getting back some of that 4% - performance we lost. At the end of runtime init we can mark *all* - objects as immortal and avoid the extra cost in incref/decref. We - only need to worry about immutability with objects that we plan on - sharing between threads without a GIL. - - (FYI, we still need to look closely at the impact of this approach on GC.) - -* an immortal object's type is effectively immortal - -.. - > Should this be enforced? - - There is nothing to enforce. The object holds a reference to its type - so the type will never be cleaned up as long as the immortal object - isn't. Hence the type of an immortal object is effectively immortal. - We don't need the type to actually be marked as immortal. - Specification ============= @@ -550,23 +503,72 @@ is immortal. Ideally, things like the interned string dictionary or the constants from a pyc file will be not merely immortal, but stored in an immortal-only memory page, so that they won't be flushed or CoW-ed when a nearby non-immortal object is modified. Getting those details right will make a difference to performance, and you don't want to be locked in to the first draft. - Immortal Mutable Objects ------------------------ -Many of the use cases for immortal objects center on immutability. -However, it may be appropriate for a mutable object (e.g. a dict) to be -marked as immortal. For example, there may be sufficient guarantee -that the object won't actually be modified. +Any object can be marked as immortal. We do not propose any +restrictions or checks. However, in practice the value of making an +object immortal relates to its mutability and depends on the likelihood +it would be used for a sufficient portion of the application's lifetime. +Marking a mutable object as immortal can make sense in some situations. -A particularly relevant case relates to mutable objects that we do -expect to be modified. We may know that such an object will never be -released (until runtime finalization). In that case we can mark the -object as immortal in order to avoid the extra overhead in -``Py_INCREF()`` and ``Py_DECREF()``. However, unlike immutable immortal -objects, such an object still can't be shared between two threads, at -least not without a lock to guard modification (e.g. the GIL or some -granular lock). +Many of the use cases for immortal objects center on immutability, so +that threads can safely and efficiently share such objects without +locking. For this reason a mutable object, like a dict or list, would +never be shared (and thus no immortality). However, immortality may +be appropriate if there is sufficient guarantee that the normally +mutable object won't actually be modified. + +On the other hand, some mutable objects will never be shared between +threads (at least not without a lock like the GIL). In some cases it +may be practical to make some of those immortal too. For example, +``sys.modules`` is a per-interpreter dict that we do not expect to ever +get freed until the corresponding interpreter is finalized. By making +it immortal, we no longer incur the extra overhead during incref/decref. + +We explore this idea further in the `mitigations`_ section below. + +(Note that we are still investigating the impact on GC +of immortalizing containers.) + +Implicitly Immortal Objects +--------------------------- + +If an immortal object holds a reference to a normal (mortal) object +then that held object is effectively immortal. This is because that +object's refcount can never reach 0 until the immortal object releases +it. + +Examples: + +* containers like ``dict`` and ``list`` +* objects that hold references internally like ``PyTypeObject.tp_subclasses`` +* an object's type (held in ``ob_type``) + +Such held objects are thus implicitly immortal for as long as they are +held. In practice, this should have no real consequences since it +really isn't a change in behavior. The only difference is that the +immortal object (holding the reference) doesn't ever get cleaned up. + +We do not propose that such implicitly immortal objects be changed +in any way. They should not be explicitly marked as immortal just +because they are held by an immortal object. That would provide +no advantage over doing nothing. + +Un-Immortalizing Objects +------------------------ + +This proposal does not include any mechanism for taking an immortal +object and returning it to a "normal" condition. Currently there +is no need for such an ability. + +On top of that, the obvious approach is to simply set the refcount +to a small value. However, at that point there is no way in knowing +which value would be safe. Ideally we'd set it to the value that it +would have been if it hadn't been made immortal. However, that value +has long been lost. Hence the complexities involved make it less +likely that an object could safely be un-immortalized, even if we +had a good reason to do so. _Py_IMMORTAL_REFCNT ------------------- @@ -682,6 +684,13 @@ to try to recover some of the lost performance: * ... +We can apply the concept from `Immortal Mutable Objects`_ in the pursuit +of getting back some of that 4% performance we lose with the naive +implementation of immortal objects. At the end of runtime init we +can mark *all* objects as immortal and avoid the extra cost in +incref/decref. We only need to worry about immutability with +objects that we plan on sharing between threads without a GIL. + Note that these are not part of the proposal. They are included here for clarity. From bf90be8979c18261265cce11c00c458cb72c7bba Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 22:53:42 -0700 Subject: [PATCH 28/41] Clarify about selection and mitigation. --- pep-0683.rst | 68 ++++++++++++++++++---------------------------------- 1 file changed, 23 insertions(+), 45 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 9c29bc47766..7f37a6bc103 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -610,44 +610,21 @@ will not be affected.) Immortal Global Objects ----------------------- -The following objects will be made immortal: +All objects that we expect to be shared globally (between interpreters) +will be made immortal. That includes the following: * singletons (``None``, ``True``, ``False``, ``Ellipsis``, ``NotImplemented``) * all static types (e.g. ``PyLong_Type``, ``PyExc_Exception``) * all static objects in ``_PyRuntimeState.global_objects`` (e.g. identifiers, small ints) -There will likely be others we have not enumerated here. - -.. - > How will the candidates be chosen? - - Any objects that we would expect to share globally (ergo otherwise - immutable) will be made immortal. That means the static types, the - builtin singletons, the objects in _PyRuntimeState.global_objects, - etc. - -.. - > Should the intern dict be belonging to runtime, or (sub)interpreter? - > - > If the interned dict is belonging to runtime, all interned dict should - > be immortal to be shared between subinterpreters. - - Excellent questions. Making immutable objects immortal is relatively - simple. For the most part, mutable objects should not be shared - between interpreters without protection (e.g. the GIL). The interned - dict isn't exposed to Python code or the C-API, so there's less risk, - but it still wouldn't work without cleverness. So it should be - per-interpreter. It would be nice if it were global though. :) - -.. - > What about __subclasses__/tp_subclasses? - - That's one we'll have to deal with specially, e.g. for core static - types we'd store the object on PyInterpreterState. Then the - __subclasses__ getter would do a lookup on the current interpreter, - instead of using tp_subclasses. We could get rid of tp_subclasses or - perhaps use it only for the main interpreter. +All such objects will be immutable. In the case of the static types, +they will be effectively immutable. ``PyTypeObject`` has some mutable +start (``tp_dict`` and ``tp_subclasses``), but we can work around this +by storing that state on ``PyInterpreterState`` instead of on the +respective static type object. Then the ``__dict__``, etc. getter +will do a lookup on the current interpreter, if appropriate, instead +of using ``tp_dict``. Object Cleanup -------------- @@ -680,19 +657,23 @@ Performance Regression Mitigation --------------------------------- In the interest of clarify, here are some of the ways we are going -to try to recover some of the lost performance: +to try to recover some of the lost `performance `_: -* ... +* at the end of runtime init, mark all objects as immortal +* drop refcount operations in code where we know the object is immortal + (e.g. ``Py_RETURN_NONE``) +* specialize for immortal objects in the eval loop (see `Pyston`_) -We can apply the concept from `Immortal Mutable Objects`_ in the pursuit -of getting back some of that 4% performance we lose with the naive -implementation of immortal objects. At the end of runtime init we -can mark *all* objects as immortal and avoid the extra cost in -incref/decref. We only need to worry about immutability with -objects that we plan on sharing between threads without a GIL. +Regarding that first point, we can apply the concept from +`Immortal Mutable Objects`_ in the pursuit of getting back some of +that 4% performance we lose with the naive implementation of immortal +objects. At the end of runtime init we can mark *all* objects as +immortal and avoid the extra cost in incref/decref. We only need +to worry about immutability with objects that we plan on sharing +between threads without a GIL. -Note that these are not part of the proposal. They are included here -for clarity. +Note that none of this section is part of the proposal. +The above is included here for clarity. Possible Changes ---------------- @@ -701,9 +682,6 @@ Possible Changes * mark the "interned" dict as immortal if shared else share all interned strings * (Larry,MvL) mark all constants unmarshalled for a module as immortal * (Larry,MvL) allocate (immutable) immortal objects in their own memory page(s) -* drop refcount operations in code where we know the object is immortal - (e.g. ``Py_RETURN_NONE``) -* specialize for immortal objects in the eval loop (see `Pyston`_) Documentation ------------- From 19befb035560ba4997358c30e43466496134967a Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 22:56:13 -0700 Subject: [PATCH 29/41] Add a short into to the specification section. --- pep-0683.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/pep-0683.rst b/pep-0683.rst index 7f37a6bc103..243d9b32184 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -339,6 +339,10 @@ makes it immortal. (There are other minor, internal changes which are not described here.) +In the following sub-sections we dive into the details. First we will +cover some conceptual topics, followed by more concrete aspects like +specific affected APIs. + Public Refcount Details ----------------------- From 6e475c41f6530b90c947d1693058756c8c84199f Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 23:33:24 -0700 Subject: [PATCH 30/41] Add a Prior Art section. --- pep-0683.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/pep-0683.rst b/pep-0683.rst index 243d9b32184..f183d653973 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -734,6 +734,11 @@ References .. _Pyston: https://mail.python.org/archives/list/python-dev@python.org/message/TPLEYDCXFQ4AMTW6F6OQFINSIFYBRFCR/ +Prior Art +--------- + +* `Pyston`_ + Discussions ----------- From 4fd812cd4c5a146eb52ce5eab4b99462c6878414 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 23:41:13 -0700 Subject: [PATCH 31/41] Wrap up the Scope section. --- pep-0683.rst | 185 +++++++++++++++++---------------------------------- 1 file changed, 60 insertions(+), 125 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index f183d653973..b17f1efb7d9 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -37,7 +37,8 @@ now an object can be truly immutable. (This proposal is meant to be CPython-specific and to affect only internal implementation details. There are some slight exceptions -to that which are explained below. See `scope`_.) +to that which are explained below. See `Backward Compatibilty`_, +`Public Refcount Details`_, and `scope`_.) Motivation @@ -229,6 +230,32 @@ performance. Backward Compatibility ---------------------- +.. + >> I think that it's safe to assume that this will break people's code, + > + > Do you have some use case in mind, or an example? From my perspective + > I'm having a hard time seeing what this proposed change would break. + + IMO, the reasoning should start from the assumption that things will + break, and explain why they won't (or why the breakage is acceptable). + +.. + > Py_INCREF increments the reference count. + > Py_REFCNT gets the reference count. + > For immortal objects, Py_INCREF will no longer function as documented in + > 3.10, and Py_REFCNT can be used to witness it. Both are public API. + + You are right that "Increment the reference count for object o." (as + documented) will not be true for an immortal object. Instead it would + be something like "indicate that there is an additional reference for + object o". I'll be sure to update the PEP, to add that change to the + docs wording. + + Regardless, how important is that distinction? If it matters then + clearly this proposal needs to change. As an exercise, we can + consider one of the most used objects, None, and that we would make it + immortal. How would that impact users of Py_INCREF() and Py_REFCNT()? + This proposal is meant to be completely compatible. It focuses strictly on internal implementation details. It does not involve changes to any public API, other a few minor changes in behavior related to refcounts @@ -253,7 +280,7 @@ unless it checks for hard-coded small values relative to some immortal object. The problems noticed by `Pyston`_ shouldn't apply here since we do not modify the refcount. -See `scope`_ below for further discussion. +See `Public Refcount Details`_ and `scope`_ below for further discussion. Stable ABI ---------- @@ -345,6 +372,25 @@ specific affected APIs. Public Refcount Details ----------------------- +.. + >> The refcount has public getters and setters, + > + > Agreed. However, what behavior do users expect and what guarantees do + > we make? Do we indicate how to interpret the refcount value they + > receive? What are the use cases under which a user would set an + > object's refcount to a specific value? Are users setting the refcount + > of objects they did not create? + + That's what I hoped the PEP would tell me. Instead of simply claiming + that there won't be issues, it should explain why we won't have any issues. + + >> and you need a pretty good + >> grasp of the concept to write a C extension. + > + > I would not expect this to be affected by this PEP, except in cases + > where users are checking/modifying refcounts for objects they did not + > create (since none of their objects will be immortal). + As part of this proposal, we must make sure that users can clearly understand on which parts of the refcount behavior they can rely and @@ -380,132 +426,21 @@ Constraints Scope of Changes ---------------- -This is not meant to be a public feature but rather an internal one. -So the proposal does *not* including adding any new public C-API, -nor any Python API. However, this does not prevent us from -adding (publicly accessible) private API to do things -like immortalize an object or tell if one -is immortal. +Object immortality is not meant to be a public feature but rather an +internal one. So the proposal does *not* including adding any new +public C-API, nor any Python API. However, this does not prevent +us from adding (publicly accessible) private API to do things +like immortalize an object or tell if one is immortal. -.. - >> This proposal is CPython-specific and, effectively, describes - >> internal implementation details. - > - > I think that is a naïve statement. Refcounting is - > implementation-specific, but it's hardly an *internal* detail. - - Sorry for any confusion. I didn't mean to say that refcounting is an - internal detail. Rather, I was talking about how the proposed change - in refcounting behavior doesn't affect any guaranteed/documented - behavior, hence "internal". - - >> There is - >> code that targets CPython specifically, and relies on the details. - > - > Could you elaborate? Do you mean such code relies on specific refcount values? - > - >> The refcount has public getters and setters, - > - > Agreed. However, what behavior do users expect and what guarantees do - > we make? Do we indicate how to interpret the refcount value they - > receive? What are the use cases under which a user would set an - > object's refcount to a specific value? Are users setting the refcount - > of objects they did not create? - - That's what I hoped the PEP would tell me. Instead of simply claiming - that there won't be issues, it should explain why we won't have any issues. - - >> and you need a pretty good - >> grasp of the concept to write a C extension. - > - > I would not expect this to be affected by this PEP, except in cases - > where users are checking/modifying refcounts for objects they did not - > create (since none of their objects will be immortal). - > - >> I think that it's safe to assume that this will break people's code, - > - > Do you have some use case in mind, or an example? From my perspective - > I'm having a hard time seeing what this proposed change would break. - - IMO, the reasoning should start from the assumption that things will - break, and explain why they won't (or why the breakage is acceptable). - If the PEP simply tells me upfront that things will be OK, I have a hard - time trusting it. - - IOW, it's clear you've thought about this a lot (especially after - reading your replies here), but it's not clear from the PEP. - That might be editorial nitpicking, if it wasn't for the fact that I - want find any gaps in your research and reasoning, and invite everyone - else to look for them as well. - - >> and this PEP should convince us that the breakage is worth it rather than - >> dismiss the issue. - > - > Sorry, I didn't mean to be dismissive. I agree that if there is - > breakage this PEP must address it. - -.. - >> This is not meant to be a public feature but rather an internal one. - >> So the proposal does *not* including adding any new public C-API, - >> nor any Python API. However, this does not prevent us from - >> adding (publicly accessible) private API to do things - >> like immortalize an object or tell if one - >> is immortal. - > - > This is a public change. - - I agree that the change to the implementation of some public API is - certainly public, as is the change in behavior for immortal objects, - as is the potential <4% performance regression. By "public feature" I - was referring to immortal objects. We are not exposing that to users, - other than that they might notice some objects now have a really high - refcount that does not change. - - > Py_INCREF increments the reference count. - > Py_REFCNT gets the reference count. - > For immortal objects, Py_INCREF will no longer function as documented in - > 3.10, and Py_REFCNT can be used to witness it. Both are public API. - - You are right that "Increment the reference count for object o." (as - documented) will not be true for an immortal object. Instead it would - be something like "indicate that there is an additional reference for - object o". I'll be sure to update the PEP, to add that change to the - docs wording. - - Regardless, how important is that distinction? If it matters then - clearly this proposal needs to change. As an exercise, we can - consider one of the most used objects, None, and that we would make it - immortal. How would that impact users of Py_INCREF() and Py_REFCNT()? - -.. - > Importantly, our system allows for the reference count of immortal objects to change, as long as it doesn't go below half of the original very-high value. So extension code with no concept of immortality will still update the reference counts of immortal objects, but this is fine. Because of this we haven't seen any issues with extension modules. - - As Guido noted, we are taking a similar approach for the sake of older - extensions built with the limited API. As a precaution, we start the - refcount for immortal objects basically at _Py_IMMORTAL_REFCNT * 1.5. - Then we only need to check the high bit of _Py_IMMORTAL_REFCNT to see - if an object is immortal. - - > - > The small amount of compatibility challenges we've run into have been in testing code that checks for memory leaks. For example this code breaks on Pyston: - > - > [snip] - > - > This might work with this PEP, but we've also seen code that asserts that the refcount increases by a specific value, which I believe wouldn't. - > - > For Pyston we've simply disabled these tests, figuring that our users still have CPython to test on. Personally I consider this breakage to be small, but I hadn't seen anyone mention the potential usage of sys.getrefcount() so I thought I'd bring it up. - -.. - I suggest being a little more explicit (even blatant) that the particular details of: - - (1) which subset of functionally immortal objects are marked as immortal - (2) how to mark something as immortal - (3) how to recognize something as immortal - (4) which memory-management activities are skipped or modified for immortal objects +The particular details of: - are not only Cpython-specific, but are also private implementation details that are expected to change in subsequent versions. +* how to mark something as immortal +* how to recognize something as immortal +* which subset of functionally immortal objects are marked as immortal +* which memory-management activities are skipped or modified for immortal objects - Ideally, things like the interned string dictionary or the constants from a pyc file will be not merely immortal, but stored in an immortal-only memory page, so that they won't be flushed or CoW-ed when a nearby non-immortal object is modified. Getting those details right will make a difference to performance, and you don't want to be locked in to the first draft. +are not only Cpython-specific but are also private implementation +details that are expected to change in subsequent versions. Immortal Mutable Objects ------------------------ From 9384454c7f88d77a8901647bd8b24e4ce72ebf6b Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Fri, 18 Feb 2022 23:55:34 -0700 Subject: [PATCH 32/41] Fix the Public Refcount Details section. --- pep-0683.rst | 63 ++++++++++++++-------------------------------------- 1 file changed, 17 insertions(+), 46 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index b17f1efb7d9..8904c0f07d5 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -230,32 +230,6 @@ performance. Backward Compatibility ---------------------- -.. - >> I think that it's safe to assume that this will break people's code, - > - > Do you have some use case in mind, or an example? From my perspective - > I'm having a hard time seeing what this proposed change would break. - - IMO, the reasoning should start from the assumption that things will - break, and explain why they won't (or why the breakage is acceptable). - -.. - > Py_INCREF increments the reference count. - > Py_REFCNT gets the reference count. - > For immortal objects, Py_INCREF will no longer function as documented in - > 3.10, and Py_REFCNT can be used to witness it. Both are public API. - - You are right that "Increment the reference count for object o." (as - documented) will not be true for an immortal object. Instead it would - be something like "indicate that there is an additional reference for - object o". I'll be sure to update the PEP, to add that change to the - docs wording. - - Regardless, how important is that distinction? If it matters then - clearly this proposal needs to change. As an exercise, we can - consider one of the most used objects, None, and that we would make it - immortal. How would that impact users of Py_INCREF() and Py_REFCNT()? - This proposal is meant to be completely compatible. It focuses strictly on internal implementation details. It does not involve changes to any public API, other a few minor changes in behavior related to refcounts @@ -372,25 +346,22 @@ specific affected APIs. Public Refcount Details ----------------------- -.. - >> The refcount has public getters and setters, - > - > Agreed. However, what behavior do users expect and what guarantees do - > we make? Do we indicate how to interpret the refcount value they - > receive? What are the use cases under which a user would set an - > object's refcount to a specific value? Are users setting the refcount - > of objects they did not create? - - That's what I hoped the PEP would tell me. Instead of simply claiming - that there won't be issues, it should explain why we won't have any issues. - - >> and you need a pretty good - >> grasp of the concept to write a C extension. - > - > I would not expect this to be affected by this PEP, except in cases - > where users are checking/modifying refcounts for objects they did not - > create (since none of their objects will be immortal). +In `Backward Compatibility`_ we introduced possible ways that user code +might be broken by the change in this proposal. Any contributing +misunderstanding by users is likely due in large part to the names of +the refcount-related API and to how the documentation explains those +API (and refcounting in general). + +Between the names and the docs, we can clearly see answers +to the following questions: + +* what behavior do users expect? +* what guarantees do we make? +* do we indicate how to interpret the refcount value they receive? +* what are the use cases under which a user would set an object's + refcount to a specific value? +* are users setting the refcount of objects they did not create? As part of this proposal, we must make sure that users can clearly understand on which parts of the refcount behavior they can rely and @@ -398,6 +369,8 @@ which are considered implementation details. Specifically, they should use the existing public refcount-related API and the only refcount value with any meaning is 0. All other values are considered "not 0". +This information will be clarified in the `documentation `_. + Arguably, the existing refcount-related API should be modified to reflect what we want users to expect. Something like the following: @@ -410,8 +383,6 @@ However, such a change is not a part of this proposal. It is included here to demonstrate the tighter focus for user expectations that would benefit this change. -See `Documentation`_ for more details. - Constraints ----------- From dcf72a328d4a67c9f22dc37593851a1ac343d579 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Sat, 19 Feb 2022 00:00:45 -0700 Subject: [PATCH 33/41] Add an open question about copy-on-write. --- pep-0683.rst | 30 +----------------------------- 1 file changed, 1 insertion(+), 29 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 8904c0f07d5..af4ca9fbb32 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -137,35 +137,6 @@ that uses copy-on-write semantics. Anything that uses ``mmap`` relies on copy-on-write, including sharing data from shared objects files between processes. -.. - > Anyway, I don't believe stopping refcounting will fix the CoW issue - > yet. See this article [1] again. - > - > [1] https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172 - - That's definitely an important point, given that the main objective of - the proposal is to allow disabling mutation of runtime-internal object - state so that some objects can be made truly immutable. - - I'm sure Eddie has some good insight on the matter (and may have even - been involved in writing that article). Eddie? - - > Note that they failed to fix CoW by stopping refcounting code objects! (*) - > Most CoW was caused by cyclic GC and finalization caused most CoW. - - That's a good observation! - - > (*) It is not surprising to me because eval loop don't incre/decref - > most code attributes. They borrow reference from the code object. - - +1 - - > So we need a sample application and profile it, before saying it fixes CoW. - > Could you provide some data, or drop the CoW issue from this PEP until - > it is proved? - - We'll look into that. - Rationale ========= @@ -633,6 +604,7 @@ Open Issues =========== * is there any other impact on GC? +* `are the copy-on-write benefits real? `__ References From de21141338adaa61cdada72f8103dabbc8952000 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Sat, 19 Feb 2022 00:06:05 -0700 Subject: [PATCH 34/41] Add notes on the criteria for acceptance. --- pep-0683.rst | 30 ++++++------------------------ 1 file changed, 6 insertions(+), 24 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index af4ca9fbb32..d8c5d937953 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -32,8 +32,11 @@ finalization). Additionally, all other per-object runtime state for an immortal object will be considered immutable. This approach has some possible negative impact, which is explained -below, along with mitigations. The fundamental improvement is that -now an object can be truly immutable. +below, along with mitigations. A critical requirement for this change +is that the performance regression be no more than 2-3%. Anything worse +the performance-neutral requires that the other benefits are proportionally +large. Aside from specific applications, the fundamental improvement +here is that now an object can be truly immutable. (This proposal is meant to be CPython-specific and to affect only internal implementation details. There are some slight exceptions @@ -99,28 +102,6 @@ otherwise immutable objects. With immortal objects, support for a per-interpreter GIL becomes much simpler. -.. - >>> Weren't you planning a PEP on subinterpreter GIL as well? Do you want to - >>> submit them together? - >> - >> IMO, as it is, the PEP's motivation doesn't really stand on its own. - >> It's only worth it as a step towards per-interpreter GIL. - >> - >> I'd have to think about that. The other PEP I'm writing for - >> per-interpreter GIL doesn't require immortal objects. They just - >> simplify a number of things. That's my motivation for writing this - >> PEP, in fact. :) - > - > Please think about it. - > If you removed the benefits for per-interpreter GIL, the motivation - > section would be reduced to is memory savings for fork/CoW. (And lots of - > performance improvements that are great in theory but sum up to a 4% loss.) - - Sounds good. Would this involve more than a note at the top of the PEP? - - And just to be clear, I don't think the fate of a per-interpreter GIL - PEP should not depend on this one. - Avoiding Copy-on-Write ---------------------- @@ -605,6 +586,7 @@ Open Issues * is there any other impact on GC? * `are the copy-on-write benefits real? `__ +* must the fate of this PEP be tied to acceptance of a per-interpreter GIL PEP? References From 6bab8a9bcb8c070f7d064c14be1a5e9f27ff0969 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Sat, 19 Feb 2022 00:13:01 -0700 Subject: [PATCH 35/41] Fix ref labels. --- pep-0683.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index d8c5d937953..1ae2e420843 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -344,7 +344,7 @@ Constraints to persist until runtime finalization. * be careful when immortalizing objects that are not otherwise immutable -.. scope: +.. _scope: Scope of Changes ---------------- @@ -513,7 +513,7 @@ accessible on the runtime state, e.g. in a ``_PyRuntimeState`` or ``PyInterpreterState`` field. We may need to add a tracking mechanism to the runtime state for a small number of objects. -.. mitigation: +.. _mitigation: Performance Regression Mitigation --------------------------------- From 6238e729a73a13e1d64efac5609cb7abf30e31d0 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Sat, 19 Feb 2022 00:20:40 -0700 Subject: [PATCH 36/41] Fix spacing for literal blocks. --- pep-0683.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/pep-0683.rst b/pep-0683.rst index 1ae2e420843..2220eacba7b 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -436,12 +436,14 @@ _Py_IMMORTAL_REFCNT ------------------- We will add two internal constants:: + #define _Py_IMMORTAL_BIT (1LL << (8 * sizeof(Py_ssize_t) - 4)) #define _Py_IMMORTAL_REFCNT (_Py_IMMORTAL_BIT + (_Py_IMMORTAL_BIT / 2)) The refcount for immortal objects will be set to ``_Py_IMMORTAL_REFCNT``. However, to check if an object is immortal we will compare its refcount against just the bit:: + (op->ob_refcnt & _Py_IMMORTAL_BIT) != 0 The difference means that an immortal object will still be considered From b879b28755ce4b0423f12415d4c1fe4f30c5be48 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Sat, 19 Feb 2022 00:25:02 -0700 Subject: [PATCH 37/41] Fix a link. --- pep-0683.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0683.rst b/pep-0683.rst index 2220eacba7b..fbf12cd4444 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -628,7 +628,7 @@ and ``_PyObject_HEAD_EXTRA`` are part of ``PyObject_HEAD``. .. _PyObject.ob_refcnt: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L107 .. _PyGC_Head: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/internal/pycore_gc.h#L11-L20 -.. __PyObject_HEAD_EXTRA: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L68-L72 +.. _PyObject_HEAD_EXTRA: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L68-L72 .. _refcounting: From 097c79b44fab490f7b0769610069932975ebad87 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Sat, 19 Feb 2022 00:28:01 -0700 Subject: [PATCH 38/41] Fix a typo in a link. --- pep-0683.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0683.rst b/pep-0683.rst index fbf12cd4444..6b4f574ae63 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -40,7 +40,7 @@ here is that now an object can be truly immutable. (This proposal is meant to be CPython-specific and to affect only internal implementation details. There are some slight exceptions -to that which are explained below. See `Backward Compatibilty`_, +to that which are explained below. See `Backward Compatibility`_, `Public Refcount Details`_, and `scope`_.) From 835748e5d9584a828c1e69efea91f0c0721bca32 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Sat, 19 Feb 2022 00:31:58 -0700 Subject: [PATCH 39/41] Fix another typo in a link. --- pep-0683.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pep-0683.rst b/pep-0683.rst index 6b4f574ae63..d3fe35d1003 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -388,7 +388,7 @@ may be practical to make some of those immortal too. For example, get freed until the corresponding interpreter is finalized. By making it immortal, we no longer incur the extra overhead during incref/decref. -We explore this idea further in the `mitigations`_ section below. +We explore this idea further in the `mitigation`_ section below. (Note that we are still investigating the impact on GC of immortalizing containers.) From 7e8247063f360f5cd916f8fd460a892cfe345ca5 Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Sat, 19 Feb 2022 00:36:14 -0700 Subject: [PATCH 40/41] Add missing leading underscore. --- pep-0683.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index d3fe35d1003..87d9aee3c96 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -627,8 +627,8 @@ container types (e.g. ``list``). Also note that ``PyObject.ob_refcnt`` and ``_PyObject_HEAD_EXTRA`` are part of ``PyObject_HEAD``. .. _PyObject.ob_refcnt: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L107 -.. _PyGC_Head: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/internal/pycore_gc.h#L11-L20 -.. _PyObject_HEAD_EXTRA: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L68-L72 +.. __PyGC_Head: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/internal/pycore_gc.h#L11-L20 +.. __PyObject_HEAD_EXTRA: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L68-L72 .. _refcounting: From b90b76a890572bfe08ac2b304f8be95e6a098f8e Mon Sep 17 00:00:00 2001 From: Eric Snow Date: Sat, 19 Feb 2022 00:39:13 -0700 Subject: [PATCH 41/41] try again --- pep-0683.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pep-0683.rst b/pep-0683.rst index 87d9aee3c96..dbd8e8fa41f 100644 --- a/pep-0683.rst +++ b/pep-0683.rst @@ -616,8 +616,8 @@ Here is the internal state that the CPython runtime keeps for each Python object: * `PyObject.ob_refcnt`_: the object's `refcount `_ -* `_PyGC_Head`_: (optional) the object's node in a list of `"GC" objects `_ -* `_PyObject_HEAD_EXTRA`_: (optional) the object's node in the list of heap objects +* `_PyGC_Head `_: (optional) the object's node in a list of `"GC" objects `_ +* `_PyObject_HEAD_EXTRA `_: (optional) the object's node in the list of heap objects ``ob_refcnt`` is part of the memory allocated for every object. However, ``_PyObject_HEAD_EXTRA`` is allocated only if CPython was built @@ -627,8 +627,8 @@ container types (e.g. ``list``). Also note that ``PyObject.ob_refcnt`` and ``_PyObject_HEAD_EXTRA`` are part of ``PyObject_HEAD``. .. _PyObject.ob_refcnt: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L107 -.. __PyGC_Head: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/internal/pycore_gc.h#L11-L20 -.. __PyObject_HEAD_EXTRA: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L68-L72 +.. _PyGC_Head: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/internal/pycore_gc.h#L11-L20 +.. _PyObject_HEAD_EXTRA: https://github.com/python/cpython/blob/80a9ba537f1f1666a9e6c5eceef4683f86967a1f/Include/object.h#L68-L72 .. _refcounting: