Python

Python

Posted by Eirik on 5 Mar, 2024

Cpython

Py_INCREF() and Py_DECREF()

While Py_INCREF() and Py_DECREF() are essential components of Python’s memory management system, they’re not the ideal way to directly count how many times a Python variable is used. Let me explain why and offer better alternatives:

Role: These functions are at the heart of Python’s reference counting garbage collection. Py_INCREF(object): Increments the reference count of an object. This indicates that the object is in use and prevents it from being prematurely destroyed. Py_DECREF(object): Decrements the reference count of an object. If the count reaches zero, Python knows the object is no longer needed and deallocates its memory. Why They’re Not Ideal for Counting Usage:

Indirect: Reference counts don’t directly equate to variable usage. A variable might hold a reference to an object, but other variables or data structures could also hold references to the same object. Internal Management: Reference counting is primarily an internal mechanism for Python’s garbage collector. Manipulating it directly can lead to memory leaks or premature deallocation if not done very carefully. Better Alternatives for Counting Variable Usage

Simple Counter: Maintain a separate counter variable. Increment it whenever you use the variable you’re interested in tracking.

my_var = 5
usage_count = 0

# Each time you use my_var:
usage_count += 1

Function Wrapping/Decorators: Create a wrapper function or a decorator that automatically increments a counter before or after the original function is called. This is useful if you want to track how often a specific function using the variable is executed.

def track_usage(func):
    def wrapper(*args, **kwargs):
        wrapper.counter += 1
        return func(*args, **kwargs)
    wrapper.counter = 0
    return wrapper

Profiling Tools: For sophisticated analysis, including precise measurements of where and how often your code uses different variables, use Python profiling tools like:

cProfile: Built-in module for profiling Python code. line_profiler: Provides line-by-line profiling information.

PyObject attribute

Complexity: Modifying the CPython source requires familiarity with C and a good understanding of Python’s internal object representation. Overhead: These modifications will introduce overhead to your Python execution. The impact depends on the complexity of your changes. Targeted Tracking: This method allows fine-grained tracking but necessitates recompiling your Python interpreter. Steps

Obtain CPython Source: Download the source code of the specific Python version you’re using from https://www.python.org/downloads/source/ Locate Relevant Structures: The primary structure to focus on is PyObject. It represents the base of all Python objects. You’ll likely need to modify this structure or related code within the Objects directory of the CPython source. Add Counters: Consider adding members to the PyObject structure to serve as counters: assignment_count modification_count access_count Instrument Code: Carefully modify the relevant portions of the CPython source where assignments, modifications, and object retrievals occur. Key places to look: Objects/object.c: Contains core object manipulation functions. Include/object.h: Header file for PyObject. Bytecode instructions within the interpreter loop (ceval.c) Recompilation: Follow the CPython build instructions to recompile your modified interpreter. Challenges and Notes

Synchronization: If your Python code is multithreaded, ensure your counter updates are thread-safe. Differentiating Modification: Distinguishing between true modifications (altering the object’s value) versus simple reassignments might require additional logic. Example (Simplified)

Let’s imagine a highly simplified modification within Objects/object.c to track object creation (similar to assignments):

/* ...existing code... */

typedef struct _object {
    /* ...existing members... */
    int creation_count; 
} PyObject;

/* ... somewhere where objects are created ... */

PyObject *new_object = /* ... object creation logic ... */
new_object->creation_count++;

Before Undertaking This

Consider whether less invasive methods might achieve your goal:

Could profiling tools provide the insights you need without CPython modifications? Would targeted function wrapping or decorators (as suggested previously) be sufficient?

venv (virtual environment)

pip show

pip list

Create venv command

python -m venv venv_name

Activate venv

cd venv/scripts ./activate

pip list

pip install flask

pip show flask

Static/Class/Abstract Methods 實現

  • Instance method：實例方法，即帶有 instance 為參數的 method，為大家最常使用的 method
  • Static method：靜態方法，不帶 instance 以及 class 為參數的 method
  • Class method：類方法，帶有 class 為參數的 method
  • Abstract method：抽象方法，尚未被實作且 children class 一定要用 override來實作之，用來實現類似 Java interface 或 C++ pure virtual function
CATALOG
    Tags