Merging Classes: an alternative path to the SOLIDness

Image by the Author

Is it considered a taboo in Python to merge classes dynamically at runtime, as one would do with dictionaries?

Short answer: there should be no inherent taboos in coding. However, simply merging classes dynamically in Python may have more drawbacks than benefits unless you employ specific techniques. Hence, I came up with the idea of creating DynDesign, a Python package that brings together all the necessary techniques and tools to dynamically merge classes in Python.

Motivation

When writing code at an advanced level, you bear in mind principles like SOLID, DRY, KISS, YAGNI , (…) as mantras. Indeed, it can be difficult to apply these principles consistently, particularly in complex projects where you need to make trade-offs and compromises to strike a balance between them. Even when you use specific Design Patterns as a solution, you may realize that it may lead you closer to certain principles but farther from others.

In contrast, the process of merging dictionaries is relatively straightforward , even for beginners, and it is not exclusive to Python but rather a standard across many programming languages. Thus, it is logical to extend this concept to classes, especially since in Python classes are internally managed exactly using __dict__ .

Merging vs Inheriting

The next question that may arise is: why not simply use Python class inheritance, a built-in standard feature that is generally employed? The answer lies in the “Dyn” prefix of the package name: you can merge classes dynamically, whereas you cannot inherit from classes dynamically since class inheritance is purely static in Python. In fact, another way to view class merging is as a sort of dynamic class inheritance .

Furthermore, DynDesign’s implementation of class merging synergizes with the capabilities of static class inheritance to broaden the range of potential use cases, as documented here.

Class Merging in action

Now, let’s observe class merging in action: the chosen testing ground is the Collatz sequences. The Collatz sequences are generated by a mathematical function that takes a positive integer as input and recursively applies a series of operations to it. If the input is even, the function divides it by 2, while if the input is odd, it multiplies it by 3 and adds 1.

The function continues to operate on the resulting number in the same way, producing a sequence of numbers that eventually reaches the value 1, regardless of the starting number. For example, if the starting number is 27, applying the function recursively leads to the sequence reaching up to 9.232 before descending to 1.

Below is a possible implementation of a Python script taking an initial number as argument and applying the Collatz function until it reaches 1 (the code for the examples presented in this article can also be found here):

# collatz.py

import argparse

class Collatz:
    FUNCTION_NAME = 'Collatz'

    def __init__(self, n):
        self.n = int(n)

    def get_next_n(self, n):
        if n % 2 == 0:
            return n // 2
        else:
            return n*3 +1

    def get_collatz_sequence(self, n):
        while n != 1:
            n = self.get_next_n(n)
            yield n

    def output_number(self, n):
        print(f"{n}; ", end='')

    def output_sequence(self):
        print(f"{self.FUNCTION_NAME} sequence starting from {self.n} is:")
        self.output_number(self.n)
        for n in self.get_collatz_sequence(self.n):
            self.output_number(n)
        print()


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description='Calculate Collatz sequence.')
    parser.add_argument('n')
    args = parser.parse_args()
    collatz = Collatz(args.n)
    collatz.output_sequence()

If the script is tested with a starting number of 57, the result obtained is:

$ python collatz.py 57
Collatz sequence starting from 57 is:
57; 172; 86; 43; 130; 65; 196; 98; 49; 148; 74; 37; 112; 56; 28; 14; 7; 22;
11; 34; 17; 52; 26; 13; 40; 20; 10; 5; 16; 8; 4; 2; 1;

It is easy to note that the class “Collatz” does not conform to the Single Responsibility Principle (SRP) of the SOLID principles because it includes both methods to calculate the sequence and methods to output the result.

Thanks to DynDesign, class “Collatz” can be swiftly split into two distinct specialized classes, namely “CollatzSequence” and “CollatzOutput”.

class CollatzSequence:
    FUNCTION_NAME = 'Collatz'

    def __init__(self, n):
        self.n = int(n)

    def get_next_n(self, n):
        if n % 2 == 0:
            return n // 2
        else:
            return n*3 +1

    def get_collatz_sequence(self, n):
        while n != 1:
            n = self.get_next_n(n)
            yield n


class CollatzOutput:
    def output_number(self, n):
        print(f"{n}; ", end='')

    def output_sequence(self):
        print(f"{self.FUNCTION_NAME} sequence of {self.n} is:")
        self.output_number(self.n)
        for n in self.get_collatz_sequence(self.n):
            self.output_number(n)
        print()

Then, the classes “CollatzSequence” and “CollatzOutput” are simply merged into a new class called “CollatzMerged” using mergeclasses , before instantiating the class:

from dyndesign import mergeclasses

...
CollatzMerged = mergeclasses(CollatzSequence, CollatzOutput)
collatz = CollatzMerged(args.n)
collatz.output_sequence()

In this example, class “CollatzMerged” is created by merging the attributes and methods of class “CollatzSequence” with those of class “CollatzOutput”, so that the instance “collatz” of the resulting class can safely access the class and instance attributes (such as “self.n”) and the methods from both merged classes.

NOTE: This example is only intended to show the fundamental functionality of *mergeclasses* , and achieving the same result is clearly also possible through conventional techniques such as class composition. However, in the next example, the additional benefits of merging classes dynamically are clearly demonstrated.

Conditional Class Merging

Suppose that the script needs to support an alternate version of the Collatz function with similar properties, and an optional argument “-c” must be added so as to switch between the original Collatz function and the alternate version. The alternate Collatz-like function, being based on a tripartition of integers, is referred to as “ternary”.

This change can be easily implemented byfurther extending the merged class with a new class “CollatzCustom”,

class CollatzCustom:
    FUNCTION_NAME = 'Collatz-like ternary'

    def get_next_n(self, n):
        if n % 3 == 0:
            return n // 3
        elif n % 3 == 1:
            return n*2 +1
        else:
            return n*3 -2

which is merged only if the argument “-c” is passed to the script:

...
parser.add_argument('-c', action='store_true', dest='custom_collatz',
                    help='use custom Collatz function instead')
args = parser.parse_args()
CollatzMerged = mergeclasses(CollatzSequence, CollatzOutput)
if args.custom_collatz:
    CollatzMerged = mergeclasses(CollatzMerged, CollatzCustom)
collatz = CollatzMerged(args.n)
collatz.output_sequence()

It is noted that: - Merged classes (i.e., “CollatzMerged”) can be merged in turn with other classes (i.e., “CollatzCustom”). - Class attribute “FUNCTION_NAME” is implemented in both component classes “CollatzCustom” and “CollatzSequence”. In this case the class merger follows a rightmost-win rule similar to the one applied when merging dictionaries: the attribute assumes the value assigned in the rightmost merged class, following the order in which the classes are merged. - Method “get_next_n” is implemented in both component classes as well. The rightmost-win rule is also applied when merging methods: the methods in the rightmost merged classes override those in the leftmost classes.

If the script is tested with the same input, it produces the same output as before. Otherwise, if the “-c” option is added, the same number 57 is processed using the ternary function, and the number sequence shows a shorter lifespan before reaching 1:

$ python collatz.py -c 57
Collatz-like ternary sequence starting from 57 is:
57; 19; 39; 13; 27; 9; 3; 1;

Invoking multiple constructors

Next, the behavior of the Collatz functions is visualized by using an optional argument “-g” in the script to output the result to a graph instead of sending to terminal.

Similar to the previous example, adding this new feature to the script mainly involves creating a new class and merging it with the others if the corresponding option is selected. Below is the new class’s code:

import matplotlib.pyplot as plt

class CollatzGraph:
    def __init__(self):
        self.sequence = []

    def output_number(self, n):
        self.sequence.append(n)

    def output_graph(self):
        plt.plot(self.sequence)
        plt.xlabel("Step")
        plt.ylabel("Value")
        plt.title(f"{self.FUNCTION_NAME} Function")
        plt.show()

The body of the script is modified as follows,

...
parser.add_argument('-g', action='store_true', dest='collatz_graph')

...
if args.collatz_graph:
    CollatzMerged = mergeclasses(CollatzMerged, CollatzGraph)
collatz = CollatzMerged(args.n)
collatz.output_sequence()
if args.collatz_graph:
    collatz.output_graph()

where it is noted that method “output_graph” is explicitly invoked before the end of the script if option “-g” is set.

The first thing that catches the eye is that the class “CollatzGraph” has another constructor __init__ in addition to the one in the class “CollatzSequence”. This highlights the fundamental rule followed when merging classes: all methods and attributes are overloaded by default using the rightmost-win rule, except for the constructor __init__ , which is attempted to be invoked for all the instances of the merged classes.

Arguments passed to each constructor are adaptively filtered based on the constructor signature so that each constructor takes just the arguments it needs.

In this example, the “CollatzGraph” constructor is called after the “CollatzSequence” constructor, and it is observed that “CollatzGraph” constructor does not take any arguments (excepted self ) while “CollatzSequence” constructor takes “n” as argument. If “CollatzGraph” were to be directly instantiated with CollatzGraph(args.n) , it would raise the exception

TypeError: CollatzGraph.__init__() takes 1 positional argument but 2 were given

Conversely, thanks to the adaptive argument mechanism implemented in DynDesign, the exceeding arguments are filtered out and the constructor of “CollatzGraph” is executed normally, with “self.sequence” initialized to “[]”. Furthermore, it is worth noting that the method “output_number” is overloaded to store the current sequence number in “self.sequence” instead of printing it to the screen. For this reason, if the script is launched with the “-g” option it results in

$ python collatz.py -g  57
Collatz sequence starting from 57 is:

$ python collatz.py -gc  57
Collatz-like ternary sequence starting from 57 is:

and the graphs below are displayed:

Invoking multiple instances of methods

Actually, the output printed by the previous version of the script might be considered not acceptable: the method “output_number” in class “CollatzGraph” overrides the method with the same name in class “CollatzOutput”, causing the sequence numbers to be missed after the messages “Collatz(-like ternary) sequence starting from … is:” are printed. Depending on the interpretation, the output may be considered acceptable given that the sequence output is displayed in the graph instead of being printed to the terminal, or, on the other hand, it may be deemed a bug.

In the latter case, the bug would be super-easy to fix by leveraging the optional argument invoke_all of mergeclasses .

The invoke_all option enables passing a list of method names whose instances need to be invoked all, as opposed to being overloaded, similar to what happens by default for __init__ . By passing “output_number” in the list, the instance of the method in class “CollatzOutput” is also executed at the method call and the sequence numbers are printed as well with the “-g” option set:

...
if args.collatz_graph:
    CollatzMerged = mergeclasses(
        CollatzMerged,
        CollatzGraph,
        invoke_all=['output_number']
    )

For example, running collatz.py -gc 57 now results in printing the numbers as well as in displaying the graph:

$ python collatz.py -gc 57
Collatz-like ternary sequence starting from 57 is:
57; 19; 39; 13; 27; 9; 3; 1;

Conclusion

DynDesign provides a powerful and flexible alternative approach to class design and composition, enabling developers to easily create and manage complex and modular code while still adhering to SOLID principles.

In the basic examples discussed in this article, it is shown how to associate each optional argument of a script with a corresponding class that implements the option’s functionalities. The class can be merged with a set of base classes or not, depending on whether the corresponding option is set or not. It is also shown that such functionalities combine together with any combination of script options.

Written by Patrizio Gelosi