What Is The Design Reason For The Fact That If __new__ Does Not Return An Instance Of Cls, Python Does Not Invoke __init__?

There are many questions on SO asking why python doesn't always call __init__ after object creation. The answer, of course, is found in this excerpt from the documentation: If __n

Solution 1:

__init__ does act like a constructor. It needs an instance to do its job like setting attributes and so on. If __new__ doesn't return explicitly returns an instance, then None is returned by default.

Imagine that what will happen when __init__ gets a None as an input and trying to set attributes? It will raise an exception called "AttributeError: 'NoneType' object has no attribute xxxxx".

So I think it's natural that not to invoke __init__ when __new__ returns None.

Solution 2:

In Python 2, you can't actually call a regular method with the first argument being anything other than an instance of the class (or a subclass):

class Foo(object):
    def __init__(self):
        pass

Foo.__init__()
# TypeError: unbound method __init__() must be calledwith Foo instance asfirst argument (got nothing instead)

Foo.__init__(3)
# TypeError: unbound method __init__() must be calledwith Foo instance asfirst argument (got int instance instead)

So __init__ isn't called because it cannot possibly do anything other than immediately raise an exception. Not trying to call it is strictly more useful (though I don't think I've ever seen code take advantage of this).

Python 3 has a slightly simpler method implementation, and this restriction is no longer in place, but the __new__ semantics are the same. It doesn't make a lot of sense to try to run a class's initializer on a foreign object, anyway.

---

For a more designy answer, rather than a "because it's this way" answer:

Overriding __new__ is already a weird thing to do. By default, it returns an uninitialized object, which is a concept that Python tries very hard to hide. If you override it, you're probably doing something like this:

classFoo(object):
    def __new__(cls, some_arg):
        if some_arg == 15:
            # 15is a magic number for some reason!
            return Bar()
        else:
            returnsuper(Foo, cls).__new__(cls, some_arg)

Let's imagine a Python variant that unconditionally called __init__ on the return value. I immediately see a number of problems.

When you return Bar(), should Python call Bar.__init__ (which has already been called in this case) or Foo.__init__ (which is for a completely different type and would break whatever guarantees Bar makes)?

The answer surely has to be that Bar.__init__ is called. Does that mean that you have to return an uninitialized Bar instance, using the mouthful return Bar.__new__(Bar) instead? Python very rarely requires you to call dunder methods outside of using super, so this would be highly unusual.

Where would Bar.__init__'s arguments come from? Both Foo.__new__ and Foo.__init__ are passed the same arguments — those passed to type.__call__, which is what handles Foo(...). But if you explicitly call Bar.__new__, there's nowhere to remember the arguments you wanted to pass to Bar.__init__. You can't store them on the new Bar object, because that's what Bar.__init__ is supposed to do! And if you just said it gets the same arguments that were passed to Foo, you severely limit what types can be returned from __new__.

Or, what if you wanted to return an object that already exists? Python has no way to indicate that an object is "already" initialized — since uninitialized objects are a transient and mostly-internal thing only of interest to __new__ — so you'd have no way to say not to call __init__ again.

The current approach is a little clumsy, but I don't think there's any better alternative. __new__ is supposed to create storage space for the new object, and returning a different type of object altogether is just a really weird thing to do; this is the least-surprising and most-useful way Python can handle it.

If this limitation is getting in your way, remember that the entire __new__ and __init__ dance is just the behavior of type.__call__. You're perfectly free to define your own __call__ behavior on a metaclass, or just swap your class out for a factory function.

Solution 3:

This is probably not THE design reason, but a nifty consequence if this design decision is that a class does not HAVE to return an instance of itself:

classFactoryClass():
    def __new__(klass, *args, **kwargs):
        if args or kwargs:
            return OtherClass(*args, **kwargs)
        returnsuper().__new__(klass)

This could be useful sometimes. Obviously if you're returning some other class object, you don't want the init method of the factory class to be called.

The real reason which I hope the above example illustrates, and the bottom line, is that any other design decision would be nonsensical. New is the constructor for Python class objects, not init. Init is no different from any other class method (other than it being auto magically called after the object has been constructed); whether or not any of them get called depends on what happens in the constructor. This is just a natural way to do things. Any other way would be broken.

Solution 4:

If __new__ doesn't return an instance of cls, then passing the return value to cls.__init__ could lead to very bad things.

In particular, Python doesn't require you to return an instance of the same type of class.

Take this contrived example:

In [1]: class SillyInt(int):
   ...:     def __new__(cls):
   ...:         return "hello"
   ...:     

In [2]: si = SillyInt()

In [3]: si
Out[3]: 'hello'

So we've created a subclass of int, but our __new__ method returned a str object.

It wouldn't make a great deal of sense to then pass that string to our inherited int.__init__ method for instantiation. Even if, in the particular case in question, it 'works' (in the sense of not throwing any errors), we may have created an object in an inconsistent state.

---

Ok, so what about a more concrete (albeit still contrived) example?

Say we create two classes which set the same attribute as part of their initialisation (in the __init__ method):

In [1]: classTree():
   ...:     def__init__(self):
   ...:         self.desc = "Tree"
   ...:         

In [2]: classBranch():
   ...:     def__init__(self):
   ...:         self.desc = "Branch"
   ...:         

If we now create a subclass which, in a custom __new__ method, wants to return a different type of object (which may sometimes make sense in practice, even if it doesn't here!):

In [3]: class SillyTree(Tree):
   ...:     def __new__(cls):
   ...:         return Branch()
   ...:     

When instantiating this subclass, we can see that we've got an object of the expected type, and that the Branch initialiser was the one that was called:

In [4]: sillyTree = SillyTree()

In [5]: sillyTree.desc
Out[5]: 'Branch'

What would have happened if Python had unconditionally called the inherited initialiser?

Well, we can actually test this by calling the initialiser directly.

What we end up with is an instance of Branch which was initialised as a Tree, leaving the object in a very unexpected state:

In [6]: SillyTree.__init__(sillyTree)

In [7]: sillyTree.desc
Out[7]: 'Tree'In [8]: isinstance(sillyTree, Branch)
Out[8]: TrueIn [9]: isinstance(sillyTree, Tree)
Out[9]: False