Static variable vs Class variable

I've got a question on the differences and how to define static and
class variables. AFAIK, class methods are the ones which receives the
class itself as an argument, while static methods are the one which
runs statically with the defining class.

Hence, my understanding is that static variables must be bound to the
class defining the variables and shared by children of parent class
where the variable is defined. But, please have a look at this code in
which a guy told me that the variable a is static:

>>> class Foo:
a = 1
@classmethod
def increment(cls):
cls.a += 1
print cls.a

Here, I am defining variable a which, I believe is class variable,
i.e., variable that is not bound to Foo itself. Rather, a is bound to
the class which is accessing the variable. The code that corroborates
this idea is as follows:

>>> class Child1(Foo):
pass

>>> Child1.increment()
4

>>> class Child2(Foo):
pass

>>> Child2.increment()
4

This means that Child1 and Child2 does not share variable a which
means that variable a is class variable rather than static variable.

Could you please comment on this? Is a static or class variable?
What's the most recent way of defining 'class' and 'static' variables?

Thanks.
- Minkoo

cls.a += 1

It does the following thing:

#1. read cls.a
#2. add one
#3. assign this value to cls.a

In point #3, you really bind a name to a value. As you probably know, in
Python, there are names and objects. The initial value of the name 'a'
is 1. It is an immutable object. The "+=" operator usually increments a
value of an object. However, because the 'int' type is immutable, the +=
operator will rather rebind this variable to a newly created value. I
believe this is what is happening here. Your question "is variable a
static or class variable?" has no real answer. After running the
increment() method on a descendant class, e.g. Child1 will rebind the
name Child1.a, creating a new name in the namespace of the class. So the
variable Foo.a is still there, but you are accessing Child1.a instead.

If you want to HANDLE a as a static variable, you can handle it with a
static method. That won't bind a new name in the descendant class.
(However, you can still rebind it, e.g. "Child.a=42")

Now here is a good question: how do you handle a variable as static,
from a class (not static) method? Here is an example:

>>> class Foo(object):
... a = 1
... @classmethod
... def increment(cls):
... Foo.a += 1
... print cls.a
...
>>> class Child1(Foo):
... pass
...
>>> Child1.increment()
2
>>> Child1.increment()
3
>>> Foo.a
3
>>> Child1.a = 10
>>> Child1.increment()
10
>>> Child1.increment()
10
>>> Child1.increment()
10
>>> Foo.a
6
>>>

However, the question is: why would you do this? :-)

BTW you should use new style classes whenever it is possible. Old style
classes will have gone...

Hope this helps,

Laszlo

First you have to define what you mean by "static".

`classmethod`\s receive the class as first arguments, `staticmethod`\s are
just functions bound to the class object.

Ask the guy what he means by "static".

No you define a class attribute that *is* bound to the class `Foo`.

Four!? Hard to believe.

There is no such thing as a "static" variable. Think of attributes that
are bound to objects. All dynamically.

What happens is: you bind a 1 to the attribute `Foo.a` in the `Foo` class
definition.

When you call `Child1.increment()` the class method will be called with
`Child1` as first argument. Now ``cls.a += 1`` is executed which is
somewhat like a short form of ``cls.a = cls.a + 1``. So this is reading
the attribute `a` from `Child1` and then bind the result to `Child1`.
`Child1` doesn't have an attribute `a`, so it is looked up in the parent
class. But the result is then bound to `Child1`. So you are reading from
`Foo` and writing to `Child1`. That's it.

Ciao,
Marc 'BlackJack' Rintsch

(I'm sorry, probably you already knew this.)

Laszlo

In Python `a' is considered a class variable. By modifying `a' using a
class method though, you are essentially re-binding `a' to the class
that it is called from. So, it will be shared by multiple instances of
the same class, but not derivatives. The re-binding only occurs when
`increment' is called. This makes for some very confusing behavior.
I'm not aware of a way, in python, to explicitly specify a variable as
static. For the behavior I think you are looking for you just need to
modify it carefully. This can be done with a static, class or instance
method. Though, using a class method is kind of silly. I'm sure some
would argue that using an instance method is also silly.

code:

class C(object):
a = 0
def inc(self):
C.a += 1 # directly modify the variable on the base class that it is
attached to
return C.a

# or you could use a static method, which is fine since the instance
isn't being used. This
# will also allow the method to be called without instantiating the
class.

class C(object):
a = 0
@staticmethod
def inc():
C.a += 1
return C.a

Matt

Your believes aside, this is simply wrong. The statement

a += x

always leads to a rebinding of a to the result of the operation +. I
presume you got confused by the somewhat arbitrary difference between

__add__

and

__iadd__

that somehow suggest there is an in-place-modification going on in case
of mutables.

but as the following snippet shows - that's not the case:

class Foo(object):
def __add__(self, o):
return "__add__"

def __iadd__(self, o):
return "__iadd__"

a = Foo()
a += 1
print a

a = Foo()
b = Foo()
c = a + b
print c

So you see, the first += overrides a with the returned value of __iadd__.

The reason for the difference though is most probably what you yourself
expected: thus it's possible to alter a mutable in place.

Diez

Not true.

>>> L = []
>>> id(L)
3083496716L
>>> L += [1]
>>> id(L)
3083496716L

It's the same L, not rebound at all.

The __iFOO__ methods are supposed to do in-place modification if
possible, but it is up to the class writer to make them do so. In the
case of your example, you specifically created an __iadd__ method that
didn't even attempt in-place modification. What did you expect it to do?

That's because you told it to do that. If you told it to do something
more sensible, it would have done so. Lists know how to do in-place
modification:

>>> id(L) # from above
3083496716L
>>> L *= 5
>>> id(L)
3083496716L
>>> L = L*5
>>> id(L)
3083496972L

--
Steven.

I'm not sure about the language level. It is told that "+=" is an
inplace operator, but how "in place" is defined? If we say "an inplace
operator will change the state of an object, and will never create a new
object" then "+=" is NOT an inplace operator. Well, at least not when
you watch it from the language level. But what about the implementation
level? Here is an example:

a = 1
a+= 1 # The compiler will probably optimize this and the Python bytecode
interpreter will not rebind 'a' here, just increment the integer in memory.

This topic starts to be very interesting. Language lawyers and
developers, please help us! How "in place operator" is defined in
Python? What makes it an "in place operator"? Is it the syntax, or what?

Laszlo

It *is* rebound. To the same object, but it *is* assigned to `L` and not
just mutated in place.

In [107]: class A:
.....: a = list()
.....:

In [108]: class B(A):
.....: pass
.....:

In [109]: B.a += [42]

In [110]: A.a
Out[110]: [42]

In [111]: B.a
Out[111]: [42]

If it was just mutation then `B.a` would have triggered an `AttributeError`.

Ciao,
Marc 'BlackJack' Rintsch

Yes, it is.

Just because the __iadd__-operation in the list implemented chose to
returen a reference to itself - certainly a sensible choice, I never
doubted that - doesn't change the fact that python rebinds the name on
the left with whatever __iadd__ returned.

Diez

I'm afraid not.

As I admitted in my reply to Marc, I overstated my case by saying that L
isn't rebound at all. Of course it is rebound, but to itself.

However, it is not true that += "always leads to a rebinding of a to the
result of the operation +". The + operator for lists creates a new list.
+= for lists does an in-place modification:

>>> L = []
>>> M = L
>>> L += [1]
>>> M
[1]

Compare with:

>>> L = []
>>> M = L
>>> L = L + [1]
>>> M
[]

You said:

modification going on in case of mutables but as the following snippet
shows - that's not the case: ..."

That's an explicit denial that in-place modification takes place, and
that's *way* off the mark. I was concentrating so hard on showing in-
place modification that I glossed over the "return self" part.

--
Steven.

No. This is Python, not C. You can't increment integers in memory.
Integers are immutable objects, not raw bytes:

>>> x = 1
>>> id(x)
150830896
>>> x += 1
>>> id(x)
150830884

--
Steven.

> Picky picky.
>
> Yes, technically there is an assignment of L to itself. I was sloppy to
> say "not rebound at all", because when you write an augmented assignment
> method you have to return self if you want to implement in-place
> mutation. But I hardly call "rebinding to itself" any sort of rebinding
> worth the name :)

Maybe picky but that detail was the source of the OPs confusion because it
introduced a new attribute on the subclass.

Ciao,
Marc 'BlackJack' Rintsch

Doh. Stupid me. Of course it is - but to a ref to the same object...

>>> class A:
... l = []
...
>>> class B(A): pass
...
>>> B.__dict__
{'__module__': '__main__', '__doc__': None}
>>> B.l
[]
>>> B.l += [1]
>>> B.__dict__
{'__module__': '__main__', '__doc__': None, 'l': [1]}
>>>

Thanks Diez for pointing this out. And as far as I'm concerned: time to
bed :(

Nope.

>>> A.l
[]
>>> A.l += [1]
>>> A.l
[1]
>>> B.l
[1]
>>>
>>> B.l is A.l
True

>>> B.l += [2]
>>> A.l
[1, 2]
>>> B.l
[1, 2]
>>> A.l is B.l
True
>>>

What's a "static" variable ? A variable that doesn't move ?-)

You mean "class attributes". Thinking in terms of another language won't
help understanding Python's objec model.

Then don't listen what this guy says.

a class attribute

Yes it is. It's an attribute of the class object Foo.

False.
>>> dir(Foo)
['__doc__', '__module__', 'a', 'increment']
>>> Foo.__dict__
{'a': 1, '__module__': '__main__', '__doc__': None, 'increment':
<classmethod object at 0x402d141c>}

???

I don't have the same result here. But anyway: the increment method is
rebinding 'a', so the first time it's called on a child class, it
creates an attribute 'a' for this class. Just change your code this way
and you'll see what happens:

def increment(cls):
print dir(cls)
cls.a += 1
print dir(cls)

FWIW, if you don't get this, then you may encounter another common gotcha:

class Truc:
a = 1
def gotcha(self):
self.a += 1
print self.a

print Truc.a
t = Truc()
print t.a
t.gotcha()
print Truc.a
t2 = Truc()
print t2.a
t2.gotcha()
print Truc.a
t.gotcha()
print t.a
print t2.a
print Truc.a

Enjoy !-)

Now back to your snippet : if you don't want this behaviour, wrap 'a'
into a mutable container:

@classmethod
def increment(cls):
cls.a[0] += 1
print cls.a[0]

class Child(Foo):
pass

And if you want to hide that, use properties (warning: you need nex
style classes here):

class Foo(object):
_a = [1]

@classmethod
def increment(cls):
cls._a[0] += 1
print cls._a[0]

@apply
def a():
def fget(self):
return type(self)._a[0]
def fset(self, val):
type(self)._a[0] = val
return property(**locals())

class Child(Foo):
pass

Nope. This only means that you don't understand the (somewhat peculiar)
semantics of bindings, immutable objects and attribute lookup rules in
Python.

It's a class attribute.

No 'static variable' in Python. And what you call a 'class variable' is
nothing else than an ordinary of the class object

It still is true.

a += b

rebinds a. Period. Which is the _essential_ thing in my post, because
this rebinding semantics are what confused the OP.

Admittedly, I miss _one_ word here: necessarily before the "an".

And I was concetrating so hard on the rebinding-part, I glossed over the
in-place-modification part.

Diez

Which I just came around to show in a somewhat enhanced example I could
have used the first time:

class Foo(object):

a = []

@classmethod
def increment(cls):
print cls
cls.a += [1]

class Bar(Foo):
pass

Foo.increment()
Bar.increment()

print Foo.a
print Bar.a
Bar.a = []
print Foo.a
print Bar.a

192:~/projects/SoundCloud/ViewAnimationTest deets$ python /tmp/test.py
<class '__main__.Foo'>
<class '__main__.Bar'>
[1, 1]
[1, 1]
[1, 1]
[]

Which makes the rebinding-part of __iadd__ pretty much an issue, don't
you think?

Diez

Doesn't this depend on wether "a" supports __iadd__ or not? Section
3.4.7 of the docs say

"""
If a specific method is not defined, the augmented operation falls
back to the normal methods. For instance, to evaluate the expression x
+=y, where x is an instance of a class that has an __iadd__() method,
x.__iadd__(y) is called. If x is an instance of a class that does not
define a __iadd__() method, x.__add__(y) and y.__radd__(x) are
considered, as with the evaluation of x+y.
"""

So if a.__iadd__ exists, a += b is executed as a.__iadd__(b), in which
case there's no reason to rebind a.

However, this confuses the heck out of me:

>>> B.l.append('1')
>>> B.l
['1']

>>> B.l.__iadd__('2')
['1', '2']
>>> B.l
['1', '2']

>>> B.l += '3'
>>> B.__dict__
{'__module__': '__main__', '__doc__': None, 'l': ['1', '2', '3']}

Why is B.l set for the += case only? B.l.__iadd__ obviously exists.

Paul

No. As shown several times in this thread already.

`__iadd__` *may* doing the addition in place and return `self` but it is
also allowed to return a different object. So there is always a rebinding.

Here you see that the method actually returns an object!

Because there is always a rebinding involved.

Ciao,
Marc 'BlackJack' Rintsch

<snip>

You misunderstand the documentation, what you quoted doesn't say that
the assignment is suppressed. If a.__iadd__ exists, a += b is executed
as a = a.__iadd__(b)

The options for a+=b are:

a = a.__iadd__(b)
a = a.__add__(b)
a = b.__radd__(a)

but the assignment always happens, it is only what gets executed for the
right hand side which varies.

Curious, do you have the relevant section in the docs that describes
this behaviour?

Paul

Yes, but mostly by implication. In section 3.4.7 of the docs, the sentence
before the one you quoted says:

These methods should attempt to do the operation in-place (modifying
self) and return the result (which could be, but does not have to be,
self).

The 'does not have to be self' tells you that the result of __iadd__ is
used, i.e there is still an assignment going on.

Just read all of that paragraph carefully. It says that if there is no
__iadd__ method it considers calling __add__/__radd__. Nowhere does it say
that it handles the result of calling the methods differently.

Right, the paragraph is actually pretty clear after a second reading.
I find it surprising nonetheless, as it's easy to forget to return a
result when you're implementing a method that does an in-place
operation, like __iadd__:

>>> class C:
... def __init__(self, v):
... self.v = v
... def __iadd__(self, other):
... self.v += other
...
>>> c=C(1)
>>> c.v
1
>>> c += 3
>>> c
>>> c is None
True

Paul

I've recently been bitten by that, and I don't understand the
reasoning behind __iadd__'s design. I mean, what is the point of an
*in-place* add operation (and others) if it doesn't always work
in-place?

A very common use case is using it to increment a number:

x += 1

If += always had to work inplace then this would throw an exception: an
inplace addition would be meaningless for Python numbers.

> Hrvoje Niksic <hni...@xemacs.org> wrote:
>
>> I've recently been bitten by [rebinding the var to what __iadd__
>> returns], and I don't understand the reasoning behind __iadd__'s

I'm aware of that; but remember that there's still __add__. It would
be sufficient for numbers not to implement __iadd__. And, in fact,
they already don't:

>>> 1 .__add__(1)
2
>>> 1 .__iadd__(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'int' object has no attribute '__iadd__'

The current implementation of += uses __add__ for addition and
__iadd__ for addition that may or may not be in-place. I'd like to
know the rationale for that design.

Simply not to introduce special cases I guess. If you write ``x.a += b``
then `x.a` will be rebound whether an `a.__iadd__()` exists or not.
Otherwise one would get interesting subtle differences with properties for
example. If `x.a` is a property that checks if the value satisfies some
constraints ``x.a += b`` would trigger the set method only if there is no
`__iadd__()` involved if there's no rebinding.

Ciao,
Marc 'BlackJack' Rintsch

Everything you need is in the PEP:

http://www.python.org/dev/peps/pep-0203/

--
Steven.

Apart from the obvious short answer of being consistent (so you don't
have to guess whether or not a+=b is going to do an assignment), I think
the decision was partly to keep the implementation clean.

Right now an inplace operation follows one of three patterns:

load a value LOAD_FAST or LOAD_GLOBAL
do the inplace operation (INPLACE_ADD etc)
store the result (STORE_FAST or STORE_GLOBAL)

or:

compute an expression
DUP_TOP
LOAD_ATTR
do the inplace operation
ROT_TWO
STORE_ATTR

or:

compute two expressions
DUP_TOPX 2
BINARY_SUBSCR
do the inplace operation
ROT_THREE
STORE_SUBSCR

I'm not sure I've got all the patterns here, but we have at least three
different cases with 0, 1, or 2 objects on the stack and at least 4
different store opcodes.

If the inplace operation wasn't always going to store the result, then
either we would need 4 times as many INPLACE_XXX opcodes, or it would
have to do something messy to pop the right number of items off the
stack and skip the following 1 or 2 instructions.

I see from PEP203 that ROT_FOUR was added as part of the implementation,
so I guess I must have missed at least one other bytecode pattern for
augmented assignment (which turns out to be slice assignment with a
stride).

Unfortunately that doesn't work very well. If the validation rejects the
new value the original is still modified:

>>> class C(object):
def setx(self, value):
if len(value)>2:
raise ValueError
self._x = value
def getx(self):
return self._x
x = property(getx, setx)

>>> o = C()
>>> o.x = []
>>> o.x += ['a']
>>> o.x += ['b']
>>> o.x += ['c']

File "<pyshell#27>", line 1, in <module>
o.x += ['c']
File "<pyshell#22>", line 4, in setx
raise ValueError
ValueError
>>> o.x
['a', 'b', 'c']
>>>

Now that's really interesting. I added a print "before" and print
"after" statement just before and after the self._x = value and these
*do not get called* after the exception is raised when the third
element is added.

Well, of course not. Did you really expect that!? Why?

Ciao,
Marc 'BlackJack' Rintsch

Because o.x *is* updated, i.e. the net result of self._x = value is
executed.

Paul

Argh, the getter is of course used to append the third element, after
which the rebinding triggers the exception. Got it now...

Paul

Note that such constraint is easily invalidated simply with:

foo = x.a
foo += b

The implementation, by necessity, supports a lot of historical cruft,
so keeping it clean was not a priority. (This is not a criticism, I
like how they kept backward compatibility.)

[...]

Thanks for pointing it out; I didn't realize just how much work went
into the new assignment opcodes. Given the complexity, it's a wonder
they're there at all!

"Steven D'Aprano" <st...@REMOVE-THIS-cybersource.com.au> wrote in message
news:13hbvu6...@corp.supernews.com...

Which illustrates that the proposal, while simplified for implementation,
is not exactly what was desired*

""" is both more readable and less error prone, because it is
instantly obvious to the reader that it is <x> that is being
changed, and not <x> that is being replaced
"""

As we see from this thread, it is not instantly obvious to the reader;
the meaning of "changed, not replaced" is ambiguous.

[david]

* That is, unless ambiguity was the desideratum