Python List Comprehension Vs For

Why does list comprehension have better performance than a for loop, in Python? list comprehension: new_items = [a for a in items if a > 10] for loop: new_items = [] for a in i

Solution 1:

Essentially, list comprehension and for loops does pretty similar things, with list comprehension doing away some overheads and making it look pretty. To understand why this is faster, you should look in Efficiency of list comprehensions and to quote the relevant part for your problem:

List comprehensions perform better here because you don’t need to load the append attribute off of the list (loop program, bytecode 28) and call it as a function (loop program, bytecode 38). Instead, in a comprehension, a specialized LIST_APPEND bytecode is generated for a fast append onto the result list (comprehension program, bytecode 33).

In the loop_faster program, you avoid the overhead of the append attribute lookup by hoisting it out of the loop and placing the result in a fastlocal (bytecode 9-12), so it loops more quickly; however, the comprehension uses a specialized LIST_APPEND bytecode instead of incurring the overhead of a function call, so it still trumps.

The link also details some of the possible pitfalls associated with lc and I would recommend you to go through it once.

Solution 2:

Assuming we're talking CPython here, you could use the dis module to compare the generated bytecodes:

>>defone():
       return [a for a in items if a > 10]

>>deftwo():
       res = []
       for a initems:if a > 10:
               res.append(a)

>> dis.dis(one)

  20 BUILD_LIST               03 LOAD_GLOBAL              0 (items)
              6 GET_ITER
        >>    7 FOR_ITER                24 (to 34)
             10 STORE_FAST               0 (a)
             13 LOAD_FAST                0 (a)
             16 LOAD_CONST               1 (10)
             19 COMPARE_OP               4 (>)
             22 POP_JUMP_IF_FALSE        725 LOAD_FAST                0 (a)
             28 LIST_APPEND              231 JUMP_ABSOLUTE            7
        >>   34 RETURN_VALUE

>> dis.dis(two)
  20 BUILD_LIST               03 STORE_FAST               0 (res)

  36 SETUP_LOOP              42 (to 51)
              9 LOAD_GLOBAL              0 (items)
             12 GET_ITER
        >>   13 FOR_ITER                34 (to 50)
             16 STORE_FAST               1 (a)

  419 LOAD_FAST                1 (a)
             22 LOAD_CONST               1 (10)
             25 COMPARE_OP               4 (>)
             28 POP_JUMP_IF_FALSE       13531 LOAD_FAST                0 (res)
             34 LOAD_ATTR                1 (append)
             37 LOAD_FAST                1 (a)
             40 CALL_FUNCTION            143 POP_TOP
             44 JUMP_ABSOLUTE           1347 JUMP_ABSOLUTE           13
        >>   50 POP_BLOCK
        >>   51 LOAD_CONST               0 (None)
             54 RETURN_VALUE

So for one thing, the list comprehension takes advantage of the dedicated LIST_APPEND opcode which isn't being used by the for loop.