Python loops: the lesson nobody teaches you

#Why it matters

Here is something nobody tells you when you start Python: you will write loops for the rest of your career, and most of the slow, buggy code you meet at work is a loop that someone wrote without thinking.

Not a fancy algorithm. Not a bad library choice. A plain for loop doing the wrong thing a million times.

Job descriptions say things like "writes efficient, maintainable Python". In a code review, that mostly means: do your loops do the least work possible, and can the next person read them? This lesson is about getting that right.

#The concept: a loop is just a conversation

When you write this:

for n in [3, 1, 4]:
    print(n)

Python is not "going through a list". It is having a short conversation. It asks the list for an iterator, then keeps saying "next, please" until the iterator says "I'm done".

You can have that conversation yourself:

nums = [3, 1, 4]
it = iter(nums)
print(next(it), next(it), next(it))
try:
    next(it)
except StopIteration:
    print("StopIteration: the loop ends here")

Output:

3 1 4
StopIteration: the loop ends here

That is the whole secret. Every for loop in Python is iter() once, then next() until StopIteration. Once you see it this way, the rest of this lesson stops being a list of tricks and starts making sense.

#Lesson 1: the loop is rarely slow. What is inside it is.

People say "loops in Python are slow". That is half true. The bigger problem is usually one line inside the loop that quietly does a loop of its own.

Here is a real situation. You have 20,000 orders and a list of 20,000 blocked customer ids. You want the orders from blocked customers.

import random, time
random.seed(7)
orders = [random.randint(1, 2_000_000) for _ in range(20_000)]
blocked = [random.randint(1, 2_000_000) for _ in range(20_000)]

t = time.perf_counter()
flagged = [o for o in orders if o in blocked]          # list lookup
slow = time.perf_counter() - t

t = time.perf_counter()
blocked_set = set(blocked)
flagged2 = [o for o in orders if o in blocked_set]     # set lookup
fast = time.perf_counter() - t

print(len(flagged), len(flagged2))
print(f"list: {slow:.2f} s   set: {fast*1000:.1f} ms   ~{slow/fast:,.0f}x faster")

Output on my machine (Python 3.13):

192 192
list: 2.92 s   set: 3.7 ms   ~785x faster

Same answer. Same loop. One changed line.

Why? o in blocked on a list checks every item, one by one. So you are secretly running 20,000 × 20,000 = 400 million comparisons. o in blocked_set jumps straight to the answer using a hash, so it costs roughly the same no matter how big the set is.

#Lesson 2: you do not always need the whole list

Look at these two lines. They look almost the same:

import sys
squares_list = [n * n for n in range(1_000_000)]
squares_gen = (n * n for n in range(1_000_000))
print(sys.getsizeof(squares_list), "bytes")
print(sys.getsizeof(squares_gen), "bytes")
print(sum(squares_gen))
print(sum(squares_gen))

Output:

8448728 bytes
200 bytes
333332833333500000
0

The square brackets build all one million numbers and keep them in memory (about 8 MB for the list alone). The round brackets build a generator: it makes each number only when the loop says "next, please", and forgets it right after.

Notice the last line. The second sum gives 0. That is not a bug, it is the conversation again: the generator already said "I'm done", and it will not start over. A generator is a one-way trip.

So the rule is simple:

• Need to use the values once, in order (summing, writing to a file, sending to an API)? Use a generator.

• Need to use them again, index into them, or check len()? Use a list.

This matters a lot with real data. Reading a 5 GB log file with for line in open("app.log") works on a laptop with 8 GB of RAM because a file is already an iterator. Calling .readlines() first does not.

#Lesson 3: Python already wrote the loop for you

A lot of beginner loops are trying to do something Python has a built-in for. Using the built-in is shorter, harder to get wrong, and usually faster because the loop runs in C.

i = 0
for name in names:
    print(i, name)
    i += 1

for i, name in enumerate(names):
    print(i, name)

A few more you should know by heart:

any() and all() also stop early. any() quits the moment it finds one True, so it never touches the rest of the data.

#Lesson 4: the else on a loop (yes, really)

This one surprises even people with years of Python. A for loop can have an else, and it runs only if the loop finished without hitting break.

users = ["asha", "ravi", "meera"]
for name in users:
    if name == "kabir":
        print("found kabir")
        break
else:
    print("kabir is not here")

Output:

kabir is not here

Think of the else as "no break". It replaces the old pattern of setting found = False before the loop and checking it after. Use it for searches, and leave a short comment the first few times so your teammates do not trip on it.

#Common mistakes

#The lambda that remembers the wrong thing

buttons = []
for i in range(3):
    buttons.append(lambda: i)
print([b() for b in buttons])

buttons = []
for i in range(3):
    buttons.append(lambda i=i: i)
print([b() for b in buttons])

Output:

[2, 2, 2]
[0, 1, 2]

The first version does not save the value of i. It saves the name i, and looks it up later, when the loop is already over and i is 2. The i=i trick freezes the value at the moment the function is made. This bug shows up in callbacks, GUI code and anything that creates functions in a loop.

#zip that silently drops data

names = ["Asha", "Ravi", "Meera"]
marks = [81, 67]
for name, mark in zip(names, marks):
    print(name, mark)

Asha 81
Ravi 67

Meera just vanished. No error, no warning. zip stops at the shortest input. Since Python 3.10, pass strict=True when the lists are supposed to match:

ValueError: zip() argument 2 is shorter than argument 1

A loud error today beats a missing student in a report next month.

#Changing a list while looping over it

Removing items from the list you are looping over makes Python skip elements, because the positions shift under the loop. Build a new list instead: keep = [x for x in items if x.active].

#Using range(len(...)) by reflex

for i in range(len(items)): items[i] works, but it hides what you mean. Loop over the items directly, and reach for enumerate when you truly need the index.

#Interview angle

Loops are where interviewers check whether you understand cost, not syntax. Expect questions like:

• "This function is slow on large inputs. Why?" The answer is very often a list lookup (in) or a .index() call inside a loop. Say the complexity out loud: "this is O(n × m), turning blocked into a set makes it O(n + m)".

• "What is the difference between a list comprehension and a generator expression?" Talk about memory, laziness and the one-way trip.

• "What does this print?" with the lambda-in-a-loop code above. Now you know.

#Practice task

• To do:

  Take any script you wrote in the last month. Find every in inside a loop and check whether the right side should be a set.

• To do:

  Rewrite one loop that builds a big list only to sum() or write it out, so it uses a generator instead. Measure memory with sys.getsizeof or tracemalloc.

• To do:

  Replace one found = False flag with a for ... else.

• To do:

  Add strict=True to every zip where the inputs should be the same length, and see if anything breaks.

#Further reading

• Python docs: iterator types

• Python docs: generator expressions

• Python wiki: time complexity of built-in types

• PEP 618: zip(strict=True)

If you are newer to Python, start with Python basics for beginners on a real dataset and come back here.