Comprehensions and iteration tools

Make a day11 folder for today’s files, open it in your editor, and run each script from its terminal, as on earlier days.

Since Day 6 you have built many lists the same way: start with an empty list, loop over something, and append a result on each pass. Since Day 8 you have built dictionaries the same way too. The pattern works, but it takes three or four lines to say one thing: “a list of this for each item in that”. Today you learn Python’s one-line way to say it, the comprehension, plus a handful of built-in tools that make loops shorter and clearer: zip, enumerate, reversed, sum, any, and all. You will also learn the opposite skill, which matters just as much: recognizing when a plain loop is the better choice.

A weather app stores temperatures in Celsius and needs them in Fahrenheit. With what you know from Days 5 and 6, you would write the accumulator pattern for a list: an empty list before the loop, append inside it, and the finished list after it.

celsius = [0, 12, 25, 31]
fahrenheit = []
for c in celsius:
    fahrenheit.append(c * 9 / 5 + 32)
print(fahrenheit)

[32.0, 53.6, 77.0, 87.8]

Look at what the four lines actually say: “fahrenheit is c * 9 / 5 + 32 for each c in celsius”. Only two pieces of information matter: what to put in the new list (c * 9 / 5 + 32) and where the values come from (for c in celsius). The empty list and the append are scaffolding that every list-building loop repeats.

A list comprehension builds a new list by writing those two pieces inside square brackets. Here is the same program:

celsius = [0, 12, 25, 31]
fahrenheit = [c * 9 / 5 + 32 for c in celsius]
print(fahrenheit)

[32.0, 53.6, 77.0, 87.8]

The output is identical. Line 2 does the whole job of lines 2 to 4 in the loop version. Take it apart:

[expression for name in sequence]

• The square brackets say “the result is a new list”, just as [] made an empty list on Day 6.
• The expression comes first: the value to put in the new list on each pass. Here, c * 9 / 5 + 32.
• for name in sequence is a for loop header without the colon. name is the loop variable, and it can be used in the expression.

Read it out loud left to right: “a list of c * 9 / 5 + 32 for each c in celsius”. The expression is written first even though Python works it out on each pass, because the first thing a reader wants to know is what the list will hold.

A comprehension can loop over anything a for loop can (any iterable, to use Day 10’s word): a list, a string, a range, the words from split(). The expression can be any expression, including a method call or a function call:

>>> words = "to be or not".split()
>>> [word.upper() for word in words]
['TO', 'BE', 'OR', 'NOT']
>>> [len(word) for word in words]
[2, 2, 2, 3]
>>> [n * 10 for n in range(1, 5)]
[10, 20, 30, 40]
>>> [ch for ch in "hey"]
['h', 'e', 'y']
>>> [n * 10 for n in []]
[]

• [word.upper() for word in words] calls a string method on each word.
• [len(word) for word in words] calls a function on each word, giving a list of lengths.
• [n * 10 for n in range(1, 5)] loops over a range.
• [ch for ch in "hey"] uses the loop variable itself as the expression, so each character becomes an item.
• Looping over an empty list gives an empty list. There are no passes, so nothing is added; no error.

prices = [4, 7, 10]
[price * 2 for price in prices]
print(prices)
doubled = [price * 2 for price in prices]
print(doubled)

[4, 7, 10]
[8, 14, 20]

animals = ["cat", "horse", "ox"]
sizes = [len(animal) - 1 for animal in animals]
print(sizes)
print(animals)

[2, 4, 1]
['cat', 'horse', 'ox']

Often you want only some of the items: the passing scores, the short words, the names that start with A. On Day 5 and Day 6 you did this with an if inside the loop, so only some passes called append:

scores = [72, 45, 90, 58, 61]
passing = []
for score in scores:
    if score >= 60:
        passing.append(score)
print(passing)

[72, 90, 61]

A comprehension can do the same. Add an if and a condition after the for part:

scores = [72, 45, 90, 58, 61]
passing = [score for score in scores if score >= 60]
print(passing)

[72, 90, 61]

[expression for name in sequence if condition]

For each item, Python first checks the condition. If it is truthy, the expression is worked out and added; if not, the item is skipped and nothing is added. The if part is a filter: it decides which items get through. Read line 2 as “a list of score for each score in scores, if score >= 60”.

The filter and the expression can both do work. Here the names arrive with stray spaces. The condition uses strip() (Day 3) to measure the real name, and the expression cleans it up and makes it capitals:

names = [" ana ", "Bo", "carmen", "Li", "dmitri "]
long_names = [name.strip().upper() for name in names if len(name.strip()) > 2]
print(long_names)

['ANA', 'CARMEN', 'DMITRI']

"Bo" and "Li" are 2 characters after stripping, so the filter drops them. The other three pass the filter and the expression turns " ana " into "ANA". The condition is checked first, so the expression only runs for items that pass.

Filtering versus choosing

There is a second, different way an if can appear in a comprehension: as a conditional expression (Day 4), a if condition else b, which picks one of two values. That goes in the expression spot, at the front, and it does not drop anything: every item gives one value or the other.

scores = [72, 45, 90, 58, 61]
passing = [score for score in scores if score >= 60]
labels = ["pass" if score >= 60 else "fail" for score in scores]
print(passing)
print(labels)

[72, 90, 61]
['pass', 'fail', 'pass', 'fail', 'pass']

scores = [72, 45, 90, 58, 61]
labels = [score for score in scores if score >= 60 else "fail"]
print(labels)

  File ".../day11/filter_else_wrong.py", line 2
    labels = [score for score in scores if score >= 60 else "fail"]
                                                       ^^^^
SyntaxError: invalid syntax

numbers = [3, 8, 5, 6]
print([n for n in numbers if n % 2 == 1])
print([n if n % 2 == 1 else 0 for n in numbers])

[3, 5]
[3, 0, 5, 0]

Today’s goal is to convert between a loop and a comprehension in both directions. Going from loop to comprehension lets you write shorter code; going from comprehension to loop lets you read code other people wrote, and debug it by adding print calls inside the loop.

The two are connected by a fixed recipe. Here is a loop and its comprehension, side by side:

words = "the quick brown fox jumps over a lazy dog".split()
short = []
for word in words:
    if len(word) <= 3:
        short.append(word.upper())
print(short)

words = "the quick brown fox jumps over a lazy dog".split()
short = [word.upper() for word in words if len(word) <= 3]
print(short)

Both print:

['THE', 'FOX', 'A', 'DOG']

Loop to comprehension. A loop can become a comprehension only if it has exactly this shape: an empty list, a for, at most an if, and a single append. Then:

1. Write the square brackets.
2. Copy what is inside append(…) to the front.
3. Copy the for line, without its colon.
4. If there is an if, copy it after the for, without its colon.
5. Assign the result to the list’s name.

Comprehension to loop. Run the recipe backwards. Start with name = []. Turn everything after the expression into indented loop lines, in the same order, adding the colons back. Put name.append(expression) at the innermost level.

Two for clauses

That “same order” rule explains comprehensions with more than one for. Day 6 stored a grid as a list of lists. To get every number in one flat list, you need a nested loop (Day 5):

grid = [[1, 2, 3], [4, 5, 6]]
flat = []
for row in grid:
    for number in row:
        flat.append(number)
print(flat)

The comprehension keeps the two for lines in the same order, outer first:

grid = [[1, 2, 3], [4, 5, 6]]
flat = [number for row in grid for number in row]
print(flat)

Both print:

[1, 2, 3, 4, 5, 6]

Read [number for row in grid for number in row] as the nested loop: “for each row in grid, for each number in row, add number”. The second for can use row because it is nested inside the first, just as the inner loop can.

A comprehension’s expression can itself be a comprehension. That builds a list of lists, such as a small times table:

grid = [[row * col for col in range(1, 4)] for row in range(1, 4)]
print(grid)
for line in grid:
    print(line)

[[1, 2, 3], [2, 4, 6], [3, 6, 9]]
[1, 2, 3]
[2, 4, 6]
[3, 6, 9]

Work from the outside in. The outer comprehension makes one item for each row from 1 to 3. That item is the inner comprehension, [row * col for col in range(1, 4)], which is a whole list: the row’s three products. So grid holds three lists. This is as far as a comprehension should go; any deeper and a loop is easier to read (11.10).

grid = [[1, 2, 3], [4, 5, 6]]
flat = [number for number in row for row in grid]
print(flat)

Traceback (most recent call last):
  File ".../day11/flatten_wrong_order.py", line 2, in <module>
    flat = [number for number in row for row in grid]
                                 ^^^
NameError: name 'row' is not defined. Did you mean: 'pow'?

prices = [3.5, 12.0, 8.25, 20.0]
cheap = []
for price in prices:
    if price < 10:
        cheap.append(price * 2)
print(cheap)

prices = [3.5, 12.0, 8.25, 20.0]
cheap = [price * 2 for price in prices if price < 10]
print(cheap)

[7.0, 16.5]

On Day 8 you built dictionaries in a loop: start with {}, then d[key] = value on each pass. That is the same “empty, loop, add” scaffolding as a list, so Python has a comprehension for it too.

A dictionary comprehension builds a new dictionary. It uses curly braces, like a dictionary, and its expression is a key: value pair:

{key_expression: value_expression for name in sequence}

The colon between the two expressions is what makes it a dictionary. On each pass, Python works out the key and the value and adds that entry, exactly like d[key] = value.

names = ["Ana", "Bo", "Carmen"]
lengths = {name: len(name) for name in names}
print(lengths)
print(lengths["Carmen"])

{'Ana': 3, 'Bo': 2, 'Carmen': 6}
6

The most common source for a dictionary comprehension is another dictionary. items() (Day 8) gives each key and value as a pair, and unpacking (Day 7) puts them in two names. A café puts everything 50 cents off, and also wants just the items under 5:

menu = {"tea": 2.5, "cake": 4.0, "soup": 6.0}
sale = {item: price - 0.5 for item, price in menu.items()}
print(sale)
under_5 = {item: price for item, price in menu.items() if price < 5}
print(under_5)

{'tea': 2.0, 'cake': 3.5, 'soup': 5.5}
{'tea': 2.5, 'cake': 4.0}

Line 2 keeps every key and changes every value. Line 4 adds a filter at the end, exactly as in 11.2, so only items under 5 get an entry; the values are unchanged. menu itself is not touched by either line.

Because you choose both the key and the value, you can also swap them. An airport code table becomes a city-to-code table:

codes = {"SFO": "San Francisco", "JFK": "New York", "LHR": "London"}
cities = {city: code for code, city in codes.items()}
print(cities)
print(cities["London"])

{'San Francisco': 'SFO', 'New York': 'JFK', 'London': 'LHR'}
LHR

The loop unpacks each pair as code, city, and the expression puts city first, as the key.

words = ["cat", "dog", "horse"]
by_length = {len(word): word for word in words}
print(by_length)

{3: 'dog', 5: 'horse'}

stock = {"pens": 12, "pads": 0, "tape": 3, "glue": 0}
reorder = {item: 10 - count for item, count in stock.items() if count < 5}
print(reorder)

{'pads': 10, 'tape': 7, 'glue': 10}

On Day 7 you met sets: collections that keep only one copy of each value. To find the different words in a sentence, ignoring capitals, you want a set of lowercase words.

A set comprehension builds a new set. It uses curly braces like a dictionary comprehension, but its expression is a single value, with no colon:

{expression for name in sequence}

text = "The cat saw the dog and the dog saw THE cat"
words = text.split()
unique = {word.lower() for word in words}
print(len(words), "words,", len(unique), "different")
print(sorted(unique))

11 words, 5 different
['and', 'cat', 'dog', 'saw', 'the']

text.split() gives 11 words. The set comprehension lowercases each one and adds it to the set; adding a value the set already holds changes nothing. So "The", "the", "the", and "THE" all end up as one "the". Five different words remain.

Why print sorted(unique) and not unique? A set has no order (Day 7), and the order Python shows a set of strings in can change from one run to the next. sorted() gives back a list in alphabetical order, which is the same every time.

Curly braces now do three jobs, and the difference is easy to miss:

>>> letters = {ch for ch in "banana"}
>>> sorted(letters)
['a', 'b', 'n']
>>> counts = {ch: "banana".count(ch) for ch in "banana"}
>>> counts
{'b': 1, 'a': 3, 'n': 2}
>>> type({})
<class 'dict'>
>>> type(set())
<class 'set'>

• {ch for ch in "banana"} has no colon, so it is a set: three different letters.
• {ch: "banana".count(ch) for ch in "banana"} has a colon, so it is a dictionary. (It counts each letter more than once on the way, which is wasteful, but the later entry for a repeated letter simply replaces the earlier one with the same count.)
• Empty {} is an empty dictionary, not an empty set. An empty set is written set(). type() shows which one you have.

names = ["Ana", "ali", "Bo", "Carmen", "cleo", "ben"]
firsts = {name[0].upper() for name in names}
print(sorted(firsts))
print(len(firsts))

['A', 'B', 'C']
3

Suppose you need the total of the squares of the numbers from 1 to a million. With a list comprehension, you can hand the list to sum() (Day 6):

as_list = sum([n * n for n in range(1, 1000001)])
as_generator = sum(n * n for n in range(1, 1000001))
print(as_list)
print(as_generator)

333333833333500000
333333833333500000

Line 1 works, but think about what it does. The comprehension first builds a complete list with a million numbers in it, holding them all in the computer’s memory at once. Only then does sum() start adding. Yet sum() never needs more than one number at a time: it adds the next number to its running total and moves on. The whole list is built just to be thrown away.

Line 2 uses round brackets instead of square ones, and gives the same answer without ever building the list. That is a generator expression: it looks like a list comprehension in parentheses, but instead of making all the values up front, it makes them one at a time, when asked. The object it gives back is called a generator. A generator is like a recipe card for a list rather than a tray of cooked food: it holds the instructions and where it is up to, and produces the next value only when something asks for it.

(expression for name in sequence if condition)

Everything you learned about the parts still applies, including the filter. Only the brackets change.

Watching a generator be lazy

You cannot see memory directly, but you can see when values are made. This program uses a function (Day 9) that prints a message every time it is called, so you can watch:

def square(n):
    print("making", n)
    return n * n


squares = (square(n) for n in range(1, 4))
print("generator ready")
for value in squares:
    print("got", value)

generator ready
making 1
got 1
making 2
got 4
making 3
got 9

Now change only the brackets to square ones:

def square(n):
    print("making", n)
    return n * n


squares = [square(n) for n in range(1, 4)]
print("list ready")
for value in squares:
    print("got", value)

making 1
making 2
making 3
list ready
got 1
got 4
got 9

The list comprehension makes all three values before line 7 runs. That is the whole difference: a list comprehension does all the work at once and keeps every result; a generator expression does the work as the values are used and keeps none of them. With three values it does not matter; with a million, the list version holds a million numbers in memory and the generator version holds one.

Printing a generator does not show its values. Python shows a short description that starts <generator object and ends with a memory address, which changes from run to run. To see the values, turn it into a list with list():

>>> doubled = (n * 2 for n in [5, 6, 7])
>>> list(doubled)
[10, 12, 14]
>>> list(doubled)
[]

The second list(doubled) is empty. That is not a mistake in the transcript; it is the next rule.

A generator can be used only once

A generator remembers where it is up to, and once it has handed over its last value, it is used up: asking again gives nothing.

squares = (n * n for n in range(1, 4))
print(sum(squares))
print(sum(squares))
print(list(squares))

14
0
[]

The first sum uses all three values (1 + 4 + 9 = 14). The second sum finds nothing left, and the sum of no numbers is 0. list(squares) gives an empty list. If you need the values more than once, use a list comprehension and keep the list.

When a generator expression is the only argument to a function, you can drop its own parentheses: sum(n * n for n in numbers) is sum((n * n for n in numbers)) with one pair of brackets doing both jobs. Round brackets with a for inside never make a tuple; Python has no tuple comprehension.

names = ["bo", "Ana", "carmen"]
ordered = sorted(name.lower() for name in names, reverse=True)
print(ordered)

  File ".../day11/two_args_wrong.py", line 2
    ordered = sorted(name.lower() for name in names, reverse=True)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized

names = ["bo", "Ana", "carmen"]
ordered = sorted((name.lower() for name in names), reverse=True)
print(ordered)

['carmen', 'bo', 'ana']

evens = (n for n in range(10) if n % 2 == 0)
print(max(evens))
print(len([n for n in range(10) if n % 2 == 0]))
print(sum(evens))

8
5
0

Generator expressions are most useful as the argument to a function that walks through values and boils them down to one answer. You know sum(), min(), and max() from Day 6. Two more built-in functions answer yes-or-no questions about a whole collection:

• any(values) gives True if at least one value is truthy, and False otherwise.
• all(values) gives True if every value is truthy, and False otherwise.

Combined with a generator expression, each of these replaces a whole accumulator loop from Day 5:

scores = [72, 45, 90, 58, 61]
print(sum(1 for score in scores if score >= 60))
print(max(score - 50 for score in scores))
print(any(score >= 90 for score in scores))
print(all(score >= 50 for score in scores))

3
40
True
False

Line 2 is the counting pattern in one line: sum(1 for … if …) adds a 1 for every item that passes the filter, which is exactly what a Day 5 counter did with count += 1. Here is the Day 5 vowel counter rewritten that way:

text = "Programming is fun"
vowels = sum(1 for ch in text.lower() if ch in "aeiou")
print("Vowels:", vowels)

Vowels: 5

Lines 4 and 5 have a condition as the expression, so the generator gives True or False for each score, and any or all checks those.

any() and all() stop early

any and all work like or and and (Day 4): they stop as soon as the answer is decided. any stops at the first truthy value, because one is enough; all stops at the first falsy value, because one failure is enough. Combined with a generator, that means the remaining values are never even made:

def too_cold(temp):
    print("checking", temp)
    return temp < 0


temps = [4, -2, 7, 1]
print(any(too_cold(t) for t in temps))
print(all(t > 0 for t in temps))

checking 4
checking -2
True
False

too_cold is called for 4 (not below 0), then for -2 (below 0). That is enough for any, so 7 and 1 are never checked. If you wrote any([too_cold(t) for t in temps]) with square brackets, the list comprehension would call too_cold for all four temperatures first; this is one more reason to pass a generator expression rather than a list.

Empty collections

What should “did everyone pass?” say about a class with no students? Python’s answer is True, because there is no student who failed. any of nothing is False, because there is no value that is true:

>>> any([])
False
>>> all([])
True
>>> scores = []
>>> all(score >= 60 for score in scores)
True
>>> len(scores) > 0 and all(score >= 60 for score in scores)
False

This is logically right but can surprise you. If an empty list should count as “not passed”, check that the list has items first, as the last line does. The and stops at len(scores) > 0, which is False.

password = "Sunny5day"
has_digit = any(ch in "0123456789" for ch in password)
all_lower = all(ch == ch.lower() for ch in password)
long_enough = len(password) >= 8
print(has_digit, all_lower, long_enough)

True False True

Often two lists belong together: names[0] goes with scores[0], names[1] with scores[1], and so on. Lists that line up position by position like this are called parallel lists. With what you knew before today, you would loop over the indexes:

names = ["Ana", "Bo", "Carmen"]
scores = [88, 92, 79]
for i in range(len(names)):
    print(names[i], scores[i])

Ana 88
Bo 92
Carmen 79

It works, but the index i is only a means to an end, and every use of it is a chance to write the wrong list or the wrong index. You want to go through both lists together.

The built-in function zip() takes two or more sequences and pairs up their items by position: the first item of each, then the second of each, and so on. Each pair is a tuple, so a for loop can unpack it (Day 7) into two names:

names = ["Ana", "Bo", "Carmen"]
scores = [88, 92, 79]
for name, score in zip(names, scores):
    print(name, score)
print(list(zip(names, scores)))

Ana 88
Bo 92
Carmen 79
[('Ana', 88), ('Bo', 92), ('Carmen', 79)]

Think of a zipper: two rows of teeth, joined one pair at a time. Line 3 unpacks each tuple into name and score. Line 5 uses list(), which builds a list from the pairs, so you can see what zip makes. Like a generator, zip makes its pairs one at a time as they are asked for; printing a zip directly shows only a description, not the pairs.

zip takes as many sequences as you like. With three, each item is a tuple of three, and you unpack it into three names. Because zip gives pairs one at a time, it works inside a generator expression too:

items = ["pen", "pad", "tape"]
prices = [1.5, 3.0, 2.25]
counts = [4, 2, 1]
for item, price, count in zip(items, prices, counts):
    print(f"{count} x {item}: {price * count:.2f}")
total = sum(price * count for price, count in zip(prices, counts))
print(f"Total: {total:.2f}")

4 x pen: 6.00
2 x pad: 6.00
1 x tape: 2.25
Total: 14.25

Line 6 multiplies each price by its count and sums the results, with no index anywhere.

Two parallel lists of keys and values are exactly what a dictionary needs. A dictionary comprehension can build one, and so can dict(), which builds a dictionary directly from a sequence of key-value pairs:

names = ["Ana", "Bo", "Carmen"]
scores = [88, 92, 79]
grades = {name: score for name, score in zip(names, scores)}
print(grades)
print(dict(zip(names, scores)))

{'Ana': 88, 'Bo': 92, 'Carmen': 79}
{'Ana': 88, 'Bo': 92, 'Carmen': 79}

Comparing neighbors

zip can also pair a list with a shifted copy of itself. temps[1:] (a Day 6 slice) is the list without its first item, so zipping temps with temps[1:] pairs each value with the next one. Then all can check a rule for every neighboring pair:

temps = [11, 13, 13, 16]
pairs = list(zip(temps, temps[1:]))
print(pairs)
print(all(later > earlier for earlier, later in pairs))
print(all(later >= earlier for earlier, later in pairs))

[(11, 13), (13, 13), (13, 16)]
False
True

Four temperatures give three neighboring pairs. “Each day strictly warmer” fails at (13, 13); “never colder” holds for all three.

names = ["Ana", "Bo", "Carmen", "Dee"]
scores = [88, 92, 79]
for name, score in zip(names, scores):
    print(name, score)
print(len(names), len(scores))

Ana 88
Bo 92
Carmen 79
4 3

first = ["a", "b", "c", "d"]
second = [1, 2, 3]
print([letter * number for letter, number in zip(first, second)])

['a', 'bb', 'ccc']

On Day 5 you used enumerate() on strings: for index, item in enumerate(sequence): gives each item with its index, from 0, or from the number you pass as a second argument. Now that you have lists and unpacking, you can see what it really does, and use it in more places.

enumerate works on a list exactly as on a string. And like zip, it makes tuples, one (index, item) pair per item; that is why the two-name for line works, and why Day 5’s single-name loop printed pairs in parentheses:

tasks = ["buy milk", "call Bo", "water plants"]
for number, task in enumerate(tasks, 1):
    print(f"{number}. {task}")
print(list(enumerate(tasks, 1)))

1. buy milk
2. call Bo
3. water plants
[(1, 'buy milk'), (2, 'call Bo'), (3, 'water plants')]

Because it gives pairs one at a time, enumerate also works inside a comprehension. Unpack the pair in the for part, and use either name in the expression or the filter:

tasks = ["buy milk", "call Bo", "water plants"]
lines = [f"{number}. {task}" for number, task in enumerate(tasks, 1)]
print(lines)
every_other = [task for index, task in enumerate(tasks) if index % 2 == 0]
print(every_other)

['1. buy milk', '2. call Bo', '3. water plants']
['buy milk', 'water plants']

Line 2 builds the numbered lines without printing them, ready to use later. Line 4 filters on the index: index % 2 == 0 keeps positions 0 and 2, which is every other task.

Going backwards

To go through a list from the end, you have seen two tools on Day 6: the reverse() method and the slice [::-1]. A third, the built-in function reversed(), gives the items of a sequence from last to first, one at a time, without changing or copying the sequence:

laps = [62, 60, 59]
for lap in reversed(laps):
    print(lap)
print(laps)
print(list(reversed(laps)))
print(list(reversed("stop")))
print(list(reversed(range(1, 4))))

59
60
62
[62, 60, 59]
[59, 60, 62]
['p', 'o', 't', 's']
[3, 2, 1]

The loop visits the laps from last to first, and printing laps afterwards shows it is unchanged. reversed works on a string or a range too. Like zip, it hands its items over one at a time, so wrap it in list() to see them all at once.

All three ways of going backwards side by side:

laps = [62, 60, 59]
copy = laps[::-1]
print(copy, laps)
backwards = list(reversed(laps))
print(backwards, laps)
result = laps.reverse()
print(result, laps)

[59, 60, 62] [62, 60, 59]
[59, 60, 62] [62, 60, 59]
None [59, 60, 62]

laps = [62, 60, 59]
for lap in laps.reverse():
    print(lap)

Traceback (most recent call last):
  File ".../day11/reverse_none.py", line 2, in <module>
    for lap in laps.reverse():
               ~~~~~~~~~~~~^^
TypeError: 'NoneType' object is not iterable

steps = ["mix", "bake", "cool"]
for number, step in enumerate(reversed(steps), 1):
    print(number, step)

1 cool
2 bake
3 mix

Comprehensions are shorter, but shorter is not the goal; clear is. Code is read far more often than it is written, by other people and by you a month from now. A comprehension is clearer than a loop only when the reader can take in the whole line at once.

A comprehension is a good fit when all of these are true:

• The goal is to build a new collection (or feed sum, any, all, min, or max).
• Each item gives one value from one expression.
• There is at most one simple filter, and at most two for clauses.

Use a plain loop when any of these is true:

Here is the first case. It is tempting to “save lines” by writing a comprehension whose expression is a print:

names = ["Ana", "Bo"]
result = [print("Hello,", name) for name in names]
print(result)

Hello, Ana
Hello, Bo
[None, None]

names = ["Ana", "Bo"]
for name in names:
    print("Hello,", name)

Hello, Ana
Hello, Bo

The second case: turning scores into letter grades has four outcomes. Conditional expressions can be chained, so a comprehension can do it:

scores = [95, 82, 71, 64]
grades = [
    "A" if s >= 90 else "B" if s >= 80 else "C" if s >= 70 else "F"
    for s in scores
]
print(grades)

['A', 'B', 'C', 'F']

It works, but to check it you have to untangle "A" if … else "B" if … else … in your head. The Day 4 if / elif chain says the same thing one rule per line:

scores = [95, 82, 71, 64]
grades = []
for score in scores:
    if score >= 90:
        grades.append("A")
    elif score >= 80:
        grades.append("B")
    elif score >= 70:
        grades.append("C")
    else:
        grades.append("F")
print(grades)

['A', 'B', 'C', 'F']

Same result, longer code, much easier to read and to change. An even better option, from Day 9, is to put the if / elif chain in a function such as letter_grade(score); then the comprehension [letter_grade(score) for score in scores] is short and clear, because the hard part has a name.

A teacher has two quiz scores (out of 10) for five students, kept in three parallel lists. The program reports each student’s total, the class average, who improved, whether anyone was perfect on quiz 2, whether everyone reached 12, and the top three.

names = ["Ana", "Bo", "Carmen", "Dee", "Eli"]
quiz1 = [8, 6, 9, 4, 7]
quiz2 = [9, 5, 10, 6, 7]

totals = {name: a + b for name, a, b in zip(names, quiz1, quiz2)}
print(totals)

average = sum(totals.values()) / len(totals)
print(f"Average: {average:.1f}")

improved = [name for name, a, b in zip(names, quiz1, quiz2) if b > a]
print("Improved:", improved)

print("Anyone perfect on quiz 2?", any(score == 10 for score in quiz2))
print("Everyone passed?", all(total >= 12 for total in totals.values()))

ranking = sorted(totals, key=lambda name: totals[name], reverse=True)
for place, name in enumerate(ranking[:3], 1):
    print(f"{place}. {name} ({totals[name]})")

{'Ana': 17, 'Bo': 11, 'Carmen': 19, 'Dee': 10, 'Eli': 14}
Average: 14.2
Improved: ['Ana', 'Carmen', 'Dee']
Anyone perfect on quiz 2? True
Everyone passed? False
1. Carmen (19)
2. Ana (17)
3. Eli (14)

• Every collection is built in one line, but the final report uses a loop, because printing is an action (11.10).
• Line 11 zips the lists again rather than using totals, because it needs both quiz scores, not their sum.
• Line 17 loops over totals, which gives its keys (Day 8), so ranking is a list of names; the lambda looks up each name’s total to sort by.

• A list comprehension, [expression for name in sequence if condition], builds a new list in one line; the optional if at the end is a filter. An if … else at the front is a conditional expression that keeps every item.
• A comprehension and a loop convert by a fixed recipe: the clauses after the expression are the loop lines in the same order, and the expression is what gets appended.
• Dictionary comprehensions {key: value for …} and set comprehensions {value for …} use curly braces; the colon makes the difference.
• A generator expression (… for …) makes values one at a time, so it saves memory, but it can be used only once. It is the natural argument for sum, min, max, any, and all, and any and all stop as soon as the answer is known.
• zip walks parallel lists together, enumerate numbers items, and reversed goes backwards without changing the list. Use a plain loop when the work is an action, has several cases or steps, or needs break.

Do these in your day11 folder. Type each program yourself, run it, and only then open the answer.

text = "Programming is fun"

# Day 5: count the vowels
count = 0
for ch in text.lower():
    if ch in "aeiou":
        count += 1
print(count, sum(1 for ch in text.lower() if ch in "aeiou"))

# Day 5: remove the spaces
result = ""
for ch in text:
    if ch != " ":
        result += ch
print(result, "".join(ch for ch in text if ch != " "))

# Day 6: square every number in a list
numbers = [2, 5, 7]
squares = []
for n in numbers:
    squares.append(n * n)
print(squares, [n * n for n in numbers])

5 5
Programmingisfun Programmingisfun
[4, 25, 49] [4, 25, 49]

sentence = "Python makes short work of long lists"
lengths = {word: len(word) for word in sentence.lower().split()}
print(lengths)

{'python': 6, 'makes': 5, 'short': 5, 'work': 4, 'of': 2, 'long': 4, 'lists': 5}

planned = [30, 45, 20, 60]
actual = [25, 45, 18, 50]

pairs = list(zip(planned, actual))
print(pairs)

for plan, real in pairs:
    print(f"planned {plan}, took {real}")

on_time = all(real <= plan for plan, real in pairs)
print("Every task on time:", on_time)

[(30, 25), (45, 45), (20, 18), (60, 50)]
planned 30, took 25
planned 45, took 45
planned 20, took 18
planned 60, took 50
Every task on time: True

items = ["pens", "pads", "tape", "glue"]
counts = [12, 0, 3, 7]
low = {item: count for item, count in zip(items, counts) if count < 8}
print(low)
print(any(count == 0 for count in low.values()))

{'pads': 0, 'tape': 3, 'glue': 7}
True

text = "72,90,61"
scores = (int(part) for part in text.split(","))
print(sum(scores) / len(scores))

TypeError: object of type 'generator' has no len()

text = "72,90,61"
scores = [int(part) for part in text.split(",")]
print(f"{sum(scores) / len(scores):.1f}")

74.3

pairs = [(a, b) for a in range(1, 4) for b in range(1, 4) if a < b]
print(pairs)

pairs = []
for a in range(1, 4):
    for b in range(1, 4):
        if a < b:
            pairs.append((a, b))
print(pairs)

[(1, 2), (1, 3), (2, 3)]

laps = [62, 60, 61, 58]
faster = [later < earlier for earlier, later in zip(laps, laps[1:])]
print(faster)
for number, lap in enumerate(reversed(laps), 1):
    if number > 2:
        break
    print(number, lap)

[True, False, True]
1 58
2 61

colors = ["red", "Blue", "red", "green", "BLUE", "red"]
as_list = [c.lower() for c in colors]
as_set = {c.lower() for c in colors}
print(len(as_list), len(as_set))

6 3

words = ["apple", "kiwi", "banana", "fig", "cherry"]
picked = [w.upper() if len(w) > 5 else w for w in words if "a" in w or "i" in w]
print(picked)

['apple', 'kiwi', 'BANANA', 'fig']

words = ["apple", "kiwi", "banana", "fig", "cherry"]
picked = []
for w in words:
    if "a" in w or "i" in w:
        if len(w) > 5:
            picked.append(w.upper())
        else:
            picked.append(w)
print(picked)

The official Python Tutorial covers today’s topics in Data Structures: List Comprehensions, Nested List Comprehensions, and Looping Techniques, which shows zip, enumerate, and reversed together.

Check each statement. If the answer is “not yet,” reread the linked subsection.

• I can write a list comprehension that builds a new list from another. Reread
• I can filter with an if at the end and tell it apart from a conditional expression at the front. Reread
• I can turn a loop into a comprehension and a comprehension back into a loop. Reread
• I can build a dictionary with a dictionary comprehension. Reread
• I can build a set with a set comprehension and tell it apart from a dictionary. Reread
• I can explain why a generator expression saves memory and why it can be used only once. Reread
• I can count, total, and check rules with sum(), any(), and all(). Reread
• I can walk parallel lists together with zip(). Reread
• I can number items with enumerate() and go backwards with reversed(). Reread
• I can decide when a plain loop is clearer than a comprehension. Reread