Python List Comprehensions

Week 1 | Lesson 5.3

LEARNING OBJECTIVES

After this lesson, you will be able to:

• Understand list comprehensions in python and what they are useful for
• Use list comprehensions to efficiently manipulate list data
• Use list comprehensions to construct dictionaries

STUDENT PRE-WORK

Before this lesson, you should already be familiar with:

• python fundamental concepts and data types
• python control statements
• python function definitions
• python arrays and dictionaries
• lambda functions

STARTER CODE

List Comprehension

LESSON GUIDE

Introduction: List Comprehensions (5 mins)

Python list comprehensions are a simple and powerful syntax that, once mastered, allow for fast, efficient, and intuitive manipulation of array-like data types.

Though list comprehensions may seem confusing at first, they are easy to get used to and once understood make otherwise complex code readable and concise.

List comprehensions are essentially replacements for iteration control statements. I will explain why this is the case below, and give the non-list-comprehension alternative code to help you understand what they are doing (and make it clear why they are so much better!).

Guided Practice: List Comprehension Basics (20 mins)

Follow along for the demos in this notebook:

python guided practice notebook

What are list comprehensions?

List comprehensions are statements that perform some kind of operation on each element of a list. Let's start with a simple array of numbers:

numbers = [0,1,2,3,4,5,6,7,8,9]

Imagine that we want to add 1 to every element of the list. We could do this a couple of ways without the use of list comprehensions. We could use a for loop:

nums_plus_one = []
for num in numbers:
  nums_plus_one.append(num+1)

We could also use python's "map" with a lambda function. Map iterates over each element of a list and applies a function to it:

nums_plus_one = map(lambda x: x+1, numbers)

These solutions each have pros and cons. The for loop is more readable and explicit (if you aren't familiar with how map and lambda works, at least), and the map with lambda is concise but arcane. Luckily list comprehensions combine the best of both worlds:

nums_plus_one = [x+1 for x in numbers]

Let's go over how that works in more granular detail.

• Like the map statement, nums_plus_one is assigned on the left as a new variable.
• List comprehensions return a list, and the internal statement is wrapped in the list brackets: [...]
• Within the brackets these elements are similar to a for loop:
  1. The operation per element comes first: x+1
  2. Next is the for loop variable assignment: for x
  3. Last comes the list of elements to iterate over: in numbers

Conditional logic in list comprehensions

List comprehensions can be extended to cover more of the functionality of a for loop than just an operation over elements. Let's say we wanted to "binarize" a variable based on whether the elements are greater or less than the mean over all elements. The for loop could look something like this:

import numpy as np
n = [1, 2, 7, 21, 3, 1, 62, 3, 34, 12, 73, 44, 12, 11, 9]
n_bin = []
n_mean = np.mean(n)
for x in n:
  if x >= n_mean:
    n_bin.append(1)
  else:
    n_bin.append(0)

But that's pretty verbose. A list comprehension can do the same thing much easier:

We can even do chained conditionals! This returns swaps 1s to 0s and vice versa in a list, otherwise sets the items to none:

Guided Practice: Advanced List Comprehensions (30 mins)

Nested List Comprehensions

As some of you may have suspected by now, we can embed list comprehensions within other list comprehensions for even more power.

For example, let's say we want the square and the square root for every non-negative element in a list:

Note that the if statement in the embedded list comes after the in statement in this example. When your condition is meant to be a filter the conditional comes after.

List Comprehensions with Functions

We can also do operations on multiple lists. I often use the zip and enumerate functions in combination with list comprehensions. First let's go over what each of the functions does.

zip goes through each element of two lists iteratively at the same time:

a = ['a','b','c','d']
z = ['z','y','x','w']

zipped = []
for a_i, z_i in zip(a, z):
  zipped.append([a_i, z_i])

[['a', 'z'], ['b', 'y'], ['c', 'x'], ['d', 'w']]]

Check Do this as a list comprehension.

enumerate keeps track of the index of each element of a list:

a = ['a','b','c','d']

enumerated = []
for i, a_i in enumerate(a):
  enumerated.append([i, a_i])

[[0, 'a'], [1, 'b'], [2, 'c'], [3, 'd']]

Check Do this as a list comprehension.

Keep note that that with enumerate the index is returned first and the element second.

Let's multiply the element of the first list by the index, then divide that by the element of the second list:

Nested Loops

Here's a list comprehension that returns syllables (defined by consonants followed by a vowel) in a flattened list:

Check: Does anyone understand how this is working?

This is a complicated list comprehension with nested for loops, and brings up one of the more confusing aspects of list comprehensions. To understand let's first write out the comprehension more explicitly:

# simple list comprehension to get non-vowel letters:
consonants = [x for x in alphabet if x not in vowels]

# get all the syllables for each consonant + vowel pair in nested consonant-syllable lists:
syllables = [[c + v for v in vowels] for c in consonants]

syllables = [
  s
  for syls in syllables
  for s in syls
]

The trick here is that the nested list comprehension for loops are in the same order as they would be in standard nested for loops, except the retrieved element comes first!

flat_syllables = []
for syls in syllables:
  for s in syls:
    flat_syllables.append(s)

Guided Practice: Dictionary Comprehensions (10 mins)

Comprehensions are not limited to lists. You can also use comprehensions to create dictionaries with key:value pairs.

Below, for example, we can create a dictionary with the integer value of each character in a string with the string as a key (the ord function returns the integer value of a character).

This can be particularly useful for creating pandas dataframes.

Independent Practice: Topic (20 minutes)

• Practice list comprehensions on your own
• Problems are separated into easy, medium, and hard categories

independent practice problems

Conclusion (5 mins)