Generators are a type of iterable in Python that can be used to generate a sequence of values. They are defined using a function that contains one or more yield expressions. When a generator function is called, it returns a generator object that can be iterated over lazily, yielding one value at a time.
Here's an example of a generator function that generates a sequence of even numbers:
def even_numbers(n: int) -> int:
"""
Generate a sequence of even numbers up to n.
Args:
n (int): The upper limit of the sequence (exclusive).
Yields:
int: The next even number in the sequence.
"""
for i in range(n):
if i % 2 == 0:
yield i
The yield keyword is used to return a value from the generator function and pause the execution of the function. The generator object can then be used to continue the execution of the function from the point where it was paused.
| Traditional Loop | Generator Yield |
|---|
| Computes all values at once | Generates values on-demand |
| Stores all values in memory | Stores only the current value |
| Uses more memory for large datasets | Memory-efficient for large datasets |
| Faster for small datasets | Slightly slower due to overhead |
| Cannot pause execution | Can pause and resume execution |
| Good for finite, small sequences | Excellent for large or infinite sequences |
To get a value from the generator, you can use the next() function.
even_nums = even_numbers(10)
print(next(even_nums))
print(next(even_nums))
print(next(even_nums))
You can also use a for loop to iterate over the generator object.
def print_even_numbers(limit: int) -> None:
"""
Print even numbers up to the given limit.
Args:
limit (int): The upper limit for generating even numbers (exclusive).
Returns:
None
"""
for num in even_numbers(limit):
print(num)
print_even_numbers(10)
You can also convert a generator object to a list. (But don't do this)
even_nums = even_numbers(10)
print(list(even_nums))
Note: Please don't convert a generator object to a list if you only need to iterate over it once. This will defeat the purpose of using a generator.
Note: The generator object is not a list, it is an iterator. You can only iterate over it once. If you need to use the values multiple times, you should convert the generator object to a list.
The Generator always remembers its state between calls and you can resume from there with no issues or state errors.
Benefits of Generator
- Memory Efficiency: Generators give values on-the-fly, so they don't require as much memory as traditional loops. This makes them ideal for processing large datasets.
- Performance: Generators can be faster than traditional loops because they don't require as much memory.
- Pause and Resume: Generators can pause and resume execution, which makes them ideal for processing large datasets.
Note: A generator doesn't raise a StopIteration exception, it is a feature of iterators and you don't have to worry about it. It just exits to make it clear that the generator has reached the end of the sequence.
Generator Expression
Generator expressions are similar to list comprehensions, but they return a generator object instead of a list.
Note: Remember, in python, if its iterable, then there is a comprehension. e.g List Comprehension, Set Comprehension, Dictionary Comprehension, and now Generator Comprehension.
even_nums = (i for i in range(10) if i % 2 == 0)
print(even_nums)
Here is a comparison of a generator expression and a list comprehension.
even_nums = (i for i in range(10) if i % 2 == 0)
even_nums_list = [i for i in range(10) if i % 2 == 0]
print(even_nums)
print(even_nums_list)
Notice generator expression is wrapped in parentheses (), while list comprehension is wrapped in square brackets [].
Use Cases for Generator Expressions
User generator expressions when you need to generate a sequence of values on-the-fly, but don't need to store all the values in memory. Consider the cost of storing intermediate values in memory when choosing between a generator expression and a list comprehension.
Note: When writting any comprehension, if the code spans more than one line, you may be better off with traditional for loop. For the following reasons:
- Readability
- Debugging
- Complexity
- PEP8
- Flake8
However, consider the following exceptions to the rule:
- Line lenght limit rules
- Complex operations
- Team code guide/rule
- Generator expressions are always more efficient for large datasets than a loop that build a list.
Examples
squares = [x**2 for x in range(10)]
Multi-line comprehension (might be better as a loop)
complex_calculation = [
(x**2 + y**2) / (x + y)
for x in range(1, 10)
for y in range(1, 10)
if (x + y) % 2 == 0
]
The above might be clearer as:
complex_calculation = []
for x in range(1, 10):
for y in range(1, 10):
if (x + y) % 2 == 0:
result = (x**2 + y**2) / (x + y)
complex_calculation.append(result)
Thank you for reading this article. I hope you found it helpful. Next lesson, we will discuss map, filter vs list comprehension.
Please follow, like and subscribe to my channel and social media pages.
