Course Content

Optimization Techniques in Python

1. Understanding and Measuring Performance

Introduction to Python Performance Timing and Benchmarking Basics Measuring Function Performance Challenge: Implementing Benchmarking

2. Efficient Use of Data Structures

Lists and NumPy Arrays Sets and Tuples Challenge: Choosing Optimal Data Structures Using the collections Module Challenge: Handling Customer Requests

3. Enhancing Performance with Built-in Tools

Leveraging map() and List Comprehensions Handling Large Files Maximizing Sorting Efficiency Efficient String Operations

Leveraging map() and List Comprehensions

Whenever possible, it's better to rely on built-in functions, methods, or library functions rather than creating algorithms from scratch. Built-in and library functions are typically optimized for performance, run more efficiently, and are designed to manage a wide range of edge cases effectively.

Let’s begin with powerful tools like the map() function and list comprehensions, which can significantly speed up your code. Since list comprehensions are more optimized for performance and map() are implemented in C, they generally execute faster than manually written loops.

List Comprehensions

List comprehensions offer a more concise and often faster way to create lists compared to traditional for loops. Besides, they can make your code easier to read and understand.

Let's compare the performance of a for loop and a list comprehension by generating a list where each number from 1 to 10000000 is multiplied by 2:


              123456789101112131415161718192021
            
import os
os.system('wget https://codefinity-content-media-v2.s3.eu-west-1.amazonaws.com/courses/8d21890f-d960-4129-bc88-096e24211d53/section_1/chapter_3/decorators.py 2>/dev/null')
from decorators import timeit_decorator

numbers = list(range(1, 10000001))

@timeit_decorator(number=3)
def list_comprehension():
    return [x * 2 for x in numbers]

@timeit_decorator(number=3)
def for_loop():
    result = []
    for x in numbers:
        result.append(x * 2)
    return result

list_1 = list_comprehension()
list_2 = for_loop()

print(list_1 == list_2)

Using map()

The map function applies a specified function to each item in an iterable (e.g., a list).

Besides being more concise than writing loops, it often runs faster because of C-level optimizations in its implementation, which reduce the overhead of repeated function calls.

While map can offer some performance benefits, especially when working with built-in functions, the speed difference compared to list comprehensions is often negligible. Both are highly optimized, and for simple operations, the performance advantage of map is barely noticeable. In practice, the choice between the two should be based more on readability and specific use cases, such as memory efficiency with map.

Let's compare the performance of map, for loops, and list comprehensions when formatting 1000000 customer names to title case (e.g., 'john doe' -> 'John Doe'):


              1234567891011121314151617181920212223242526
            
import os
os.system('wget https://codefinity-content-media-v2.s3.eu-west-1.amazonaws.com/courses/8d21890f-d960-4129-bc88-096e24211d53/section_1/chapter_3/decorators.py 2>/dev/null')
from decorators import timeit_decorator

customer_names = ['john doe', 'jane smith', 'alex johnson', 'ivan smith'] * 250000

@timeit_decorator(number=10)
def map_function():
    return list(map(str.title, customer_names))

@timeit_decorator(number=10)
def list_comprehension():
    return [name.title() for name in customer_names]

@timeit_decorator(number=10)
def for_loop():
    result = []
    for name in customer_names:
        result.append(name.title())
    return result

formatted_with_map = map_function()
formatted_with_comprehension = list_comprehension()
formatted_with_for_loop = for_loop()

print(formatted_with_map == formatted_with_comprehension == formatted_with_for_loop)

As expected, the for loop is the slowest approach of the three. In this example, map turned out to be faster than list comprehension, although in practice, performance can vary depending on the specific use case. Sometimes, list comprehensions may even outperform map, especially for simpler expressions.

Everything was clear?

Thanks for your feedback!

Section 3. Chapter 1