Course Content

Optimization Techniques in Python

1. Understanding and Measuring Performance

Introduction to Python Performance Timing and Benchmarking Basics Measuring Function Performance Challenge: Implementing Benchmarking

2. Efficient Use of Data Structures

Lists and NumPy Arrays Sets and Tuples Challenge: Choosing Optimal Data Structures Using the collections Module Challenge: Handling Customer Requests

3. Enhancing Performance with Built-in Tools

Leveraging map() and List Comprehensions Handling Large Files Maximizing Sorting Efficiency Efficient String Operations

Using the collections Module

Although built-in data types and NumPy arrays handle most common tasks, the collections module offers specialized data structures designed for specific use cases. Among these, deque (double-ended queue) stands out for its significant performance benefits in certain scenarios.

Unlike lists, which require shifting elements when inserting or removing from the beginning, a deque allows efficient operations on both ends. Therefore, if you frequently need to add or remove elements from either end of a collection, a deque is a better choice.

Now, let's compare the performance of list and deque in a practical scenario:


              123456789101112131415161718192021222324252627282930
            
import os
os.system('wget https://codefinity-content-media-v2.s3.eu-west-1.amazonaws.com/courses/8d21890f-d960-4129-bc88-096e24211d53/section_1/chapter_3/decorators.py 2>/dev/null')
from decorators import timeit_decorator
from collections import deque

numbers_list = list(range(1, 10000001))
numbers_deque = deque(numbers_list)

@timeit_decorator(number=1000)
def list_append_left():
    # Insert -1 at the beginning
    numbers_list.insert(0, -1)

@timeit_decorator(number=1000)
def deque_append_left():
    numbers_deque.appendleft(-1)

@timeit_decorator(number=1000)
def list_pop_left():
    # Remove the element at index 0 (first element)
    numbers_list.pop(0)

@timeit_decorator(number=1000)
def deque_pop_left():
    numbers_deque.popleft()

list_append_left()
deque_append_left()
list_pop_left()
deque_pop_left()

In this example, we've created a list and a deque, each containing 1000000 numbers from 1 to 1000000 inclusive. As shown, inserting and removing elements at the beginning is much faster in a deque than in a list and remains efficient regardless of the size.

When it comes to inserting or removing elements at the beginning, both list and deque can handle these tasks efficiently. Therefore, using a deque solely for this purpose often doesn’t provide a significant advantage.


              1234567891011121314151617181920212223
            
import os
os.system('wget https://codefinity-content-media-v2.s3.eu-west-1.amazonaws.com/courses/8d21890f-d960-4129-bc88-096e24211d53/section_1/chapter_3/decorators.py 2>/dev/null')
from decorators import timeit_decorator
from collections import deque

numbers_list = list(range(1, 10000001))
numbers_deque = deque(numbers_list)

@timeit_decorator(number=1000)
def append_right(data_structure):
    data_structure.append(-1)

@timeit_decorator(number=1000)
def pop_right(data_structure):
    data_structure.pop()

print('List performance:')
append_right(numbers_list)
pop_right(numbers_list)

print('Dequeue performance:')
append_right(numbers_deque)
pop_right(numbers_deque)

The performance results are indeed similar for both data structures. However, appending to a list is slightly slower than appending to a deque because lists, implemented as dynamic arrays, occasionally need to resize by allocating a larger memory block and copying elements over. In contrast, deque's block-based structure avoids resizing, making appends consistently faster. This difference, however, becomes noticeable only with relatively large lists.

Everything was clear?

Thanks for your feedback!

Section 2. Chapter 4