15 Tips to Skyrocket Your Python Performance

Introduction

Python has long been criticized for its slow execution speed. However, it’s undeniable that Python remains a powerful tool in both our learning and work. Therefore, our relationship with Python is one of “love and hate.”

This article summarizes some small tips that can help improve Python’s execution speed and optimize performance. All the following techniques have been verified by me and can be safely applied.

Conclusion First:

Use map() for function mapping.
Use set() to find intersections.
Use sort() or sorted() for sorting.
Use collections.Counter() for counting.
Use list comprehensions.
Use join() to concatenate strings.
Use x, y = y, x to swap variables.
Use while 1 instead of while True.
Use decorator caching.
Reduce the use of the dot operator (.).
Use for loops instead of while loops.
Use Numba.jit to accelerate computations.
Use Numpy vectorized arrays.
Use in to check list membership.
Use the itertools library for iteration.

How to Measure Program Execution Time?

Accurately measuring a program’s execution time in Python seems simple but is actually quite complex. This is because execution time is influenced by many factors, such as the operating system, Python version, and related hardware (CPU performance, memory read/write speed). When running the same language version on the same computer, these factors are fixed. However, the program’s sleep time still varies, and other programs running on the computer can interfere with the experiment. Strictly speaking, this makes the experiment non-reproducible.

Two representative libraries I know for timing are time and timeit.

The time library has three functions for timing (in seconds): time(), perf_counter(), and process_time(). Appending _ns indicates timing in nanoseconds (since Python 3.7). Before this, there was the clock() function, but it was removed after Python 3.3. The differences between the three are as follows:

time(): Relatively lower precision and affected by the system. Suitable for representing date/time or timing large programs.
perf_counter(): Suitable for testing smaller programs, includes sleep() time.
process_time(): Suitable for testing smaller programs, excludes sleep() time.

Compared to the time library, timeit has two advantages:

timeit selects the best timer based on your operating system and Python version.
timeit temporarily disables garbage collection during timing.

timeit.timeit(stmt='pass', setup='pass', timer=<default timer>, number=1000000, globals=None) Parameter Description:

stmt='pass': The statement or function to be timed.
setup='pass': Code to run before executing stmt. Typically used to import modules or declare necessary variables.
timer=<default timer>: The timer function, defaults to time.perf_counter().
number=1000000: The number of times to execute the timed statement, defaults to one million times.
globals=None: Specifies the namespace for executing the code.

All timings in this article use the timeit method with the default of one million executions.

Why one million times? Because our test programs are very short; without this many executions, it would be impossible to see any difference.

Main Content

1. Use map() for Function Mapping

✅ Exp1: Convert lowercase letters in a string list to uppercase.

Test list: oldlist = ['life', 'is', 'short', 'i', 'choose', 'python']

Method 1

python

newlist = []
for word in oldlist:
    newlist.append(word.upper())

Method 2

python

list(map(str.upper, oldlist))

Method 1 time: 0.5267724000000005s
Method 2 time: 0.41462569999999843s
Performance improvement: 21.29% 🚀

2. Use set() to Find Intersections

✅ Exp2: Find the intersection of two lists.

Test lists: a = [1,2,3,4,5], b = [2,4,6,8,10]

Method 1

python

overlaps = []
for x in a:
    for y in b:
        if x == y:
            overlaps.append(x)

Method 2

python

list(set(a) & set(b))

Method 1 time: 0.9507264000000006s
Method 2 time: 0.6148200999999993s
Performance improvement: 35.33% 🚀

Note: |, &, - for union, intersection, and difference respectively.

3. Use sort() or sorted() for Sorting

Method 1 (Quick Sort)

python

def quick_sort(lists,i,j):
    # ... (quick sort implementation)
    return lists

Method 2

python

lists.sort()

Method 1 time: 2.4796975000000003s
Method 2 time: 0.05551999999999424s
Performance improvement: 97.76% 🚀

sorted() took 0.1339823999987857s.

Extension: How to define the key for sort() or sorted()?

Using lambda:pythonstudents.sort(key = lambda student: student[0]) # Sort by name
Using operator:pythonimport operator students.sort(key=operator.itemgetter(0)) sorted(students, key = operator.itemgetter(1, 0)) # Sort by grade, then name
Using cmp_to_key() (Most flexible):pythonimport functools def cmp(a,b): # … custom comparison logic sorted(students, key = functools.cmp_to_key(cmp))

4. Use collections.Counter() for Counting

✅ Exp4: Count the occurrences of each character in a string.

Test string: sentence='life is short, i choose python'

Method 1

python

counts = {}
for char in sentence:
    counts[char] = counts.get(char, 0) + 1

Method 2

python

from collections import Counter
Counter(sentence)

Method 1 time: 2.8105250000000055s
Method 2 time: 1.6317423000000062s
Performance improvement: 41.94% 🚀

5. Use List Comprehensions

✅ Exp5: Square the odd numbers in a list, leave evens unchanged.

Test list: oldlist = range(10)

Method 1

python

newlist = []
for x in oldlist:
    if x % 2 == 1:
        newlist.append(x**2)

Method 2

python

[x**2 for x in oldlist if x%2 == 1]

Method 1 time: 1.5342976000000021s
Method 2 time: 1.4181957999999923s
Performance improvement: 7.57% 🚀

6. Use join() to Concatenate Strings

✅ Exp6: Concatenate elements in a string list.

Test list: oldlist = ['life', 'is', 'short', 'i', 'choose', 'python']

Method 1

python

sentence = ""
for word in oldlist:
    sentence += word

Method 2

python

"".join(oldlist)

Method 1 time: 0.27489080000000854s
Method 2 time: 0.08166570000000206s
Performance improvement: 70.29% 🚀

join can also specify a separator:

python

"//".join(oldlist) # Result: 'life//is//short//i//choose//python'

7. Use x, y = y, x to Swap Variables

✅ Exp7: Swap the values of x and y.

Test data: x, y = 100, 200

Method 1

python

temp = x
x = y
y = temp

Method 2

python

x, y = y, x

Method 1 time: 0.027853900000010867s
Method 2 time: 0.02398730000000171s
Performance improvement: 13.88% 🚀

8. Use while 1 Instead of while True

✅ Exp8: Loop 100 times using while True vs while 1.

Method 1

python

i = 0
while True:
    i += 1
    if i > 100:
        break

Method 2

python

i = 0
while 1:
    i += 1
    if i > 100:
        break

Method 1 time: 3.679268300000004s
Method 2 time: 3.607847499999991s
Performance improvement: 1.94% 🚀

9. Use Decorator Caching

✅ Exp9: Calculate the Fibonacci sequence.

Test data: fibonacci(7)

Method 1

python

def fibonacci(n):
    if n == 0:
        return 0
    elif n == 1:
        return 1
    return fibonacci(n - 1) + fibonacci(n-2)

Method 2

python

import functools

@functools.lru_cache(maxsize=128)
def fibonacci(n):
    if n == 0:
        return 0
    elif n == 1:
        return 1
    return fibonacci(n - 1) + fibonacci(n-2)

Method 1 time: 3.955014900000009s
Method 2 time: 0.05077979999998661s
Performance improvement: 98.72% 🚀

Important Notes:

The cache uses parameters as keys. If parameters don’t change, the decorated function executes only once.
All parameters must be hashable. For example, list cannot be a parameter.

10. Reduce the Use of the Dot Operator (.)

✅ Exp10: Convert lowercase letters in a string list to uppercase.

Test list: oldlist = ['life', 'is', 'short', 'i', 'choose', 'python']

Method 1

python

newlist = []
for word in oldlist:
    newlist.append(str.upper(word))

Method 2

python

newlist = []
upper = str.upper
for word in oldlist:
    newlist.append(upper(word))

Method 1 time: 0.7235491999999795s
Method 2 time: 0.5475435999999831s
Performance improvement: 24.33% 🚀

11. Use for Loops Instead of while Loops

✅ Exp11: Loop 100 times using for vs while.

Method 1

python

i = 0
while i < 100:
    i += 1

Method 2

python

for _ in range(100):
    pass

Method 1 time: 3.894683299999997s
Method 2 time: 1.0198077999999953s
Performance improvement: 73.82% 🚀

12. Use Numba.jit to Accelerate Computations

✅ Exp12: Calculate the sum from 1 to 100.

Method 1

python

def my_sum(n):
    x = 0
    for i in range(1, n+1):
        x += i
    return x

Method 2

python

from numba import jit

@jit(nopython=True)
def numba_sum(n):
    x = 0
    for i in range(1, n+1):
        x += i
    return x

Method 1 time: 3.7199997000000167s
Method 2 time: 0.23769430000001535s
Performance improvement: 93.61% 🚀

13. Use Numpy Vectorized Arrays

✅ Exp13: Multiply two sequences element-wise.

Test lists: a = [1,2,3,4,5], b = [2,4,6,8,10]

Method 1

python

[a[i]*b[i] for i in range(len(a))]

Method 2

python

import numpy as np
a = np.array([1,2,3,4,5])
b = np.array([2,4,6,8,10])
a*b

Method 1 time: 0.6706845000000214s
Method 2 time: 0.3070132000000001s
Performance improvement: 54.22% 🚀

14. Use in to Check List Membership

✅ Exp14: Check if a list contains a specific member.

Test list: lists = ['life', 'is', 'short', 'i', 'choose', 'python']

Method 1

python

def check_member(target, lists):
    for member in lists:
        if member == target:
            return True
    return False

Method 2

python

if target in lists:
    pass

Method 1 time: 0.16038449999999216s
Method 2 time: 0.04139250000000061s
Performance improvement: 74.19% 🚀

15. Use the itertools Library for Iteration

✅ Exp15: Return all permutations of a list.

Test list: ["Alice", "Bob", "Carol"]

Method 1

python

def permutations(lst):
    # ... custom permutation implementation
    return result

Method 2

python

import itertools
list(itertools.permutations(["Alice", "Bob", "Carol"]))

Method 1 time: 3.867292899999484s
Method 2 time: 0.3875405000007959s
Performance improvement: 89.98% 🚀

Easy Python

15 Tips to Skyrocket Your Python Performance

New Article

Related articles