Python

ProcessPoolExecutor for CPU-Bound Work

admin by @admin ADMIN
Jun 16, 2026
May 31, 2026
Public
0 0 up · 0 down Sign in to vote
Use processes (not threads) for CPU-bound work to sidestep the GIL. Functions must be picklable — define them at module level, not as locals or lambdas.
Python
Raw
from concurrent.futures import ProcessPoolExecutor
import hashlib

def heavy(n: int) -> str:
    """Pure-CPU work — many threads would serialize on the GIL."""
    h = hashlib.sha256(b"seed").digest()
    for _ in range(n):
        h = hashlib.sha256(h).digest()
    return h.hex()[:12]

if __name__ == "__main__":
    inputs = [200_000] * 8
    with ProcessPoolExecutor() as pool:
        for digest in pool.map(heavy, inputs):
            print(digest)
    # On a 4-core machine: ~4x faster than threads for this kind of work
Tags

Save your own code snippets

Create a free account and build your private vault. Share publicly whenever you want.