Python

unique_everseen — Order-Preserving Dedupe

admin by @admin ADMIN
1d ago
May 31, 2026
Public
0 0 up · 0 down Sign in to vote
Remove duplicates while preserving original order (set() loses order on collisions). Optional key function lets you dedupe by a derived value — e.g., lowercased email.
Python
Raw
from typing import Callable, Hashable, Iterable, Iterator, TypeVar

T = TypeVar("T")

def unique_everseen[T](items: Iterable[T], key: Callable[[T], Hashable] | None = None) -> Iterator[T]:
    seen: set[Hashable] = set()
    for item in items:
        k = key(item) if key else item   # type: ignore[arg-type]
        if k not in seen:
            seen.add(k)
            yield item

list(unique_everseen([1, 2, 1, 3, 2, 4]))                 # [1, 2, 3, 4]

emails = ["Bob@X.com", "alice@x.com", "BOB@x.com"]
list(unique_everseen(emails, key=str.lower))              # ['Bob@X.com', 'alice@x.com']
Tags

Save your own code snippets

Create a free account and build your private vault. Share publicly whenever you want.