Python Collections: Lists, Tuples, Sets, and Map/Reduce Trade-offs
Choose Python collection types by invariants and access patterns, then use map, comprehensions, and reductions without obscuring intent.
On this page
Key takeaways
- Use lists for ordered mutable sequences, tuples for fixed records, and sets for unique membership tests.
- Hash-based set and dictionary keys need stable equality and hash behavior.
- Comprehensions are often clearer than map for simple transformations; generators avoid unnecessary materialization.
- Reduce is useful for associative accumulation, but a named loop can be easier to debug than a clever fold.
Pick the data structure that states the rule
A list preserves order and allows duplicates; use it for sequences that change. A tuple is an immutable sequence and can communicate a fixed record such as (host, port), though a dataclass or named tuple can make fields clearer. A set stores unique hashable values and is useful for membership and set operations. A dict maps hashable keys to values.
Sets and dictionaries rely on equality and hashing. If a custom object is used as a key, its hash must remain stable while it is stored. Mutable objects whose equality-relevant fields can change are poor keys because lookup may no longer find the stored entry.
seen_ids = set()
unique_in_order = []
for record in records:
if record.id not in seen_ids:
seen_ids.add(record.id)
unique_in_order.append(record)
This retains first-seen order while using a set for membership checks. It makes both requirements visible instead of hoping a collection choice implicitly supplies them.
map, comprehensions, and generators
map(function, values) applies a function lazily in Python 3. A list comprehension is often more readable for a simple transformation and can express filtering in one place: [normalize(value) for value in values if value]. A generator expression keeps the same flow lazy when the consumer can process values incrementally.
Avoid building a large intermediate list when a downstream operation can consume a generator. Conversely, materialize intentionally if values will be traversed more than once or if retaining a stable snapshot is part of the design.
Reduce and algebraic assumptions
functools.reduce combines values into one result. It is a good fit for associative operations such as summing values, but many transformations are clearer as a loop or a specialized built-in such as sum, min, or any. Floating-point addition is not perfectly associative, so changing grouping can change low-order bits.
A debugging-friendly accumulation often names the state explicitly:
total_by_team = {}
for event in events:
total_by_team[event.team] = total_by_team.get(event.team, 0) + event.cost
For parallel reductions, the operation generally needs a well-defined identity and associative combine rule. Hidden side effects in a mapping or reducer make ordering and parallel execution hard to reason about.
Complexity is part of the choice
List membership scans in linear time; set and dictionary membership are expected constant time but depend on hashing and can degrade under pathological collisions. Sorting a list costs more than keeping data in insertion order. Choose from measured operations and expected data size, not folklore.
Review checklist
Ask whether order matters, duplicates are meaningful, mutation is expected, keys remain stable, and the data must be consumed once or repeatedly. Those answers usually identify the right type and make the code's intent easier for the next engineer to maintain.