Clojure Parallel Processing: Mastering `pmap` and `future`

future: One asynchronous computation whose result will be available later.
pmap: A parallel lazy mapping tool for independent, usually CPU-bound work across a collection.

future and pmap are both simple, but they solve different problems. Confusing them leads to the two classic mistakes:

• using future to build a whole data-processing topology
• using pmap on work that is too small, too lazy, or too effect-heavy to benefit

future Is a One-Task Tool

future is a good fit when one unit of work can be launched and joined later:

1(def parsed
2  (future
3    (parse-large-file path)))
4
5;; later
6@parsed

This is useful for:

• speculative work
• one expensive preload
• one background fetch
• overlapping independent operations

It is not a job scheduler, queue, or stream processor. Once you start building coordination around many futures, you usually need a more explicit model.

pmap Is Collection Parallelism

pmap is best when:

• each item is independent
• each function call is fairly expensive
• order of results should still match input order
• the function is mostly CPU-bound and side-effect-light

1(def scores
2  (doall
3    (pmap score-document docs)))

The doall matters because pmap is lazy. If you do not realize it, you may think work is parallelized while actually forcing results slowly at the consumer edge.

Why pmap Often Disappoints

pmap is frequently slower than expected because:

• per-item work is too cheap
• laziness hides actual execution timing
• the function performs blocking I/O
• downstream consumption is serial and late

pmap is not a magic speedup switch. It works when the workload is large enough to amortize coordination cost.

CPU-Bound vs Blocking Work

Use pmap for independent CPU-bound transformations. For blocking I/O, it is usually better to use:

• future
• thread
• explicit executors
• pipeline-blocking

Blocking work has different saturation behavior from CPU-bound work, so the execution model should reflect that.

Laziness Changes the Result

Because pmap is lazy, downstream realization determines when work is actually forced. That has two practical consequences:

• you may launch less parallel work than you assume until results are consumed
• a slow consumer can accidentally serialize the benefit

This is why doall, dorun, or an explicit downstream collection boundary often appears in correct pmap usage. The laziness is not wrong, but it changes how parallelism shows up in the program.

When to Prefer future

Choose future when:

• there are only a few tasks
• each task is substantial
• coordination is simple
• you want explicit joins

Choose pmap when:

• the work is naturally “map over many values”
• each call is independent
• the parallelism is really about collection processing

Common Misuses

The common mistakes are:

• using pmap on tiny units of work where coordination cost dominates
• running blocking network or filesystem calls inside pmap
• nesting many futures because there is no clearer concurrency model in the codebase
• forgetting that pmap preserves collection order, which may force a slower shape than an unordered worker design

If the workload is not naturally an expensive parallel map, another abstraction usually explains the system better.

Realization Boundaries Decide Throughput

pmap often surprises people because the parallel work is still tied to consumption. If downstream code realizes only a few elements at a time, the apparent “parallel map” behaves more like a lazily paced computation than a fully active worker farm.

That means review should ask:

• who is consuming the lazy result?
• how eagerly is it realized?
• whether batching or doall is actually intended
• whether a queue-and-worker design would express the throughput goal more honestly

Practical Rule

Use future for one background computation. Use pmap for independent, expensive collection transformations. If the workload needs queues, buffering, cancellation policy, or blocking-stage control, you have moved beyond both and should use a richer concurrency model.