Formatting, Output, and What Actually Limits Throughput

Async logging is often presented as an obvious optimization:

• synchronous logging is slow
• async logging is fast
• therefore async logging is better

In practice, this is an oversimplification.

Async logging does not remove the cost of logging.
It only redistributes it — across threads, queues, memory, and time.

To understand its real behavior, we need to separate what async logging actually consists of.

---

1. Async Logging = Two Independent Problems

Async logging is not a single mechanism. It is a combination of two distinct subsystems:

• Async formatting
• Async output

These are often conflated, but they have very different constraints and trade-offs.

---

Async Formatting

This includes:

• capturing arguments
• copying or serializing data
• optionally formatting later (deferred formatting)

The key question is:

Can the backend safely reconstruct the data later?

If not, formatting cannot be deferred.

---

Async Output

This includes:

• queues
• backend worker thread(s)
• sinks (file, console, network)
• flushing and fsync

This part determines throughput and I/O behavior.

---

2. Deferred Formatting Is Conditional, Not Universal

Deferred formatting sounds attractive:

enqueue format string + arguments → format later on a backend thread

But this only works if arguments remain valid.

---

When It Breaks

If only the string_view is stored, the data becomes invalid before the backend processes it.

The same problem appears with:

• char* to temporary buffers
• references to mutable objects
• objects with internal pointers
• container views
• externally owned data

This is not a corner case — it is a fundamental limitation.

---

3. How Systems Actually Solve It

There are only a few real solutions.

---

1. Copy the Data (Safe Capture)

The logging system copies data at the call site:

• string_view → copied into owned buffer
• char* → copied
• objects → copied or serialized

This guarantees correctness.

But it also means:

Deferred formatting requires eager data capture — i.e. copying.

---

2. Format Immediately

If copying is unsafe or too complex:

The system formats immediately on the producer thread and stores the resulting string.

This avoids lifetime issues but increases hot-path latency.

---

3. Custom Serialization

For complex types:

• user-defined serialization
• custom codecs
• explicit control over what is captured

This is flexible — but increases complexity.

---

4. The Hidden Cost of Deferred Formatting

At this point, the pipeline is not:

It is:

So the real trade-off becomes:

• replacing formatting cost with copying + serialization
• introducing memory overhead
• adding complexity

In many cases:

copy + deferred format ≈ immediate format

---

5. Async Output Does Not Increase Throughput

Async logging decouples the caller from I/O.
It does not make I/O faster.

---

The Bottleneck Still Exists

If your sinks process:

and producers generate:

then:

This is inevitable.

---

The Queue Is Not Magic

A queue does not remove work.

It only delays when you notice that the work cannot be completed.

The queue is not acceleration — it is debt storage.

---

6. Single Backend Thread: A Hard Limit

Many async logging systems use:

• multiple producers
• a single backend thread

---

No Automatic Scaling

Even with many producer threads:

• there is still one consumer
• throughput is limited by that thread

Async logging does not scale across CPU cores automatically.

---

Multiple Sinks Multiply Cost

For each log message:

Total cost = sum of all sinks.

---

Critical Consequence

Adding a slow sink slows down all logging.

Example:

• file: 10 µs
• console: 200 µs

→ total: 210 µs

The slowest sink dominates.

---

7. What Happens Under Sustained Overload

When:

the system must choose how to behave.

---

Possible Strategies

1. Grow Buffers

• queues expand
• memory usage increases

Temporary solution only.

---

2. Block Producers

• caller threads slow down
• async advantage disappears

---

3. Drop Messages

• system remains responsive
• logs become incomplete

---

4. Overwrite Old Data

• bounded memory
• loss of history

---

Key Insight

Async logging systems do not eliminate overload — they define how overload is handled.

---

8. Why Deferred Formatting Is Often Overrated

Deferred formatting is often marketed as a major optimization.

In reality, its benefits are conditional.

---

CPU Redistribution, Not Elimination

Deferred formatting moves work:

• from producer threads
• to the backend thread

It does not eliminate it.

---

Backend Becomes the Bottleneck

If formatting is deferred:

• producers become lighter
• backend becomes heavier

If backend is already the bottleneck, this can reduce total throughput.

---

Real Benefit

The actual benefit is:

lower latency on the hot path

Not lower total cost.

---

When It Helps

• high-frequency logging
• many producer threads
• simple argument types
• burst workloads

---

When It Does Not

• slow sinks dominate
• complex argument types
• sustained overload
• backend already saturated

---

9. Conceptual Complexity and New Failure Modes

Async formatting changes the mental model of logging.

---

Synchronous Model

“What you log is what you see.”

---

Async Model

Correctness depends on:

• argument lifetime
• safe capture
• serialization rules
• backend timing

---

New Classes of Bugs

• dangling references
• partially updated objects
• corrupted log output
• nondeterministic behavior

These bugs are rare, timing-dependent, and hard to reproduce.

---

Increased User Responsibility

Users must now consider:

• ownership vs references
• lifetime guarantees
• copy vs no-copy trade-offs
• when to force immediate formatting

In some systems, this requires explicit APIs or wrappers.

---

10. Practical Design Convergence

Because of these constraints, many systems converge to:

synchronous formatting + asynchronous output

This approach:

• avoids lifetime issues
• simplifies reasoning
• reduces user burden
• keeps async benefits for I/O

---

Trade-off Summary

---

11. Real-World Implementations

To make this discussion concrete, consider several widely used C++ logging libraries:

• spdlog — https://github.com/gabime/spdlog
• Quill — https://github.com/odygrd/quill
• logme — https://github.com/efmsoft/logme

All of them support asynchronous logging, but with different design choices:

• some focus on async output
• some push more work into the backend (including formatting)
• some balance simplicity and predictability

---

12. What Benchmarks Actually Show

To validate these ideas, we can look at real measurements from:

• logbench benchmark project — https://github.com/efmsoft/logbench

---

Synchronous Mode Performance

Libraries such as:

• spdlog
• logme

often demonstrate very strong performance in synchronous mode, sometimes outperforming async configurations.

---

What This Means

In synchronous mode:

• formatting happens on the caller thread
• output is immediate
• no queues, no background workers

So:

The core logging path is already highly optimized.

---

Interpreting This Correctly

Async logging introduces additional layers:

• queues
• synchronization
• memory management
• background threads

So total cost becomes:

---

Key Insight

Async logging does not make a slow logging core fast.

It only redistributes where the work happens.

---

Important Clarification

When we force all libraries into synchronous mode:

• we are not “making them faster”
• we are removing work distribution

All work happens in one place:

---

Connecting Back to the Core Model

This reinforces the central idea:

Async logging is not about reducing work — it is about redistributing it between threads.

---

13. Case Study: Quill

Quill is a good example of a fully async design.

It uses:

• per-thread queues
• a single backend thread
• configurable overflow policies

---

Multi-Level Buffering

Quill introduces:

• frontend queues
• backend transit buffer

Under overload:

1. backend buffer fills
2. backend stops reading frontend queues
3. frontend queues fill
4. system blocks or drops

---

Formatting Modes

Quill explicitly separates:

• deferred formatting (safe only with proper capture)
• immediate formatting (when required)

---

Comparison With Simpler Models

Compared to:

• spdlog
• logme

Quill pushes more work into the backend thread.

This reduces producer latency, but increases the risk that:

• backend becomes the bottleneck
• queues grow under sustained load

---

14. Final Takeaways

• Async logging reduces caller latency, not total work
• Throughput is limited by sinks, not producers
• Queues delay problems — they do not eliminate them
• Deferred formatting requires safe capture (usually copying)
• Multiple sinks multiply cost
• Single backend thread is a scalability limit

---

Final Thought

Async logging is not about making logging free.

It is about choosing where and how you pay for it.

Real-world benchmarks confirm this:

• well-optimized synchronous logging is already very fast
• async logging adds flexibility, not free performance

Async logging does not remove the cost — it only moves it.