• 0. What Does This Article Answer Next?
• 1. What Guarantees Does Java volatile Provide at the Language Level?
  • 1.1 Visibility and Ordering
  • 1.2 volatile Does Not Make Compound Operations Atomic
• 2. How Does HotSpot Implement volatile?
  • 2.1 Layers
  • 2.2 End-to-End Implementation Path for a volatile Write and Read
  • 2.3 Guarantees at the Runtime / Language Implementation Layer
    • 2.3.1 Visibility
    • 2.3.2 Ordering
  • 2.4 Guarantees at the Hardware Layer
• 3. Why Don't Go and CPython Have Java-style volatile?
  • 3.1 Go: A Language Design Choice
  • 3.2 CPython: Synchronization Remains a Runtime Responsibility
• 4. How volatile, Atomics, and Mutexes Differ
• 5. Next: From Mutexes to Read-Write Locks

0. What Does This Article Answer Next?

The previous two articles traced atomic operations from language-level semantics down to the CPU, showing how Atomicity, Visibility, and Ordering are implemented.

This article follows the same path for volatile.

volatile does not make a compound update such as counter++ atomic. It is better suited to publication and observation:

Thread A                         Thread B

counter = 1                     if (ready) {
ready = true                        print(counter)
                                }

The required guarantee is:

If Thread B observes ready == true, a subsequent read of counter must observe 1.

We first establish this guarantee in the Java Memory Model, then trace how HotSpot preserves it down to the CPU.

1. What Guarantees Does Java volatile Provide at the Language Level?

1.1 Visibility and Ordering

Declare ready as volatile:

int counter = 0;
volatile boolean ready = false;

// Thread A
counter = 1;
ready = true;

// Thread B
if (ready) {
    System.out.println(counter);
}

counter remains an ordinary field.

JLS §17.4.4 states:

“A write to a volatile variable v synchronizes-with all subsequent reads of v by any thread.”

So if Thread B's volatile read observes the value true written by Thread A, the relationship is:

Thread A                              Thread B

counter = 1
    │
    │ program order
    ▼
volatile ready = true ── synchronizes-with ──► read volatile ready == true
                                                   │
                                                   │ program order
                                                   ▼
                                               read counter

By transitivity of happens-before:

write counter
    ↓ happens-before
write volatile ready
    ↓ synchronizes-with
read volatile ready
    ↓ happens-before
read counter

Once B observes ready == true, the earlier counter = 1 write must be visible to B. The following outcome is therefore forbidden:

ready == true
counter == 0

That gives us two guarantees:

• Visibility: writes completed by A before the volatile write become visible to B after B observes that write;
• Ordering: the operations must be observed in a way that is consistent with the happens-before relationship.

1.2 volatile Does Not Make Compound Operations Atomic

Consider:

volatile int counter = 0;

counter++;

counter++ is still a read-modify-write operation:

volatile read
      ↓
     add
      ↓
volatile write

volatile constrains the visibility and ordering of the individual read and write, but it does not make the entire read → add → write sequence indivisible.

With multiple threads incrementing concurrently, updates can still be lost.

If you need an atomic RMW, use AtomicInteger, CAS, or a mutex.

2. How Does HotSpot Implement volatile?

2.1 Layers

As in the Mutex and Atomic articles, first establish the implementation stack:

Java Source Code
Example: volatile boolean ready
Role: declare a field with volatile memory semantics
        │
        ▼
JVM Bytecode
Example: getfield / putfield (or getstatic / putstatic)
Field metadata: ACC_VOLATILE
Role: perform ordinary field access; volatility is carried by field metadata
        │
        ▼
JVM Implementation (HotSpot)
Example: field->is_volatile() / MO_SEQ_CST / MemBar
Role: preserve and implement volatile memory-ordering semantics
        │
        ▼
x86-64 Hardware
Example: Load / Store / Cache Coherence / Fence
Role: provide the hardware basis for Visibility and Ordering

This differs from both synchronized and AtomicInteger:

synchronized
  → dedicated monitorenter / monitorexit bytecodes

AtomicInteger
  → ordinary invokevirtual
  → no Atomic-specific bytecode

volatile
  → still uses getfield / putfield
  → no dedicated volatile-load / volatile-store bytecode
  → the volatile property is recorded in ACC_VOLATILE field metadata

2.2 End-to-End Implementation Path for a volatile Write and Read

Continue with the same counter / ready example:

The key point is not a particular machine instruction. HotSpot must recognize that ready is volatile and preserve its memory semantics all the way to the target architecture.

2.3 Guarantees at the Runtime / Language Implementation Layer

2.3.1 Visibility

HotSpot distinguishes ordinary fields from volatile fields while parsing field accesses. In the current C2 path, volatile accesses carry memory-order information such as MO_SEQ_CST, while ordinary fields use unordered access semantics.

Conceptually:

ordinary field
  → ordinary load / store

volatile field
  → volatile load / store
  → preserve cross-thread observable memory semantics

HotSpot therefore cannot optimize a volatile access as if it were an ordinary field access—for example, by eliminating, merging, or moving it across synchronization boundaries that matter.

2.3.2 Ordering

HotSpot must also constrain reordering around volatile accesses.

In C2's IR, the typical structure includes:

volatile store
  ← ordering constraints such as MemBarRelease

volatile load
  → ordering constraints such as MemBarAcquire

Cases that require stronger ordering—such as volatile store → volatile load—also require the corresponding StoreLoad / volatile-barrier constraint.

These are IR nodes and compiler-ordering constraints, not a one-to-one promise that every barrier becomes a separate CPU fence instruction.

Relevant OpenJDK sources include parse3.cpp, memnode.hpp, and templateTable_x86.cpp.

2.4 Guarantees at the Hardware Layer

At the hardware layer, these guarantees reduce to the same two capabilities introduced earlier in the series:

Visibility
  → Cache Coherence
  → other CPU cores cannot keep using an invalidated old cache line indefinitely

Ordering
  → CPU Memory Ordering + Fence / equivalent ordering constraints
  → preserve the memory-access order required at volatile boundaries

x86-64 already provides relatively strong memory ordering, so many Acquire / Release constraints do not require a standalone hardware fence instruction.

When a StoreLoad fence is required, HotSpot's Linux x86 path can use:

lock; addl $0, 0(%rsp)

to provide a full-fence effect.

See OpenJDK orderAccess_linux_x86.hpp.

3. Why Don't Go and CPython Have Java-style volatile?

Go and Python do not expose Java's field-level volatile mechanism. The reasons are different, so consider them separately.

3.1 Go: A Language Design Choice

Effective Go summarizes one of Go's central concurrency design principles with a well-known rule:

“Do not communicate by sharing memory; instead, share memory by communicating.”

That reflects a language-design choice: synchronization should be expressed through explicit concurrency primitives rather than implicitly through field modifiers.

Accordingly, Go does not provide a Java-style field-level volatile modifier. It expresses the same kinds of synchronization through channels, mutexes, and atomic operations.

For the use case closest to Java volatile, Go uses sync/atomic. The Go Memory Model states:

“This definition provides the same semantics as C++'s sequentially consistent atomics and Java's volatile variables.”

Use the same counter / ready example:

var counter int
var ready atomic.Bool

// Goroutine A
counter = 1
ready.Store(true)

// Goroutine B
if ready.Load() {
    fmt.Println(counter)
}

If ready.Load() observes the value written by ready.Store(true), the two atomic operations establish a synchronized-before relationship:

Goroutine A                          Goroutine B

counter = 1
    │
    │ sequenced-before
    ▼
ready.Store(true) ── synchronized-before ──► ready.Load() == true
                                                 │
                                                 │ sequenced-before
                                                 ▼
                                             read counter

So once B observes ready == true, the earlier counter = 1 write must be visible to B.

3.2 CPython: Synchronization Remains a Runtime Responsibility

Python does not expose Java-style volatile. The key reason is not syntax; synchronization has historically been a responsibility of the CPython runtime.

There are two main points.

1. Traditional CPython uses the GIL to cover many Visibility / Ordering concerns

The CPython C API documentation states:

“only a thread that holds the GIL may operate on Python objects or invoke Python’s C API.”

In traditional CPython, the GIL serializes many operations on Python objects.

Releasing and reacquiring the GIL also forms a thread-synchronization boundary, so in practice it addresses many of the Visibility and Ordering problems that volatile is used to solve.

The abstractions are still different:

Java volatile
  → field-level language semantics

CPython GIL
  → runtime-level synchronization mechanism

2. Free-threaded CPython still uses runtime locks and atomics

Making the GIL optional did not introduce field-level volatile semantics.

PEP 703 instead says:

“This PEP proposes using per-object locks...”

The runtime also uses atomic operations internally.

So the implementation shifts from:

GIL

to:

Per-object lock + atomic

Synchronization therefore remains a runtime responsibility rather than becoming part of Python's field-level language semantics.

At the application level, publication and observation are expressed through synchronization objects. The documentation for threading.Event says:

“one thread signals an event and other threads wait for it.”

Use the same counter / ready example:

counter = 0
ready = threading.Event()

# Thread A
counter = 1
ready.set()

# Thread B
ready.wait()
print(counter)

Here, Event plays the synchronization role that ready played in the Java example:

Thread A                         Thread B

counter = 1
    │
    ▼
ready.set()  ───────────────►  ready.wait() returns
                                  │
                                  ▼
                              read counter

So Python application code uses Event / Lock for synchronization, while lower-level atomics and memory ordering remain implementation details of the CPython runtime.

4. How volatile, Atomics, and Mutexes Differ

The three abstractions operate at different granularities:

A compact way to view the relationship is:

volatile
  → establish Visibility / Ordering between a publication and a later observation

Atomic
  → make one shared-state operation indivisible
  → also provide the corresponding Visibility / Ordering

Mutex
  → use lower-level primitives such as atomics to arbitrate lock state
  → then protect an entire critical section

These are not simply stronger and weaker versions of the same tool. They solve different problems at different granularities.

5. Next: From Mutexes to Read-Write Locks

A Mutex allows only one execution unit to enter a critical section at a time.

If most operations only read shared state, making readers exclude one another is unnecessary.

The next article looks at read-write locks: multiple readers can proceed concurrently, while writers still retain exclusive access.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

© ThinkerQAQ. Original articles, notes and accompanying technical diagrams use CC BY-NC-SA 4.0: noncommercial reuse and adaptations require attribution, change notices and ShareAlike; commercial use requires permission. Photographs and standalone illustrations are All Rights Reserved; third-party rights remain with their owners.

• 0. What Does This Article Answer Next?
• 1. How Does HotSpot Implement AtomicInteger?
  • 1.1 Layers
  • 1.2 End-to-End Implementation Path for AtomicInteger
  • 1.3 Guarantees at the Runtime / Language Implementation Layer
    • 1.3.1 Atomicity
    • 1.3.2 Visibility and Ordering
  • 1.4 Guarantees at the Hardware Layer
• 2. How Is sync/atomic Implemented in Go?
  • 2.1 Layers
  • 2.2 End-to-End Implementation Path for atomic.Int64
  • 2.3 Guarantees at the Runtime / Language Implementation Layer
    • 2.3.1 Atomicity
    • 2.3.2 Visibility and Ordering
  • 2.4 Guarantees at the Hardware Layer
• 3. How Does CPython Implement Internal Atomics?
  • 3.1 Layers
  • 3.2 End-to-End Implementation Path for CPython Atomics
  • 3.3 Guarantees at the Runtime / Language Implementation Layer
    • 3.3.1 Atomicity
    • 3.3.2 Visibility and Ordering
  • 3.4 Guarantees at the Hardware Layer
• 4. The Common Pattern Across the Three Implementations
  • 4.1 How Does a Single-variable Update Become Indivisible? — Atomic RMW
  • 4.2 What Happens When a Conditional Update Fails? — CAS / Retry
  • 4.3 Why Can Atomics Also Establish Visibility and Ordering?
  • 4.4 How Do Atomics Differ from Mutexes?
    • 4.4.1 One State Update, Two Approaches
    • 4.4.2 Why Can't Multiple Atomics Replace One Mutex?
    • 4.4.3 Which Should You Use?
• 5. Next: volatile

0. What Does This Article Answer Next?

The previous article examined how atomic operations provide Atomicity, Visibility, and Ordering through the rules of language memory models.

This article goes deeper: how do Java, Go, and CPython implement these guarantees at the runtime and CPU layers?

1. How Does HotSpot Implement AtomicInteger?

1.1 Layers

Java Source Code
Example: AtomicInteger.incrementAndGet() / compareAndSet() / get()
Role: express atomic operations
        │
        ▼
JVM Bytecode
Example: invokevirtual AtomicInteger.incrementAndGet() / compareAndSet() / get()
Role: invoke AtomicInteger methods; there is no dedicated bytecode for atomic operations
        │
        ▼
JVM Implementation (HotSpot)
Example: Intrinsic / C1 / C2
Role: lower atomic operations to the target architecture
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Fence
Role: provide the hardware basis for Atomicity, Visibility, and Ordering

1.2 End-to-End Implementation Path for AtomicInteger

The main atomic operations can be expanded as follows:

1.3 Guarantees at the Runtime / Language Implementation Layer

1.3.1 Atomicity

At the Runtime / Language Implementation layer, Atomicity is preserved through two kinds of indivisible operations:

Atomic Add
  → Unsafe.getAndAddInt
  → HotSpot Intrinsic / C1 / C2
  → Atomic Get-And-Add

CAS
  → Unsafe.compareAndSetInt
  → HotSpot Intrinsic / C1 / C2
  → Atomic CAS

HotSpot preserves their atomic semantics through compilation and lowering; the target architecture then provides the concrete machine instructions.

1.3.2 Visibility and Ordering

Visibility and Ordering depend on HotSpot preserving these memory effects:

Write side: Atomic Add / CAS
Read side: volatile read

The JMM / VarHandle memory effects define the memory semantics of these operations. HotSpot must preserve the corresponding ordering constraints during compilation and optimization.

1.4 Guarantees at the Hardware Layer

At the hardware layer, CPU cache coherence and memory ordering implement these guarantees.

For Java, the mapping to the hardware model in the first article is:

Atomicity
  → Atomic Instruction
  → HotSpot / x86-64: LOCK XADD / LOCK CMPXCHG

Visibility
  → Cache Coherence
  → Typical x86-64 CPUs: MESI-family (e.g. MESIF / MOESI)

Ordering
  → Fence / equivalent ordering constraints
  → x86-64 memory ordering + LOCKed RMW / Load rules

For implementation details, see OpenJDK's AtomicInteger.java, Unsafe.java, vmIntrinsics.hpp, c1_GraphBuilder.cpp, and c2compiler.cpp.

2. How Is sync/atomic Implemented in Go?

2.1 Layers

Go Source Code
Example: atomic.Int64.Add() / CompareAndSwap() / Load()
Role: express atomic operations
        │
        ▼
Go Implementation
Example: sync/atomic + internal/runtime/atomic
Role: implement the atomic API and target-specific atomic primitives
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Fence
Role: provide the hardware basis for Atomicity, Visibility, and Ordering

2.2 End-to-End Implementation Path for atomic.Int64

The main atomic operations can be expanded as follows:

2.3 Guarantees at the Runtime / Language Implementation Layer

2.3.1 Atomicity

At the Runtime / Language Implementation layer, Atomicity is represented by Go's atomic primitives:

Atomic Add
  → sync/atomic
  → internal/runtime/atomic.Xadd64

CAS
  → sync/atomic
  → internal/runtime/atomic.Cas64

These operations already carry indivisible atomic semantics at this layer; the target architecture provides the concrete machine instructions.

2.3.2 Visibility and Ordering

Visibility and Ordering depend on the Go implementation preserving atomic semantics:

Write side: Xadd64 / Cas64
Read side: Load64

The Go Memory Model defines the synchronization and ordering semantics of atomic operations. The compiler and Runtime must preserve those semantics.

2.4 Guarantees at the Hardware Layer

At the hardware layer, CPU cache coherence and memory ordering implement these guarantees.

For Go, the mapping to the hardware model in the first article is:

Atomicity
  → Atomic Instruction
  → Go / x86-64: LOCK XADDQ / LOCK CMPXCHGQ

Visibility
  → Cache Coherence
  → Typical x86-64 CPUs: MESI-family (e.g. MESIF / MOESI)

Ordering
  → Fence / equivalent ordering constraints
  → x86-64 memory ordering + LOCKed RMW / Load rules

For implementation details, see sync/atomic/type.go, sync/atomic/asm.s, and internal/runtime/atomic/atomic_amd64.s.

3. How Does CPython Implement Internal Atomics?

Python's standard library has no general-purpose integer atomic API equivalent to AtomicInteger or atomic.Int64, so this section starts inside the CPython runtime.

3.1 Layers

CPython Runtime Source Code
Example: internal shared-state updates
Role: call internal atomic operations
        │
        ▼
CPython Implementation
Example: _Py_atomic_* + GCC / Clang __atomic_*
Role: implement atomic operations and memory-order semantics, then lower them to the target architecture
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Fence
Role: provide the hardware basis for Atomicity, Visibility, and Ordering

3.2 End-to-End Implementation Path for CPython Atomics

The main internal atomic operations can be expanded as follows:

3.3 Guarantees at the Runtime / Language Implementation Layer

3.3.1 Atomicity

At the Runtime / Language Implementation layer, CPython expresses Atomicity through its internal atomic abstraction and the compiler's atomic builtins:

Atomic Add
  → _Py_atomic_add_*
  → __atomic_fetch_add

CAS
  → _Py_atomic_compare_exchange_*
  → __atomic_compare_exchange_n

These operations carry explicit atomic semantics before the compiler selects the final instruction sequence for the target architecture.

3.3.2 Visibility and Ordering

Visibility and Ordering correspond to the atomic write and read boundaries in the flowchart:

Write side: Atomic Add / CAS
Read side: Atomic Load

_Py_atomic_* passes the memory order to GCC / Clang's __atomic_* builtins, preserving the corresponding ordering semantics at the implementation layer.

3.4 Guarantees at the Hardware Layer

The compiler then maps these atomic operations and memory orders to the target architecture.

For CPython, the mapping to the hardware model in the first article is:

Atomicity
  → Atomic Instruction
  → x86-64: LOCK XADD / LOCK CMPXCHG

Visibility
  → Cache Coherence
  → Typical x86-64 CPUs: MESI-family (e.g. MESIF / MOESI)

Ordering
  → Fence / equivalent ordering constraints
  → x86-64 ordering rules + compiler-selected instruction sequence

This describes the CPython runtime's implementation; it does not turn those details into a memory model for Python application code.

For implementation details, see pyatomic.h and pyatomic_gcc.h.

4. The Common Pattern Across the Three Implementations

After examining Java, Go, and CPython, we can set aside the specific APIs: atomic operations perform essentially the same tasks.

4.1 How Does a Single-Variable Update Become Indivisible? — Atomic RMW

An ordinary read → modify → write consists of three separate steps:

Load
  ↓
Modify
  ↓
Store

Other execution units can interleave with those steps. For example, both CPUs can read the old value and overwrite one another's increment:

CPU A                       CPU B
Load 0                      Load 0
Add 1                       Add 1
Store 1                     Store 1

Atomic Add, Exchange, CAS, and other atomic RMW operations make the shared-state update indivisible:

Atomic RMW
   ├── Add
   ├── Swap
   └── CAS

At the lowest level, CPU atomic instructions carry out these operations.

4.2 What Happens When a Conditional Update Fails? — CAS / Retry

With Atomic Add, two competing increments both take effect, regardless of their order:

counter = 0
CPU A                       CPU B
Atomic Add
0 → 1
                            Atomic Add
                            1 → 2

CAS behaves differently: when both CPUs compare against the same expected value, only one can succeed:

CPU A                       CPU B
CAS 0 → 1                   CAS 0 → 1
    │                           │
    ▼                           ▼
  success                     failed

When CAS fails, the higher-level code decides what to do:

CAS
 │
 ├── success ──> done
 │
 └── failed
       │
       ├── return false
       │
       └── reload → retry

Atomic operations therefore do not follow the fixed Spin → Park → Wakeup path of a mutex. Add completes an atomic update directly; CAS returns failure when it fails, leaving the decision to retry to higher-level code.

4.3 Why Can Atomics Also Establish Visibility and Ordering?

Atomicity guarantees only that this update cannot be split apart.

Ordinary reads and writes around an atomic operation must also satisfy the memory ordering specified by the higher layers:

earlier ordinary writes
      │
      ▼
Atomic Update
      │
      ▼
Atomic Read
      │
      ▼
later ordinary reads

Three layers implement this relationship:

Language / Runtime Memory Semantics
        ↓
Compiler Ordering
        ↓
Hardware Memory Ordering + Cache Coherence

Despite their different APIs, all three implementations ultimately address the same three concerns:

make each update indivisible
        +
handle conflicts
        +
preserve the required order of surrounding memory accesses

4.4 How Do Atomics Differ from Mutexes?

The simplest distinction is:

An atomic operation makes one shared-state operation indivisible; a Mutex protects a block of code.

4.4.1 One State Update, Two Approaches

Suppose all we need is to increment one counter.

With a Mutex:

Lock
  ↓
counter++
  ↓
Unlock

With an Atomic:

Atomic Add(counter, 1)

The Mutex approach is:

let one execution unit enter
        ↓
run the protected code
        ↓
leave the critical section

The Atomic approach is:

make this update itself
indivisible

If the problem is truly one Add, CAS, Swap, Load, or Store, there is no need to wrap that atomic operation in an additional critical section.

4.4.2 Why Can't Multiple Atomics Replace One Mutex?

Suppose one logical operation must update two pieces of state:

balance -= 100
count++

Making both variables atomic gives us:

Atomic balance -= 100
        ↓
        ← another thread may observe this intermediate state
        ↓
Atomic count++

Each update is indivisible on its own, but the two updates are not atomic as a group.

A Mutex can protect both updates as one critical section:

Lock
  ↓
balance -= 100
count++
  ↓
Unlock

Until that critical section completes, another contender cannot enter the same protected region.

So once the requirement changes from “update one state” to “these steps must complete together,” a Mutex usually expresses the intent more directly.

4.4.3 Which Should You Use?

Look at what needs protection:

one state operation
Add / CAS / Swap / Load / Store
        ↓
      Atomic

a block of logic
read → decide → update multiple states
        ↓
      Mutex

The two are not independent. A Mutex commonly uses atomic operations or CAS internally to decide who acquires the lock:

Atomic / CAS
    ↓
compete for lock state
    ↓
Mutex
    ↓
protect a critical section

So the relationship can be summarized as:

Atomic: one shared-state operation
Mutex:  a critical section built on lower-level primitives such as atomic operations / CAS

5. Next: volatile

Atomic operations make RMW updates such as counter++ atomic.

If no RMW is needed, and the goal is simply to make one thread's writes visible to another in the correct order, we reach the problem addressed by volatile.

The next article returns to language-level rules to discuss Visibility and Ordering.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

© ThinkerQAQ. Original articles, notes and accompanying technical diagrams use CC BY-NC-SA 4.0 (https://creativecommons.org/licenses/by-nc-sa/4.0/): noncommercial reuse and adaptations require attribution, change notices and ShareAlike; commercial use requires permission. Photographs and standalone illustrations are All Rights Reserved; third-party rights remain with their owners. Original article

• 0. Continue from the Previous Article
• 1. What Guarantees Do the Atomic Rules in Each Language Provide?
  • 1.1 Java: AtomicInteger
  • 1.2 Go: sync/atomic
  • 1.3 CPython: No Symmetric Application-level Atomic API
• 2. Next: How Are Atomics Implemented?

0. Continue from the Previous Article

This article discusses Atomics directly from the rules defined by language memory models and public APIs.

1. What Guarantees Do the Atomic Rules in Each Language Provide?

1.1 Java: AtomicInteger

Start with the rules.

AtomicInteger.incrementAndGet() states:

“Atomically increments the current value, with memory effects as specified by VarHandle.getAndAdd.”

This is the Atomicity guarantee: reading the old value, adding one, and writing the result back happen as one indivisible atomic read-modify-write operation.

For Visibility and Ordering, ordinary reads and writes through AtomicInteger have volatile semantics: get() uses VarHandle.getVolatile, while set() uses VarHandle.setVolatile.

JLS §17.4.4 Synchronization Order states:

“A write to a volatile variable v synchronizes-with all subsequent reads of v by any thread.”

So a volatile write to an Atomic variable can establish a synchronizes-with relationship with a later volatile read in another thread that observes that write.

JLS §17.4.5 Happens-before Order further states:

“If one action happens-before another, then the first is visible to and ordered before the second.”

That happens-before relationship gives us both Visibility and Ordering: earlier writes become visible to later reads, and those operations must be observed in the order required by happens-before.

Now apply those rules to the same two examples used earlier in the series.

Atomicity

With an ordinary increment:

Thread A                    Thread B

count++                     count++

both threads can read the old value 0, causing a lost update and leaving the final value at count = 1.

Now make the same update atomic:

AtomicInteger count = new AtomicInteger(0);

// Thread A
count.incrementAndGet();

// Thread B
count.incrementAndGet();

The two updates are applied to the same value one after the other:

Thread A                    Thread B

incrementAndGet()           incrementAndGet()
        │                           │
        ▼                           ▼
      0 → 1                       1 → 2

The final result is:

count = 2

Here, Atomicity comes from incrementAndGet() itself being an atomic RMW operation. Unlike a Mutex, it does not serialize an entire critical section.

Visibility / Ordering

Now return to the same counter / ready example. With plain reads and writes, observing ready == true alone does not establish the ordering needed to guarantee that the earlier counter = 1 write is visible.

Make only ready atomic:

int counter = 0;
AtomicInteger ready = new AtomicInteger(0);

// Thread A
counter = 1;        // A1
ready.set(1);       // A2

// Thread B
if (ready.get() == 1) {     // B1
    System.out.println(counter); // B2
}

If B's ready.get() observes the value written by A's ready.set(1), then:

Thread A                              Thread B

A1  counter = 1
        │
        │ program order
        ▼
A2  ready.set(1)
        │
        │ synchronizes-with
        ▼
                                    B1  ready.get() == 1
                                        │
                                        │ program order
                                        ▼
                                    B2  read counter

By happens-before transitivity:

A1 happens-before A2
A2 happens-before B1
B1 happens-before B2

        ↓

A1 happens-before B2

So once B observes ready == 1 through the Atomic variable, the earlier counter = 1 write is visible to B, and this result is not allowed:

ready == 1
counter == 0

That is how the same Atomic operation provides Visibility and Ordering in this example.

1.2 Go: sync/atomic

Start with the rules.

atomic.Int64.Add states:

“Add atomically adds delta to x and returns the new value.”

This is the Atomicity guarantee: Add(1) performs the read, increment, and write-back as one atomic update.

For Visibility and Ordering, The Go Memory Model - Atomic Values states:

“If the effect of an atomic operation A is observed by atomic operation B, then A is synchronized before B.”

In other words, if B's atomic operation observes the result of A's atomic operation, A is synchronized-before B.

The same section also states that all atomic operations behave as though they were in:

“some sequentially consistent order”

The Go Memory Model defines happens-before as the transitive closure of sequenced-before and synchronized-before.

So atomic operations do more than update values indivisibly. When one atomic operation observes another, they can connect the ordinary memory operations around them into a happens-before chain, providing Visibility and Ordering as well.

Now apply those rules to the same two examples.

Atomicity

A plain count++ can lose an update. Replace it with:

var count atomic.Int64

// Goroutine A
count.Add(1)

// Goroutine B
count.Add(1)

Neither update is lost:

Goroutine A                 Goroutine B

Add(1)                      Add(1)
  │                           │
  ▼                           ▼
0 → 1                       1 → 2

The final result is:

count = 2

Here, Atomicity comes directly from the atomic semantics of Add(1).

Visibility / Ordering

Now return to the same counter / ready example, making only ready atomic:

var counter int
var ready atomic.Bool

// Goroutine A
counter = 1          // A1
ready.Store(true)    // A2

// Goroutine B
if ready.Load() {    // B1
    fmt.Println(counter) // B2
}

If B's ready.Load() observes the value written by A's ready.Store(true), then:

Goroutine A                           Goroutine B

A1  counter = 1
        │
        │ sequenced-before
        ▼
A2  ready.Store(true)
        │
        │ synchronized-before
        ▼
                                    B1  ready.Load() == true
                                        │
                                        │ sequenced-before
                                        ▼
                                    B2  read counter

Therefore:

A1 happens-before A2
A2 happens-before B1
B1 happens-before B2

        ↓

A1 happens-before B2

So once B observes ready == true through the Atomic variable, the earlier counter = 1 write is visible to B, and this result is not allowed:

ready == true
counter == 0

That is how Go Atomics provide Visibility and Ordering in this example.

1.3 CPython: No Symmetric Application-level Atomic API

Python's standard library currently has no general-purpose integer Atomic API equivalent to Java's AtomicInteger or Go's atomic.Int64.

Application code therefore has no corresponding set of public Atomic rules to cite. An ordinary counter += 1 is not a language-defined Atomic API. The existence of the GIL in a particular CPython configuration, or the use of _Py_atomic_* inside the Runtime, does not turn it into a cross-implementation guarantee of Atomicity, Visibility, or Ordering for application code.

CPython itself needs Atomics for shared Runtime state. How _Py_atomic_* reaches the compiler and CPU is an implementation question for the next article, not part of Python's public application-level Atomic semantics.

2. Next: How Are Atomics Implemented?

This article stops at the language level: Java and Go define Atomicity, Visibility, and Ordering through public Atomic APIs and their memory models, while Python application code has no equivalent general-purpose Atomic API.

The next article turns those language-level rules into implementation-level problems by following Java AtomicInteger, Go sync/atomic, and CPython's internal _Py_atomic_* path from the Runtime down to the CPU.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

• 0. What Does This Article Answer Next?
• 1. How Does HotSpot Implement synchronized?
  • 1.1 Layers
  • 1.2 End-to-End Implementation Path for synchronized
  • 1.3 Guarantees at the Runtime / Language Implementation Layer
    • 1.3.1 Atomicity
    • 1.3.2 Visibility and Ordering
  • 1.4 Guarantees at the Hardware Layer
• 2. How Is sync.Mutex Implemented in Go?
  • 2.1 Layers
  • 2.2 End-to-End Implementation Path for sync.Mutex
  • 2.3 Guarantees at the Runtime / Language Implementation Layer
    • 2.3.1 Atomicity
    • 2.3.2 Visibility and Ordering
  • 2.4 Guarantees at the Hardware Layer
• 3. How Does CPython Implement threading.Lock?
  • 3.1 Layers
  • 3.2 End-to-End Implementation Path for threading.Lock
  • 3.3 Guarantees at the Runtime / Language Implementation Layer
    • 3.3.1 Atomicity
    • 3.3.2 Visibility and Ordering
  • 3.4 Guarantees at the Hardware Layer
• 4. The Common Pattern Across the Three Implementations
  • 4.1 Who Gets In? — Atomically Updating Lock State
  • 4.2 What Happens If You Don't Get the Lock? — Spin / Park / Wakeup
  • 4.3 After Acquiring the Lock, Why Are the Previous Holder's Writes Visible?
• 5. Next: Atomic

0. What Does This Article Answer Next?

The previous article used language-memory-model rules to explain how a mutex provides Atomicity, Visibility, and Ordering.

This article goes one layer deeper and follows how Java, Go, and CPython turn those guarantees into concrete behavior across the Runtime, OS Thread, and CPU layers.

1. How Does HotSpot Implement synchronized?

1.1 Layers

First, establish the implementation stack for Java:

Java Source Code
Example: synchronized
Role: mark a synchronized region
        │
        ▼
JVM Bytecode
Example: monitorenter / monitorexit (synchronized blocks)
Role: encode Monitor entry / exit
        │
        ▼
JVM Implementation (HotSpot)
Example: Synchronization Runtime
Role: implement the Monitor semantics defined by the JVM
        │
        ▼
OS Thread / Thread Parking
Example: Platform Thread / ParkEvent
Role: carry Park / Wakeup when contention persists
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Fence
Role: provide the hardware foundation for Atomicity, Visibility, and Ordering

1.2 End-to-End Implementation Path for synchronized

The sequence diagram above shows only the main cross-layer path. Here, OS Thread represents the path that actually blocks a platform thread. Current HotSpot can first try to unmount a virtual thread, so not every contended synchronized operation blocks an OS thread.

The contention path inside the Runtime can be expanded further:

1.3 Guarantees at the Runtime / Language Implementation Layer

1.3.1 Atomicity

Atomicity corresponds to the two atomic lock-acquisition attempts in the flowchart above:

Fast Path: CAS markWord lock bits [Acquire]
Inflated Monitor: CAS _owner [Acquire]

Both update lock state atomically. When multiple threads contend for the lock, only one can acquire it and enter the critical section.

1.3.2 Visibility and Ordering

Visibility and Ordering correspond to the lock boundaries in the flowchart:

Acquire lock: CAS ... [Acquire]
Release lock: release lock state [Release]

Together, Release and Acquire connect the two critical sections, ensuring that writes from the previous lock holder are observed by the next holder in the required order.

1.4 Guarantees at the Hardware Layer

At the hardware layer, these guarantees are realized through CPU Cache Coherence and memory-ordering constraints.

For Java, this corresponds to the hardware model introduced in the first article:

Atomicity
  → Atomic Instruction
  → HotSpot / x86-64: LOCK CMPXCHG

Visibility
  → Cache Coherence
  → Typical x86-64 CPUs: MESI-family (e.g. MESIF / MOESI)

Ordering
  → Fence
  → HotSpot / x86-64: LOCK ADDL $0, 0(%rsp)

LOCK ADDL $0, 0(%rsp) is a concrete path HotSpot uses to implement a full fence on Linux/x86. This does not mean every synchronized operation executes an extra instance of that instruction: x86 memory-ordering rules and LOCKed RMW operations also contribute to the required ordering guarantees.

For implementation details, see OpenJDK's markWord.hpp, objectMonitor.cpp, objectMonitor.inline.hpp, graphKit.cpp, and orderAccess_linux_x86.hpp.

2. How Is sync.Mutex Implemented in Go?

2.1 Layers

Go Source Code
Example: sync.Mutex / Lock() / Unlock()
Role: define a mutual-exclusion boundary
        │
        ▼
Go Implementation
Example: sync / internal/sync
Role: implement the Mutex state machine and fast / slow paths
        │
        ▼
Go Runtime / Scheduler
Example: runtime_SemacquireMutex / gopark / goready
Role: park / wake goroutines
        │
        ▼
OS Thread (M)
Example: M in the G-M-P model
Role: execute goroutines; gopark does not block M
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Fence
Role: provide the hardware foundation for Atomicity, Visibility, and Ordering

2.2 End-to-End Implementation Path for sync.Mutex

The sequence diagram above shows only the main cross-layer path. The key difference from Java and CPython is that sync.Mutex waits through gopark: it parks the goroutine (G), not the OS thread (M), so that M can continue running another runnable goroutine.

The contention path inside sync.Mutex can be expanded further:

2.3 Guarantees at the Runtime / Language Implementation Layer

2.3.1 Atomicity

Atomicity corresponds to the atomic lock-state modifications in the flow above:

Acquire lock: CompareAndSwapInt32(&state, 0, mutexLocked)
Release lock: AddInt32(&state, -mutexLocked)

When multiple goroutines contend for the mutex, only one can atomically change state from unlocked to locked and enter the critical section.

On amd64, these atomic operations are implemented with LOCK CMPXCHGL and LOCK XADDL.

2.3.2 Visibility and Ordering

Visibility and Ordering come from the synchronization boundary formed by Lock / Unlock. After one holder completes Unlock, a goroutine that subsequently acquires the mutex with Lock can observe writes from the previous critical section in the required order.

2.4 Guarantees at the Hardware Layer

At the hardware layer, these guarantees are realized through CPU Cache Coherence, x86 memory ordering, and the ordering constraints of the atomic instructions themselves.

For Go, this corresponds to the hardware model introduced in the first article:

Atomicity
  → Atomic Instruction
  → Go / x86-64: LOCK CMPXCHGL / LOCK XADDL

Visibility
  → Cache Coherence
  → Typical x86-64 CPUs: MESI-family (e.g. MESIF / MOESI)

Ordering
  → Fence / equivalent ordering constraint
  → Go / x86-64: LOCK CMPXCHGL / LOCK XADDL

This amd64 Mutex path does not need an additional MFENCE; the LOCKed RMW operations already provide both the atomic state update and the ordering constraints required here.

For implementation details, see internal/sync/mutex.go, runtime/sema.go, runtime/HACKING.md, and internal/runtime/atomic/atomic_amd64.s.

3. How Does CPython Implement threading.Lock?

This section focuses on the current CPython implementation.

3.1 Layers

Python Source Code
Example: threading.Lock / acquire() / release()
Role: declare a mutual-exclusion boundary
        │
        ▼
CPython Binding
Example: _thread.lock
Role: map the Python Lock API to CPython's lock implementation
        │
        ▼
CPython Implementation
Example: PyMutex / Parking Lot
Role: implement lock state and waiter queues
        │
        ▼
OS Thread / OS Wait Primitive
Example: WaitForMultipleObjects / sem_wait / pthread_cond_wait
Role: block / wake the thread when the Parking Lot waits
        │
        ▼
x86-64 Hardware
Example: Atomic Instruction / Cache Coherence / Fence
Role: provide the hardware foundation for Atomicity, Visibility, and Ordering

3.2 End-to-End Implementation Path for threading.Lock

The sequence diagram above shows only the main cross-layer path. CPython's internal contention path can be expanded further:

3.3 Guarantees at the Runtime / Language Implementation Layer

3.3.1 Atomicity

Atomicity comes from the atomic state changes that PyMutex performs on _bits:

Acquire lock: _Py_atomic_compare_exchange_uint8(..., _Py_LOCKED)
Release lock (no waiter): _Py_atomic_compare_exchange_uint8(..., _Py_UNLOCKED)

When waiters exist, the release path enters _PyParkingLot_Unpark(), whose callback atomically updates _bits.

When multiple threads contend for the lock, only one can successfully set _Py_LOCKED and enter the critical section.

3.3.2 Visibility and Ordering

Visibility and Ordering come from the synchronization boundary formed by acquire() / release(). In current CPython, the default atomic compare-exchange and store operations use __ATOMIC_SEQ_CST in the GCC / Clang implementation.

3.4 Guarantees at the Hardware Layer

At the hardware layer, these guarantees are realized through CPU Cache Coherence, x86 memory ordering, and those sequentially consistent atomic operations.

For current CPython, this corresponds to the hardware model introduced in the first article:

Atomicity
  → Atomic Instruction
  → CPython / x86-64: LOCK CMPXCHGB

Visibility
  → Cache Coherence
  → Typical x86-64 CPUs: MESI-family (e.g. MESIF / MOESI)

Ordering
  → Fence / equivalent ordering constraint
  → CPython / x86-64: SEQ_CST atomic
    (typically LOCK CMPXCHGB / memory XCHGB)

This likewise does not require every Lock operation to execute an additional MFENCE: on x86-64, LOCKed RMW operations and sequentially consistent atomics already provide the required ordering constraints.

For implementation details, see Modules/_threadmodule.c, Python/lock.c, Python/parking_lot.c, and Include/cpython/pyatomic_gcc.h.

4. The Common Pattern Across the Three Implementations

With Java, Go, and CPython in view, we can strip away the implementation-specific details and focus on the three problems every mutex implementation must solve.

4.1 Who Gets In? — Atomically Updating Lock State

Suppose a lock is represented by a single state variable:

0 = unlocked
1 = locked

A and B must not both read 0 and then both write 1; otherwise, each would believe it had acquired the lock.

Therefore, checking the lock state and changing it to locked must be one atomic operation, such as Compare-And-Swap:

Compare-And-Swap(lock_state, 0, 1)

When two execution units compete:

CPU A                         CPU B

CAS 0 -> 1                   CAS 0 -> 1
    │                             │
    ▼                             ▼
  success                       failed

Only one contender can successfully modify the lock state. A mutex builds on this small hardware atomic primitive to protect an arbitrary critical section such as counter++.

4.2 What Happens If You Don't Get the Lock? — Spin / Park / Wakeup

After CAS fails, an execution unit should not spin on the lock state forever.

A common path is:

try atomic lock acquisition
        │
        ├── success ──> enter critical section
        │
        └── failure
              │
              ├── Spin briefly
              │      │
              │      └── retry
              │
              └── Park / wait
                         │
                         └── wake after lock release

Spinning works well when the expected wait is very short because it avoids immediately entering a blocking path. If contention persists, parking prevents the waiter from continuously consuming CPU time.

4.3 After Acquiring the Lock, Why Are the Previous Holder's Writes Visible?

The first two questions answer who gets the lock and what happens when acquisition fails. The third is about memory visibility: why are writes from the previous holder's critical section visible to the next holder?

Across all three implementations, the common mechanism is a Release / Acquire synchronization boundary between lock release and the next lock acquisition:

previous lock holder
critical-section writes
    │
    ▼
unlock [Release]
    │
    ▼
lock [Acquire]
    │
    ▼
next lock holder
critical-section reads

Release constrains memory operations before the unlock, while Acquire constrains memory operations after the subsequent lock.

Therefore, when the next execution unit acquires the same lock, writes from the earlier critical section are ordered before reads in the later one. This is the memory-ordering guarantee that every mutex implementation must establish.

The first three sections showed how each Runtime and CPU pair realizes this Release / Acquire boundary.

5. Next: Atomic

A mutex only needs a small amount of atomic state to enforce mutual exclusion over a critical section of arbitrary size.

Atomic narrows the scope further: instead of protecting an entire critical section, it makes a single read, write, or update of shared state indivisible.

The next article examines how this smaller unit of synchronization is implemented.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

• 0. Continue from the Previous Article
• 1. What Guarantees Do the Lock Rules in Each Language Provide?
  • 1.1 Java: synchronized and Monitors
  • 1.2 Go: sync.Mutex
  • 1.3 CPython: threading.Lock
• 2. Next: How Is a Mutex Implemented?

0. Continue from the Previous Article

This article examines Mutexes through the rules defined by language memory models.

1. What Guarantees Do the Lock Rules in Each Language Provide?

1.1 Java: synchronized and Monitors

Java's synchronized locks the monitor associated with an object. JLS §17.1 states:

“Only one thread at a time may hold a lock on a monitor.”

This rule provides the exclusion needed for atomicity. At most one thread can hold the monitor, so only that thread may execute the protected critical section while competitors wait.

JLS §17.4.5 also states:

“An unlock on a monitor happens-before every subsequent lock on that monitor.”

This rule provides both visibility and ordering.

For visibility, A's writes are before its unlock, while B's reads are after the subsequent lock. Through happens-before transitivity, the writes in A's critical section become visible to B.

For ordering, A's program order, the unlock → lock relationship, and B's program order form one happens-before chain. B therefore cannot observe a result such as ready = true, counter = 0 when that would violate the established ordering.

Now reuse the same two examples from the previous article.

Atomicity

Previously, two threads executing:

Thread A                    Thread B

count++                     count++

could both read the old value 0 and leave the final value at count = 1.

Now place the same count++ inside a critical section guarded by the same monitor:

synchronized (lock) {
    count++;
}

The execution is serialized:

initial: count = 0

Thread A                         Thread B

acquire lock
                                 try to acquire lock
                                 wait
read count -> 0
compute count + 1 -> 1
write count -> 1
release lock
                                 acquire lock
                                 read count -> 1
                                 compute count + 1 -> 2
                                 write count -> 2
                                 release lock

The final result is:

count = 2

The ordinary count++ statement has not become an atomic CPU instruction. The monitor makes the entire critical section execute with mutual exclusion relative to other code using the same monitor.

Visibility / Ordering

The other example from the previous article was:

Thread A                    Thread B

counter = 1
ready = true                read ready == true
                                  │
                                  ▼
                             read counter

Now access that state through the same monitor:

// Thread A
synchronized (lock) {
    counter = 1;
    ready = true;
}

// Thread B
synchronized (lock) {
    if (ready) {
        System.out.println(counter);
    }
}

If Thread B acquires lock after Thread A releases it:

counter = 1
ready = true
    │
    ▼
unlock
    │
    │ happens-before
    ▼
lock
    │
    ▼
read ready == true
read counter == 1

So once B observes ready == true after that lock acquisition, it also observes the earlier counter = 1; it cannot observe:

ready == true
counter == 0

If B acquires the lock before A, it may of course observe ready == false; that does not violate the rule.

1.2 Go: sync.Mutex

Go's sync.Mutex is not owned by a particular goroutine and is not reentrant. sync.Mutex.Lock states:

“If the lock is already in use, the calling goroutine blocks until the mutex is available.”

This gives the exclusion needed for atomicity. While one goroutine holds the mutex, other goroutines attempting to acquire the same mutex wait rather than entering the critical section.

The Go Memory Model - Locks further states:

“For any sync.Mutex or sync.RWMutex variable l and n < m, call n of l.Unlock() is synchronized before call m of l.Lock() returns.”

This rule provides visibility and ordering.

For visibility, A's writes are sequenced-before Unlock; Unlock is synchronized-before B's Lock returns; B's reads are sequenced after that return. Together these relationships form happens-before, so A's writes are visible to B.

For ordering, the same happens-before chain connects A's writes, Unlock → Lock, and B's reads. B likewise cannot observe ready = true, counter = 0 when the synchronization relationship requires the earlier counter write to be visible.

Use the same two examples again.

Atomicity

Place the previous count++ inside a critical section guarded by the same sync.Mutex:

mu.Lock()
count++
mu.Unlock()

When two goroutines execute it concurrently:

initial: count = 0

Goroutine A                      Goroutine B

Lock()
                                 Lock()
                                 wait
read count -> 0
compute count + 1 -> 1
write count -> 1
Unlock()
                                 Lock() returns
                                 read count -> 1
                                 compute count + 1 -> 2
                                 write count -> 2
                                 Unlock()

The final result is:

count = 2

Again, the atomicity comes from serializing the critical section with the mutex, not from ordinary count++ itself.

Visibility / Ordering

Now reuse the same counter / ready example:

// Goroutine A
mu.Lock()
counter = 1
ready = true
mu.Unlock()

// Goroutine B
mu.Lock()
if ready {
    fmt.Println(counter)
}
mu.Unlock()

If B's Lock() returns after A's Unlock():

counter = 1
ready = true
    │
    ▼
Unlock()
    │
    │ synchronized-before
    ▼
Lock() returns
    │
    ▼
read ready == true
read counter == 1

So if B observes ready == true, it also observes counter == 1; the result ready = true, counter = 0 is not allowed by this synchronization chain.

1.3 CPython: threading.Lock

Python's official threading.Lock documentation states:

“A primitive lock is a synchronization primitive that is not owned by a particular thread when locked.”

“All methods are executed atomically.”

The second rule corresponds to Atomicity. When the lock is already held, other threads calling acquire() wait; only one thread at a time can therefore enter the critical section protected by that lock.

For Visibility and Ordering, Python differs from Java and Go: it does not define a formal language-level release → acquire happens-before rule, so these guarantees cannot be derived from a language memory model in the same way.

Now put the same examples behind threading.Lock.

Atomicity

with lock:
    count += 1

When two threads use the same Lock:

initial: count = 0

Thread A                         Thread B

acquire()
                                 acquire()
                                 wait
read count -> 0
compute count + 1 -> 1
write count -> 1
release()
                                 acquire() returns
                                 read count -> 1
                                 compute count + 1 -> 2
                                 write count -> 2
                                 release()

The final result is:

count = 2

Visibility / Ordering

Protect the shared counter / ready state with the same Lock as well:

# Thread A
with lock:
    counter = 1
    ready = True

# Thread B
with lock:
    if ready:
        print(counter)

In CPython, the same Lock serializes these two critical sections: A writes counter = 1 and ready = True, releases the Lock, and B then acquires that same Lock before reading the shared state.

So in this example, if B reads ready == True, the corresponding counter value is already 1.

The next article explores how CPython implements release → acquire to establish visibility and ordering across critical sections.

2. Next: How Is a Mutex Implemented?

This article stops at the language level: it looks at what guarantees a Mutex provides to programmers from the perspectives of Atomicity, Visibility, and Ordering.

The next article turns those language-level guarantees into implementation-level problems, following Java synchronized, Go sync.Mutex, and CPython threading.Lock from the Runtime down to the CPU.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

• 0. Continue from the Previous Article
• 1. Why Does the Language Need a Memory Model Layer?
• 2. How Should Programmers Understand Memory Model Rules?
  • 2.1 Java Memory Model
  • 2.2 Go Memory Model
  • 2.3 CPython Concurrency Semantics
    • 2.3.1 GIL Mode
    • 2.3.2 Free-threaded Mode
    • 2.3.3 So What Should Python Programs Rely On?
• 3. Next: Mutexes

0. Continue from the Previous Article

The previous article on runtime scheduling explained how execution units get CPU time. Here, we turn to the language rules governing concurrent memory accesses.

1. Why Does the Language Need a Memory Model Layer?

Source code passes through a compiler and Runtime before it finally executes on a CPU. Compilers optimize code, and processors such as x86-64 and ARM64 do not allow exactly the same memory-access orderings.

If programmers had to study every compiler, Runtime, and CPU separately, portable concurrent programming would be difficult to sustain.

We therefore also need a language memory model layer. It defines the rules programmers need to understand, while the underlying Compiler / Runtime implements those rules differently for each platform:

Java                                                Go                                                Python
 │                                                   │                                                   │
 ▼                                                   ▼                                                   ▼
synchronized / volatile / AtomicInteger             sync.Mutex / sync/atomic / channel                 threading.Lock / queue.Queue
 │                                                   │                                                   │
 ▼                                                   ▼                                                   ▼
Java Memory Model                                   Go Memory Model                                     CPython Concurrency Semantics
 │                                                   │                                                   │
 └───────────────────────────────────────────────────┼───────────────────────────────────────────────────┘
                                                     │
                                       Compiler / Runtime implementation
                                       abstracts away hardware differences
                                                     │
                                            ┌────────┴────────┐
                                            │                 │
                                            ▼                 ▼
                                         x86-64             ARM64

2. How Should Programmers Understand Memory Model Rules?

There are three dimensions to look at:

• Atomicity
• Visibility
• Ordering

In theory, each concurrency primitive has corresponding memory-model rules that describe which of these properties it provides.

We will not go deeper here. Instead, we will reuse the same examples from earlier:

count++

and:

counter = 1
ready = true

We will use the language memory model to answer three questions:

2.1 Java Memory Model

Atomicity

Suppose a process contains this variable:

count = 0

Now there are two execution units:

Thread A                    Thread B

count++                     count++

If A and B each execute once, is the final result guaranteed to be:

count = 2

No.

Logically, count++ can be broken down into three steps:

read count
compute count + 1
write count back

So the following interleaving is possible:

initial: count = 0

Thread A                    Thread B

read count -> 0
compute count + 1 -> 1
                            read count -> 0
                            compute count + 1 -> 1
write count -> 1
                            write count -> 1

The final value becomes:

count = 1

Both threads complete one +1, but because they read the same old value 0, both compute 1, and one of the updates is overwritten.

Therefore, ordinary count++ cannot be treated as atomic.

Visibility

Now consider:

int counter = 0;

// Thread A
counter = 1;

// Thread B
System.out.println(counter);

That is:

Thread A                    Thread B

counter = 1                 read counter

In Java, Visibility is commonly expressed in terms of happens-before: whether a write is guaranteed to be observed by another thread depends on whether a happens-before relationship exists between the write and the read.

JLS §17.4.5 gives a very direct description of happens-before:

“If one action happens-before another, then the first is visible to and ordered before the second.”

In other words, if we can establish:

Thread A                    Thread B

counter = 1                 read counter
     │                           ▲
     │                           │
     └──── happens-before ───────┘

then Thread A's write is guaranteed to be visible to Thread B's read.

But in the current code:

Thread A                    Thread B

counter = 1                 read counter
     │                           ▲
     │                           │
     └── no happens-before ──────┘

no such cross-thread happens-before relationship has been established.

Therefore, Thread B is not guaranteed to observe 1.

Ordering

Return to the second example:

// Thread A
counter = 1;
ready = true;

// Thread B
if (ready) {
    System.out.println(counter);
}

That is:

Thread A                    Thread B

counter = 1
ready = true                read ready == true
                                  │
                                  ▼
                             read counter

In Java, Ordering can also be expressed in terms of happens-before. If one action happens-before another, then the first action—and memory writes that precede it in the same thread—are ordered before the second action. Through the transitivity of happens-before, those earlier writes are also visible to reads that follow the second action.

Applied to this example: within Thread A, counter = 1 comes before ready = true; within Thread B, reading ready comes before reading counter. But the two threads are still missing the cross-thread happens-before relationship from ready = true to read ready == true.

So the current code still does not establish a complete happens-before chain between Thread A and Thread B:

Thread A                    Thread B

counter = 1
ready = true                read ready == true
     │                            │
     │                            ▼
     │                       read counter
     │
     └──── no cross-thread happens-before

Therefore, even if B observes:

ready == true

B is still not guaranteed to observe:

counter == 1

2.2 Go Memory Model

Atomicity

In Go, we can approach Atomicity through the Go Memory Model's definition of a data race: it first identifies which concurrent accesses constitute a data race, then excludes atomic accesses provided by sync/atomic from that definition.

The Go Memory Model states:

“Programs that modify data being simultaneously accessed by multiple goroutines must serialize such access.”

It then explains that synchronization mechanisms such as channels, sync, and sync/atomic can serialize these accesses. Its definition of a data race explicitly excludes atomic accesses provided by sync/atomic.

But:

counter++

is only an ordinary increment operation, not an atomic access provided by sync/atomic.

Therefore, if two goroutines concurrently execute:

counter++

they perform concurrent reads and writes on the same counter, which constitutes a data race.

So ordinary counter++ cannot be treated as an atomic concurrent update.

Visibility

Continue with:

var counter int

// Goroutine A
counter = 1

// Goroutine B
fmt.Println(counter)

Split into two columns:

Goroutine A                 Goroutine B

counter = 1                 read counter

In Go, Visibility can also be expressed in terms of happens-before. For an ordinary read r to observe a write w, the Go Memory Model requires w to be visible to r; the first condition is:

“w happens before r.”

Applied to our example, if we can establish:

Goroutine A                 Goroutine B

counter = 1                 read counter
     │                           ▲
     │                           │
     └──── happens-before ───────┘

then the later read can reliably observe that write.

But in the current code:

Goroutine A                 Goroutine B

counter = 1                 read counter
     │                           ▲
     │                           │
     └── no happens-before ──────┘

no such relationship has been established, and the program contains a data race.

Therefore, the second goroutine cannot rely on always observing:

counter == 1

Ordering

Now consider:

// Goroutine A
counter = 1
ready = true

// Goroutine B
if ready {
    fmt.Println(counter)
}

Split into two columns:

Goroutine A                 Goroutine B

counter = 1
ready = true                read ready == true
                                  │
                                  ▼
                             read counter

In Go, Ordering can likewise be expressed in terms of happens-before. This is essentially the same pattern used by the Go Memory Model under Incorrect synchronization: one goroutine writes data and then a flag; another goroutine observes the flag and then reads the earlier data.

The Go Memory Model states:

“Even if this occurs, it does not imply that reads happening after r will observe writes that happened before w.”

Applied to our example:

Goroutine A                 Goroutine B

counter = 1
ready = true                read ready == true
                                  │
                                  ▼
                             read counter

        no happens-before guarantee

So:

observing ready == true

does not imply:

counter must be observed as 1

This is very similar to the Java example.

Both Java and Go use happens-before to describe important Visibility and Ordering guarantees.

2.3 CPython Concurrency Semantics

Python differs from Java and Go in one important respect:

Python does not define one unified concurrency memory model for all interpreter implementations at the same level as the JMM or Go Memory Model.

So this article focuses on the most widely used implementation: CPython.

We will continue to use the same two examples:

counter += 1

and:

counter = 1
ready = True

and examine Atomicity, Visibility, and Ordering.

2.3.1 GIL Mode

Before returning to counter, first clarify one question:

What does the GIL actually protect?

The GIL is first and foremost a mechanism used by the CPython Runtime to protect Python objects and interpreter-internal state.

The official Python C API documentation states the requirement directly:

“only a thread that holds the GIL may operate on Python objects or invoke Python’s C API.”

In other words, in default GIL-enabled CPython, a thread must hold the GIL before it can operate on Python objects or call the Python C API.

The documentation then uses Reference Count to explain why such a lock is needed: if two threads increment the same object's reference count at the same time, the result may be incremented only once instead of twice.

Suppose a Python object currently has:

refcount = 10

If two threads could modify that reference count concurrently:

Thread A                    Thread B

read refcount = 10          read refcount = 10

refcount + 1                refcount + 1

write refcount = 11         write refcount = 11

Both threads increment the reference count once, yet the final result may be:

expected: refcount = 12
actual:   refcount = 11

That would break CPython's management of Python object lifetimes.

So the GIL primarily protects:

             GIL
              │
              ▼
      CPython Runtime
              │
      ┌───────┴────────┐
      ▼                ▼
Python Objects     Runtime State
      │
      ├── Reference Count
      └── Object Internals

There are two different layers here:

• CPython Runtime layer: how to safely access and maintain Python objects under concurrency
• Application layer: how to safely access and modify shared state under concurrency

The GIL primarily addresses the former. It does not make application code automatically thread-safe.

Atomicity

Now return to:

counter += 1

The official Python FAQ directly addresses this kind of operation.

When discussing which operations are atomic in GIL-enabled CPython, it explicitly lists:

i = i + 1

as a non-atomic operation:

“These aren’t: i = i+1”

Therefore, even with the GIL, we cannot treat:

counter += 1

as a single atomic concurrent update.

We need to distinguish:

GIL
 │
 └── protection for CPython Runtime / Python Objects

counter += 1
 │
 └── a compound state update defined by the application

Visibility

Continue with the same example:

counter = 0

# Thread A
counter = 1

# Thread B
print(counter)

Split into two columns:

Thread A                    Thread B

counter = 1                 read counter

Here the distinction between CPython and Java / Go appears.

With Java and Go, we can continue to ask:

write
 │
 │ happens-before ?
 ▼
read

because those languages define formal Memory Models.

Python, however, does not define a corresponding happens-before model that applies uniformly to all Python implementations.

So we cannot draw the CPython code above as:

Thread A                    Thread B

counter = 1                 read counter
     │                           ▲
     │                           │
     └──── Python happens-before ┘

and then derive a conclusion from some Python Language Memory Model rule, because there is no corresponding unified model of that kind.

Conversely, because no separate rule specifies such a guarantee, we should regard the outcome as uncertain. That uncertainty means it does not satisfy the Visibility requirement.

Ordering

Finally, consider:

# Thread A
counter = 1
ready = True

# Thread B
if ready:
    print(counter)

Again, split into two columns:

Thread A                    Thread B

counter = 1
ready = True                read ready == True
                                  │
                                  ▼
                             read counter

Likewise, there is no corresponding rule that specifies such a guarantee here, so we should regard the outcome as uncertain. That uncertainty means it does not satisfy the Ordering requirement.

2.3.2 Free-threaded Mode

Starting with Python 3.13, CPython provides free-threaded builds in which the GIL can be disabled.

In GIL-enabled CPython:

Thread A ──┐
           │
           ├── GIL ───> Python Code
           │
Thread B ──┘

In free-threaded CPython:

Thread A ─────────────> CPU Core 0

Thread B ─────────────> CPU Core 1

Multiple threads can execute Python code in parallel.

But removing the GIL does not mean CPython no longer needs to protect its objects and Runtime state.

Python's official free-threading documentation explicitly states that built-in types such as dict, list, and set use internal locks to protect against concurrent modification:

“Built-in types like dict, list, and set use internal locks”

These internal locks are an implementation mechanism of free-threaded CPython. They should not be interpreted as Python the language defining a unified Memory Model.

So the change is from:

GIL-enabled

one global GIL
      │
      ▼
protect CPython Runtime

to:

Free-threaded

finer-grained internal synchronization
      │
      ▼
protect CPython Runtime

For application code, the same three questions still exist.

Different paths, same destination. Removing the GIL does not mean CPython no longer needs synchronization; it moves from one global lock toward finer-grained locks, atomic operations, and Runtime coordination. In this respect, its implementation approach increasingly resembles Java and Go.

2.3.3 So What Should Python Programs Rely On?

In summary, Java, Go, and CPython differ at this layer in an important way:

Java
  │
  ▼
Java Memory Model
  │
  └── happens-before

Go
  │
  ▼
Go Memory Model
  │
  └── happens-before

Python
  │
  ▼
Concurrency semantics of different Interpreters
  │
  └── CPython
        ├── GIL-enabled
        └── Free-threaded

The summary is simple: Python does not have one unified Memory Model, at the same level as the JMM or Go Memory Model, that applies to every interpreter implementation. Therefore, when discussing concurrency safety in Python, first identify which Interpreter is being used, and rely on explicit synchronization tools and the concurrency guarantees they provide.

3. Next: Mutexes

Earlier, we discussed the hardware layer; this article explains the rules at the language memory model layer.

The next article begins with the first concrete synchronization tool, Mutex, and follows it from the language memory model layer all the way down to the hardware layer.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

• 0. From Language-Level Execution Units to the CPU
• 1. How Does Java Schedule Threads?
  • 1.1 From Platform Threads to Virtual Threads
    • 1.1.1 The Simplest Model: One Java Thread per OS Thread
    • 1.1.2 The Problem: Resource and Scheduling Costs of Many OS Threads
    • 1.1.3 The Solution: Let the JVM Schedule Lightweight Threads
    • 1.1.4 The Result: Platform Threads and Virtual Threads
  • 1.2 Execution Layers
  • 1.3 From Java Thread Lifecycles to Their Implementation
    • 1.3.1 Creation and Start: Created → Runnable
    • 1.3.2 Waiting to Be Scheduled: Runnable
    • 1.3.3 Running: Runnable → Running
    • 1.3.4 Parking and Waiting: Running → Waiting
    • 1.3.5 Wakeup and Resumption: Waiting → Runnable → Running
    • 1.3.6 Completion and Cleanup: Running → Terminated
  • 1.4 End-to-End Sequence Diagrams
  • 1.5 Platform Threads vs. Virtual Threads
  • 1.6 Returning to the Opening Question
• 2. How Does Go Schedule Goroutines?
  • 2.1 From G-M to G-M-P
    • 2.1.1 The Simplest Model: A Goroutine Runs on an OS Thread
    • 2.1.2 The Problem: Many Goroutines, Fewer OS Threads
    • 2.1.3 The Solution: Evolving from G-M to G-M-P
    • 2.1.4 The Final Model: G, M, and P
  • 2.2 Execution Layers
  • 2.3 From a Goroutine's Lifecycle to Its Runtime Implementation
    • 2.3.1 Creation: Created → Runnable
    • 2.3.2 Waiting for Scheduling: Runnable
    • 2.3.3 Execution: Runnable → Running
    • 2.3.4 Parking and Waiting: Running → Waiting
    • 2.3.5 Wakeup and Resumption: Waiting → Runnable → Running
    • 2.3.6 System Calls and Preemption
    • 2.3.7 Completion and Cleanup: Running → Dead
  • 2.4 End-to-End Sequence Diagram
  • 2.5 Returning to the Opening Question
• 3. How Does CPython Schedule Threads?
  • 3.1 From Native OS Threads to the GIL and Free-Threaded Builds
    • 3.1.1 The Simplest Model: One Python Thread per OS Thread
    • 3.1.2 The Problem: Sharing Interpreter State Across OS Threads
    • 3.1.3 The Default Solution: The GIL Restricts Interpreter Execution
    • 3.1.4 The Next Step: Free-Threaded CPython
    • 3.1.5 The Final Model: Thread, PyThreadState, and GIL
  • 3.2 Execution Layers
  • 3.3 From CPython Thread Lifecycles to Their Implementation
    • 3.3.1 Creation and Start: Created → Runnable
    • 3.3.2 Waiting for Scheduling: Runnable
    • 3.3.3 Execution: Runnable → Running
    • 3.3.4 Parking and Waiting: Running → Waiting
    • 3.3.5 Wakeup and Resumption: Waiting → Runnable → Running
    • 3.3.6 Completion and Cleanup: Running → Terminated
  • 3.4 End-to-End Sequence Diagram
  • 3.5 Returning to the Opening Question
• 4. What the Three Implementations Have in Common
  • 4.1 How Does an Execution Unit Reach the CPU? — Establish Its Carrier
  • 4.2 Who Selects What Runs Next? — Separate the Scheduling Layers
  • 4.3 What Happens After a Wait? — Regain an Execution Opportunity
• 5. Next: Language Memory Models

0. From Language-Level Execution Units to the CPU

The previous article examined how the CPU executes count++ and how atomic instructions, cache coherence, and fences provide the hardware foundations for atomicity, visibility, and ordering.

This article moves one layer up, to the operating system. It asks one central question:

How do Java threads, Go goroutines, and CPython threads pass through their runtimes and the operating system to execute on a CPU?

Answering it requires understanding three things:

1. Execution mapping: How does a language-level execution unit map to an OS thread?
2. Scheduling responsibilities: What do the runtime scheduler and the operating system scheduler each control?
3. Waiting and resumption: When an execution unit waits, wakes, or resumes, what is suspended, and which threads can continue running?

Next, we will examine Java, Go, and CPython in turn. For each, we will trace the evolution of its threading model, then use execution layers, lifecycles, and sequence diagrams to answer these three questions. Finally, we will draw out the language-independent mechanisms they share and return to the opening question.

1. How Does Java Schedule Threads?

1.1 From Platform Threads to Virtual Threads

1.1.1 The Simplest Model: One Java Thread per OS Thread

When a program has only a few concurrent tasks, the most straightforward approach is to map each Java thread to an OS thread. Java creates the thread, and the operating system schedules the corresponding OS thread.

With relatively few threads, this one-to-one mapping is easy to understand and debug.

1.1.2 The Problem: Resource and Scheduling Costs of Many OS Threads

Suppose a service handles many requests, each on its own thread:

Request 1 → Thread 1 → OS Thread 1
Request 2 → Thread 2 → OS Thread 2
Request 3 → Thread 3 → OS Thread 3
...
Request N → Thread N → OS Thread N

Many request-handling tasks spend most of their time waiting:

Read from a database
Wait for a network response
Wait for a message
Wait for other I/O

If Java threads continue to map one-to-one to OS threads, two costs become significant:

1. Native-thread resources: Even while a task is waiting, the operating system must retain resources such as its kernel thread and native stack. These costs grow with the thread count.
2. Scheduling overhead: When many threads wake frequently and become Runnable, the operating system must schedule among more runnable threads. Context switches and scheduler work consume additional CPU time.

1.1.3 The Solution: Let the JVM Schedule Lightweight Threads

How can we address these costs?

In this setting, logical tasks spend most of their time waiting for I/O and relatively little time using the CPU. They do not need to occupy a dedicated OS thread for their entire lifetime. The key is to decouple logical execution units from OS threads.

Once they are decoupled, the JVM can introduce its own scheduler:

Logical Thread A ─┐
Logical Thread B ─┼─→ JVM Scheduler ─→ Carrier Thread 1 ⇔ OS Thread 1
Logical Thread C ─┤                  └→ Carrier Thread 2 ⇔ OS Thread 2
Logical Thread D ─┘

A Carrier Thread is a Platform Thread in the carrier role. The ⇔ symbol indicates its one-to-one native mapping to an OS thread, not a method call.

The JVM can manage many logical threads while the operating system schedules a smaller number of underlying OS threads.

Java calls this lightweight logical thread a Virtual Thread.

1.1.4 The Result: Platform Threads and Virtual Threads

A Platform Thread maps one-to-one to an OS Thread; the JVM Scheduler dispatches Virtual Threads onto Platform Threads acting as carriers, and the OS Scheduler schedules the underlying OS Threads.

1.2 Execution Layers

1.3 From Java Thread Lifecycles to Their Implementation

A Java thread's conceptual lifecycle is:

This is a conceptual lifecycle, not an exact mapping to Java's Thread.State. In particular, RUNNABLE covers both threads awaiting CPU time and threads currently running.

1.3.1 Creation and Start: Created → Runnable

A platform thread creates an OS thread at start(); a virtual thread submits its continuation to the JVM scheduler.

(1) Platform Thread

(2) Virtual Thread

1.3.2 Waiting to Be Scheduled: Runnable

A platform thread waits for OS scheduling; a virtual thread waits for JVM dispatch to a carrier thread.

(1) Platform Thread

(2) Virtual Thread

1.3.3 Running: Runnable → Running

A platform thread executes Thread.run() on its OS thread; a virtual thread mounts and runs its continuation on a carrier.

(1) Platform Thread

(2) Virtual Thread

1.3.4 Parking and Waiting: Running → Waiting

A waiting platform thread normally retains its OS thread; when unmounting is possible, a virtual thread releases its carrier.

(1) Platform Thread

(2) Virtual Thread

1.3.5 Wakeup and Resumption: Waiting → Runnable → Running

A platform thread resumes on the same OS thread; a virtual thread reenters the JVM scheduler and may resume on a different carrier.

(1) Platform Thread

(2) Virtual Thread

1.3.6 Completion and Cleanup: Running → Terminated

A platform thread cleans up its JavaThread and OS thread; a virtual thread terminates without terminating its carrier.

(1) Platform Thread

(2) Virtual Thread

1.4 End-to-End Sequence Diagrams

The following diagrams trace the native startup path for a platform thread and the essential mount, unmount, and resume operations for a virtual thread. The virtual-thread sequence assumes its carrier already has CPU time, so it does not repeat OS scheduling.

1.4.1 Platform Thread

1.4.2 Virtual Thread

1.5 Platform Threads vs. Virtual Threads

Their lifecycle differences can be summarized as follows:

The central distinction is the execution relationship: platform threads remain associated with their OS threads, while the JVM dispatches virtual threads to carriers. When a virtual thread can unmount, its carrier becomes available for other tasks.

For implementation details, see OpenJDK's Thread.java, jvm.cpp, javaThread.cpp, and VirtualThread.java. For the virtual-thread design, see JEP 444.

1.6 Returning to the Opening Question

As Section 1.2, Execution Layers shows, a Platform Thread maps one-to-one to an OS Thread, while a Virtual Thread reaches an OS Thread through a Carrier Thread. Therefore, both thread models ultimately execute on OS Threads.

Section 1.3.2, Waiting to Be Scheduled and Section 1.3.3, Running show that the JVM Scheduler dispatches Virtual Threads to Carrier Threads, while the OS Scheduler schedules the underlying OS Threads. Therefore, the scheduling paths differ, but the OS Scheduler ultimately grants CPU execution time to both.

Section 1.3.4, Parking and Waiting and Section 1.3.5, Wakeup and Resumption show that a Virtual Thread can release its Carrier Thread when waiting permits unmounting, then reenter JVM scheduling when ready. Therefore, many waiting tasks need not each occupy an OS Thread for their entire lifetime.

Answering the opening question: Java retains the native-thread model of Platform Threads while allowing Virtual Threads to reuse Carrier Threads, so many concurrent tasks can ultimately execute on the CPU using a relatively small number of OS Threads.

2. How Does Go Schedule Goroutines?

2.1 From G-M to G-M-P

2.1.1 The Simplest Model: A Goroutine Runs on an OS Thread

For a small number of concurrent tasks, the simplest approach is:

Goroutine
    ↓
OS Thread
    ↓
Operating System Scheduler
    ↓
CPU Core

One Goroutine maps to one OS Thread; the Go runtime manages the Goroutine, while the operating system schedules its OS Thread.

At modest thread counts, this one-to-one model is easy to understand and debug.

2.1.2 The Problem: Many Goroutines, Fewer OS Threads

Suppose a service handles many requests, each on its own Goroutine:

Request 1 → Goroutine 1 → OS Thread 1
Request 2 → Goroutine 2 → OS Thread 2
Request 3 → Goroutine 3 → OS Thread 3
...
Request N → Goroutine N → OS Thread N

Many service tasks spend most of their time waiting for databases, network responses, or other I/O.

If Goroutines remained mapped one-to-one to OS Threads, they would face the same two costs as Java Platform Threads:

1. Native-thread resources: a waiting task would still retain a kernel thread, native stack, and other resources.
2. Scheduling overhead: many OS Threads repeatedly becoming runnable would increase context switching and OS scheduling work.

2.1.3 The Solution: Evolving from G-M to G-M-P

How can the runtime address this?

These logical tasks spend much of their time waiting for I/O rather than using the CPU. They therefore do not need to retain a dedicated OS Thread for their entire lifetime.

The solution is to decouple the logical execution unit from the OS Thread: the Go runtime schedules Goroutines so a limited number of OS Threads can be reused.

2.1.3.1 Step One: G-M with a Global Run Queue

Start with G, M, and a global run queue:

Here, G represents a goroutine and M represents an OS thread.

Runnable Gs primarily enter a shared global run queue, from which multiple Ms fetch work.

As concurrency grows, two problems emerge:

Multiple M instances
  ↓
Access the same Global Run Queue
  ↓
More synchronization contention

And:

Scheduling state for G is centralized
        ↓
Poorer locality

To reduce contention on the global queue, some scheduling state and runnable work can be distributed across execution contexts:

What if each execution context had its own scheduling resources and local run queue?

2.1.3.2 Step Two: Introduce P and Local Run Queues

To reduce contention for the global queue, the Go runtime introduces P, with each P maintaining its own local run queue:

P holds the runtime resources needed to execute Go user code and maintains its own local run queue. The actual G-M-P scheduler also retains a global queue; goroutines are not permanently assigned to a particular P. We will examine other work sources next.

This gives us the defining constraint of G-M-P:

An M needs a P to execute Go user code.

Scheduling state is no longer concentrated in one queue contested by all M instances; each P maintains its own local run queue.

2.1.3.3 Third Question: What If a P's Local Queue Is Empty?

Local queues reduce contention but can leave work unevenly distributed: one P is busy while another has nothing queued. An M holding an idle P can use work stealing to obtain runnable goroutines from another P's queue.

The scheduler can also find work from other sources:

In the actual implementation, findRunnable() considers fairness, timers, the network poller, and other conditions when searching for runnable Gs. It does not rigidly follow the same sequence every time.

2.1.3.4 Fourth Question: What If M Blocks in a System Call?

Another constraint comes from blocking system calls:

If an M holding a P blocks in the OS, must the P become idle too?

If P remains permanently attached to that M, other goroutines cannot run using its scheduling resources while the M is blocked.

P and M therefore need to be separable:

G0 remains blocked in the system call on M0's OS thread. Meanwhile the runtime can reclaim P0 so M1 can run another runnable G. When M0 returns from the system call, it must try to acquire a P again before resuming Go user code. P and M must therefore be free to separate and recombine.

2.1.3.5 Fifth Question: What If One G Runs for Too Long?

Even without I/O, a goroutine that runs continuously for a long time can reduce the execution opportunities available to other runnable goroutines.

The runtime therefore also needs preemption:

G Running
    ↓
Preempt
    ↓
G Runnable
    ↓
Scheduler
    ↓
Another G gets CPU time

Modern Go supports preemption, including asynchronous preemption, to reduce the chance that a long-running goroutine blocks progress elsewhere. Preemption does not, by itself, guarantee strict fairness.

2.1.4 The Final Model: G, M, and P

2.1.4.1 G: Goroutine

G represents a goroutine: a logical execution unit managed by the Go runtime.

It holds:

• Stack;
• PC / scheduler context;
• Scheduling state such as Runnable, Running, and Waiting.

go f() creates a G, not a new OS thread.

2.1.4.2 M: Machine

M represents a real OS thread.

M
↓
OS Thread
↓
OS Scheduler
↓
CPU

The OS scheduler schedules the OS thread represented by M onto a CPU.

Each M also has a special g0 goroutine. It uses M's system stack, on which the runtime performs scheduling, stack management, and other low-level work.

2.1.4.3 P: Processor

P represents the runtime resources and permission an M needs to execute Go user code.

It contains scheduler and allocator state, a local run queue, and runnext for preferential scheduling.

The number of P instances is determined by GOMAXPROCS:

GOMAXPROCS
    ↓
Number of P instances
    ↓
Maximum parallelism for Go user code

Their roles can be summarized in one sentence:

G is the task to run; M is the OS thread; P provides the runtime resources an M needs to execute Go code.

2.2 Execution Layers

The Go runtime scheduler and OS scheduler operate at different layers. Switching goroutines on the same M does not require an OS thread context switch.

2.3 From a Goroutine's Lifecycle to Its Runtime Implementation

A goroutine's conceptual lifecycle is:

The diagram uses conceptual states such as Created and Waiting. The Go runtime uses more specific internal states, including _Grunnable and _Gwaiting.

2.3.1 Creation: Created → Runnable

go f() creates a G, marks it runnable, and enqueues it for scheduling.

2.3.2 Waiting for Scheduling: Runnable

The G is in a run queue, but an M holding a P has not yet selected it.

2.3.3 Execution: Runnable → Running

An M holding a P selects a runnable G through the runtime scheduler, then runs it through execute(G) and gogo. The OS scheduler independently schedules M's underlying OS thread.

2.3.4 Parking and Waiting: Running → Waiting

gopark() suspends only G, letting its M continue running other runnable goroutines while holding P.

2.3.5 Wakeup and Resumption: Waiting → Runnable → Running

When the wait ends, goready() requeues G so the scheduler can dispatch it to an available M.

2.3.6 System Calls and Preemption

A system call can block M while P becomes available to another M; preemption makes a long-running G eligible for rescheduling.

2.3.7 Completion and Cleanup: Running → Dead

When G's function returns, the runtime cleans up G, marks it dead, and M schedules other work.

2.4 End-to-End Sequence Diagram

This sequence follows one G through creation, execution, waiting, and termination. OS scheduling and goroutine switching are separate layers: the diagram shows M being scheduled onto a CPU once, but later goroutine switches on that same M need not trigger another OS scheduling event.

For implementation details, see Go's proc.go and runtime2.go.

2.5 Returning to the Opening Question

Section 2.1.4, The Final Model and Section 2.2, Execution Layers establish that G is a logical execution unit, M corresponds to an OS thread, and M needs P to execute Go code. Therefore, a G executes through an M holding a P, not as a dedicated OS thread.

Section 2.1.3.3, Work Sources, Section 2.3.2, Waiting for Scheduling, and Section 2.3.3, Execution show the Go scheduler selecting runnable Gs from queues, work stealing, or netpoll for an M, while the OS scheduler schedules the underlying OS thread. Therefore, the two schedulers decide which G runs and which OS thread receives CPU time, respectively.

Section 2.3.4, Parking and Section 2.3.5, Resumption show that M can run other Gs while one G waits. Section 2.3.6, System Calls and Preemption shows that another M may take P if a system call blocks the original M, and a long-running G may be preempted. Therefore, waiting or blocking need not leave the associated scheduling resources idle indefinitely.

Answering the opening question: Go uses G-M-P to schedule many goroutines on M instances holding P, while the OS scheduler runs their underlying OS threads on CPUs, allowing concurrent tasks to make progress on a limited number of CPU cores.

3. How Does CPython Schedule Threads?

3.1 From Native OS Threads to the GIL and Free-Threaded Builds

3.1.1 The Simplest Model: One Python Thread per OS Thread

For a small number of concurrent tasks, the execution relationship is straightforward:

Python Thread
      ↓
OS Thread
      ↓
Operating System Scheduler
      ↓
CPU Core

The OS scheduler decides which OS thread receives CPU time.

3.1.2 The Problem: Sharing Interpreter State Across OS Threads

For example, the OS might schedule two Python threads simultaneously on different CPU cores:

Python Thread A → OS Thread A → CPU Core A
Python Thread B → OS Thread B → CPU Core B

The operating system can schedule both threads in parallel. Inside CPython, however, the threads share interpreter state, including object internals, reference counts, and runtime data structures.

This raises a separate runtime question:

How can multiple OS threads safely access shared state while executing Python interpreter code?

The default GIL-enabled CPython build uses the GIL to protect shared interpreter state. Whether a thread can execute protected Python code therefore depends on two independent conditions: OS scheduling and the GIL.

OS Scheduler
Decides which OS Thread gets CPU time

GIL
Restricts which Thread can execute protected Python interpreter code

3.1.3 The Default Solution: The GIL Restricts Interpreter Execution

In a default GIL-enabled build, a thread must hold the GIL while executing interpreter code protected by it:

OS Thread
    ↓
OS Scheduler
    ↓
CPU Time

Python Thread
    ↓
Acquire GIL
    ↓
Python Interpreter Execution

Executing Python bytecode therefore requires two things: the OS thread must have CPU time, and the thread must hold the GIL.

If another thread already holds the GIL:

Thread A
  ↓
Holds the GIL
  ↓
Executes Python bytecode

Thread B
  ↓
Attempts to acquire the GIL
  ↓
Waits
  ↓
The GIL is released
  ↓
Competes to acquire it again

Their responsibilities remain distinct:

• The OS scheduler governs OS Thread → CPU.
• The GIL gates Thread → Python Interpreter Execution in a default CPython build.

3.1.4 The Next Step: Free-Threaded CPython

To allow multiple Python threads to execute Python code in parallel, the normal execution path must no longer depend on that global lock:

Python Thread A → OS Thread A → CPU Core A
Python Thread B → OS Thread B → CPU Core B

A free-threaded build does not rely on the GIL as a global execution lock on its normal path. Instead, finer-grained synchronization protects CPython's internal state, allowing multiple Python threads to execute code in parallel. What changes is interpreter synchronization; the OS scheduler still schedules the same kind of native threads.

CPython thus offers two principal execution models:

Some incompatible extensions may still cause a free-threaded runtime to enable the GIL.

3.1.5 The Final Model: Thread, PyThreadState, and GIL

3.1.5.1 Python Thread

Python programs create and manage threads through threading.Thread.

In CPython, _thread ultimately creates a real OS thread.

3.1.5.2 OS Thread

The OS scheduler determines when an OS thread runs on a CPU.

Unlike Go's G-M-P or Java virtual threads, CPython does not multiplex large numbers of threading.Thread objects onto a smaller pool of OS threads.

3.1.5.3 PyThreadState

PyThreadState stores the interpreter-side runtime state associated with a Python thread.

A thread's execution context in CPython consists of two parts:

OS Thread
    +
PyThreadState
    ↓
Execution context for this Thread inside the CPython Runtime

3.1.5.4 GIL

In the default GIL-enabled build, a thread must hold the GIL to execute protected Python interpreter code.

Even when its OS thread has CPU time, it must still satisfy the GIL requirement before executing the corresponding Python bytecode.

3.2 Execution Layers

3.3 From CPython Thread Lifecycles to Their Implementation

CPython relies on OS scheduling for threading.Thread. The default GIL-enabled build also requires GIL ownership to execute protected interpreter code; free-threaded builds use different synchronization mechanisms.

3.3.1 Creation and Start: Created → Runnable

Thread.start() creates an OS thread and its PyThreadState; CPU execution still depends on OS scheduling.

3.3.2 Waiting for Scheduling: Runnable

The OS thread underlying a Python thread waits for OS scheduling; the default build must also satisfy its GIL requirement.

3.3.3 Execution: Runnable → Running

The OS scheduler grants CPU time to the native thread; in the default GIL-enabled build, the thread acquires the GIL before executing Python code.

3.3.4 Parking and Waiting: Running → Waiting

Waiting for I/O, locks, or conditions may block the underlying OS thread. In the default build, operations that release the GIL also allow another thread to execute protected interpreter code.

3.3.5 Wakeup and Resumption: Waiting → Runnable → Running

When ready, the OS thread reenters OS scheduling; the default build must reacquire the GIL before resuming protected Python execution if it was released.

3.3.6 Completion and Cleanup: Running → Terminated

After Thread.run() returns, the runtime cleans up PyThreadState, the native thread exits, and its ThreadHandle reaches the completed state.

3.4 End-to-End Sequence Diagram

Here is the thread lifecycle in default GIL-enabled CPython:

For implementation details, see CPython's _threadmodule.c, pystate.c, and ceval_gil.c; for free-threading, see the official guide.

3.5 Returning to the Opening Question

Section 3.1.5, Final Model and Section 3.2, Execution Layers show that threading.Thread maps to a native OS thread, while PyThreadState records its interpreter state. Therefore, the OS scheduler directly grants CPU execution opportunities; CPython has no separate user-space thread scheduler like Go's G-M-P.

Section 3.1.3, GIL, Section 3.3.2, Waiting for Scheduling, and Section 3.3.3, Execution show that a thread in the default GIL-enabled build must acquire the GIL before executing protected Python interpreter code. Therefore, CPU time alone does not guarantee Python-code execution; the GIL is an execution constraint, not another thread scheduler.

Section 3.3.4, Parking and Waiting, Section 3.3.5, Resumption, and Section 3.1.4, Free-Threaded establish that a blocking operation can suspend an OS thread and release the GIL, whereas free-threaded builds change Python-code synchronization without changing the threading.Thread to OS-thread mapping. Therefore, the interpreter's concurrency constraints change, but the execution carrier does not.

Answering the opening question: CPython relies on the OS scheduler for native threads. The default build uses the GIL to limit parallel execution of Python code within one interpreter, whereas free-threaded builds permit multiple threads to execute Python code in parallel; neither model multiplexes large numbers of threading.Thread instances onto a few OS threads.

4. What the Three Implementations Have in Common

After examining Java, Go, and CPython, we can set aside their specific APIs and runtime names. Three questions remain: what carries an execution unit, who decides when it runs, and how does it resume after waiting?

4.1 How Does an Execution Unit Reach the CPU? — Establish Its Carrier

First, distinguish the execution unit defined by a language or runtime from the OS thread that the operating system schedules. They do not necessarily have a one-to-one relationship.

Suppose a program has three execution units, A, B, and C. The most direct approach gives each its own OS thread:

Execution Unit A → OS Thread 1
Execution Unit B → OS Thread 2
Execution Unit C → OS Thread 3

Tasks may, however, spend most of their time waiting for I/O. If every task permanently occupies an OS thread, native-thread resources must remain allocated even for waiting tasks.

Another approach separates tasks from underlying threads:

Runnable Task A ─┐
Runnable Task B ─┼─→ Runtime Dispatch → OS Thread 1
Runnable Task C ─┘                 (reused over time)

One OS thread can execute different tasks at different times. Models that do not require user-space scheduling can keep the direct one-to-one mapping. Both ultimately reach the same execution path:

Therefore, execution units can have different carriers, but the OS scheduler ultimately dispatches OS threads—not language-level tasks—to the CPU.

4.2 Who Selects What Runs Next? — Separate the Scheduling Layers

Establishing a carrier does not mean that a task immediately runs. Two different decisions remain:

• Task selection: If a runtime manages lightweight runnable tasks, it chooses one to assign to an underlying thread.
• Thread scheduling: The OS scheduler selects a runnable OS thread to receive CPU time.

Consider a single CPU core. OS threads T1 and T2 are both runnable, and a runtime has already assigned task A to T1:

Runtime: select Task A → assign to T1

OS Scheduler: T1 and T2 are both Runnable
                         ↓
                    select T2 first
                         ↓
                   CPU runs T2

Although the runtime has selected task A, it cannot make progress until T1 receives CPU time. Conversely, an OS thread receiving CPU time does not guarantee that a language-level synchronization condition is satisfied; such a condition is not another thread scheduler.

With direct one-to-one mapping, a separate runtime thread scheduler may be unnecessary, but OS scheduling still applies. Long-running work may also have to yield or be preempted so other tasks can make progress.

Therefore, runtime task dispatch and OS thread scheduling make choices at different levels; Runnable is not the same as Running.

4.3 What Happens After a Wait? — Regain an Execution Opportunity

Suppose task A is executing and then starts an I/O operation, while task B is already runnable. The central question is: can A's underlying OS thread run B while A waits?

Task A running → waits for I/O
                     │
                     ├─ Suspend only logical Task A
                     │       └─ OS thread can run Task B
                     │
                     └─ Block the underlying OS thread
                             └─ That thread waits; other OS threads can run

Either path is possible, depending on the execution model and whether the specific wait releases its carrier. Suspending a logical task does not necessarily block its OS thread; blocking one OS thread does not stop the entire process.

When the I/O completes, task A is not guaranteed to run immediately. It becomes eligible for scheduling again:

Waiting
   ↓ condition ready / wakeup
Runnable
   ↓ optionally queue in runtime and wait for a carrier
   ↓ OS thread receives CPU time
Running

If only the logical task was suspended, the OS thread could make progress on other tasks. If the OS thread itself was blocked, it must become runnable and receive CPU time again. Waiting and resumption determine whether execution resources can be reused; wakeup does not mean immediate execution.

Returning to the opening question: a language-level execution unit must first be associated with an OS thread. When present, a runtime scheduler selects the task; the OS scheduler assigns CPU time to its underlying thread. Waiting and resumption determine whether that thread can serve other tasks in the meantime. Together, these mechanisms explain how concurrent execution units reach the CPU.

5. Next: Language Memory Models

The previous chapter explained how the CPU executes instructions. This chapter addresses the missing link: how runtimes and operating systems give concurrent execution units CPU time and resume them after waiting.

With the execution path established, the next chapter turns to the Language Memory Model—the rules governing concurrent reads and writes, which provide the foundation for later discussions of Mutex, Atomic, Volatile, and other synchronization mechanisms.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

• 1. Start with the von Neumann Architecture
• 2. What Does count++ Become?
• 3. Why Do We Need Caches?
• 4. Concurrency on One Core: Why Can count++ Lose an Update?
• 5. From One Core to Multiple Cores: Visibility
• 6. What About the Order of Multiple Memory Operations?
  • 6.1 Store Buffer
  • 6.2 Out-of-Order Execution
  • 6.3 Store Buffer vs Out-of-Order Execution
• 7. We Have Actually Encountered Three Problems
• 8. How Does Hardware Address Them?
  • 8.1 Atomicity: Atomic Instruction
  • 8.2 Visibility: Cache Coherence
  • 8.3 Memory Ordering: Fence
• 9. Summary

This article continues to use count, but focuses only on the hardware layer:

What concurrency problems do the CPU and memory system introduce, and what capabilities does the hardware provide to address them?

1. Start with the von Neumann Architecture

The von Neumann model treats programs as data: instructions and data are stored in the same general way. At a high level, a computer can be divided into a control unit, arithmetic unit, memory, input devices, and output devices.

The arithmetic and control units form the CPU, which also contains registers. The Program Counter (PC) stores the address of the next instruction; the Instruction Register (IR) stores the instruction currently being processed; general-purpose registers such as R1 hold data and intermediate results.

┌──────────────────┐                    ┌──────────────────┐
│   Input Device   │                    │  Output Device   │
└─────────┬────────┘                    └─────────▲────────┘
          │                                       │
          └───────────────────┬───────────────────┘
                              │
                          System Bus
                              │
             ┌────────────────┴─────────────────┐
             │                                  │
             ▼                                  ▼
┌────────────────────────┐         ┌────────────────────────┐
│          CPU           │         │         Memory         │
│                        │         │                        │
│      Control Unit      │         │  Instructions + Data   │
│     Arithmetic Unit    │         │                        │
│       Registers        │         │                        │
│       PC / IR / R1     │         │                        │
└────────────────────────┘         └────────────────────────┘

The execution of one instruction can be simplified as:

        ┌────────────────────────────────────┐
        │                                    │
PC provides the next instruction address     │
        ↓                                    │
Fetch: load the instruction from memory      │
        ↓                                    │
IR: hold the current instruction             │
        ↓                                    │
Decode: decode the current instruction       │
        ↓                                    │
Execute: execute the current instruction     │
        ↓                                    │
PC advances to the next instruction ─────────┘

2. What Does count++ Become?

A source-level expression such as count++ must first become machine instructions, and each of those instructions then goes through the execution loop from the previous section.

For discussion, we can simplify count++ into three machine-level operations:

LOAD  R1, [count]    // load count into R1
ADD   R1, 1          // increment R1 inside the CPU
STORE [count], R1    // write the result back to count

Connecting this to the instruction cycle from the previous section:

count++
        ↓ simplified compiled / translated form

LOAD  R1, [count]
ADD   R1, 1
STORE [count], R1

        ↓ each machine instruction goes through the previous cycle

PC → address of LOAD
Fetch LOAD
IR ← LOAD R1, [count]
Decode LOAD
Execute LOAD

        ↓

PC → address of ADD
Fetch ADD
IR ← ADD R1, 1
Decode ADD
Execute ADD

        ↓

PC → address of STORE
Fetch STORE
IR ← STORE [count], R1
Decode STORE
Execute STORE

Both LOAD and STORE access data. That leads naturally to the next question:

What would happen if every LOAD and STORE had to wait directly for main memory?

3. Why Do We Need Caches?

A CPU executes far faster than main memory can be accessed. If every LOAD had to wait for main memory to return data and every STORE had to wait until memory completed the write, the CPU would spend a large amount of time idle.

Modern processors therefore place smaller, faster cache levels between a CPU core and main memory. A simplified hierarchy looks like this; exact levels and which caches are shared depend on the processor design:

CPU
 │
 ▼
L1 Cache
 │
 ▼
L2 Cache
 │
 ▼
L3 Cache
 │
 ▼
Memory

When the CPU accesses count, it normally loads the cache line containing it into a cache. Simplifying again:

CPU
 │
 ▼
Cache: count = 0
 │
 ▼
Memory: count = 0

Later accesses to count may hit in the cache instead of going to main memory every time.

So:

Caches address the speed gap between the CPU and memory.

That is fundamentally a performance problem.

Many concurrency issues are related to caches, but we will leave those details aside for now and move directly to the concurrency problems themselves.

4. Concurrency on One Core: Why Can count++ Lose an Update?

Suppose the machine has only one CPU core:

Thread A ──┐
           │
           ├──> Core 0 ──> Cache ──> Memory
           │
Thread B ──┘

The two threads cannot literally execute at the same instant on that core, but the scheduler can interleave them.

Start with:

count = 0

Then the following schedule is possible:

Thread A                              Thread B
   │                                    │
   ├─ LOAD count -> 0                   │
   │                                    │
   ├─────── context switch ────────────>│
   │                                    ├─ LOAD  count -> 0
   │                                    ├─ ADD   1
   │                                    ├─ STORE count -> 1
   │                                    │
   │<────── context switch ─────────────┤
   ├─ ADD   1                           │
   ├─ STORE count -> 1                  │

During a context switch, the operating system preserves Thread A's execution context, including intermediate state. When A resumes, it can continue computing from the 0 it read earlier.

The final result is:

count = 1

The reason is that:

LOAD
ADD
STORE

are not one indivisible operation.

If another execution unit can run between those steps, a lost update becomes possible.

This gives us the first problem:

Problem 1: How can a compound operation execute indivisibly?

That is the problem of:

Atomicity

We will not answer it yet. First, move from one core to multiple cores.

5. From One Core to Multiple Cores: Visibility

With multiple CPU cores, two threads may truly execute in parallel:

Thread A                       Thread B
   │                              │
   ▼                              ▼
Core A                         Core B
   │                              │
   ▼                              ▼
Cache A                        Cache B
   │                              │
   └──────────────┬───────────────┘
                  ▼
                Memory

The original count++ race still exists, but multiple cores introduce another issue.

Suppose:

count = 0

Both Core A and Core B have read count. The same data may now exist in separate caches:

Core A Cache                    Core B Cache

count = 0                       count = 0

               Memory
              count = 0

Now Core A changes count:

Core A Cache                    Core B Cache

count = 1                       count = 0

The question becomes:

May Core B keep using its old copy?

Or more generally:

Problem 2: After one core writes data, when can other cores observe the new value?

That is:

Visibility

Again, hold the answer for a moment and move to the third class of problem.

6. What About the Order of Multiple Memory Operations?

Continue with the same count example:

count = 0
ready = false

Thread A                         Thread B
   │                                │
   ├─ count = 1                     ├─ read ready
   └─ ready = true                  └─ if true, read count

A programmer naturally wants the following guarantee:

If Thread B has already observed ready = true, it should also observe the earlier write count = 1.

Ideally, B should only see one of these outcomes:

Outcome 1:

Thread A                         Thread B

                                 read ready -> false
                                 do not enter if

Outcome 2:

Thread A                         Thread B

count = 1
ready = true                     read ready -> true
                                 enter if
                                 read count -> 1
                                 print 1

We do not want:

Thread A                         Thread B

count = 1
(write not yet observed by B)
ready = true                     read ready -> true
                                 enter if
                                 read count -> 0
                                 print 0

So the question is:

Once B has read ready = true, how do we prevent it from still reading the old count = 0?

This is no longer merely a question about multiple cached copies of the same location. We now have two different memory locations:

count
ready

and a relationship between two writes:

count = 1
ready = true

This gives us the third problem:

Problem 3: In what order may memory operations become observable to other cores?

That is:

Memory Ordering

Why does hardware make this problem more complicated?

For the same reason as before: modern CPUs introduce various optimizations for performance.

6.1 Store Buffer

Continue with count++. Suppose count = 0 and Core A executes:

LOAD  count -> 0
ADD   1     -> 1
STORE [count], 1

The STORE of count = 1 may first enter a Store Buffer, as shown below:

CPU (Core)
    │
    ▼
Store Buffer
    │
    ▼
Cache
    │
    ▼
Memory

Note: Store Buffer and Cache are not the same thing. The Store Buffer is a temporary structure on a Core's write path, while the Cache stores cached data. For simplicity, we will continue with the following abstraction:

CPU (Core A)
executes STORE [count], 1
        │
        ▼
Store Buffer
holds count = 1 temporarily
        │
        ▼
Memory

Core A can continue executing before the buffered write becomes visible to other cores:

initial: count = 0

Core A                               Core B
   │                                    │
   ├─ LOAD  count -> 0                  │
   ├─ ADD   1     -> 1                  │
   ├─ STORE count -> 1                  │
   │  enters Store Buffer               │
   ├─ continue with later instructions  │
   │                                    ├─ LOAD count -> 0
   │                                    │  still sees old value
   └─ count = 1 becomes visible         │

6.2 Out-of-Order Execution

Modern CPUs also rearrange internal execution, where dependencies allow it, to keep execution units busy.

As long as this does not change the result observed by the current thread, such optimization is valid from that thread's perspective. The difficulty appears when multiple cores interact.

Continue with the earlier count publication example:

Thread A / Core A                 Thread B / Core B

count = 1                        if ready {
ready = true                         print(count)
                                 }

The source code writes count before ready. But without additional ordering constraints, on hardware whose memory model permits it, another core may observe:

initial: count = 0, ready = false

Thread A / Core A                 Thread B / Core B
   │                                  │
   ├─ STORE count = 1                 │
   │  not yet observed by Core B      │
   ├─ STORE ready = true              │
   │                                  ├─ LOAD ready -> true
   │                                  └─ LOAD count -> 0

In other words, A issued the count write first in source order, yet B observed ready = true first and then still read the old count = 0.

The underlying question is:

In what order is another core allowed to observe these memory operations?

6.3 Store Buffer vs Out-of-Order Execution

These two mechanisms are easy to confuse, so compare the questions they answer directly:

Their common effect is that both can influence the order in which memory operations are ultimately observed by other Cores.

7. We Have Actually Encountered Three Problems

The earlier examples can be summarized as three categories:

Next, let's look at the basic capabilities provided by hardware:

What mechanisms does modern hardware provide for these three problems?

8. How Does Hardware Address Them?

8.1 Atomicity: Atomic Instruction

A normal LOAD + ADD + STORE consists of several operations. To make a Read-Modify-Write operation appear indivisible to other execution units, hardware provides atomic operations such as:

Compare-And-Swap
Exchange
Fetch-And-Add

Internally, these operations are not necessarily a single microscopic step. “Atomic” describes what competing execution units are allowed to observe:

┌──────────────────┐
│ read old value   │
│      │           │
│      ▼           │
│ modify           │
│      │           │
│      ▼           │
│ write new value  │
└────────┬─────────┘
         │
         └── appears as one indivisible atomic operation to competitors

Return to count++. If a language implements the increment with a hardware-supported atomic Read-Modify-Write, we can reason about the result as:

initial: count = 0

Thread A                         Thread B

atomically change count 0 -> 1
                                 atomically change count 1 -> 2

final: count = 2

It does not matter whether A or B wins first. Each atomic update must operate on a definite previous value, so both threads cannot independently read 0 and then both write 1.

8.2 Visibility: Cache Coherence

Multicore processors use cache-coherence protocols to coordinate cached copies of the same memory location across cores.

MESI is one of the classic coherence protocols:

M - Modified
E - Exclusive
S - Shared
I - Invalid

Real processors may use MESI or extensions and variants such as:

MESI
MOESI
MESIF
...

We do not need every state transition here; focus on the problem coherence solves.

Suppose both cores cache the line containing count:

Core A Cache              Core B Cache

count = 0                 count = 0
Shared                    Shared

If Core A wants to modify it to:

count = 1

the hardware must coordinate ownership of that cache line and the state of other copies.

A simplified view is:

Core A wants to modify count
        │
        ▼
obtain write permission for the cache line
        │
        ▼
invalidate copies that may no longer be used
        │
        ▼
Core A performs the modification

After Core A obtains write permission, Core B's old count = 0 copy is invalidated. The next time B reads count, it cannot continue using that stale copy.

8.3 Memory Ordering: Fence

Suppose count = 1 has not yet been observed by B. On hardware that permits this outcome, we may see:

Thread A / Core A                       Thread B / Core B
|                                       |
+-- STORE count = 1                     |
+-- STORE ready = true                  |
|                                       +-- LOAD ready -> true
|                                       +-- LOAD count -> 0

To forbid that result, fences can constrain the order of memory operations on each side:

Thread A / Core A                       Thread B / Core B
|                                       |
+-- STORE count = 1                     |
+-- FENCE                               |
+-- STORE ready = true                  |
|                                       +-- LOAD ready -> true
|                                       +-- FENCE
|                                       +-- LOAD count -> 1

The writer-side fence prevents count = 1 from being ordered after ready = true. The reader-side fence prevents the read of count from being ordered before the read of ready.

A fence does not guarantee that B will read ready = true. But once B has observed ready = true under the required synchronization pattern, the later read of count cannot still return the stale 0.

9. Summary

Hardware provides different foundations for the three concurrency problems:

Next: Thread and runtime scheduling — how threads and goroutines get CPU time.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.

• 1. Define the Scope First
• 2. Start with a Shared Variable
• 3. Two Main Coordination Models
  • 3.1 Shared Memory
  • 3.2 Message Passing
• 4. Why Does This Code Work Correctly?
• 5. Languages Need to Define Concurrency Semantics
• 6. Why Do We Still Need to Go Down to the Hardware?
• 7. A Single Diagram for the Layers Covered in This Series
• 8. Next: Start with the Hardware

1. Define the Scope First

This series focuses only on:

Concurrency within a single process on a single machine.

In other words, we are concerned with multiple execution units inside the same process.

For example:

• Java Thread;
• Go Goroutine;
• Python Thread / asyncio Task.

Inter-process communication, distributed systems, and network communication are outside the scope of this series.

2. Start with a Shared Variable

Suppose a process contains this variable:

count = 0

Now there are two execution units:

Thread A                    Thread B

count++                     count++

If A and B each execute once, is the final result guaranteed to be:

count = 2

No.

Logically, count++ can be broken down into three steps:

read count
compute count + 1
write count back

So the following interleaving is possible:

initial: count = 0

Thread A                    Thread B

read count -> 0
compute count + 1 -> 1
                            read count -> 0
                            compute count + 1 -> 1
write count -> 1
                            write count -> 1

The final value becomes:

count = 1

Both threads complete one +1, but because they read the same old value 0, both compute 1, and one of the updates is overwritten.

The key problem is that two Threads modify count at the same time:

Multiple execution units access and modify the same mutable data concurrently.

Within a single process on a single machine, there are two main models for handling this safely under concurrency:

• Shared Memory: multiple execution units directly share mutable data and coordinate concurrent access through synchronization mechanisms such as locks and atomic operations;
• Message Passing: execution units avoid directly sharing mutable data where possible and exchange data through messages instead.

We will keep using count++ to see how these two models handle the same concurrency problem.

3. Two Main Coordination Models

3.1 Shared Memory

Continue with count.

In Java, the shared state can be protected with synchronization, for example with synchronized:

class Counter {
    private int count = 0;

    public synchronized void increment() {
        count++;
    }
}

Both threads still access the same Counter:

Thread A                    Thread B

counter.increment()         counter.increment()

However, because of synchronized, the two threads cannot enter the critical section in increment() at the same time. In this example, that critical section is count++.

Suppose Thread A acquires the lock first. The execution becomes:

initial: count = 0

Thread A                         Thread B

request to enter increment()
acquire lock
                                 request to enter increment()
                                 cannot acquire the same lock; wait
read count       -> 0
compute count+1  -> 1
write count      -> 1
leave critical section and release lock
                                 acquire lock
                                 read count       -> 1
                                 compute count+1  -> 2
                                 write count      -> 2
                                 leave critical section and release lock

final: count = 2

If Thread B acquires the lock first, the order of A and B is reversed, but the result is still 2.

What matters is not which thread runs first. What matters is that before one thread leaves the critical section, another thread cannot enter the same critical section. The interleaving in which both threads read 0 therefore cannot occur.

Go and Python provide similar mechanisms:

Go      -> Mutex
Python  -> Lock

They share the same idea:

Mutable data is shared directly, but access to it is coordinated through synchronization mechanisms such as locks and atomic operations.

3.2 Message Passing

Continue with count.

For example, in Go:

increments := make(chan int)

go func() { // Counter Owner Goroutine
    count := 0

    for delta := range increments {
        count += delta
    }
}()

The two producer Goroutines do not modify count directly. Instead, they send increments to the same Channel:

go func() { // Goroutine A
    increments <- 1
}()

go func() { // Goroutine B
    increments <- 1
}()

We can abstract this as:

Goroutine A ── +1 ──┐
                     ├──> Channel ──> Counter Owner Goroutine ──> count++
Goroutine B ── +1 ──┘

Both Goroutines only send messages. The actual count is modified by a dedicated Counter Owner Goroutine.

Java and Python provide similar message-passing tools:

• Java BlockingQueue
• Python queue.Queue / asyncio.Queue

They share the same idea:

Multiple execution units coordinate through messages instead of directly modifying the same state at the same time.

4. Why Does This Code Work Correctly?

So far, we have handled the original count++ problem in two ways.

With Shared Memory, we used synchronization mechanisms:

synchronized / Lock / Mutex

With Message Passing, we used:

Channel / Queue

But there is a more fundamental question:

Why does this code work correctly?

To answer it, we need to look at three fundamental properties of concurrent programs:

• Atomicity: when an execution unit performs one or more operations, whether those operations appear to other execution units as one indivisible whole; other execution units cannot observe intermediate states, only the state before or after the operation.
• Visibility: after one execution unit writes data, whether other execution units can observe that write.
• Ordering: when one execution unit performs multiple operations in sequence, whether other execution units observe those operations taking effect in an order consistent with that sequence.

Programmers need to know what guarantees they can rely on across these three dimensions.

That leads us to:

The concurrency semantics defined by the programming language.

5. Languages Need to Define Concurrency Semantics

A programming language needs to define:

When multiple execution units access and modify the same mutable data concurrently, which behaviors are allowed and which rules programmers may rely on.

Java has the:

Java Memory Model

Go has the:

Go Memory Model

Python is somewhat different. In this series, we mainly discuss:

Python / CPython Concurrency Semantics

These rules need to answer three questions:

6. Why Do We Still Need to Go Down to the Operating System and Hardware?

Languages define concurrency rules, and those rules are ultimately realized by the operating system and the underlying hardware.

The Operating System manages the execution units on which programs actually run, including thread scheduling, blocking, and wake-up, and provides synchronization primitives such as semaphores, futexes, and condition variables.

Hardware provides lower-level primitives and mechanisms, including atomic instructions, memory-ordering guarantees, CPU caches, and store buffers.

7. A Single Diagram for the Layers Covered in This Series

First, consider the conventional software-to-hardware stack:

Source Code
├──→ Java: Java Application / JDK Source
├──→ Go: Go Application / Standard Library Source
└──→ Python: Python Application / Standard Library Source
        ↓
Compiler / Translator
├──→ Java: javac
├──→ Go: Go Compiler
└──→ Python: CPython Compiler
        ↓
Runtime / Execution Environment
├──→ Java: HotSpot JVM (Interpreter / JIT / Runtime)
├──→ Go: Go Runtime
└──→ Python: CPython Interpreter / Runtime
        ↓
Operating System
└──→ Linux / Windows / macOS
        ↓
Hardware
        │
        ├── ISA ──→ x86-64 / ARM64
        │      ↓
        ├── Microarchitecture ──→ Cache / Store Buffer / Reorder Buffer / Out-of-Order Execution
        │      ↓
        └── Physical Hardware ──→ CPU / DRAM / Interconnect

The Language Memory Model / Concurrency Semantics forms a cross-layer constraint: the Compiler, Runtime, Operating System, and Hardware work together to realize the concurrency guarantees defined by the language.

This series focuses on concurrency. To give the later Mutex, Atomic, Volatile, and related articles a consistent structure, we combine the concurrency rules with the conventional software / hardware stack into one discussion model:

Concurrency Tools
├──→ Java: synchronized / Lock / Atomic / BlockingQueue
├──→ Go: Mutex / Atomic / Channel
└──→ Python: Lock / Queue / asyncio
        ↓
Language Memory Model / Concurrency Semantics
├──→ Java: Java Memory Model
├──→ Go: Go Memory Model
└──→ Python: Python / CPython Concurrency Semantics
        ↓
Runtime / Language Implementation
├──→ Java: HotSpot JVM (Interpreter / JIT / Runtime)
├──→ Go: Go Runtime
└──→ Python: CPython Interpreter / Runtime
        ↓
Operating System
└──→ Threads / Scheduler / Semaphores / Futexes / Condition Variables
        ↓
Hardware
├──→ ISA: x86-64 / ARM64
├──→ Microarchitecture: Cache / Store Buffer / Reorder Buffer / Out-of-Order Execution
└──→ Physical Hardware: CPU / DRAM / Interconnect

8. Next: Start with the Hardware

The next article moves down to the Hardware layer.

It focuses on:

• why CPUs need caches;
• how multiple CPU Cores coordinate cached data;
• why Store Buffers and Out-of-Order Execution affect Memory-access ordering;
• what Atomic Instructions provide.

This article was first published on ThinkerQAQ's personal blog and syndicated here by the author. The original article may be revised over time; please refer to the personal blog for the latest version.