Concurrency: Traps, Failure Modes & Interview Q&A

Building scalable concurrent systems requires anticipating subtle failure modes across operating system thread schedulers, process boundaries, and asynchronous event loops.

This lesson explores edge cases like the GIL Convoy Effect, provides a definitive concurrency selection matrix, and walks through high-frequency senior technical interview flashcards.

1. Advanced Concurrency Failure Modes#

Trap 1: The GIL Convoy Effect#

On multi-core architectures, running a mix of CPU-bound threads and I/O-bound threads can trigger the GIL Convoy Effect:

When an I/O thread finishes waiting for a packet and requests the GIL, the CPU thread is often already in a tight loop attempting to re-acquire the lock. The resulting contention forces operating system thread context switches, cache line invalidations, and introduces severe latency spikes for network requests.

Trap 2: Event Loop Starvation#

Because asyncio is single-threaded, any synchronous operation running for more than a few milliseconds starves all other tasks on the loop:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import asyncio
import time

# Antipattern: Synchronous blocking call inside async function!
async def handle_request(request_id: int):
    print(f"Handling request {request_id}")
    # BUG: time.sleep() freezes the ENTIRE event loop!
    # No other request can be received or processed for 2 seconds!
    time.sleep(2.0)
    return "Done"

# Idiomatic Solution: Async non-blocking await
async def handle_request_clean(request_id: int):
    print(f"Handling request {request_id}")
    # Non-blocking: yields control back to the loop!
    await asyncio.sleep(2.0)
    return "Done"

Trap 3: Modern Structured Concurrency via TaskGroup (Python 3.11+)#

In earlier versions of Python, managing multiple concurrent tasks using asyncio.gather() often leaked background tasks if one task raised an unhandled exception.

PEP 654 introduced asyncio.TaskGroup, which guarantees Structured Concurrency: if any task inside the group fails, all remaining child tasks are cancelled automatically:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import asyncio

async def fetch_profile(user_id: int):
    await asyncio.sleep(0.1)
    return {"id": user_id, "name": "Alice"}

async def fetch_orders(user_id: int):
    await asyncio.sleep(0.15)
    return [{"order_id": 101, "total": 49.99}]

async def main():
    # Exception-safe structured concurrency block
    async with asyncio.TaskGroup() as tg:
        t1 = tg.create_task(fetch_profile(1))
        t2 = tg.create_task(fetch_orders(1))

    # Execution reaches here ONLY when all tasks finish cleanly!
    print(f"Profile: {t1.result()}")
    print(f"Orders : {t2.result()}")

asyncio.run(main())

2. "When to Use What" Concurrency Decision Matrix#

3. Senior Technical Interview Flashcards#

4. Concept Verification#

Test your understanding of concurrency trade-offs: