Advanced Coroutines & Delegation (yield from)

Generators expand far beyond simple read-only data streams. By incorporating bidirectional value passing, exception injection, and transparent sub-iterator delegation, Python transforms generators into lightweight cooperative coroutines.

This bidirectional execution pipeline forms the direct evolutionary bridge between simple Python iterators and modern async/await asynchronous programming.

1. Bidirectional Communication via send()#

While next(gen) resumes execution and receives the next yielded item, gen.send(value) allows the caller to pass data back into the generator.

The injected value becomes the return value of the yield expression inside the generator:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
def running_averager():
    """Coroutine computing running average of numbers sent to it."""
    total = 0.0
    count = 0
    average = None

    while True:
        # yield returns the value passed into send()
        term = yield average
        if term is None:
            break
        total += term
        count += 1
        average = total / count

# Usage
avg = running_averager()
# Prime the coroutine to advance it to the first yield
next(avg)

print(avg.send(10))  # Output: 10.0
print(avg.send(20))  # Output: 15.0
print(avg.send(30))  # Output: 20.0

2. Exception Handling: throw() & close()#

Generators can handle external signals and errors through two specialized methods:

1. gen.throw(ExcType, value=None, traceback=None)#

Injects an exception directly into the generator frame at the exact instruction pointer where it is currently suspended. The generator can catch the exception using standard try/except blocks:

1
2
3
4
5
6
7
8
9
10
def resilient_task():
    try:
        while True:
            yield "working"
    except ValueError:
        yield "recovered from ValueError"

task = resilient_task()
next(task)  # "working"
print(task.throw(ValueError))  # "recovered from ValueError"

2. gen.close()#

Injects a GeneratorExit exception into the generator. If the generator catches GeneratorExit, it must either exit or return; if it attempts to yield another value, Python raises RuntimeError: generator ignored GeneratorExit.

3. Sub-Iterator Delegation: yield from#

Prior to Python 3.3, delegating iteration to a nested sub-generator required verbose boilerplate to manually propagate send(), throw(), and return values.

PEP 380 introduced yield from, which establishes a direct, bidirectional communication tunnel between the outer caller and the inner sub-generator:

Key Responsibilities of yield from:#

1. Direct Value Forwarding: Any value yielded by the sub-generator is routed directly to the outer caller.
2. Transparent Input Forwarding: Any value passed via send() or exception injected via throw() is forwarded straight to the sub-generator.
3. Return Value Extraction: When the sub-generator raises StopIteration(value=R), yield from catches the exception and evaluates to R as its return expression!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
def sub_accumulator():
    total = 0
    while True:
        val = yield
        if val is None:
            # The return value is embedded in StopIteration!
            return total
        total += val

def delegating_manager():
    # yield from extracts the return value upon completion
    result = yield from sub_accumulator()
    print(f"Sub-accumulator finished with total: {result}")
    return result

mgr = delegating_manager()
next(mgr)  # Prime
mgr.send(10)
mgr.send(25)
try:
    mgr.send(None)  # Terminate sub_accumulator
except StopIteration as e:
    print(f"Delegating manager returned: {e.value}")

4. Concept Verification#

Test your understanding of advanced generator coroutines: