What's The Difference Between The Call/return Protocol Of Oldstyle And Newstyle Coroutines In Python?

I'm transitioning from old-style coroutines (where 'yield' returns a value supplied by 'send', but which are otherwise essentially generators) to new-style coroutines with 'async d

Solution 1:

It is possible, and perhaps instructive, to relate the two models pretty directly. Modern coroutines are actually implemented, like the old, in terms of the (generalized) iterator protocol. The difference is that return values from the iterator are automatically propagated upwards through any number of coroutine callers (via an implicit yield from), whereas actual return values are packaged into StopIteration exceptions.

The purpose of this choreography is to inform the driver (the presumed “event loop”) of the conditions under which a coroutine may be resumed. That driver may resume the coroutine from an unrelated stack frame and may send data back into the execution—via the awaited object, since it is the only channel known to the driver—again, much as send communicates transparently through a yield from.

An example of such bidirectional communication:

class Send:
  def __call__(self,x): self.value=x
  def __await__(self):
    yield self  # same object for awaiter and driver
    raise StopIteration(self.value)

async def add(x):
  return await Send()+x

def plus(a,b):  # a driver
  c=add(b)
  # Equivalent to next(c.__await__())
  c.send(None)(a)
  try: c.send(None)
  except StopIteration as si: return si.value
  raise RuntimeError("Didn't resume/finish")

A real driver would of course decide to call the result of send only after recognizing it as a Send.

Practically speaking, you don’t want to drive modern coroutines yourself; they are syntax optimized for exactly the opposite approach. It would, however, be straightforward to use a queue to handle one direction of the communication (as you already noted):

async def avg(q):
  n=s=0
  while True:
    x=await q.get()
    if x is None: break
    n+=1; s+=x
    yield s/n

async def test():
  q=asyncio.Queue()
  i=iter([10,30,5])
  await q.put(next(i))
  async for a in avg(q):
    print(a)
    await q.put(next(i,None))

Providing values that way is a bit painful, but it’s easy if they’re coming from another Queue or so.