Non-blocking busio

[kamocat]

I would like non-blocking calls for the busio I2C, SPI, and UART modules.

This would be awesome for ports that support asyncio, but could also be used to interleave communication and processing on ports with a little less resources.

I opened PR #10854 to implement this, but Dan suggested the discussion would be better in an issue.

The question is, what should the API look like to best support all possible ports?

[kamocat]

For I2C it could be a start_read() and start_write() which return tracking objects and can be checked with corresponding read_complete() and write_complete() methods.

For SPI the reads and writes happen simultaneously, so maybe just a start_readwrite() with a corresponding readwrite_complete() method. Ideally this would transfer the greater of the written or read buffers, but I suppose it's fine if we require the buffers to be the same size for simplicity, so long as one of them can be None.

Non-blocking UART read can already be accomplished by first checking the bytes available to read and only reading that quantity. Non-blocking write would need a start_write() and write_complete() just like I2C.

[tannewt]

My preference would be to make a new abusio module and have the normal methods return coroutines that can be awaited. I know this is the first time we've done this so it'll be tricky. BUT, I think we'll want to do this in many places to make our API async-native.

[kamocat]

I agree that the abusio should be a separate module, but I don't think the awaitable methods can be written in C. That means it must be a Python module just like asyncio.
Furthermore, I would like to preserve the blocking methods for the likely event that someone will use a blocking and non-blocking driver on the same bus.

[dhalbert]

but I don't think the awaitable methods can be written in C

That was my conclusion the last time I was working on asyncio. There is not native support in MicroPython for C async methods or functions.

[kamocat]

What about a method to tell how many bytes are remaining to transfer? Or one to abort a transfer?

[tannewt]

but I don't think the awaitable methods can be written in C

That was my conclusion the last time I was working on asyncio. There is not native support in MicroPython for C async methods or functions.

Any idea what it would take to make it work? It's the way it should work IMO.

[kamocat]

but I don't think the awaitable methods can be written in C

That was my conclusion the last time I was working on asyncio. There is not native support in MicroPython for C async methods or functions.

Any idea what it would take to make it work? It's the way it should work IMO.

If you can find an example I'm happy to try it. In my research I was not able to find one.
Also, what would you do about the asyncio dependency, which itself is written in Python?

[dhalbert]

Any idea what it would take to make it work?

I don't know. I would say it's a question to take up with the MicroPython folks. It's a bit hard to search for, because "native async" can mean @micropython.native which is about native code generation from Python.

Here is a several-year old MicroPython discussion related to this issue which @tannewt contributed to: https://github.com/orgs/micropython/discussions/13081

[tannewt]

I don't care about use without asyncio (like andrew in that discussion) and I'd be willing to migrate actual asyncio to native as well so it'd just work.

This is what pi/Codex came up with and it seems plausible to me. I could give it a shot tomorrow:

Great question. In this tree, asyncio support is split into language/runtime in py/
and scheduler in _asyncio + frozen asyncio Python code.

How py/ supports async/await

1. Compiler marks async funcs as coroutine-generators
   • ../../py/compile.c (compile_async_stmt) sets MP_SCOPE_FLAG_GENERATOR |
     MP_SCOPE_FLAG_ASYNC.
2. Function objects get coroutine wrapper types
   • ../../py/emitglue.c maps async code to mp_type_coro_wrap /
     mp_type_native_coro_wrap.
3. Calling an async function returns a coroutine instance
   • ../../py/objgenerator.c creates mp_type_coro_instance.
   • Coroutines have send/throw/close and await (returns self).
4. await x in this fork is CPython-style
   • In ../../py/compile.c (compile_atom_expr_await), it emits:
     - call x.await()
     - then YIELD_FROM
   • (CircuitPython change: await requires await, not bare generator behavior.)
5. VM drives awaitables with mp_resume
   • ../../py/vm.c (MP_BC_YIELD_FROM) calls mp_resume.
   • ../../py/runtime.c (mp_resume) resumes via:
     - coroutine/generator fast path, or
     - iterator (iternext) when sending None, or
     - .send() / .throw() for full protocol.

────────────────────────────────────────────────────────────────────────────────

For native C methods that return awaitables

You usually don’t construct mp_type_coro_instance directly.
Instead, return a custom C object that is awaitable.

Minimum contract for your returned object

• await method (must exist in this fork)
• returns an object resumable by mp_resume (often self)
• implement iterator/coroutine protocol:
  • iternext (or next)
  • preferably send and throw too (for compatibility/cancellation)

Practical pattern

• Type with MP_TYPE_FLAG_ITER_IS_ITERNEXT
• iter slot = your iternext
• locals dict includes await, send, throw
• your C API method allocates and returns that object

If you want it to integrate nicely with asyncio scheduling (not busy-yield), copy the
pattern in ../../extmod/modasyncio.c (Task type): it parks/wakes tasks using the task
queue.