Bluetooth on the X1, and why isochronous audio is hard

The Path OS page says bluetooth and isochronous audio proved by far the hardest part of bringing the OS up on real hardware, and that I should write up why. This is the first instalment of that, from the X1 rather than the X220.

Finding the right channel

Once again, problems with RFCOMM channel identification, and with buffer lengths. And then with timeouts, and with queues not being flushed correctly. None of them is interesting on its own.

When it works it looks like this:

acl> ... 35 03 19 11 1f ...          ← our SDP query for the Hands-Free Gateway UUID (0x111F)
acl< 112 | ... 35 5e 36 00 5b ...    ← the phone's service record comes back
bt   its gateway is on RFCOMM channel 4 - opening it
acl> ... 02 04 04 00 03 00 42 00     ← we open RFCOMM (PSM 3)
acl< ... 02 08 04 00 01 00 4b 00     ← and the phone opens SDP to US to read our record

bt -> AT+CIND?      bt <- +CIND: 0,0,1,2,0,4,0
bt -> AT+CHLD=?     bt <- +CHLD: (0,1,2,3)
bt -> AT+CLIP=1     bt -> AT+COPS?
bt <- +COPS: 0,0,"EE"          ← it's reading your network operator
bt    its messages are on RFCOMM channel 6
bt    messages: Ready          ← MAP (SMS) is up too

The channel number is not a constant, it is whatever the phone published in its service record, which is the whole reason the SDP query at the top exists. Guessing it works right up until the phone changes.

Two cards that disagree

Bitten by a subtle difference between the Intel and Broadcom cards. Broadcom generates zero-length packets, which the Intel path would take as being in error.

That is the kind of difference that stays invisible until the day you swap a card. Neither card is wrong. The Intel one just never does the thing, so the assumption that it never happens survives every test you run on that hardware.

Call audio, which is where it got difficult

Implementing audio is more involved on the X1 than on the X220. The audio subsystem is EHCI based, which means bi-directional audio on a call needs dual-stream xHCI isochronous SCO.

There is also an intermittent problem with HDA reset, which means the codec does not always come back on reboot. A bit of delving suggested specific clock-gating would fix it, and that alone turned out not to be enough. It does seem to be sufficient paired with a hard reboot: once locked up we cannot recover with a soft boot, but setting the clock gating every time prevents the lock-up occurring in the first place. Should it occur anyway there is no software solution beyond a hard reboot, and I have not found one.

Isochronous buffers are hard. The rates rarely match natively and often are not exact multiples of one another. Get it wrong and the end product is dropped audio, distortion, or crackling over the top of the audio. Balancing buffer size against the need for minimal latency is a challenge in itself, and the two pull in opposite directions.

In the end, two independent crystals running at different speeds are always going to have some phase drift, so no fixed rate is ever going to be the right one. I switched to monitoring the final buffer's low-water mark rather than driving the audio through at a fixed rate, and that improved the crackle. The final cure was to move the audio to a dedicated, non-pre-emptive core - which is the idea from the last note, put to use sooner than expected.

An accidental neural network

Surprised to find that the Kaby-Lake era X1 has a basic neural network processor built in, a "GMM device" according to ChatGPT. It does not actually seem to be useful for very much. It was originally intended to support voice recognition in some capacity, so it might be useful for either voice activity detection or wake-word spotting, both of which the speech work would want. That is speculation and I have not written a line against it.

What is left on the X1

The built-in GPU, an Intel HD 620, which is easy enough having done something similar on the X220. Thunderbolt, which is potentially interesting because I have a Tesla P40 in a Thunderbolt cage available, although using it would most likely also involve writing some homebrew version of CUDA, and that is a huge side-mission rather than an afternoon. Thermal controls, and power management. The built-in xHCI USB hubs need some additional work too, but that part is fairly vanilla.