Patrick Lidstone
Self-hosted

ARDOP: a native OFDM/4FSK HF ARQ data protocol

A self-contained, dependency-light re-implementation of the Amateur Radio Digital Open Protocol — the multi-carrier PSK/QAM + 4FSK frame waveform, its Reed–Solomon + CRC-16 per-carrier FEC, the leader/matched-filter sync and frame-type header, the ISS/IRS half-duplex ARQ state machine, and an ardopcf-compatible TCP host/TNC interface.

Rafe project · app/radio/ardop_native.py, ardop_arq.py, ardop_tnc.py · clean-room from the MIT ardopcf reference · NumPy at runtime

Abstract

ARDOP (Amateur Radio Digital Open Protocol) is an HF connected-mode data protocol: a family of short OFDM-ish PSK/QAM and 4-FSK burst waveforms wrapped in an automatic-repeat-request (ARQ) session layer, used by Winlink and other keyboard / email-over-radio clients as an alternative to PACTOR and VARA. This document specifies Rafe's native, in-repo ARDOP: a clean-room port of the open-source ardopcf modem (MIT — Rick Muething KN6KB, John Wiseman G8BPQ, Peter LaRue), built in three layers.

  1. The modem/PHY (ardop_native.py) — a 12 kHz waveform engine. It generates and detects the two-tone leader, the redundant 4-FSK frame-type header, and the data body, which is one of a 17-entry table of data modes spanning 4PSK/8PSK/16QAM on 1, 2, 4 or 8 parallel audio carriers plus three single-carrier 4-FSK modes. Every carrier's payload is protected independently by a frame-type-keyed CRC-16 and a systematic Reed–Solomon code over GF(2⁸). A streaming receiver locates each burst by normalized FFT cross-correlation against the leader and decodes it by frame type.

  2. The ARQ layer (ardop_arq.py) — the half-duplex ISS/IRS state machine: connect (ConReq → ConAck), even/odd-sequenced data with ACK/NAK retransmission, a gearshift that trades speed for robustness on ACK/NAK history, a BREAK turnaround, and DISC/END teardown.

  3. The TNC (ardop_tnc.py) — the ardopcf TCP host interface (command port 8515, data port 8516) that Pat and Winlink Express speak, so the native stack is a drop-in for the external ardopcf daemon.

The whole stack is exercised by three test modules (§8) that round-trip every data mode through AWGN, recover a NAK-dropped frame, and push a multi-frame Winlink message end-to-end through the real audio modem. Constants are protocol facts transcribed from ardopcf; no code is copied.

1. Motivation

Winlink over HF needs an ARQ modem, and in the field that means running the external ardopcf daemon as a subprocess, piping its PTT keystrings and 12 kHz audio through PulseAudio (app/radio/ardop.py does exactly this — a pty CAT device, pacat/parec bridges, and the marginal ~180 ms WLAN audio-buffer timing noted in that file's docstring). Rafe has moved its digital text and voice modes in-house — native FT8, the WSJT-X family, the fldigi family, RVQ-Voice — for the usual reasons: no build dependencies, full control of the DSP, and something genuinely part of the project. ARDOP is the ARQ member of that family.

The interesting engineering content is that ARDOP is not one waveform but a graceful-degradation ladder of them, negotiated on the fly by an ARQ session: the same session can be running fat 16QAM-on-8-carriers when the channel is clean and drop to a tough single-carrier 4-FSK burst when it is not, without dropping the connection. Re-implementing it means getting three fairly independent machines right — a multi-mode burst modem, a Reed–Solomon/CRC block coder, and a retransmission protocol — and making them interoperate with the reference at every seam (the CRC keying, the frame-type table, the host protocol).

2. Background

2.1 What ARDOP is

ARDOP is a frame-based half-duplex ARQ protocol for HF. Unlike a continuous keyboard mode (PSK31, RTTY) it transmits discrete bursts: each burst is a self-contained frame that begins with an acquisition leader, declares its type, and (for data frames) carries a fixed-size FEC-protected payload. The two stations take turns: the ISS (Information Sending Station) transmits a data frame and stops; the IRS (Information Receiving Station) transmits a short ACK or NAK and stops; the ISS sends the next frame or repeats. This strict listen-before-transmit turn-taking is what makes ARQ possible on a half-duplex SSB channel with no collision detection.

2.2 The waveform family

The data body is where ARDOP's flexibility lives. Two waveform families share the same leader + header machinery:

  • Multi-carrier PSK/QAM. Up to 8 audio subcarriers spaced 200 Hz apart are modulated in parallel at 100 baud with a common constellation — 4PSK (2 bits/sym), 8PSK (3 bits/sym) or 16QAM (4 bits/sym). This is a narrow, low-order OFDM: carriers are independent, each pulse-shaped by a half-sine envelope, and each carries its own FEC-protected sub-payload. More carriers and a denser constellation buy throughput; fewer carriers and 4PSK buy robustness. Phase is carried differentially per carrier (each carrier opens with a phase-0 reference symbol), so no absolute carrier-phase recovery is needed.

  • Single-carrier 4-FSK. Three rugged fallback modes key one of four tones per symbol at 50 or 100 baud. 4-FSK needs no phase reference at all and survives where PSK cannot; it is the bottom of the gearshift ladder and the medium for the shortest control-adjacent data frames.

2.3 FEC: Reed–Solomon over CRC

Every carrier's payload is a small systematic Reed–Solomon codeword over GF(2⁸): the message bytes are sent verbatim, followed by rslen parity bytes that let the decoder correct up to rslen/2 byte errors anywhere in the block. RS is byte-oriented, so it shrugs off the bursts of bit errors that HF fading and the PSK differential detector produce (a whole corrupted symbol is one or two bad bytes, not a scatter of bad bits). Underneath the RS parity sits a CRC-16 that is keyed by the frame type — the frame-type byte is folded into the CRC — so a payload only validates if it was demodulated under the correct mode, which doubles as a guard against a mis-detected leader firing a false frame.

2.4 ARQ and the gearshift

On top of the modem, the ARQ state machine turns unreliable bursts into a reliable byte stream: sequence numbers (a single alternating even/odd bit) drop duplicate retransmissions, NAKs trigger repeats, and a gearshift promotes the data mode up the ladder after a run of clean ACKs and demotes it on a NAK — the adaptive-rate behaviour that lets one connection ride out changing propagation.

3. System overview

                              TX  (ISS side)
 host bytes ─▶ ARQ segment ─▶ frame type ftype ─┐
                 (ardop_arq)                      │
   per carrier c = 0..N-1:                        ▼
     block ─▶ [count|data|CRC16|RS parity] ─▶ modulate carrier c
                (frame-type-keyed CRC, systematic RS/GF(2^8))
                                                  │
   [ leader 2-tone ] [ 4FSK type×2 ] [ N parallel PSK/QAM or 1×4FSK body ]
                                                  │
                             12 kHz ──▶ ×4 upsample ──▶ 48 kHz s16le ──▶ radio

                              RX  (IRS side)
 48 kHz ─▶ ÷4 decimate ─▶ 12 kHz ─▶ NCC vs leader (FFT) ─▶ sync ─┐
                                                                 ▼
   read 4FSK type (2 bytes, both must agree + parity) ─▶ frame_info ─┐
                                                                     ▼
   per carrier: demod ─▶ RS-correct ─▶ CRC-check ─▶ bytes  (any fail ⇒ NAK)
                                                                     │
                                    ARQ: dup-drop, ACK/NAK, gearshift, deliver

The layering is strict: ardop_native knows nothing about connections (it maps (frame_type, data) ↔︎ audio); ardop_arq knows nothing about audio (it maps (frame_type, data) frames to a reliable byte stream); ardop_tnc glues both to the radio callbacks and the host TCP protocol. Key global parameters: 12 000 Hz modem sample rate (SR, ardop_native.py), 100 baud for PSK/QAM (120 samples/symbol), 50 or 100 baud for 4-FSK (240 / 120 samples/symbol), a 240 ms leader and a 200 ms frame-type header.

4. Signal & frame structure (exact numbers)

Every on-air frame — control or data — has the same three-part skeleton:

 ┌──────────────────┬────────────────────────┬───────────────────────────┐
 │  LEADER          │  FRAME-TYPE HEADER      │  DATA BODY (data frames)  │
 │  2-tone 50 baud  │  4FSK 50 baud, 10 sym   │  PSK/QAM 100 baud, N car  │
 │  12 sym / 240 ms │  = 2 bytes × 5 sym      │   or 4FSK 50/100 baud     │
 │  2880 samples    │  = 2400 samples/200 ms  │   (control frames: none)  │
 └──────────────────┴────────────────────────┴───────────────────────────┘

4.1 The leader (acquisition preamble)

The leader is a two-tone burst — 1475 Hz + 1525 Hz summed — repeated as 50-baud symbols (_LEAD_SPS = 240 samples, i.e. 20 ms; ardop_native.py). The tone pair is fixed by _LEAD_SYM:

_LEAD_SPS = 240
_LEAD_SYM = sin(2π·1475·n/SR) + sin(2π·1525·n/SR)     # n = 0..239

The default leader is 12 symbols = 2880 samples = 240 ms (leader_ms=240). The symbol sign alternates +,−,+,−,… for a low crest factor, and the final symbol's sign is inverted relative to that alternation — a deliberate phase discontinuity that marks the exact sync instant (_leader):

def _leader(leader_ms=240):
    nsym = leader_ms // 20
    sign = -1 if (nsym & 1) else 1
    out = []
    for i in range(nsym):
        s = (-sign if i == nsym - 1 else sign)   # last symbol inverted
        out.append(s * _LEAD_SYM)
        sign = -sign
    return concatenate(out), nsym

The receiver correlates the entire 2880-sample leader waveform (including that inverted last symbol) as a matched filter; a true leader peaks at normalized cross-correlation ≈ 1.0, while a window that only partially overlaps it forms a broad ~0.5–0.6 shoulder — which is why the detector thresholds high (§5.1).

4.2 The frame-type header

Immediately after the leader, the frame type is sent as two identical bytes, each encoded as five 50-baud 4-FSK symbols: the four data dibits MS-first, then a fifth parity symbol. Ten symbols × 240 samples = 2400 samples = 200 ms (_frametype_wave):

def _frametype_wave(ftype):
    tmpl = _FSK_TEMPLATES[50]
    out, sym = [], 0
    for byte in (ftype & 0xFF, ftype & 0xFF):        # two copies
        for k in range(5):
            val = ((byte >> (2*(3-k))) & 3) if k < 4 else _type_parity(byte)
            out.append((1 if sym % 2 == 0 else -1) * tmpl[val])   # sign alternates
            sym += 1
    return concatenate(out)

The parity symbol is the XOR of the four dibits (a 2-bit value → one 4-FSK symbol; _type_parity):

def _type_parity(byte):
    p = 0
    for k in range(4):
        p ^= (byte >> (2*k)) & 3
    return p

Redundancy is deliberate: on receive both byte copies must agree and both parity symbols must pass before the type is accepted (_read_frame_type). This is the only integrity check a control frame gets — control frames carry no CRC — so it is what rejects a false leader from firing a spurious DISC/ACK.

4.3 The 4-FSK tone plans

4-FSK is used for the frame-type header (always 50 baud) and for the three 4-FSK data modes. Two tone quartets and two symbol rates:

baud samples/sym tone 0 tone 1 tone 2 tone 3
50 240 1425 1475 1525 1575
100 120 1350 1450 1550 1650 
SR = 12000
_FSK_TONES = {50: (1425, 1475, 1525, 1575), 100: (1350, 1450, 1550, 1650)}
_FSK_SPS   = {50: 240, 100: 120}

Each byte is four dibits, MS-first; consecutive symbols alternate in sign (the ardopcf crest-factor trick). Detection is non-coherent: correlate the symbol against each tone template and take the largest magnitude (mod_4fsk / demod_4fsk).

4.4 The PSK/QAM carrier plan

PSK/QAM runs at 100 baud (_PSK_SPS = 120) on a fixed grid of nine candidate carriers spaced 200 Hz apart, of which a symmetric subset is activated per carrier-count:

_PSK_SPS = 120
_PSK_CARRIERS = (800, 1000, 1200, 1400, 1500, 1600, 1800, 2000, 2200)
_PSK_CAR_IDX  = {1: (4,), 2: (3, 5), 4: (2, 3, 5, 6), 8: (0,1,2,3,5,6,7,8)}
carriers active grid indices audio frequencies (Hz)
1 4 1500
2 3, 5 1400, 1600
4 2, 3, 5, 6 1200, 1400, 1600, 1800
8 0,1,2,3,5,6,7,8 800, 1000, 1200, 1400, 1600, 1800, 2000, 2200

Note the 1-carrier mode uses the centre tone 1500 Hz, and the 8-carrier mode skips index 4 (1500 Hz) to keep the set symmetric about centre. Each carrier is pulse-shaped by a half-sine envelope over the 120-sample symbol (_PSK_ENV = sin(π·n/119)), which is what bounds the occupied bandwidth, and the whole multi-carrier sum is scaled by 1/√numcar so total power is carrier-count-independent.

The constellation is set by the mode (_PSK_BITS):

_PSK_BITS = {"4PSK": (2, 2), "8PSK": (3, 1), "16QAM": (4, 1)}   # (bits/sym, symset)

bits/sym is how many payload bits a symbol carries; symset is the differential phase multiplier. All three constellations live on the same 8-point (π/4) phase grid — 4PSK advances phase in steps of symset=2 (0/2/4/6 = the four QPSK points), 8PSK in steps of 1 (all eight), and 16QAM uses the eight phases plus a one-bit half-magnitude flag (bit 3) to reach 16 points. Details in §5.

4.5 The per-carrier data payload

Each carrier carries one fixed-length payload block (encode_data_carrier):

 ┌─────────┬───────────────┬──────────┬──────────────────┐
 │ count 1 │ data data_len │ CRC-16 2 │ RS parity rs_len │
 └─────────┴───────────────┴──────────┴──────────────────┘
  └── CRC-16 over (count+data), frame-type keyed ──┘
  └────── RS systematic over (count+data+CRC) ──────────┘
   codeword length  =  data_len + 3 + rs_len  bytes
  • count (1 byte) = number of valid data bytes in this carrier (≤ data_len).
  • data = the payload, zero-padded to data_len.
  • CRC-16 (2 bytes) over count+data, with the frame type folded into the low byte (§7.2).
  • RS parity (rs_len bytes) over everything before it — a systematic shortened RS(n, n−rs_len) with n = data_len+3+rs_len, correcting rs_len/2 byte errors.

The full frame payload is numcar such blocks laid end to end, one per carrier (§7.1 lists data_len/rs_len per mode).

4.6 Frame timings

Combining the fixed 240 ms leader + 200 ms header with each mode's body length gives the on-air frame durations (leader+header overhead is a constant 440 ms; computed from the code):

ftype mode car baud user B/car rs codeword B body sym user B/frame frame ms
0x42 4PSK 1 100 16 8 27 109 16 1530
0x48 4FSK 1 50 16 4 23 92 16 2280
0x4C 4FSK 1 100 32 8 43 172 32 2160
0x4A 4FSK 1 100 64 16 83 332 64 3760
0x40 4PSK 1 100 64 32 99 397 64 4410
0x44 8PSK 1 100 108 36 147 393 108 4370
0x46 16QAM 1 100 128 64 195 391 128 4350
0x50 4PSK 2 100 64 32 99 397 128 4410
0x52 8PSK 2 100 108 36 147 393 216 4370
0x54 16QAM 2 100 128 64 195 391 256 4350
0x60 4PSK 4 100 64 32 99 397 256 4410
0x62 8PSK 4 100 108 36 147 393 432 4370
0x64 16QAM 4 100 128 64 195 391 512 4350
0x70 4PSK 8 100 64 32 99 397 512 4410
0x72 8PSK 8 100 108 36 147 393 864 4370
0x74 16QAM 8 100 128 64 195 391 1024 4350

Because the carriers are parallel, adding carriers does not lengthen the frame — 0x40, 0x50, 0x60 and 0x70 are all 4410 ms — it multiplies the payload. Net user throughput therefore ranges from ~84 bit/s (0x42) to ~1.88 kbit/s (0x74, 1024 bytes in 4.35 s), which is the gearshift's operating range. (These data_len /rs_len values are ardopcf's FrameInfo table; §7.1.)

5. Encode / TX, step by step

encode_frame(ftype, data) (ardop_native.py) turns a frame type and its user payload into 12 kHz float audio. The pipeline for a data frame:

(1) Split across carriers. The numcar·data_len-byte payload is cut into numcar consecutive data_len-byte blocks, carrier c taking bytes [c·data_len : (c+1)·data_len] (encode_data_frame). This is the only "interleave" in the native implementation — a spatial split across parallel carriers; there is no bit/symbol interleaver. Each carrier is an independent FEC block, so a carrier lost to a narrowband notch fails alone (its CRC catches it) without smearing errors into the others.

(2) Per-carrier CRC-16. For each block, build count(1) ‖ data(data_len), and compute the frame-type-keyed CRC-16 over it (§7.2). The high CRC byte is stored verbatim; the low byte is XORed with the frame type:

c = crc16(bytes(buf[:data_len + 1]))
buf[data_len + 1] = (c >> 8) & 0xFF
buf[data_len + 2] = (c & 0xFF) ^ (ftype & 0xFF)

(3) Per-carrier Reed–Solomon. Systematic RS over count+data+CRC (data_len+3 bytes) appends rs_len parity bytes (rs_encode). The codeword is data_len+3+rs_len bytes; §7.3 gives the generator.

(4) Modulate.

  • PSK/QAM (mod_psk): the concatenated per-carrier codewords are re-split by carrier; each carrier's bytes become differential symbols (_psk_symbols) and are prefixed with a phase-0 reference symbol. Symbol s on carrier at freq is _PSK_ENV · sin((sym & 7)·π/4 + 2π·freq·k/SR), halved in amplitude if the 16QAM magnitude bit (sym >> 3) is set; all carriers are summed and the sum divided by √numcar.
  • 4FSK (mod_4fsk): the concatenated codeword bytes become dibits; each dibit keys one of the four tones for _FSK_SPS[baud] samples, sign alternating per symbol.

The differential symbol mapping (_psk_symbols) is worth spelling out. Payload bits are consumed bps at a time (MS-first); the value byt advances a running phase accumulator on the 8-phase grid, and for 16QAM its top bit rides along as the magnitude flag:

phase = (phase + byt * symset) & 7
syms.append(phase + (byt & 0x08 if is_qam else 0))

So for 4PSK each 2-bit byt ∈ {0,1,2,3} steps the phase by {0,2,4,6}·(π/4); for 8PSK each 3-bit byt steps by {0..7}·(π/4); for 16QAM the low 3 bits step the phase and bit 3 sets half magnitude.

(5) Prepend leader + header. mod_frame concatenates leader ‖ frame-type wave ‖ body. A control frame (no body) uses encode_control_frame = leader ‖ frame-type wave.

(6) Rate-convert for the radio. ArdopModem.transmit linearly interpolates 12 kHz → 48 kHz, normalizes to 0.7 full scale, and packs little-endian s16 (_to_48k).

6. Decode / RX, step by step

ArdopModem.feed(pcm48k) streams received audio in; decoded (frame_type, data) frames come back through the on_frame callback (ardop_native.py).

6.1 Decimate & buffer

Incoming 48 kHz s16 is decimated to 12 kHz by averaging groups of four (_decimate_48_to_12) and appended to a running buffer.

6.2 Leader detection (normalized cross-correlation)

_scan searches a bounded 32 000-sample head window (W) for the leader. It computes the cross-correlation by FFT (O(n log n)) and a running-sum energy, forming a normalized cross-correlation (NCC) that peaks at 1.0 on a perfectly-aligned full leader:

full = irfft(rfft(seg, nfft) * rfft(lead[::-1], nfft), nfft)
corr = full[L-1:n]                                   # 'valid' cross-correlation
cs   = concatenate([[0.0], cumsum(seg*seg)])
energy = cs[L:n+1] - cs[:n-L+1]
ncc  = abs(corr) / sqrt(maximum(energy, 1e-9) * lead_e)
cand = where(ncc >= 0.8)[0]                          # leftmost strong peak

The 0.8 threshold rejects the ~0.5–0.6 shoulders from partial overlap; the leftmost candidate is refined to the local NCC peak within one symbol. If no candidate is found the leading junk/silence is drained and the idle buffer is capped. Frames are processed leftmost-first, trimming the buffer past each so every iteration works on a fresh alignment.

6.3 Read the frame type

_read_frame_type demodulates the 10 frame-type 4-FSK symbols (50 baud) that follow the leader, reconstructs both byte copies, and accepts the type only if both bytes agree and both parity symbols check:

b0 = (syms[0]<<6)|(syms[1]<<4)|(syms[2]<<2)|syms[3]
b1 = (syms[5]<<6)|(syms[6]<<4)|(syms[7]<<2)|syms[8]
if b0 == b1 and syms[4] == _type_parity(b0) and syms[9] == _type_parity(b1):
    return b0, body_start

6.4 Dispatch on type

frame_info(ftype) looks the type up in the data-mode table (masking off the even/odd sequencing bit, base = ftype & 0xFE). Two outcomes:

  • Control frame (frame_info is None): a recognized control/ACK/NAK type is emitted with data=None, but only after a hard gate — NCC ≥ 0.85 and the type is a known control frame, an ACK (≥ 0xE0) or a NAK (≤ 0x1F) — because control frames have no CRC to catch a false positive. The leader+header is consumed and scanning continues.

  • Data frame: the body length is computed from the mode (per·8/bps symbols per carrier, +1 reference symbol for PSK/QAM), and the body is decoded.

6.5 Demodulate & FEC-correct the body

decode_data_frame demodulates the body and, per carrier, RS-corrects then CRC-checks; if any carrier fails, the whole frame is rejected (→ the ARQ layer will NAK):

  • PSK/QAM demod (demod_psk): each carrier is matched-filtered against _PSK_ENV · exp(−j2π·freq·k/SR); symbol phase is quantized to the 8-point grid (with a +π/2 correction for the sin-vs-exp reference offset) and magnitude recorded. The reference symbol (index 0) fixes full-scale magnitude; each subsequent symbol is decoded differentially (inc = (phase[si] − phase[si−1]) & 7; byt = inc // symset), and for 16QAM a magnitude below 0.7·full sets the half-magnitude bit. Bits are packed MS-first into exactly bpc bytes per carrier.

  • 4FSK demod (demod_4fsk): argmax-magnitude tone per symbol → dibits → bytes.

  • Per-carrier correction (decode_data_carrier): rs_decode fixes up to rs_len/2 byte errors, then the frame-type-keyed CRC-16 is verified; on any mismatch the carrier (and frame) is rejected.

To absorb sub-sample timing jitter left by the NCC peak, the streaming decoder retries the body decode at symbol-start offsets 0, −1, +1, −2, +2, −3, +3 samples and takes the first CRC-valid result.

6.6 Reed–Solomon decoding

rs_decode is a textbook syndrome decoder over GF(2⁸): compute syndromes at α¹…α^rslen (_syndromes), and if any is nonzero run Berlekamp–Massey for the error locator (_error_locator), Chien search for the error positions (_find_errors), and Forney for the error magnitudes (_correct). A re-check that the corrected word has zero syndrome guards against a mis-correction beyond capacity. It corrects up to rslen/2 byte errors and interoperates with the reference decoder by construction.

6.7 One-shot decode

For non-streaming use, decode_frame_auto(audio) does the whole thing on a single buffer: NCC-locate the leader, read the type, and decode by the registry — returning (frame_type, data|None).

7. Constants & tables

7.1 The frame-type table

Data modes (_DATA_MODES, ardop_native.py). The key is the even frame type; +1 is the odd sequencing variant used by ARQ. The tuple is (modulation, carriers, baud, data_len, rs_len) — the last two per carrier:

_DATA_MODES = {
    0x40: ("4PSK", 1, 100, 64, 32), 0x42: ("4PSK", 1, 100, 16, 8),
    0x44: ("8PSK", 1, 100, 108, 36), 0x46: ("16QAM", 1, 100, 128, 64),
    0x48: ("4FSK", 1, 50, 16, 4), 0x4A: ("4FSK", 1, 100, 64, 16),
    0x4C: ("4FSK", 1, 100, 32, 8),
    0x50: ("4PSK", 2, 100, 64, 32), 0x52: ("8PSK", 2, 100, 108, 36),
    0x54: ("16QAM", 2, 100, 128, 64),
    0x60: ("4PSK", 4, 100, 64, 32), 0x62: ("8PSK", 4, 100, 108, 36),
    0x64: ("16QAM", 4, 100, 128, 64),
    0x70: ("4PSK", 8, 100, 64, 32), 0x72: ("8PSK", 8, 100, 108, 36),
    0x74: ("16QAM", 8, 100, 128, 64),
}

Control frames (CONTROL_FRAMES) — these carry no data body:

CONTROL_FRAMES = {
    0x23: "BREAK", 0x24: "IDLE", 0x29: "DISC", 0x2C: "END", 0x30: "IDFRAME",
    0x31: "ConReq200M", 0x32: "ConReq500M", 0x33: "ConReq1000M", 0x34: "ConReq2000M",
    0x35: "ConReq200F", 0x36: "ConReq500F", 0x37: "ConReq1000F", 0x38: "ConReq2000F",
    0x39: "ConAck200", 0x3A: "ConAck500", 0x3B: "ConAck1000", 0x3C: "ConAck2000",
    0x3D: "PINGACK", 0x3E: "PING",
}

Reserved ranges (frame_name): frame types 0xE0–0xFF are DataACK (the low 5 bits ftype−0xE0 carry a link-quality estimate 0–31), and 0x00–0x1F are DataNAK. frame_name renders a data mode as e.g. "16QAM.8CAR".

7.2 CRC-16 (frame-type keyed)

ardopcf's GenCRC16: MSB-first, initial register 0xFFFF, polynomial constant 0x8810, with each data bit shifted into the register LSB and the polynomial XORed when the pre-shift MSB was set (crc16):

def crc16(data):
    reg, poly = 0xFFFF, 0x8810
    for byte in data:
        mask = 0x80
        for _ in range(8):
            bit = byte & mask
            mask >>= 1
            msb = reg & 0x8000
            reg = ((reg << 1) | (1 if bit else 0)) & 0xFFFF
            if msb:
                reg ^= poly
    return reg

The result is stored MS byte verbatim and LS byte XORed with the frame type (append_crc16; check_crc16), which is what binds a payload to its declared mode. Reproduction check values from this implementation:

input crc16
b"123456789" 0x0E59
b"ARDOP frame payload" 0x4C24

This is ardopcf's own CRC, not standard CRC-16-CCITT. Although the polynomial has the CCITT-style algebraic shape x¹⁶+x¹²+x⁵+1, the literal constant that reproduces ardopcf's GenCRC16 bit-for-bit is 0x8810 with the MSB-test / LSB-feed loop above — transcribe that, and do not substitute the standard CRC-16-CCITT polynomial 0x1021.

7.3 Reed–Solomon (GF(2⁸), systematic)

The field is GF(2⁸) with primitive polynomial 0x11D (x⁸+x⁴+x³+x²+1, RFC 5510) and generator element α = 2 (_GF_POLY; exp/log tables). The code is systematic with first consecutive root α¹ (_FCR = 1) and generator roots α¹ … α^rslen. The GF(2⁸) antilog/log tables are the standard α = 2 LFSR reduced modulo 0x11D, and the generator polynomial is then the product g(x) = ∏_{i=1}^{rslen}(x − αⁱ), built with an explicit GF(2⁸) multiply-accumulate loop (_rs_generator):

_GF_POLY = 0x11D                        # x^8 + x^4 + x^3 + x^2 + 1
_EXP, _LOG, x = [0] * 512, [0] * 256, 1
for i in range(255):                    # antilog (_EXP) + log (_LOG), alpha = 2
    _EXP[i], _LOG[x] = x, i
    x <<= 1
    if x & 0x100:
        x ^= _GF_POLY
for i in range(255, 512):               # duplicated tail so _EXP[a+b] never wraps
    _EXP[i] = _EXP[i - 255]

def _gmul(a, b):                        # GF(2^8) multiply via the log tables
    return 0 if (a == 0 or b == 0) else _EXP[_LOG[a] + _LOG[b]]

def _rs_generator(rslen):
    g = [1]
    for i in range(rslen):
        root = _EXP[i + 1]              # alpha^(i+1)
        ng = [0] * (len(g) + 1)
        for j in range(len(g)):         # g(x) *= (x - alpha^(i+1))
            ng[j]     ^= _gmul(g[j], root)
            ng[j + 1] ^= g[j]
        g = ng
    return g[::-1]                      # leading coeff first (monic)

Generator polynomials (leading coefficient first, first six bytes) for the rs_len values actually used by the mode table — computed from the code:

rs_len corrects generator g(x), first coeffs (hex)
4 2 01 1E D8 E7 74 …
8 4 01 E3 2C B2 47 AC …
16 8 01 76 34 67 1F 68 …
32 16 01 E8 1D BD 32 8E …
36 18 01 38 37 22 B4 DB …
64 32 01 9F 28 AB 87 CD …

Each carrier's codeword is a shortened RS(n, n−rs_len) with n = data_len+3+rs_len (the standard 255-length code with the leading zero bytes dropped), so the decoder operates on the true codeword length.

7.4 Carrier / baud / constellation summary

mode bits/sym symset baud samples/sym carrier freqs
4PSK 2 2 100 120 per _PSK_CAR_IDX (§4.4)
8PSK 3 1 100 120 per _PSK_CAR_IDX
16QAM 4 1 100 120 per _PSK_CAR_IDX + half-mag bit
4FSK 2 – 50/100 240/120 1425/1475/1525/1575 (50) · 1350/1450/1550/1650 (100)

7.5 The leader / sync pattern (recap)

Two-tone 1475 + 1525 Hz, 50 baud, 12 symbols / 240 ms default, signs alternating with the last symbol inverted to mark sync. Detected by normalized FFT cross-correlation, threshold 0.8 (data) / 0.85 (control).

8. The ARQ state machine (ISS/IRS)

ArdopARQ (ardop_arq.py) is the connected-mode protocol: transport-agnostic (it emits (ftype, data) frames through on_tx and consumes them through on_frame), so it can be driven by the real modem or a direct frame loopback. States: DISC, ISS, IRS, IDLE, IRStoISS.

8.1 Frame-type constants

_CONREQ = {"200": 0x31, "500": 0x32, "1000": 0x33, "2000": 0x34}   # bandwidth Hz
_CONACK = {"200": 0x39, "500": 0x3A, "1000": 0x3B, "2000": 0x3C}
DISC, END, BREAK, IDLE = 0x29, 0x2C, 0x23, 0x24
_DATA_ACK, _DATA_NAK   = 0xE0, 0x00      # ACK 0xE0-0xFF (quality), NAK 0x00-0x1F
_LADDER = [0x42, 0x40, 0x44, 0x46, 0x50, 0x52, 0x54, 0x62, 0x64, 0x72, 0x74]

_LADDER is the gearshift ladder of PSK/QAM data modes, slow → fast: 4PSK/16B → 4PSK/64B → 8PSK → 16QAM → the 2-, 4-, 8-carrier variants. (_mode is an index into it.)

8.2 Protocol flow

   ISS (caller)                         IRS (answerer)
   ────────────                         ──────────────
   connect(target, bw):
     ── ConReq(bw), MYCALL ─────────────▶ become IRS
                          ◀───────────── ConAck(bw), MYCALL ──  connected
   send(data):
     ── DATA(even) ─────────────────────▶ deliver, remember parity
                          ◀───────────── DataACK (quality) ──
     ── DATA(odd) ──────────────────────▶ (dup? drop) deliver
                          ◀───────────── DataACK / DataNAK ──
       (NAK ⇒ repeat same frame; 3 clean ACKs ⇒ gearshift up; NAK ⇒ gearshift down)
   turnaround (IRS has data):
                          ◀───────────── BREAK ──
     ── IDLE ───────────────────────────▶ IRS becomes ISS
   disconnect():
     ── DISC ───────────────────────────▶
                          ◀───────────── END ──         both ⇒ DISC

8.3 The mechanics

  • Connect. connect(target, bw) sets state ISS and sends ConReq[bw] with the callsign as payload. The peer, on receiving 0x31–0x38, becomes IRS, adopts the caller's callsign, and replies ConAck[bw]. The caller, on ConAck (0x39–0x3C), transitions to connected ISS and begins sending.

  • Data segmentation. _send_next_data pulls data_len·numcar bytes from the outbox, zero-pads, tags the frame type with the current parity bit (ftype = _LADDER[_mode] | _odd), marks it in-flight, and transmits. The receiver (_on_data) delivers the payload only if its parity differs from the last delivered (dropping a duplicate retransmission) and always ACKs with full quality (_DATA_ACK | 0x1F).

  • ACK/NAK. On DataACK (≥ 0xE0) the ISS clears the in-flight frame, flips parity, and sends the next (_on_ack). On DataNAK (≤ 0x1F) it repeats the in-flight frame (_on_nak). A frame that fails to decode arrives as ftype=None; if the station is IRS it answers DataNAK to request a repeat (_on_bad_frame).

  • Gearshift. Three consecutive ACKs promote the mode one rung up _LADDER (_good >= 3); any NAK demotes one rung and resets the streak. This is the adaptive-rate behaviour.

  • Turnaround & teardown. A received BREAK puts the station in IDLE and it answers IDLE, ceding the channel so the peer can send. DISC is answered with END and both drop to DISC. tick() (the no-response timeout) repeats the in-flight frame and gives up after 5 retries.

9. The TNC host interface

ArdopTNC (ardop_tnc.py) wraps the modem + ARQ in the ardopcf TCP host protocol so Pat / Winlink Express drive the native stack unchanged. Two asyncio TCP servers: a command port (default 8515) and a data port (command+1 = 8516).

  • Command port — CR-terminated text commands in, text replies + async notifications out. Recognized commands (_dispatch) include VERSION (→ "VERSION 1.0.0.0-native"), MYCALL, GRIDSQUARE, PROTOCOLMODE, ARQBW/CALLBW, LISTEN, STATE, BUFFER (queued bytes), ARQCALL <target> (→ PENDING, then connect), DISCONNECT/ABORT, and a long list of accepted-but-minimal knobs (CWID, SENDID, ARQTIMEOUT, LEADER, …) so Pat's init sequence succeeds. Async notifications pushed to every client (_arq_event): PTT TRUE/FALSE, CONNECTED, DISCONNECTED, TARGET, NEWSTATE <state>, STATUS gearshift 0x….

  • Data port — length-prefixed framing: host→TNC is [len_hi][len_lo][payload]; TNC→host is [len_hi][len_lo][3-byte tag][data] with big-endian len = 3 + len(data) and tag "ARQ"/"FEC". An optional leading ARQ/FEC tag on host data is stripped before it is queued for TX (_on_host_data).

  • Half-duplex TX pump (_tx_worker): when the ARQ layer emits a frame, it is modulated to 48 kHz audio and queued; the pump keys PTT (set_ptt(True), notify PTT TRUE), plays the audio, holds for the audio duration + 50 ms tail, then unkeys. RX audio is muted while transmitting (feed_rx) — the essential half-duplex interlock.

10. Interoperability & validation

Three pytest modules exercise the stack from field arithmetic up to a full Winlink-style transfer.

test_ardop.py — PHY & FEC.

  • test_gf_field_sane, test_crc16_frame_type, test_rs_roundtrip_and_correction, test_rs_rejects_beyond_capacity — the GF(2⁸) exp/log inverse property, the frame-type-keyed CRC (a wrong frame type or a flipped byte fails the check), RS correcting exactly rslen/2 errors and refusing to mis-correct beyond capacity.
  • test_4fsk_modulation_roundtrip, test_psk_qam_multicarrier_roundtrip — clean and AWGN (10 dB 4FSK, 20 dB PSK) round-trips of every constellation × carrier count.
  • test_data_carrier_fec, test_data_frame_roundtrip — the per-carrier [count|data|CRC|RS] block corrects 4 injected byte errors and rejects a wrong frame type; full data frames round-trip through 20 dB AWGN.
  • test_full_frame_in_stream, test_streaming_modem_*, test_frame_type_registry_roundtrip — leader + header + body found by matched-filter sync in padded/noisy streams; the streaming ArdopModem recovers every control frame and every data mode (including the 4FSK modes 0x48/0x4A/0x4C), and several mixed-mode frames back-to-back in one stream.

test_ardop_arq.py — the ARQ machine. test_connect_send_disconnect (a multi-frame message delivered exactly), test_gearshift_up_on_clean_run (upshift after a clean run), test_nak_retransmit_recovers (a frame dropped in flight is NAKed, repeated, and the message arrives intact), and test_duplicate_frame_delivered_once (an even/odd retransmission is delivered only once).

test_ardop_tnc.py — host protocol & the real RF path. test_command_protocol and test_data_port_framing validate the text commands and the length-prefixed/tagged data framing; test_arqcall_notifies_and_connects checks the ARQCALL → PENDING/TARGET flow; and test_arq_connect_transfer_disconnect_over_modem runs a full connect + multi- frame data transfer + disconnect through two real ArdopModem instances over audio loopback — b"Winlink over native ARDOP! " × 4 delivered byte-exact.

Interop vs ardopcf. The design contract is bit-for-bit compatibility with the reference: the field polynomial 0x11D/α=2, the RS root convention (α¹…), the GenCRC16 constant 0x8810 with the frame-type-keyed low byte, the two-tone leader frequencies, the 4-FSK tone plans, the PSK carrier grid, and the _DATA_MODES/CONTROL_FRAMES type values are all transcribed from ardopcf's FrameInfo, GeneratePSKTemplates, Calc1CarPSKSymbols and GenCRC16. The TNC speaks ardopcf's host protocol on ports 8515/8516, so an unmodified Pat client connects to it as though it were the daemon. app/radio/ardop.py (the legacy subprocess bridge) remains as the other end of the interop reference.

11. Limitations and future work

Honest gaps in the native implementation, roughly in order of impact:

  • No true bit/symbol interleaver. The payload is split across carriers and each carrier is an independent RS block, which is excellent against narrowband fades (one carrier fails alone) but the reference's within-carrier interleaving would add resilience to time-selective fading. RS's byte-burst tolerance covers much of this, but not all.

  • Timing recovery is coarse. Sync is the NCC leader peak plus a ±3-sample body jitter search; there is no per-symbol tracking loop, so very long frames or a large sample-clock offset between stations could walk off the symbol grid. In practice the 120/240-sample symbols and short frames keep this comfortable.

  • PSK carrier-frequency offset. Differential detection tolerates a static tone offset only up to a fraction of the 100 baud symbol rate; there is no automatic frequency correction. A large dial-frequency mismatch will fail decode rather than degrade gracefully.

  • Simplified gearshift & link quality. ACKs always report full quality (0x1F); the up/down logic is a plain 3-ACK / 1-NAK counter rather than the reference's SNR/quality-weighted mode selection. Good enough to demonstrate adaptivity, not yet tuned for a real band.

  • ARQ timing is caller-driven. tick()/retry cadence and the busy-channel detection (BUSYDET/BUSYBLOCK are accepted-but-ignored knobs) are left to the host; there is no in-modem busy-channel avoidance.

  • PING/IDFRAME/PINGACK are table entries only. The type values exist (0x3D/0x3E/0x30) but the ID/ping exchanges are not yet driven by the ARQ layer.

None of these change the architecture; they are refinements on a working, interoperable base.

12. Implementation & reproduction

file role
app/radio/ardop_native.py modem/PHY: CRC-16, RS/GF(2⁸), 4FSK, PSK/QAM, leader+header, frame table, ArdopModem
app/radio/ardop_arq.py ISS/IRS ARQ state machine, gearshift, segmentation
app/radio/ardop_tnc.py ardopcf-compatible TCP host/TNC (ports 8515/8516)
app/radio/ardop.py legacy bridge that drives the external ardopcf subprocess
test_ardop.py, test_ardop_arq.py, test_ardop_tnc.py the test suite (§8, §10)

Runtime API sketch:

from app.radio import ardop_native as A

m = A.ArdopModem(on_frame=lambda ft, data: ...)   # RX callback
m.feed(pcm48k_bytes)                               # stream RX audio in
audio48 = m.transmit(0x54, payload_256_bytes)      # TX one 16QAM/2-carrier frame
ctrl48  = m.transmit_control(0x31)                 # TX a ConReq200M control frame

ft, data = A.decode_frame_auto(audio12k)           # one-shot decode

Everything at runtime is NumPy; the constant tables (field polynomial, tone plans, carrier grid, _DATA_MODES) are protocol facts transcribed once from ardopcf, exactly as the LoRa/Meshtastic constant tables were.

13. References

  1. Rick Muething KN6KB, ARDOP — Amateur Radio Digital Open Protocol specification — the protocol the mode tables, frame types and waveform plan follow.
  2. ardopcf (John Wiseman G8BPQ, Peter LaRue AI7YN, from KN6KB's ARDOP_Win) — the MIT-licensed reference C implementation this is a clean-room port of; GenCRC16/GenCRC16FrameType, FrameInfo, GeneratePSKTemplates, Calc1CarPSKSymbols, EncodePSKData/EncodeFSKData, and the TCP host interface.
  3. RFC 5510, Reed–Solomon Forward Error Correction (FEC) Schemes — the GF(2⁸) primitive polynomial 0x11D convention used for the RS coder.
  4. S. B. Wicker, V. K. Bhargava, Reed–Solomon Codes and Their Applications — the Berlekamp–Massey / Chien / Forney decoding chain of §6.6.
  5. Winlink / Pat — the ARQ email-over-radio clients that drive the TNC host interface (§9).