Skip to content

Coprocessor Interface Control Document (ICD)

This chapter specifies the hardware parameters, firmware sub-modules, and physical bus mappings required to implement the NKS4 coprocessor emulator using an ESP32-P4 microcontroller.

Coprocessor Hardware Specification

The target coprocessor platform must meet the following hardware requirements:

Subsystem Component Target Specification Functional Role
Microcontroller (MCU) ESP32-P4 (Dual-core RISC-V, 400 MHz) Handles real-time tasks, USB stack, and scanning
USB Controller On-chip USB 2.0 OTG (Device Mode) Emulates the NKS4 High-Speed composite device
Display Panel LCD with MIPI-DSI or RGB interface Renders the 800 × 600 RGB565 framebuffer
Touch Controller Capacitive Touch Overlay (I2C interface) Captures touch coordinates
Audio DAC ES8388 (or compatible I2S DAC) Renders the 6-channel 48kHz 24-bit output stream
Audio ADC ES8388 (or compatible I2S ADC) Captures 6-channel 48kHz 24-bit input audio
UART Interface 3.3V TTL UART Keybed serial link to Host COM port

Firmware Architecture Specification

The coprocessor firmware is divided into six concurrent sub-modules to manage real-time tasks:

                  ┌──────────────────────────────────────────┐
                  │          ESP32-P4 Firmware Core          │
                  └────┬────────────────────────────────┬────┘
                       │                                │
      ┌────────────────┴──────────────┐  ┌──────────────┴──────────────┐
      ▼                               ▼  ▼                             ▼
┌──────────┐                     ┌──────────┐                     ┌──────────┐
│ USB Core │                     │  Video   │                     │  Audio   │
│ (USBD)   │                     │ (RGB565) │                     │  (I2S)   │
└────┬─────┘                     └────┬─────┘                     └────┬─────┘
     │                                │                                │
     ├──► Interface 0 (Panel INT/OUT) ├──► Decode 0xC2/0xC6 frames     └──► 6-Channel DAC
     ├──► Interface 1 (Isochronous)   └──► Render to LCD               └──► 6-Channel ADC
     └──► Interface 2 (Accessory MIDI)
  1. USB Composite Stack: Implements a USB device stack emulating three co-resident interfaces (VID 0x0944, PID 0x1005, bInterfaceSubClass=0xFF).
  2. Panel Command Processor: Parses incoming 4-byte NKS4Command Bulk OUT frames (such as LED controls and scanning configurations) and generates Interrupt IN event payloads.
  3. Video Pipeline: Receives type 0xC2 headers and type 0xC6 pixel data blocks via Bulk OUT. Performs 32-bit word-level byte-swaps, translates RGB565 pixels, and writes to the LCD controller.
  4. Real-Time Audio Streamer: Buffers incoming isochronous OUT audio blocks and outputs them via I2S DMA. Simultaneously captures audio from the ADC and writes to the isochronous IN endpoint.
  5. HMI Event Generator: Translates touch coordinates and rotary encoder transitions into standard NKS4Command frames.
  6. NV2AC/Atmel Emulation Sub-module: Responds to register-read requests (0xE1) with correct acknowledgments and key payloads (see Atmel AT88SC0204CA Coprocessor Emulation Specification).

Minimum Viable Configuration for Booting

To pass the Host driver's initial probe and prevent boot deadlocks during development, the coprocessor must implement the following minimum subset:

  1. Descriptor Verification: Enumerate successfully as a USB High-Speed device with VID 0x0944, PID 0x1005, and bInterfaceSubClass=0xFF.
  2. Idle Polling: Return only the end-of-transfer marker ([0x00, 0x00, 0x00, 0x87]) on the Interrupt IN endpoint when no events are pending.
  3. Handshake Acknowledgment:
    • CommunicationCheck (0xEE) → Respond with [0x00, 0x00, 0x66, 0x00]
    • GetVersion (0xF0) → Respond with [0x21, 0x21, 0x70, 0x00]
    • ReadPortConfiguration (0xF1) → Respond with [0x04, 0x00, 0x71, 0x01]
  4. Bulk OUT Command Sink: Accept all other panel and video configuration commands silently without generating USB protocol stall errors.

Host Physical Interconnect Specification

The connection between the Host and the coprocessor uses standard physical and logical pathways:

Interconnect Host Interface Coprocessor Interface Physical Channel Operational Role
USB Link USB 2.0 Host Port USB Device Controller Type-A to Type-B Cable Panel commands, video, audio, MIDI
Serial Link COM Port (16550 UART) Hardware UART Interface 3-Wire Bus (TX / RX / GND) Keybed scanning and pedal MIDI data

Coprocessor Interconnect Topology

             Intel Atom Host Board                                ESP32-P4 Coprocessor
          ┌──────────────────────────┐                        ┌──────────────────────────┐
          │  Standard USB Host Port  │◄───────── USB ────────►│  High-Speed USB Device   │
          │                          │                        │                          │
          │  COM Port (16550 UART)   │◄──────── UART ────────►│  Hardware UART (3.3V)    │
          │  (RS-232 / TTL Pins)     │     (3.3V Signals)     │                          │
          └──────────────────────────┘                        └────────────┬─────────────┘
                                                                           │
                                                                           ▼
                                                              ┌──────────────────────────┐
                                                              │  Velocity Keybed Matrix  │
                                                              │  (61 or 88-Key Scanner)  │
                                                              └──────────────────────────┘
  • Baud Rate and Timing: The UART interface operates at a baud rate code of 0x18 (38,400 bps), 8 data bits, no parity, and 1 stop bit.
  • Voltage Level Shifting: If the Host motherboard outputs standard RS-232 voltage levels (\(\pm 12\text{V}\)), an intermediate transceiver (e.g., MAX3232) must be integrated to shift the signals to the coprocessor's \(3.3\text{V}\) TTL limits.

Keybed Serial Protocol Interface

Initial System Handshake

  1. Upon startup, the Host sends a single synchronization byte 0xA5 to probe the keybed interface.
  2. The coprocessor must capture 0xA5 and return a 3-byte response payload: [0xA0, VER_HI, VER_LO].
  3. The response must be returned within 21 milliseconds. If verified, the Host enables real-time MIDI frame decoding.

Real-Time MIDI Stream Format

Once the handshake completes, the coprocessor continuously scans the keyboard matrix, processes note velocity from dual-contact timing, and sends standard MIDI events to the Host over the UART serial link:

Key Press:   [0x90] [Note ID] [Velocity Value (1-127)]
Key Release: [0x80] [Note ID] [Velocity Value (0-127)]
Pressure:    [0xD0] [Aftertouch Value (0-127)]
Expression:  [0xB0] [CC Number] [CC Value (0-127)]

The Host's CSTGKeybedInterface::ReceiveMessage parses these incoming bytes using an internal lookup table to identify packet boundaries.