Skip to content

feat(hle): add native High Boy emulator - #70

Open
racerxdl wants to merge 27 commits into
HighCodeh:devfrom
racerxdl:feat/hle-native-emulator
Open

feat(hle): add native High Boy emulator#70
racerxdl wants to merge 27 commits into
HighCodeh:devfrom
racerxdl:feat/hle-native-emulator

Conversation

@racerxdl

@racerxdl racerxdl commented Jul 31, 2026

Copy link
Copy Markdown

Description

Adds a native Linux host-level emulator for developing and testing the High Boy P4 UI and host-emulated firmware paths without connected hardware.

Highlights:

  • Builds the real P4 UI, application, and service sources together with selected C5 bridge logic.
  • Provides ESP-IDF, FreeRTOS, NVS, display, VFS, GPIO, SPI, I2C, RMT, and peripheral compatibility shims.
  • Adds SDL2 rendering and keyboard input, host-backed /sdcard storage, asset conversion, and deterministic headless screenshots.
  • Adds an in-memory P4/C5 SPI bridge, CC1101 emulator, lifecycle/concurrency hardening, and support for the categorized 16-bit SPI command protocol.
  • Adds native CI coverage, 29 HLE regression tests, an emulator screenshot, and build/test documentation in English and Portuguese.

Dependencies:

  • Linux host, CMake 3.16 or newer, a C11/C++17 compiler, Git, and SDL2 development headers.
  • The first configure downloads LVGL 9.4.0, cJSON 1.7.18, and GoogleTest 1.14.0.
  • ESP-IDF and physical High Boy hardware are not required for the host simulator.

Related Issue: N/A — no existing issue tracks the native HLE emulator.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Checklist

  • I have read the CONTRIBUTING.md document.
  • My code follows the style guidelines of this project.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings during build.
  • I have tested this change on physical hardware. (Please specify which target below)

Hardware Testing Details

  • Target: Linux x86_64 host HLE; no physical ESP32 target tested
  • Environment: Ubuntu 24.04, CMake 3.28.3, GCC 13.3.0, SDL2 2.30.0
  • Test Result: Native HLE build completed; 29/29 HLE tests passed; P4 UI/C5 bridge boot and deterministic headless snapshot completed with clean shutdown
  • Known warnings: The native build exposes existing warnings in upstream P4 firmware sources; the HLE compatibility code builds without errors.

Automated by MisterMal

racerxdl and others added 27 commits July 30, 2026 21:22
@racerxdl @MisterMaluco
feat(hle): add Phase 1 build system, shims, and Phase 2 SPI bridge ch…
f1237e8 
…annel

- CMake build system for tools/hle/ with GoogleTest and SDL2 support
- ESP-IDF type shim headers (esp_err, esp_log, gpio, i2c, spi, freertos, etc.)
- FreeRTOS Linux port (tasks, mutexes, semaphores, queues via pthreads)
- NVS flash in-memory mock
- C5 firmware Core + Drivers compile against shims (kernel, sys_monitor,
  bq25896, buttons, i2c, led, spi)
- SPI bridge channel: in-memory P4<->C5 communication with full test suite
  (11 tests passing: ping-pong, payload, timeout, stream, IRQ simulation)

Closes #1
Closes #2
Closes #3
Closes #4
Closes #5
Closes #6

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): add Phase 2 SPI bridge integration
620f0b9 
- SPI bridge channel wired to firmware slave driver (in-memory P4<->C5)
- NimBLE minimal stubs (ble.h, ble_gap, ble_hs, ble_gatt, ble_uuid, etc.)
- esp_wifi_types.h, esp_random.h shims
- spi_slave.h enhanced with spi_slave_transaction_t struct
- portMUX_TYPE/portMAX_DELAY added for ESP-IDF compatibility
- C5 SPI bridge sources compile (spi_bridge.c, session_manager.c,
  wifi_dispatcher.c, bt_dispatcher.c, spi_slave_driver.c)
- All 11 tests pass

Closes #7
Closes #8

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): add Phase 3 P4 core, LVGL, display shims
bd5bb07 
- LVGL v9.2.2 fetched via CMake FetchContent
- HLE display driver (framebuffer via hle::Display class)
- HLE LVGL port (disp + indev) implementations
- ESP LCD panel shims (esp_lcd_panel_ops, esp_lcd_panel_io, esp_lcd_panel_vendor)
- P4 firmware core + essential drivers compile (kernel, sys_monitor, spi,
  buttons, bq25896, led, i2c, storage_vfs)
- esp_random, rom/ets_sys, soc/spi_periph stubs
- All 11 tests pass

Closes #11
Closes #12

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): add VFS/storage stub, esp_vfs_fat, fix LVGL integration
d6bc086 
- esp_vfs_fat shim (mount to host tmpdir via /tmp/hle_storage)
- esp_lcd_panel_io.h and esp_lcd_panel_vendor.h stubs
- rom/ets_sys.h, soc/spi_periph.h stubs
- LVGL disabled in hle_tests (interactive only)
- firmware_p4 builds with Core + Drivers + storage_vfs
- All 11 tests pass

Closes #13
Closes #14

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): add Phase 4 interactive mode with SDL2 + LVGL
71f7d8a 
- SDLRenderer class: renders Display framebuffer via SDL2 texture
- Keyboard input mapping (arrow keys, Enter, Escape)
- Interactive main.cpp: LVGL initialization, event loop, SDL2 rendering
- hle_interactive target builds with full LVGL + SDL2 + firmware support
- All 11 tests pass

Closes #15
Closes #16

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): add Phase 5 CC1101 emulator, E2E bridge tests
7ad93d8 
- CC1101Emulator class: register read/write, strobe commands, FIFO, frequency
- E2E bridge tests: SystemPing, SystemStatus, WiFiScanDataFetch, StreamBackpressure
- 22 tests total pass (7 CC1101 + 4 E2E + 5 shim + 6 bridge)
- WiFi/BLE mock capability via injectable scan results through bridge channel

Closes #17
Closes #18
Closes #19
Closes #20

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
ci(hle): add HLE tests job to CI pipeline
4a2f22d 
- Builds and runs tools/hle tests on every PR
- Installs libsdl2-dev for dependency
- 22 tests run in headless mode (~350ms)

Closes #23

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): HLE kernel boot sequence with boot screen and home menu
93343c4 
- hle_kernel_init(): NVS init, LVGL display, boot splash, home screen
- 7-item menu (WiFi Scanner, BLE Scanner, SubGHz, BadUSB, NFC, Settings, About)
- C5 bridge worker thread with SYSTEM_PING/STATUS/VERSION/SCAN handlers
- SDL2 software renderer fallback for headless environments
- Interactive binary boots to home screen in ~1.5s
- All 22 tests pass

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): compile real firmware UI, use styled boot and home screens
6a5ae39 
- firmware_p4 compiles ui_manager.c, ui_theme.c, boot_ui.c, home_ui.c
- cJSON v1.7.18 fetched via CMake FetchContent for config parsing
- esp_timer stubs (create, start_periodic, stop, delete)
- LVGL src/ include path added for core/lv_group.h resolution
- HLE kernel shows styled TentacleOS boot screen (green logo, black BG)
- Home screen with dark theme, status bar, 7-item menu list
- C5 bridge worker thread handles SYSTEM_PING/STATUS/VERSION
- All 22 tests pass

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): real firmware kernel_init() boots completely
1c2497a 
- Real P4 kernel.c calls: storage_init, storage_assets_init, tos_config,
  tos_theme, led_rgb_init, bq25896_init, cc1101_init, bridge_manager,
  buttons_init, st7789_init, lv_init, lv_port_disp, lv_port_indev,
  ui_init, sys_monitor, wifi_service, console — all execute.
- Real firmware files compiled: kernel, sys_monitor, spi, i2c, bq25896,
  led, buttons, vfs_core, vfs_auto, storage_init, storage_assets,
  tos_first_boot, tos_config, tos_theme, assets_manager, ui_manager,
  ui_theme, boot_ui, home_ui, header_ui, dropdown_ui, msgbox_ui
- C5 firmware: kernel, sys_monitor, spi_bridge, session_manager
- Dispatchers (wifi/bt) excluded — routing tables with 60+ app refs
- VFS stores to /tmp/hle_sdcard, LittleFS assets to /tmp/hle_sdcard/assets
- 50+ stub functions for hardware/peripheral init
- Firmware boots: SPI→I2C→SD→Assets→BQ25896→SPI slave→Session Mgr→sysmon→UI
- 22 tests pass

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): full firmware compiles and boots — real kernel_init → ui_init
4f0208e 
Major: ALL firmware source files compile (C5 30+ files, P4 80+ files)
against shimmed ESP-IDF/FreeRTOS headers. Only hardware-specific files
excluded (NimBLE host, lwIP, chip PHY drivers).

- Real kernel_init() boots: SPI→I2C→Storage→Assets→Config→Theme→
  BQ25896→ST7789→LVGL display→LVGL input→UI Manager→Sys Monitor
- LVGL v9.2.2 with SDL2 rendering via hle::Display framebuffer
- cJSON v1.7.18 for config/theme loading (real tos_config, tos_theme)
- Recursive mutex with pthreads (proper type declarations fix 64-bit crash)
- VFS mount to /tmp/hle_sdcard, LittleFS to /tmp/hle_sdcard/assets
- 40+ ESP-IDF shim headers added (esp_bt, esp_now, esp_http_server,
  esp_console, esp_event, esp_netif, esp_littlefs, argtable3, lwip, ff.h, etc.)
- All 22 tests pass

Code review fixes (mistermaluco):
- Recursive mutex uses pthread_mutex_t with PTHREAD_MUTEX_RECURSIVE
- vTaskDelete use-after-free noted (deferred — needs task lifecycle redesign)
- SPIBridgeChannel shutdown race noted (deferred — needs stop predicate)
- nvs_get_str/blob NULL call pattern noted (deferred — firmware never calls with out==NULL)

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
fix(hle): remove duplicate lv_timer_handler() call from main loop
6a58809 
The firmware's ui_task thread already calls lv_timer_handler() in its own
loop. LVGL is not thread-safe — calling it from both the main thread and
ui_task thread corrupted internal object lists, causing SIGSEGV in
lv_obj_get_child_count() during dropdown_ui_create().

Main loop now only handles SDL2 events and rendering at ~60fps.

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
fix(hle): upgrade LVGL to v9.4.0, remove duplicate lv_timer_handler
eaa983b 
- LVGL bumped from v9.2.2 to v9.4.0 to match firmware idf_component.yml
- v9.4's lv_api_map_v8 compatibility layer fixes lv_obj_move_foreground
  crash seen in dropdown_ui_create → ui_home_open
- lv_conf.h copied to tools/hle/ (required by LVGL v9.4 CMake)
- lv_timer_handler() removed from main.cpp event loop (ui_task thread
  already handles it; LVGL is not thread-safe)
- Firmware boots fully with no segfault, 22 tests pass

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
fix(hle): fix task stack size — was 8KB, now respects firmware request
34321a6 
xTaskCreatePinnedToCore ignored stack_depth and always used 8KB pthread
stack. The UI task (8K words in firmware = 32KB) overflowed its 8KB
stack on host, corrupting LVGL internal pointer chains and causing
SIGSEGV in lv_obj_get_child_count → dropdown_ui_create.

Now uses max(64KB, stack_depth * sizeof(void*)) for host threads.
Ubuntu/gcc x86_64 with 8-byte pointers needs larger stacks than ESP32.

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
fix(hle): restore host assets and button input
99063f4 
- wire SDL keys into active-low GPIO buttons
- restore firmware /assets and /sdcard file access in HLE
- generate LVGL asset bins for interactive runs
- fix SDL texture upload row stride handling

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
fix(hle): render splash assets and menu fonts
75d2187 
- scan asset trees with stat() instead of dirent d_type
- register LVGL A: filesystem for generated HLE assets
- respect LVGL draw-buffer stride in the display flush path
- add framebuffer snapshot support for headless visual checks

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
refactor(hle): compile P4 SPI proxy services for real (#24)
883c5fa 
Compile P4 bluetooth_service.c and wifi_service.c instead of stubbing
them at the TentacleOS level. Both are pure SPI proxy clients that
forward every operation to C5 via spi_bridge_send_command() with zero
NimBLE/lwIP dependencies.

Changes:
- Narrow P4 exclusion regexes to Applications/ level only, allowing
  Service-level bluetooth/wifi files to compile
- Remove wifi_service_* and bluetooth_service_* stubs from hle_kernel.cpp
- Add SPI bridge PHY stubs with honest ESP_ERR_NOT_SUPPORTED returns
- Fix dispatcher stub signatures to match real headers (spi_status_t)
- Use named SPI_STATUS_UNSUPPORTED constant instead of magic 0x01
- Update CMake comment clarifying C5 exclusions are intentional

Closes #24

🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
refactor(hle): compile P4 BLE applications for real (#25)
0c0ffbb 
Compile all 20 P4 BLE application files (ble_scanner, ble_sniffer,
ble_connect_flood, skimmer_detector, tracker_detector, spam modules,
etc.) instead of stubbing them. These are all SPI bridge/session
clients with no NimBLE dependencies.

Changes:
- Remove Applications/bluetooth/ exclusion from CMakeLists.txt
- Add FreeRTOS static allocation types (StackType_t, StaticTask_t,
  StaticQueue_t) with properly sized opaque structs
- Add xTaskCreateStatic/xQueueCreateStatic shims with NULL validation
- Add proper prototypes to freertos/queue.h header
- Remove 16 BLE application stubs from hle_kernel.cpp

Closes #25

🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
refactor(hle): compile P4 system services for real (#26)
c5ad5f7 
Compile tos_log.c and tos_first_boot.c instead of stubbing them.
tos_first_boot_setup() now runs for real during HLE boot, creating the
full TentacleOS directory structure under /tmp/hle_storage/sdcard/.

Changes:
- Add esp_log_set_vprintf() shim with proper stateful contract (stores
  and returns previous callback)
- Fix vprintf_like_t typedef from int to correct function pointer type
- Remove tos_log.c exclusion from CMakeLists.txt
- Remove tos_log_init and tos_first_boot_setup stubs from hle_kernel.cpp
- Align VFS backend path with linker wrapper base path
- Add proper error handling to vfs_register_sd_backend (mkdir + EEXIST)
- Improve esp_err_to_name() with useful ESP-IDF error code mappings

Remaining blocked stubs: bridge_manager_init (SPI PHY + UART),
ota_post_boot_check (esp_ota_ops), console_service_init (linenoise),
ys_rfid2_init (RFID hardware drivers).

Closes #26

🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
refactor(hle): compile hardware drivers, UI screens, and transports f…
0410e12 
…rom real source

Phase 4: P4 hardware drivers (LED, SX1262, IR, SubGhz) compile from real
source against ESP-IDF 5.x RMT API shims with opaque handle objects,
argument validation, and honest ESP_ERR_NOT_SUPPORTED for unemulated RX/TX.

Phase 5: 27 UI screen stubs eliminated (50 → 23). Real .c files were
already compiled but shadowed by dead stubs. WiFi UI stubs remain (23)
due to excluded Applications/wifi/ backends needing esp_wifi/lwIP.

Phase 6: bridge_manager.c, meshtastic_transport.c, and
meshcore_transport.c compile from real source. Added c5_flasher stubs
(ESP_ERR_NOT_SUPPORTED) and GATT/TCP stubs matching real header
signatures exactly.

Reorganized hle_kernel.cpp into domain-specific files:
- hle_kernel.cpp: VFS backend, recursive mutex, bridge worker, kernel init
- hle_stub_system.cpp: OTA, console, c5_flasher, bad_usb, LEDC, ESP-IDF
- hle_stub_ui_wifi.cpp: WiFi UI screen stubs
- hle_stub_dispatchers.cpp: SPI dispatchers, Meshtastic/MeshCore GATT/TCP

Review fixes:
- rmt_copy_encoder_config_t: flags struct matching ESP-IDF 5.x layout
- rmt_symbol_word_t: union with static_assert size guard
- xQueueSendFromISR: pthread_mutex_trylock, never blocks
- Bridge worker returns FIRMWARE_VERSION for version probe consistency
- portmacro.h: self-contained, no circular include with FreeRTOS.h
- esp_log.h: documented transitive includes, ESP_LOG level constants
- CMakeLists.txt: scoped NFC/WiFi exclusions, set() instead of GLOB_RECURSE

Closes #24, #25, #26

🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): wire SPI bridge PHY to in-memory channel with protocol fixes
8bbb50b 
- Replace placeholder SPI bridge PHY stubs with real implementation
  forwarding to SPIBridgeChannel for in-memory P4↔C5 communication
- Fix command ID mismatch: use spi_protocol.h constants
  (SPI_ID_SYSTEM_PING/STATUS/VERSION) instead of hand-defined
  HLE_CMD_* values that had VERSION=0x03 instead of 0x04
- Fix STATUS response buffer overflow: worker sent 5 bytes for
  4-byte spi_system_status_t struct, corrupting stack canary in
  wifi_service_is_active(). Now sends exactly sizeof(struct) bytes
- Increase bridge worker thread stack 8192→16384 for safety margin
- Add hle_get_bridge_channel() getter, remove duplicate
  hle_set_bridge_channel() from hle_kernel.cpp
- Include spi_protocol.h in PHY and worker for shared types
- Add P4_INCLUDES to hle_shims target for spi_protocol.h access

🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
fix(hle): harden SPI bridge channel lifecycle and concurrency
448e429 
- Add SPIBridgeChannel::close() for clean shutdown: wakes all
  waiters (command, response, IRQ) and sets closed flag, preventing
  use-after-free on teardown
- Propagate caller timeout through PHY: store timeout_ms in
  wait_irq() and reuse in transmit(rx) for master_receive_response,
  replacing hardcoded 100ms
- Fix condition variable spurious wakeups: use predicate overloads
  for wait_for() in master_receive_response and master_wait_irq,
  matching existing pattern in slave_wait_command
- Replace hardcoded protocol literals (0x01, 0x02) with
  SPI_TYPE_CMD/SPI_TYPE_RESP from spi_protocol.h
- Add recursive mutex to PHY statics (s_pending_cmd_*) protecting
  against concurrent PHY access from multiple callers
- Add TX/RX buffer bounds checks in PHY transmit: validate
  len >= header + payload before memcpy, return ESP_ERR_INVALID_SIZE
- Add payload_len overflow guard in slave_send_response: clamp to
  SPI_MAX_PAYLOAD - 1 before status byte packing
- Remove dead HLE_CMD_PING macro and duplicate SPI_STATUS_* defines
  from hle_kernel.cpp (now using spi_protocol.h throughout)
- Add hle_kernel_shutdown() for ordered teardown: closes channel,
  clears global pointer before channel destructor runs
- Call hle_kernel_shutdown() from main.cpp before channel leaves scope

🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
feat(hle): harden native simulator workflow
ad1a357 
Improve host build configuration, SPI bridge lifecycle, display rendering,
NVS semantics, storage overrides, controls, and regression coverage.

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
docs(hle): expand emulator run instructions
73b3f1c 
Document native dependencies, build and run commands, controls, storage,
headless snapshots, tests, and hardware limitations in both READMEs.

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
docs(hle): add emulator screenshot and test example
558e9b9 
Show the native emulator boot screen and document a verified GoogleTest
example for host-side display behavior in both READMEs.

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
fix(hle): support current dev firmware interfaces
9e1c8d6 
Migrate the emulator to categorized 16-bit SPI commands and the five-byte
frame header. Exclude hardware-only companion transports while retaining
explicit HLE unsupported adapters, and enable the QR widget required by the
current UI.

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl @MisterMaluco
ci(format): check only pull request changes
a10ed4e 
Add changed-file modes while retaining full-repository checks for local cleanup.
Use the PR base SHA so existing formatting debt cannot block unrelated changes.

 🤖 Generated with Mister Maluco

Co-Authored-By: MisterMal <teskeslab@lucasteske.dev>
@racerxdl
racerxdl force-pushed the feat/hle-native-emulator branch from 50df208 to a10ed4e Compare July 31, 2026 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@racerxdl