Building the Firmware¶
Procedure for compiling and flashing the regulator firmware from source with the Arduino IDE. The build is a standard Arduino sketch; the items that require attention are the board settings and the library set, documented below.
Prerequisites¶
- Arduino IDE 2.x with the Espressif board package installed (Boards Manager: "esp32 by Espressif Systems").
- USB-C cable for the first flash.
- Target hardware: ESP32-S3 module with octal-interface external RAM (OPI PSRAM) and 16 MB flash (ESP32-S3-WROOM-1U-N16R8). The firmware allocates all large buffers in external RAM (
ps_malloc()throughout), so a board without OPI PSRAM will compile but fail at runtime.
Getting the Source¶
Clone the public mirror:
git clone https://github.com/markliquid1/Regulator2026-public.git Xregulator
The sketch folder must be named Xregulator; the Arduino IDE requires the folder name to match the main sketch file (Xregulator.ino). Rename the folder if it was cloned under the repository's default name.
Xregulator.ino holds the globals, setup(), and loop(). The numbered companion files (2_functions.ino through 7_functions.ino) hold supporting functions and compile as tabs of the same sketch.
Board Settings¶
Arduino IDE Tools menu settings, as of this writing:
| Setting | Value |
|---|---|
| Board | ESP32S3 Dev Module |
| USB CDC On Boot | Enabled |
| PSRAM | OPI PSRAM |
| Flash Size | 16MB |
| CPU Frequency | 240 MHz |
| Partition Scheme | Custom |
Settings not listed remain at their defaults.
USB CDC On Boot routes Serial output to the USB-C connector. It defaults to Disabled; left that way, the upload succeeds but the serial monitor shows nothing.
The PSRAM setting is mandatory. With it wrong, or on a board without OPI PSRAM, external-RAM allocations fail at runtime and produce symptoms unrelated to memory.
The custom partition scheme is defined by a partitions.csv file in the sketch folder. See Distribution status below.
Required Libraries¶
Forked dependencies
Four libraries are forks or carry local modifications. Installing the same-named library from the Arduino Library Manager installs the upstream version, which either fails to compile against this code or misbehaves at runtime. This is the most common build failure for this project. Check the table below before installing.
Modified or fork-dependent libraries¶
| Library | Function | Source | Status | What was changed |
|---|---|---|---|---|
| ESPAsyncWebServer | Serves the web dashboard | Community fork at github.com/ESP32Async/ESPAsyncWebServer (formerly under the mathieucarbou account); tested with 3.11.0 |
Fork required — the abandoned upstream is incompatible with current ESP32 cores | Nothing — used exactly as published by the ESP32Async fork |
| AsyncTCP | Network layer under the web server | Same community fork: github.com/ESP32Async/AsyncTCP; tested with 3.4.10 |
Fork required, plus one local edit | In AsyncTCP.h, the default network-task core assignment (CONFIG_ASYNC_TCP_RUNNING_CORE) is changed from -1 (any core) to 0, pinning the network task to core 0. Without the pin, the network task can preempt the alternator control loop on core 1 mid-sensor-read, producing 30–62 ms control stalls. Command-line builds can pass -DCONFIG_ASYNC_TCP_RUNNING_CORE=0 as a compiler flag instead; Arduino IDE builds do not support per-sketch flags and require the header edit. Re-apply after any library update. |
| PID_v1_xeng | Field-control PID loop | Project fork of Brett Beauregard's Arduino PID library v1.2.2 — see Distribution status | Fork required — the firmware calls methods that do not exist in the stock library | Actuator-aware tracking anti-windup: TrackAppliedOutput() reports the post-governor applied output back to the PID, ResetIntegratorTo() provides bumpless mode transfers, SetTrackingGain() / EnableTracking() configure the tracking loop, and accessors expose the individual P/I/D terms and pre-clamp output. Header renamed PID_v1.h → PID_v1_xeng.h. Stock methods are unchanged. |
| BMP388_DEV | Barometric pressure and board temperature sensor | Patched copy of Martin Lindupp's BMP388_DEV — see Distribution status | Patched — do not substitute the upstream release | Hardened I2C error handling. The stock library does not check bus transaction results; the patched I2C layer (Device.cpp) checks every transaction, verifies the expected byte count arrived, drains partial data from the bus on failure, and returns status. The sensor driver (BMP388_DEV.cpp) propagates those failures, so a faulty bus read returns an error instead of invalid pressure/temperature values. |
Standard libraries¶
Install these as published; no modifications.
| Library | Function | Source |
|---|---|---|
| NMEA2000 | NMEA 2000 (marine CAN) message library | Timo Lappalainen, github.com/ttlappalainen/NMEA2000 |
| NMEA2000_twai | ESP32-S3 CAN transport for the library above | Svante Karlsson's TWAI-driver port (provides NMEA2000_esp32.h on the S3). Note: the older, similarly named NMEA2000_esp32 library targets the original ESP32 and does not work on the S3 |
| INA228 | Battery shunt monitor (voltage/current) | Rob Tillaart, github.com/RobTillaart/INA228 |
| ADS1115_lite | Auxiliary analog inputs | Terry Myers, github.com/terryjmyers/ADS1115-Lite |
| STM32duino LSM6DSOX | Six-axis motion sensor (IMU) | STMicroelectronics, github.com/stm32duino/LSM6DSOX |
| OneWire + DallasTemperature | DS18B20 temperature probes | Arduino Library Manager |
| VeDirectFrameHandler | Victron VE.Direct serial protocol | Arduino library based on Victron's reference implementation (GitHub / Library Manager: "VeDirectFrameHandler") |
| ArduinoJson | JSON parsing and serialization | Benoit Blanchon, Arduino Library Manager |
| TinyGPSPlus | NMEA 0183 GPS sentence parsing | Mikal Hart, Arduino Library Manager. Listed for completeness; this input path is not currently active |
The remaining includes — WiFi, WiFiClientSecure/mbedTLS, LittleFS, ESPmDNS, DNSServer, NVS, esp_ota_ops — ship with the Espressif board package and require no separate installation.
The Web Dashboard Bundle¶
The browser dashboard is part of the firmware project. Source files live in web_src/ (index.html, script.js, styles.css, and the uPlot charting files). They are not flashed as-is: each file is gzip-compressed and the compressed copies are written to the device filesystem (LittleFS). The web server serves the .gz versions transparently — a request for script.js is answered with script.js.gz and a gzip Content-Encoding header. Page references are unchanged.
Consequence: an edit to a file in web_src/ has no effect on the device until the compressed copies are regenerated and reflashed. Regeneration is a gzip pass over each web_src/ file into the staging folder the filesystem image is built from; there is no bundler or build system. Do not edit the compressed output folder directly — it is overwritten on every rebuild.
First Flash¶
With the board settings and libraries in place, the first flash is the standard Arduino flow: connect over USB-C, select the serial port, Upload. The serial monitor (baud rate is printed in the boot log) shows a boot sequence that includes a line confirming OPI PSRAM detection; verify that line on first boot.
USB uploads on a previously updated device
Production devices receive firmware through an over-the-air update system (see Over-the-Air Updates). After a device has taken one over-the-air update, the bootloader prefers the slot that update was written into: a USB upload can complete successfully, the device reboots, and the previous over-the-air firmware is still what runs. If a USB-flashed change does not appear to execute, this is the probable cause. Recovery is device-specific — email joe@xengineering.net.
Forcing programming mode¶
The ESP32 can typically be programmed over USB without pulling GPIO0 low — this fallback is rarely needed.
To force programming mode: short Cable 3 pin 8 (GPIO0, brown wire) to GND before powering up, then upload firmware via USB. To reboot into normal operation: release the short and ground Cable 4 pin 13 (RESET, blue/white wire) momentarily, or power-cycle the board. Cable numbering and wire colors are in Data Cables & Pinout.
Forked Libraries and Partition Table — Distribution Status¶
The patched library copies (PID_v1_xeng, BMP388_DEV) and the custom partition table (partitions.csv) are not yet packaged for public distribution. Packaging is planned. Until it lands, email joe@xengineering.net for early access to these files.