Firmware for a Remote Controller Based on ESP32

About This

This project consists in creating a firmware modem-like for a custom device based on an ESP32-S3-WROOM-1-N16R8V module. The designed device functions as a remote controller for the main burner control system, communicating through digital I/Os and an infrared UART connection.

The firmware is built on the ESP-IDF framework, leveraging FreeRTOS as the operating system to provide a stable and scalable environment for device functionality.

One of the core components of the project is a sophisticated state machine, managing rules, commands, and system interactions. Thanks to this approach, the device is capable of:

• Receiving and processing commands through various interfaces, such as web server, USB, MQTT, and UART.

• Managing Wi-Fi connections for remote controls and updates.

• Integrating Bluetooth Low Energy (BLE) for automatic detection of registered devices and activation of specific rules.

This project includes a web server, enabling system monitoring and configuration through an intuitive interface accessible from any browser.

Key endpoints include:

• Secure login for protected access.

• Monitoring the device’s inputs and outputs.

• Sending customized commands.

• Viewing Wi-Fi, BLE, MQTT, UARTS configurations.

• Accessing status and configuration information of the burner system.

This web interface makes the device user-friendly, reducing setup and management time for end users. The web pages are retrieved from an sdcard.

Commands

Every command has its own syntax, but they always follow the pattern AT#<command_name>,<data>.

Basic Commands

• Output command

• Play tone command

• Analog output command: PWM

• Wait command: Delay in ms

• Send to COM port command: Communicate with the UART.

• HTTP request command: Send HTTP requests, supporting both GET and POST methods.

• Publish MQTT message command

• Send UDP message command

• Send TCP message command

• Append to BLE List command: Adds a new UID/name pair to the BLE observer list.

• List BLE command

• Delete from BLE List command

• Firmware Upgrade command: (not yet implemented)

Configuration Commands

Configuration commands are used to set parameters of the device’s features (Wi-Fi, BLE, MQTT, etc.), and are permanently stored in the NVS. For instance: - Wi-Fi access point configuration. - BLE settings. - MQTT settings. - UARTs settings. - Web server password and hostname configuration.

Rules Commands

• READ RULE

• EDIT/DELETE RULE

Command response

Responses are sent using a command_response function, that sends the response based on where it is needed. For example if the command was sent though usb, the response will be on the usb. Same for the web server. To do that, the command response has a context where is specified where the function to be used for the specific interface.

Web Server

The WebServer allows the presentation of predefined or customized content via a browser. It starts when: - Wi-Fi connectivity is activated. - The SD card containing the httpdocs folder is found.

Main Endpoints

• Login (POST): Authenticates users attempting to access the server.

• Input/Outputs (GET): Retrieves the current status of the onboard inputs and outputs.

• Send Commands (POST): Sends commands directly to the device.

• Message Area (GET): Returns the contents of the message area (up to 1024 bytes).

• Wi-Fi Settings (GET): Returns the current Wi-Fi configuration.

• BLE Settings (GET): Returns BLE configuration and registered devices.

• MQTT Settings (GET): Retrieves MQTT client settings.

• Burner Controller Status (GET): Extended status information from the infrared port.

• Burner Controller History (GET): Logs or records of past events.

• Burner Controller Configuration (GET): Returns the device configuration.

Project Structure

The code is split into several libraries:

1. Util: Contains shared utility functions.

2. HAL: Hardware-dependent code.

3. Lib: Hardware-independent code utilizing HAL interfaces.

State Machine and Event Handler

The project is based on a state machine organized through a global context static struct (sm_context), which includes: - Event loop: Managed by ESP-IDF. - Procedure queue: Contains parsed procedures. - Actual state: Updated during execution. - Rule database: Memorizes active rules to handle events.

Rule database

Rules are stored in the database for event matching and command execution. They can be:

• Edited by index

• Read by index

• Matched against events

Parser

Commands are parsed using flex (lexical analysis) and bison (syntactical analysis).

All commands have the prefix AT# followed by the command type.

• There can be a number of CONFIG_PROC_MAX_CMDS commands that can be parsed from a single string.

• Commands can have a maximum of MAX_ARGS arguments.

• Each command is parsed and stored into a struct.

• The struct contains the number of arguments of the command and an array of arguments of the command.

Furthermore, the parsing of the argument is done leveraging argument descriptors:

• ARGUMENT_DESCRIPTOR_STRING

• ARGUMENT_DESCRIPTOR_STRING_OPTIONAL

• ARGUMENT_DESCRIPTOR_ENUM

• ARGUMENT_DESCRIPTOR_ENUM_OPTIONAL

• ARGUMENT_DESCRIPTOR_INT

• ARGUMENT_DESCRIPTOR_INT_OPTIONAL

• ARGUMENT_DESCRIPTOR_IP

• ARGUMENT_DESCRIPTOR_IP_OPTIONAL

Procedure Queue

Commands are executed sequentially in a FIFO queue. Until the first command finishes, the next ones are queued. The procedure queue has a limited number of procedures it can schedule.

MQTT

The protocol used is the MQTT5 one. The device is able to connect to a broker through a command and publish on topics. In this way the device can exchange real-time data with centralized management systems.

Bluetooth

The configuration of the bluetooth is done using the esp-idf library that will start a ble thread. In this project we chose to use the NimBLE stack.

Through the BLE command we can swicth-on the bluetooth of the device and filter the result selecting a minimum required signal that means that we can set the further signal in which searching for the ble devices in the LBLE list.

When a device is found in the selected range the ble rule is triggered, such as proximity-based automation.

The devices to detect must be beacons, iBeacon or Eddystone, with uuids of 16 bytes. In the case of Eddystone the uuid in the list must match the namespace ID + instance ID of the beacon (32 bits in total). iBeacon has already an uid of 32 bits.

Quality Assurance

To ensure firmware reliability, were employed:

• Unit Testing using the Unity library

• Use of mocks to simulate hardware during tests (the ones provided by espressif).

• Valgrind to identify memory leaks.

• Code Coverage monitored with tools like LCOV, enabling us to achieve high coverage for core functionalities.

• Use of Gerrit and Jenkins pipelines.

Unit Testing

Tests are implemented using the Unity library and ESP-IDF mocks. CMock is used for hardware-specific mock generation.

Code Coverage

lcov is used for monitoring code coverage and integrates with VSCode during local development.

In visual studio the coverage gutters plug-in is used:

This shows at the bottom that 62% of the playtone file (buzzer) is tested. In green the covered parts, in red not covered.

This show the jenkins code coverage in CI using coverage plugin

Memory Leak Detection

Valgrind is used to detect memory leaks.

valgrind --leak-check=full --xml=yes --xml-file="$(PROJECT_TEST_BUILD_DIR)/memcheck.xml"
         "$(PROJECT_TEST_BUILD_DIR)/test_***_host.elf"

Valgrind checks on the terminal (summary):

Valgrind jenkins results:

Gerrit and Jenkins

The development is done using Gerrit and Jenkins together for code review and automated CI.

This can be summarized in 4 steps:

1. Developer Submits Patch: - Developer writes code, commits changes, and uploads a patch to Gerrit.

2. Code Review in Gerrit:

   • Reviewers inspect, comment, and approve (+1/+2) or request changes.

   • Developer updates the patch if needed.

3. Jenkins Validates Patch:

   • Gerrit triggers Jenkins to run automated tests (in this project: build, unit tests, documentation, c-lang checks and valgrind checks).

   • Jenkins reports results back to Gerrit as Verified +1 (pass) or Verified -1 (fail).

4. Merge Patch:

   • If approved (Code-Review +2 and Verified +1), the patch is merged into the target branch.

Key Benefits

• Quality: Automated testing ensures reliability.

• Efficiency: Collaboration in Gerrit with CI/CD via Jenkins streamlines the process.