Over-the-air IoT device update delivery architecture

Table of contents:

Regular firmware updates for Internet of Things (IoT) devices are a crucial aspect of ensuring continuous product development, introducing new features, and addressing bugs. Over-the-air (OTA) updates provide a convenient and functional method, enabling seamless device updates across the entire fleet without user intervention. In this article, the 2Smart team shares their experience with IoT device OTA updates for products based on ESP32 and ESP8266 microcontrollers using the ESP-IDF (Espressif IoT Development Framework).

Mechanics of OTA Updates for IoT Devices Based on ESP Microcontrollers

Preparations

Using ESP-IDF, the developer needs to make the following preparations:

1. Enable the OTA update option when compiling the firmware.
2. Consider the memory layout rules for the firmware.

Here is an example of memory layout:

nvs,     	data, nvs, 		 ,    	0x4000,
otadata, 	data, ota, 		 ,    	0x2000,
phy_init,	data, phy, 		 ,    	0x1000,
ota_0,   	app,  ota_0,     ,    	0x300000,
ota_1,   	app,  ota_1,     ,    	0x300000

In addition to the technical preparations, the developer must devise the business logic for this type of update delivery, which will serve as a foundation for further work.

Mechanics of updating firmware on a device

Returning to the memory layout for firmware, in the context of the discussed topic, it is crucial to pay attention to the memory sections “ota_0” and “ota_1,” where firmware binaries are stored, as well as “otadata,” which contains information about which of the mentioned sections the bootloader should use when starting the device.

Here’s how it works:

1. During the initial device flashing, the main binary is written to the “ota_0” memory area.
2. When delivering a new firmware version over the air, the new binary is written to the “ota_1” memory area.
3. After completing the OTA update, the firmware updates the information in “otadata” for the bootloader, so that during the next start, the bootloader launches the firmware from the “ota_1” partition.
4. During the next update, the new binary is written to the “ota_0” memory area, then again to “ota_1,” and so on. These two memory areas alternate so that the device always retains two firmware versions – the current and the previous.

Additionally, in some projects, the firmware’s business logic may include checking whether the new firmware version is operational. If the check result is negative, a rollback to the previous firmware version stored in its memory area occurs.

Compilation results

After compiling the firmware, the developer obtains a minimum of four binaries:

1. bootloader.bin – responsible for loading and executing the main application code.
2. partition-table.bin – flash memory partition map.
3. ota_data_initial.bin – OTA helper binary which contains the metadata for bootloader which factory partition should be loaded during boot.
4. factory.bin – firmware binary.

These four binaries are always present and are mandatory when OTA update support is enabled in ESP-IDF. Depending on the project, the compilation result may vary, and the overall number of binaries could be greater.

Ways to Implement Over-the-Air Updates

The OTA update process can be implemented in several ways:

1. The device sends an HTTP request to a specific URL to check for the availability of a new firmware file and downloads it if available.
2. Upload the firmware file through the device’s web admin interface.

The above describes two basic approaches to delivering over-the-air updates. However, with an understanding of this process, developers can implement any custom OTA update procedure tailored to their specific requirements.

An example of a custom OTA update procedure is a case from the practical experience of the 2Smart team. One of our solution’s customers uses a hardware configuration where two devices are interconnected and communicate over a wired bus. In this setup, only the first device (device A) has internet access via Ethernet, while the second device (device B) is not connected to the global network.

To implement OTA updates on the device B, we use the device A as an HTTP proxy server. When initiating the OTA update, device A starts a Wi-Fi access point. The device B connects to it and sends an HTTP request to the A, which redirects it to the cloud and starts downloading the updated firmware file.

This way, even a device without direct internet access can receive firmware updates over the air.

Limitations of OTA Updates

Developers should keep in mind a crucial aspect when working on the development and initial release of firmware – OTA updates only affect the firmware binary, specifically the factory.bin. The other three binaries mentioned earlier (bootloader.bin, partition-table.bin, and ota_data_initial.bin) remain unchanged.

Updating the bootloader.bin file using the built-in ESP-IDF tools is not possible. However, it’s challenging to envision a scenario where updating this file is absolutely necessary, except in the case of upgrading the ESP-IDF version. Only in this situation might there be a case where certain functionality won’t work due to the use of an older version of the bootloader.

Although updating partition-table.bin using ESP-IDF’s built-in functions is possible, it is highly discouraged. The slightest mistake could turn the device into a brick, so it is recommended to consider all development plans for the firmware and account for all risks even in the initial stages of developing the first version of the firmware.

For instance, if the early firmware version doesn’t write any information to the FS (NVS or SPIFFS), but there are plans to implement this in the future, by the time the first version is released, the partition table should already contain the necessary memory layout for the file system.

Hence, right from the outset of the project, it’s crucial to allocate all available flash memory resources for the firmware. When designing the partition table for their projects, the 2Smart team relies on this cheat sheet by Preben Dyrholm.

Security When Delivering Updates Over the Air

Security during over-the-air (OTA) firmware updates is critical, as a malicious actor could potentially replace the firmware with unauthorized code (Man-in-the-middle attack). The minimum security measure against this risk is to use the HTTPS protocol for delivering updates. In cases where using self-signed certificates for device-to-cloud communication is not feasible, built-in x509 certificates, controlled within the ESP-IDF framework, can be employed.

If OTA updates are implemented through the device’s web admin interface, developers should consider an additional user authentication procedure to restrict access to the firmware update page.

Additional Functionality for OTA Updates

With a deep understanding of the mechanics behind over-the-air (OTA) firmware updates, developers can incorporate necessary additional functionality. An example of this is staging, implemented by the 2Smart team for devices connected to the 2Smart Cloud platform.

Staging firmware refers to a type of beta version of the firmware that users can receive while it is still in the development and testing stages. To receive such firmware versions, users must enable their delivery for a specific device through the mobile application. Thanks to this mechanism, developers, testers, or business owners can receive a firmware update in advance, test its performance in real-world conditions, and provide approval for release to all instances of the device.

Wrapping Up

While the principle of delivering firmware updates for Internet of Things devices over-the-air is simple and intuitive, its practical implementation requires a comprehensive approach to firmware development, considering the future evolution of the product and potential security risks. Additionally, in some non-standard cases, unconventional solutions may be needed to facilitate automatic update delivery.

If you want to learn more about how OTA updates work on the 2Smart Cloud platform, you can refer to the relevant documentation section. And if you are interested in assistance with developing your product or using the 2Smart Cloud platform and mobile application under a white-label model, you can schedule a meeting with our experts.