Blog:
Over-The-Air Updates | Choose the best Linux OS Image Update Model

Thursday, October 10, 2024

Introduction & Background

Keeping in-field devices up-to-date and secure can be challenging, especially in scenarios of limited connectivity or restricted data usage. This blog delves into various methods for deploying Over-the-Air (OTA) updates, focusing on Linux OS image update models.

In this blog, we’ll analyze the unique advantages of Torizon in challenging update scenarios, comparing its performance with other models based on insights from Toradex’s talk at EOSS 2024.

Comparing Linux OS Image OTA Update Models

Before we begin, let’s define a few concepts to help us better understand the benchmarks we discuss below.

Currently, two primary Linux OS Image update models are used in the embedded industry to perform OTA updates: Dual A-B Partitioning, and OSTree.

Dual A-B Partitioning

This model involves alternating between two partitions to activate new system versions post-update. It's robust, allowing devices to revert to a previous state if updates fail but it requires double the storage space.

In short, it’s reliable, but your device needs a lot of storage space, especially if you have some headroom in storage in case you extend the functionality of your software.

OSTree

OSTree uses a Git-like model for managing and deploying bootable filesystem trees, optimizing storage by only downloading new and modified files during an update. It's suitable for environments where connectivity is unstable.

Additionally, since you are only downloading a few files, this model is very time-efficient as well.

The main downside of OSTree is that it can be complex to set up. Fortunately, with Torizon, we have done that work for you.

In-Depth Comparison and Benchmarks

So how do you find the right Linux OS Image update model for your use-case? Trial, error, and comparison!

Toradex Technical Solutions Architect, Drew Moseley, presented a technical talk at EOSS 2024 comparing known update systems: Mender, RAUC, SWUpdate, and Torizon. The systems chosen either use Dual A-B Partitioning or OSTree.

If you're keen on viewing the complete presentation, you can watch the replay on the EOSS YouTube channel. But if you are short on time, continue reading to get the details.

Set Up and Methodology

For reproducibility and accuracy, let's understand the setup and methodology employed during Drew’s presentation at the EOSS 2024 conference.

Setup

The hardware setup included a Toradex Verdin i.MX8M Mini System on Module and an Ixora Carrier Board equipped with block-based eMMC storage. For local/standalone deployments, Drew used a local area network (GbE). For the initial installation, he used a Toradex Easy Installer image, and Torizon Cloud for Torizon deployments.

Based on customer feedback, Drew chose to recreate common update scenarios as follows:

  • One byte change in /etc (plus MENDER_ARTIFACT_NAME) - full image update: This is the baseline to determine how much benefit each update system gets from their delta update implementation.
  • One byte change in /etc (plus MENDER_ARTIFACT_NAME) - delta update(s): This gives us an absolute minimum update size for each updater.
  • Install chromium v100.0.4896.88 - delta update(s): This is probably not a reasonable use case since most designs would include chromium in the initial install.
  • Upgrade chromium to v101.0.4951.54 - delta update(s): This is a common use case requiring an update to a large package.
  • Uninstall chromium – delta update(s): This was done just to simplify the next step.
  • Upgrade to Torizon v6.5.0 repo manifest – delta update(s): This is also a common use case for our users updating to a newer version of Torizon.

Methodology

The overriding goal was to minimize the differences between the setups as much as possible.

Drew started with the standard Torizon repo manifest, DISTRO, etc., which generated an image that was approximately 350MB. Next, he hacked it to allow the disabling of OStree.

Once that was done, he added layers for the update systems and chromium, implementing verdin-imx8mm support on SWUpdate and RAUC. Then, he built the torizon-minimal image, triggered the update, and used tcpdump and wireshark to measure the conversation size.

After the initial install from Torizon v6.2.0 repo manifest using Toradex Easy Installer – no updates deployed – Drew proceeded with the test scenarios.

Update Systems Chosen

Here are the updaters that were used in the benchmark with a bit of detail about each one.

Mender

  • Dual A-B plus persistent data partition
  • Hosted or on-prem server
  • Standalone deployments
    • Used for benchmarks
  • Binary deltas using xdelta compression tool
    • Part of the commercial offering
    • Read-only unmodified partitions required
    • Default tuning values used for benchmarks

RAUC

  • Dual A-B plus persistent data partition
  • Compatible with Hawkbit deployment server
  • Standalone deployments
    • Used for benchmarks
  • casync for adaptive updates
    • Content-addressable data synchronization tool
    • Can be used on UBI
    • Verity bundle format used for benchmarks
    • Read-only not required
  • Block-based adaptive updates
    • 4kiB blocks hashed using SHA256
    • Read-only not required

SWUpdate

  • Dual A-B plus persistent data partition
  • Asymmetric updates supported
    • Not part of the benchmark
  • Compatible with Hawkbit deployment server
  • On-device web UI for deployments
    • Used for benchmarks
  • Zchunk algorithm used for adaptive updates
    • A file format designed for highly efficient deltas while maintaining good compression
    • Read-only not required

Torizon

  • Single partition managed with libostree
  • Torizon cloud deployment server (Uptane-compliant)
    • Local deployment using treehub is possible
  • Read-only required for libostree-managed components
    • /var is completely unmanaged
    • /etc is partially managed (3-way merge)
  • Adaptive updates: only new or modified objects need to be downloaded
  • Additional size reduction using “OStree static deltas”
    • Per object binary deltas
    • Significantly reduces the number of HTTP GET requests

Our Torizon.io website has even more information about our industrial Embedded Linux distro, but for this demo, the above should be enough.

Update Scenario Demos and Results

Now, it’s time for the exciting part – the results! We've illustrated the effectiveness of the OTA update systems under realistic conditions that Drew chose in the tables and charts:

Table 1 - Minor Updates: One Byte Change in /etc (plus MENDER_ARTIFACT_NAME)

System Full Image Update (MB) Delta Update(s) (MB)
Mender 117 0.227
RAUC casync 129 18
RAUC adaptive 129 28
SWUpdate 115 42
Torizon - Standard 12 12
Torizon - OSTree static delta N/A 0.09375

Table 2 - Major Software Deployment: Install and Upgrade Chromium – Delta Update(s)

System Install Chromium v100.0.4896.88 Upgrade to Chromium v101.0.4951.54
Mender 150 MB 59 MB
RAUC casync 151 MB 128 MB
RAUC adaptive 165 MB 147 MB
SWUpdate 162 MB 148 MB
Torizon - Standard 135 MB 117 MB
Torizon - OSTree static delta 113 MB 81 MB

Table 3 - System Reversion: Uninstall Chromium – Delta Update(s)

System Data Transferred
Mender 35 MB
RAUC casync 19 MB
RAUC adaptive 28 MB
SWUpdate 48 MB
Torizon - Standard 12 MB
Torizon - OSTree static delta 0.2 MB

Table 4 - Significant System Upgrade: Torizon v6.2.0 to v6.5.0 – Delta Update(s)

System Data Transferred
Mender 76 MB
RAUC casync 114 MB
RAUC adaptive 128 MB
SWUpdate 131 MB
Torizon - Standard 113 MB
Torizon - OSTree static delta 55 MB

This chart summarizes all the above data:

Chart depicting the results of all scenarios tested
Conclusion

Choosing the right Linux OS image update model is crucial for effective and efficient OTA updates, particularly when facing resource and connectivity limitations. Torizon not only addresses these challenges but also provides a robust, developer-friendly platform built on reliability, security, and efficiency.

We hope the above analysis successfully highlights Torizon’s strengths and advantages that make it a compelling choice for developers aiming for seamless integration and minimal disruption during OTA updates.

More Info/Resources:
?Have a Question ?