TFLM vs Ekkono

The goal of this study is to compare the performance of TensorFlow Lite for Microcontrollers (TFLM) and Ekkono.

Ekkono is only able to run FC models, so we have limited our comparison to FC models only.

Slightly Modified Models

Since Ekkono is not able to do classification, we have slightly changed some models to make them suitable for regression. The changes are minimal and should not have a noticeable impact on the results.

Board: Any NUCLEO-L4R5ZI RenesasRX65N

Models

FC parameters and MACs

Error

FC - NUCLEO-L4R5ZI

FC - RenesasRX65N

Execution Time

FC - NUCLEO-L4R5ZI

FC - RenesasRX65N

Flash Size

FC - NUCLEO-L4R5ZI

FC - RenesasRX65N

RAM Usage

FC - NUCLEO-L4R5ZI

FC - RenesasRX65N

Summary

• Model Correctness:

  • Some models failed to run on the RenesasRX65N board. (1)

    1. For FC_1 and FC_2 models, the program halts for an unknown reason.

  • Ekkono and TFLM basic are perfect. TFLM int8 only has a bit of error which is acceptable.

• Execution Time: For small models, Ekkono is faster than TFLM. However, as the model size increases, TFLM becomes faster.

• Flash Size: For small models, Ekkono has a smaller flash size. However, as the model size increases, TFLM int8 only becomes more efficient. (1)

  1. The Ekkono library itself has a smaller footprint than the TFLM library. As the model grows, the TFLM library's overhead becomes negligible compared to the model size. In case of TFLM int8 only, the model size grows 1/4th compared to TFLM basic or Ekkono which results in a smaller flash size.

• RAM Usage: For small models, Ekkono requires a smaller RAM. However, as the model size increases, TFLM int8 only becomes more efficient. (1)

  1. Same story as the flash size.

• Conclusion: For small models, Ekkono is more efficient. However, as the model size increases, TFLM int8 only becomes more efficient.