Embark on a Vision Board — TensorFlow Lite Neural‑Network Journey
1. Preface
With the development of artificial intelligence, neural networks, and machine learning applications, edge computing scenarios are becoming increasingly common. Several transfer learning networks tailored for IoT devices and embedded systems have emerged. TensorFlow Lite was created in this context. Although TensorFlow Lite is small and fast enough, resource-constrained microcontrollers still face challenges, especially in image processing. Fortunately, the Vision Board has robust performance and extends a large SDRAM as data processing memory, which generally meets the requirements for recognizing small data volumes and low-quality images.
Note: This chapter will explain how to use the website edgeimpulse.com to train neural network models yourself, enabling machine learning capabilities.
2. Introduction to the Vision Board (Cortex M85 Core)
● Core: 480 MHz Arm Cortex-M85, incorporating Helium and TrustZone technology
● Storage: Built-in 2MB/1MB Flash and 1MB SRAM (including TCM, with 512KB ECC protection)
● Peripherals: Compatible with xSPI’s quad-line OSPI (supporting XIP and real-time decryption/DOTF), CAN-FD, Ethernet, USBFS/HS, 16-bit camera interface, and I3C, etc.
● Advanced Security: Excellent encryption algorithms, TrustZone, immutable storage, tamper protection against DPA/SPA attacks, secure debugging, secure factory programming, and lifecycle management support
● Performance: 6.39 CoreMark/MHz, capable of supporting demanding IoT applications requiring high computational performance, DSP, or ML functionalities
3. Preparation
Below are the software and reference materials needed for this experiment:
● Development Tools: MDK5 V5.3.8, OpenMV IDE V4.0.14
● Demo Code: https://github.com/RT-Thread-Studio/sdk-bsp-ra8d1-vision-board
● EDGE IMPULSE Website: studio.edgeimpulse.com
● Image Materials: https://github.com/JiaBing912/VisionBoard-Picture-training-material
4. Development Environment Setup
This experiment will be based on the vision_board_openmv demo. Double-click the mklinks.bat file; after executing the script, two folders named rt-thread and libraries will be generated.
Run the env, then enter menuconfig.8 Under Enable OpenMV for RT-Thread → directory, enable the “Enable tflite support” feature.
Save and exit. Then execute `scons — target=mdk5` to regenerate the MDK5 project.
Then open the MDK5 project, compile it, and program it onto the device.
5. Uploading to Edge Impulse for Training
5.1 Register an account and create an Edge Impulse project
Open the Edge Impulse website, register and log in. Then, in the project tab, create a new project:
5.2 Upload the training set
Follow these steps: click Dashboard → Add existing data → Upload data. (The training sample images can be obtained from the images in the “3. Preparation” section. For this experiment, handwritten digit images from the “mnist_lite” folder are used.)
Note: If the images in the dataset are not labeled with annotations (refer to the official documentation), you need to manually add labels or select “Enter label” to define labels. The dataset images are in 28x28 pixel BMP format, but Edge Impulse does not support BMP or 28x28 pixel formats. You need to convert the images using a Python script. The images in the “mnist_lite” folder have been converted to 256x256 PNG format.
5.3 Generate features
Click on Impulse Design → Create Impulse, then sequentially click Add a processing block and Add a learning block to select input data objects and train the model, then click Save.
Continue by clicking on the Image tab on the left, select RGB for Color Depth, and click Save. Next, it will automatically navigate to the feature generation page. Click the Generate Features button and wait for the features to be generated. Once completed, a 3D visualization will be displayed.
5.4 Transfer Learning
Click on the Transfer Learning button on the left, then set the training parameters sequentially: training epochs, learning rate, and other parameters. Then, choose the training model that best suits your experiment and click Start Training.
If the final results and accuracy do not meet your experimental requirements, you can try retraining by adjusting the parameters and selecting a different training model.
5.5 Deploy on Vision Board
Click on Deployment on the left, search for OpenMV library, then click Build and wait for the firmware to be generated.
Unzip the downloaded compressed file, including “trained.tflite”, “labels.txt”, and “ei_image_classification.py”. Rename “ei_image_classification.py” to “main.py”. Then, copy all three files to the SD card (make sure there are no other files on the SD card).
Connect the Vision Board’s USB-OTG port using a Type-C cable. Then, drag the main.py file from the SD card into the OpenMV IDE, open it, and run. At this point, you should see the recognition results and accuracy displayed in the serial terminal.
