Guide Reference: Deploy Large Model AI Terminals on Microcontrollers with RT-Thread
Are you envious of AI experts with their massive models requiring dozens of GB of memory? Today, you’ll get to experience the joy of chatting with a large language model! Who says embedded devices can only blink LEDs? Today, we’ll make them “speak eloquently!”
From “Hello World” to “Hey, AI”
I remember when I first started, getting an LED to light up felt exhilarating. Now, our embedded devices can directly converse with large AI models! This is not just a technological advancement but a revolution in our approach to development. Today, I will guide you step-by-step on how to connect the DeepSeek large model to a development board, turning your microcontroller into an “AI terminal”!
This article will detail how to use the RT-Thread operating system and its comprehensive network protocol stack to successfully connect the development board to the DeepSeek large language model, enabling real-time conversation, using the FRDM-MCXN947 as an example.
Preparation: To do a good job, you must first sharpen your tools!
Hardware Requirements
● FRDM-MCXN947 Development Board: NXP’s flagship development board, the best of the MCX series, featuring a 150 MHz Arm® Cortex®-M33 core.
● Network Connection: RT-Thread RW007 Wi-Fi module.
● USB Type-C Cable: For power and debugging.
Software Preparation
● RT-Thread V5.2.0: Version 5.2.0 of RT-Thread or the latest master branch.
● DeepSeek API Key: Required to access the DeepSeek large model’s API. (Refer to the next section for details.)
● Keil: IDE for development.
Obtaining a DeepSeek API Key
First, visit the DeepSeek official website: DeepSeek API Keys
1. Register and Log In: Create an account and log in to DeepSeek.
2. Access the Developer Console: Navigate to the developer dashboard.
3. Create a New API Key: Select the option to create a new API key.
4. Copy and Save Your API Key: (Important: This key is displayed only once, so make sure to save it!) Remember to keep your API key secure!
Environment Configuration
1. Ensure RT-Thread Runs Properly on MCXN947: We need to make sure that RT-Thread can run smoothly on the MCXN947 and that the network functionality is working. This is part of the basic configuration, and there are comprehensive tutorials available, such as the NXP FRDM-MCXN947 Getting Started Guide. We will not elaborate on this here.
2. Access the RT-Thread Environment Tool’s Menuconfig:
In the menuconfig, we need to enable the following features:Activate the RT-Thread Large Language Model package:
Navigate to:
● RT-Thread online packages → AI packages → Large Language Models (LLM) for RT-Thread .Enter your own API Key in the configuration.
● Enable WebClient Component:
To enable the WebClient component, follow these steps:
Open the RT-Thread environment tool and enter menuconfig.
Navigate to the following path in the menu:
CopyRT-Thread online packages → IoT — internet of things → WebClient: A HTTP/HTTPS Client for RT-Thread
Select MbedTLS support.
● Exit and save the configuration, then enter the command pkgs — update to pull the packages.
● Generate the Keil project using the command: scons — target=mdk5.
● After completing all configurations, you can compile and run the program. If everything goes smoothly, you should see the RT-Thread startup information in the serial terminal.
First, enter your WiFi username and password to connect the board to the internet.
Successful Connection and Interaction
When you see “Got IP address,” it indicates that the Wi-Fi module has successfully connected to the network. Next, enter llm to access the large language model (LLM) dialogue interface, and then you can start chatting with DeepSeek!
Practical Application Scenarios
Integrating LLM functionality into embedded devices can open up many interesting application scenarios:
1. Smart Home Control: Control household devices through natural language.
2. Industrial Equipment Diagnosis: Inquire about device status and troubleshoot issues.
3. Educational Assistant Devices: Create interactive learning tools.
4. Smart Customer Service Terminals: Deploy information inquiry terminals in public places.
5. Development Aid Tools: Provide code suggestions and debugging assistance during the development process.
Optimization and Improvements
● Currently, the interaction is only via a serial terminal. If we could upgrade it to voice interaction in the future, the experience could reach a whole new level, creating a super “Tmall Genie” or “Xiao Ai Tong Xue.”
● At present, this code has not been optimized in detail, so the response time of the large language model is relatively slow, typically taking 1 to 4 seconds, and it consumes quite a bit of RAM (around 160 KB). The author once attempted to port it to the FRDM-MCXA156 (1M Flash, 128 KB RAM), but there wasn’t enough RAM available, and optimization is still ongoing.
Conclusion and Complete Code Download
Through this practical implementation, we successfully transformed the FRDM-MCXN947 development board into an AI terminal that can converse with the DeepSeek large model. This not only showcases the powerful network communication capabilities of the NXP MCU but also opens up new possibilities for intelligent applications in embedded devices.
In the future, with the development of edge computing technologies, we can even expect to run lightweight LLM models directly on MCUs to achieve true local AI inference. For now, by calling cloud-based large models via API, we have already begun to empower embedded devices with “intelligence.”
