Deepseek on Microcontroller? Running LLM on RT-Thread
Introduction
While browsing the RT-Thread Club, I came across an article about microcontrollers interacting with large language models. Considering the recent surge in interest around large language models and having an RA8D1 Vision Board development kit available, I decided to give it a try by integrating a large language model API on the Vision Board to enable intelligent conversational features.
Test Platform
- Vision Board (Purchase link: https://www.aliexpress.com/item/1005006676753692.html)
- RT-Thread Studio
- RT-Thread 5.0.2
- MobaXterm
Implementation Process (Using Vision Board as an Example)
- Log in to DeekSeek-API: https://api-docs.deepseek.com/zh-cn/
- Create a DeekSeek API key:
3. Create a New Wi-Fi Example Project Based on Vision Board
4. Configure the LLM Package: Double-click on RT-Thread Settings, select “Add Package”, and search for the keyword “llm”.
5. Add the Package to the Project: After adding the package, proceed to configure it by entering the API key obtained earlier.
6. Configure WebClient in TLS Mode:
Go to RT-Thread Settings → Packages → IoT (Internet of Things) → WebClient → In the "TLS Mode" section, select MbedTLS support.
7. Enable Software RTC Functionality
8. Update Package Configuration:
In the RT-Thread Settings interface, press Ctrl+S to save the configuration. Then, compile the code and flash it to the board.
9. Manually Connect the Development Board to Wi-Fi:
Open a serial terminal tool and enter:wifi join [Wi-Fi Name] [Wi-Fi Password]
10. Enter the Chat Terminal:
Type “llm” to enter the chat terminal. Press CTRL+D to exit the chat window and return to the MSH terminal.
RT-Thread Community Developer RB contributed this article.
Check out the open-source Repository:
https://github.com/Rbb666/llm_chat
Note: Due to high usage on Deepseek’s servers, you may encounter connection failures, resulting in errors like the one shown below:
You can try switching to the ChatGPT large language model for an alternative experience.
