TinyML: Machine Learning on Raspberry Pi Pico with Tensorflow Lite Micro and Arducam (Featuring Person Detection)
The Future of Machine Learning Is Tiny
The future of machine learning is tiny.TweetFrom Pete Warden, technical lead of the mobile and embedded TensorFlow group in Google’s Brain team.
Machine learning, seen as a part of artificial intelligence, is the study of computer algorithms that improve automatically through experience.
Tiny machine learning (tinyML) is the intersection of machine learning and embedded internet of things (IoT) devices. The field has the potential to revolutionize many industries.
Before we start, here is a brief introduction to this emerging field from HarvardX.
Tensorflow Lite Micro (tflmicro)
TensorFlow Lite for Microcontrollers is designed to run machine learning models on microcontrollers and other devices with only a few kilobytes of memory.
It supports microcontroller platforms like Arduino Nano 33 BLE Sense, ESP32, STM32F746 Discovery kit, and so on. Since the release of the $4 Raspberry Pi Pico, which has gained increasing popularity among makers, Arducam has been trying to bring what’s possible on other microcontroller platforms to Pico.
Getting started with Machine Learning with Pico
This article is a tutorial on using the machine learning framework Tensorflow Lite Micro on the Pico for Person Detection. If you are more interested in the camera part, check out our Raspberry Pi Pico Camera series.
Demo 1: Person Detection on RPi Pico
Demo 2: Person Detection on Arducam Pico4ML
Demo 3: Magic Wand w/ Pico4ML
Demo 4: Wake-Word Recognition w/ Pico4ML
Demo 5: Arducam HM01B0 Camera for Raspberry Pi Pico
Components and Supplies
Complete All-in-One Pico Person Detection Bundle
Tiny Machine Learning Person Detection Bundle for Raspberry Pi Pico
Available now: Pico4ML, one dev board for all Tensorflow Light Micro Examples
RP2040 Dev Board w/ QVGA Camera, Microphone, LCD & More
Components of Pico person detection project
|1 × Arducam Mini 2MP Plus||UCTRONICS | Amazon|
|1 × Raspberry Pi Pico||RPi | Amazon|
|1 × USB to TTL Converter||Amazon|
|1 × Micro USB cable||Amazon|
Quick Pico Setup
If you are developing for Raspberry Pi Pico on Raspberry Pi 4B, or the Raspberry Pi 400, most of the installation steps in this Getting Started guide can be skipped by running the setup script. You can get this script by doing the following:
git clone https://github.com/raspberrypi/pico-setup.git
The script will:
- Create a directory called pico
- Install required dependencies
- Download the pico-sdk, pico-examples, pico-extras, and pico-playground repositories
- Define PICO_SDK_PATH, PICO_EXAMPLES_PATH, PICO_EXTRAS_PATH, and PICO_PLAYGROUND_PATH in your ~/.bashrc
- Build the blink and hello_world examples in pico-examples/build/blink and pico-examples/build/hello_world
- Download and build picotool (see Appendix B). Copy it to /usr/local/bin. • Download and build picoprobe (see Appendix A).
- Download and compile OpenOCD (for debug support)
- Download and install Visual Studio Code
- Install the required Visual Studio Code extensions (see Chapter 6 for more details)
- Configure the Raspberry Pi UART for use with Raspberry Pi Pico
Once it has run, you will need to reboot your Raspberry Pi,
- Download RPI-Pico-Cam
git clone https://github.com/ArduCAM/RPI-Pico-Cam.git
If you don’t want to compile, use the pre-built uf2 file below , you only need to wire the hardware and download uf2 to the device.
cd RPI-Pico-Cam/tflmicro mkdir build cd build cmake ..
|person_detection_int8.uf2||This is the main program of person_detection, which can be dragged onto the RP2040 USB Mass Storage Device.|
|person_detection_benchmark.uf2||This is the benchmark program of person_detection, you can use it to test the performance of person_detection on pico.|
|image_provider_benchmark.uf2||This is the benchmark program of image_provider, you can use it to test the performance of image data acquisition.|
|person_detection_int8||This is a person detection demo.|
- Hardware connection
- Load and run person_detection The simplest method to load software onto a RP2040-based board is by mounting it as a USB Mass Storage Device. Doing this allows you to drag a file onto the board to program the flash. Go ahead and connect the Raspberry Pi Pico to your Raspberry Pi using a micro-USB cable, making sure that you hold down the BOOTSEL button to force it into USB Mass Storage Mode.
If you are logged in via ssh for example, you may have to mount the mass storage device manually:
$ dmesg | tail [ 371.973555] sd 0:0:0:0: [sda] Attached SCSI removable disk $ sudo mkdir -p /mnt/pico $ sudo mount /dev/sda1 /mnt/pico
If you can see the files in
/mnt/pico then the USB Mass Storage Device has been mounted correctly:
$ ls /mnt/pico/ INDEX.HTM INFO_UF2.TXT
Copy your person_detection_int8.uf2 onto RP2040:
sudo cp examples/person_detection/person_detection_int8.uf2 /mnt/pico sudo sync
The person detection example outputs some information through usb, you can use minicom to view:
minicom -b 115200 -o -D /dev/ttyACM0
This person detection example also outputs image data and person detection results to the UART, and we provide a processing program to display them:
- The TensorFlow website has information on training, tutorials, and other resources.
- The TinyML Book is a guide to using TensorFlow Lite Micro across a variety of different systems.
- TensorFlowLite Micro: Embedded Machine Learning on TinyML Systems has more details on the design and implementation of the framework.
- Visit, star, or fork Arducam Pico Cam GitHub Repo.
What’s your take on this? Leave your comments below!