How to access and program the Pivariety Camera using Command Line, Python, OpenCV, and GStreamer

Table Of Contents

1. Libcamera and Libcamera App Installation

1.1. Use pre-compiled binaries

Note: If you don’t want to compile, you can choose to use our pre-compiled deb package.

1.1.1. Install libcamera

wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/v0.0.1/libcamera-dev.deb
sudo apt install ./libcamera-dev.deb

1.1.2. Install libcamera-apps

wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/libcamera-apps-v0.0.1/libcamera-apps.deb
sudo apt install ./libcamera-apps.deb

If you use the pre-compiled package please go to the chapter of Select the correct graphics driver.

1.2. Build from the source code

1.2.1. Install dependency

If you need to use the plugin GStreamer, please install the GStreamer dependency before compiling libcamera.

sudo apt update
sudo apt install -y libboost-dev libgnutls28-dev openssl \
	libtiff5-dev qtbase5-dev libqt5core5a libqt5gui5 \
	libqt5widgets5 meson libegl1-mesa-dev cmake \
	libboost-program-options-dev libdrm-dev \
	libexif-dev libegl1-mesa-dev
sudo pip3 install pyyaml ply
sudo pip3 install --upgrade meson

Optional dependencies (for GStreamer plugin).

sudo apt update
sudo apt install -y libgstreamer1.0-dev \
	libgstreamer-plugins-base1.0-dev

1.2.2. Download and compile libcamera

Please note that Pivariety can’t be supported by Raspistill, you need to download libcamera to make it work. You can follow the installation steps listed in the code library.

cd ~
git clone git://linuxtv.org/libcamera.git
cd libcamera
meson build
cd build
meson configure -Dpipelines=raspberrypi -Dtest=false
cd ..
ninja -C build

On some lower memory platforms (e.g. 1GB) there have been cases of ninja exhausting all the system memory and aborting (as it will attempt to use all the CPU cores). If this happens, please try replacing ninja -C build by ninja -C build -j 2 – this will restrict Ninja to only 2 cores.

After downloading and compiling libcamera, you need to replace the code ipa_rpi.so with.

cd ~
wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/ipa-v0.01/ipa_rpi.tar.xz
tar xvf ipa_rpi.tar.xz
cp ipa_rpi.so libcamera/build/src/ipa/raspberrypi/
cd libcamera && sudo ninja -C build install

1.2.3. Download and compile libepoxy

libcamera-apps requires libepoxy to be installed.

cd
git clone https://github.com/anholt/libepoxy.git
cd libepoxy
mkdir _build
cd _build
meson
ninja
sudo ninja install

1.2.4. Download and compile libcamera-apps

cd
git clone https://github.com/raspberrypi/libcamera-apps.git
cd libcamera-apps
mkdir build
cd build
cmake ..
make -j4
sudo make install

To check everything is working correctly, type ./libcamera-hello – you should see a preview window displayed for about 5 seconds.

Note for Pi 3 devices
As we saw previously, 1GB devices may need make -j2 instead of make -j4.Also, Pi 3s do not by default use the correct GL driver, so please ensure you have dtoverlay=vc4-fkms-v3d in the [all] (not in the [pi4]) section of your /boot/config.txt file.

1.2.5. Select the correct graphics driver

There are 3 different graphics drivers available on the Raspberry Pi: firmware, FKMS, and KMS.

The firmware graphics driver cannot be used with libcamera-apps. The Raspberry Pi 4 and 400 use the newer FKMS graphics driver by default: this is compatible with libcamera-apps.

For all other models of Raspberry Pi, you must select the FKMS driver by adding the following line to the /boot/config.txt file(Note: Please add it under the [all] tag ).

[all]
dtoverlay=vc4-fkms-v3d

NOTE: If you want to modify the file of /boot/config.txt, you need to reboot your device after modifying.

Tips: How to add the /boot/config.txt under the [all] tag?
a. open the file
sudo nano /boot/config.txt #Use the mouse to draw to the bottom #Add dtoverlay=vc4-fkms-v3d under the table of [all]
b. Press the ctrl+o after editing, and press enter to save your file
c. Press the Ctrl+x to exit the editor

You can refer to here to learn more.

2. Use libcamera to access the camera

2.1. Use libcamera-app to access the camera

2.1.1. Capture image

For example, preview for 5s and save the image named test.jpg.

libcamera-still -t 5000 -o test.jpg

Parameter explanation
-t 5000 means preview for 5 seconds
-o test.jpg means to take a picture after the preview is over and save it as test.jpg

2.1.2. Record video

For example, record a H.264 10s video with the frame size 1920W × 1080H.

libcamera-vid -t 10000 --width 1920 --height 1080 -o test.h264

Parameter explanation
-t 10000 means recording for 10 seconds.
–width 1920 means to set the width to 1920.
–height 1080 means to set the height to 1080.
-o means to output the encoded data stream to the test.h264 file.

Note: H.264 format only supports 1920×1080 and below resolution.

2.2. Create a virtual video node (Access by Python)

2.2.1. Install V4l2loopback

Download kernel source code

cd ~
sudo apt install git bc bison flex libssl-dev
sudo wget https://raw.githubusercontent.com/RPi-Distro/rpi-source/master/rpi-source -O /usr/local/bin/rpi-source && sudo chmod +x /usr/local/bin/rpi-source && /usr/local/bin/rpi-source -q --tag-update
rpi-source -d $(uname -r)

Compile driver

cd ~
git clone https://github.com/umlaeute/v4l2loopback.git
cd v4l2loopback
make clean && make
make && sudo make install
sudo depmod -a

2.2.2. Use v4l2loopback

sudo modprobe v4l2loopback video_nr=3
cd ~/libcamera
export GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer
gst-launch-1.0 libcamerasrc ! 'video/x-raw,width=1920,height=1080' ! videoconvert ! tee ! v4l2sink device=/dev/video3

Now you can use the video 3 node to access the camera. (vlc/OpenCV etc. )

Python OpenCV:

import cv2
cap = cv2.VideoCapture(3)
while True:
	ret, frame = cap.read()
	cv2.imshow("Arducam", frame)
	key = cv2.waitKey(1)
	if key == ord('q'):
		break

2.3. Use GStreamer to access the camera

2.3.1 Plugin GStreamer installation

Install GStreamer

sudo apt update
sudo apt install -y gstreamer1.0-tools

Preview

cd ~/libcamera
export GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer
gst-launch-1.0 libcamerasrc ! 'video/x-raw,width=1920,height=1080' ! videoconvert ! autovideosink

2.3.2. Streaming MJPEG

# server 
libcamera-vid -t 0 --width 1920 --height 1080 --codec mjpeg -n -l -o tcp://0.0.0.0:5001

# client
gst-launch-1.0 -v tcpclientsrc host=xxx.xxx.xxx.xxx port=5001 ! decodebin ! fpsdisplaysink sync=false text-overlay=false

2.3.3. Streaming H.264

# server 
libcamera-vid -t 0 --width 1920 --height 1080 --codec h264 -n -l -o tcp://0.0.0.0:5001

# client
gst-launch-1.0 -v tcpclientsrc host=xxx.xxx.xxx.xxx port=5001 ! decodebin ! fpsdisplaysink sync=false text-overlay=false

3. Control the Focusing Manually

Some Pivariety Camera is a program controllable motorized focus camera, its focal length can be adjusted via v4l2-ctl or script.

3.1. Control via v4l2-ctl

v4l2-ctl -c focus_absolute=300

3.1. Control via script

scrigit clone https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver.git
cd Arducam-Pivariety-V4L2-Driver/focus
python3 FocuserExample.py

Now you can use the up and down on the keyboard to control the focus position:

Cameras for Raspberry Pi