Introduction

This Arducam stereo camera bundle kit consists of two synched OV9281 global shutter cameras and an Arducam Camarray stereo camera HAT. As a stereo bundle system, it enables simultaneously working dual cameras with a variable baseline for better prototyping and evaluation with a single MIPI CSI camera slot: The two image sensors are on separated camera boards, and you can easily move them around for different perspectives while the Camarray HAT handles the rest of the work. This is helpful for proof-of-concept stages when you may want more flexibility than what a fed stereo camera module can offer. You will be able to find the optimal baseline for your application with this bundle, and we welcome you to send us product customization requests after that.

What is Camarray

Camarray is a series of embedded stereo cameras and multiple camera solutions from Arducam. Upgraded from Arducam Sync Stereo Camera HAT, the Camarry can disguise up to 4 synched camera modules as a single camera slot connection to embedded systems like the Raspberry Pi, Jetson nano, and Xavier NX. With Arducam camarray, your camera connectivities are no longer limited to the camera connectors on the motherboard or the carrier board. Along with Arducam-provided camera drivers, more flexibility can be added to your multi-camera applications.

Common Specs

Image Sensor

Lens Assembly

Board

Connector & Cable

Driver Performance (With Official V4L2 Driver)

Quick Start Guide

​​Hardware Setup

Driver Installation

1.Driver note

For the new version of the camera, we recommend this driver for Raspberry Pi:

Pivariety V4L2 driver

Raspberry Pi is transitioning from a legacy camera software stack based on proprietary Broadcom GPU code to an open-source stack based on libcamera. Raspberry Pi OS images from Bullseye onwards will contain only the libcamera-based stack. In order to connect the new system and libcamera, we developed the Pivariety V4L2 driver.

Pivariety V4L2 driver supports using libcamera and arducamstill to fetch images. Therefore, for some customers who still want to use Buster legacy system, we recommend using arducamstill, no need to install OBISP driver.

If you encounter any problem when using the camera, please directly contact [email protected] for help.

2.Install the Driver

2.1. Installation

wget -O install_pivariety_pkgs.sh https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/install_script/install_pivariety_pkgs.sh
chmod +x install_pivariety_pkgs.sh
./install_pivariety_pkgs.sh -p kernel_driver

You can also follow the step to install the driver by yourself

• Download V4L2 Pivariety driver

wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/Arducam_pivariety_v4l2_v1.0/arducam_pivariety_v4l2.tar.gz

• 1.2 Decompress V4L2 Pivariety driver

tar -zxvf arducam_pivariety_v4l2.tar.gz Release/

• 1.3 Install V4L2 Pivariety driver

cd Release/
./install_driver.sh

After pressing y to reboot, driver installation is complete.

Note

Please make sure the camera is connected correctly before rebooting, otherwise, you will not detect the camera.
If you want to upgrade the version of Raspberry Pi(for example you want to upgrade your pi3 or Pi3B+ to pi4 ), please reinstall the driver. Each version of Raspberry Pi when you use needs to be installed V4L2 Pivariety driver once.

2.2. V4L2 Pivariety driver detection

2.2.1 View V4L2 Pivariety driver status

dmesg | grep arducam

The driver is installed successfully if output arducam-pivariety and the camera has been detected if output firmware version.

You can see the following video nodes When everything is normal.

ls /dev/video* -l

2.2.2 View device node information

v4l2-ctl --list-formats-ext

2.3. Failed to Install V4L2 Pivariety Driver

Please check the kernel version, we only provide the driver for the latest official kernel version image when this Pivariety camera is released.

2.4. What is the next?

If you encounter any trouble, please following the DOC to troubleshooting.

3. How to build Raspberry Pi Kernel Driver

We only provide the driver for the latest official kernel version image when the Pivariety camera is released. Please check the kernel version if the driver installation fails. You can choose to compile the kernel yourself if the kernel version used is not provided by us.

There are two compilation methods:

3.1. The first way-Build from the kernel source

Please refer to the official documentation to compile the kernel.

After obtaining the Linux source code, add the Arducam driver and device tree code through the apply patch.

git clone https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver.git
cd <path>/linux
patch -p1 -i <path>/Arducam-Pivariety-V4L2-Driver/patchs/arducam_driver.patch

At this point, your kernel driver building is complete! You can go to the chapter on the next.

3.2. The second way-Build from the kernel headers

3.2.1 Download the kernel source code

cd ~
sudo apt update && sudo apt install git bc bison flex libssl-dev
sudo wget https://raw.githubusercontent.com/RPi-Distro/rpi-source/master/rpi-source -O /usr/local/bin/rpi-source && sudo chmod +x /usr/local/bin/rpi-source && /usr/local/bin/rpi-source -q --tag-update
mkdir $(uname -r)
rpi-source -d $(uname -r)

3.2.2 Compile the driver and device tree

cd ~
git clone https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver.git
cd Arducam-Pivariety-V4L2-Driver/src

# compile dtbo
patch -p1 -d /lib/modules/$(uname -r)/build -i $(pwd)/../patchs/arducam_device_tree.patch
make -C /lib/modules/$(uname -r)/build  dtbs
sudo cp /lib/modules/$(uname -r)/build/arch/arm/boot/dts/overlays/arducam.dtbo /boot/overlays/arducam.dtbo

# manully add dtoverlay=arducam in /boot/config.txt
# manully add dtparam=i2c_vc=on in /boot/config.txt

# compile arducam.ko
make && sudo make install
sudo depmod
sudo modprobe arducam

At this point, your kernel driver building is complete!

What is the next?

If you encounter any trouble, please follow the DOC to troubleshooting.

How to access the camera by using command line, Python,OpenCV, and GStreamer

1. libcamera-dev and libcamera-apps Installation

wget -O install_pivariety_pkgs.sh https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/install_script/install_pivariety_pkgs.sh
chmod +x install_pivariety_pkgs.sh
./install_pivariety_pkgs.sh -p libcamera_dev
./install_pivariety_pkgs.sh -p libcamera_apps

wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/v0.0.2/libcamera-dev.deb -O libcamera-dev.deb
sudo apt install ./libcamera-dev.deb

wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/libcamera-apps-v0.0.2/libcamera-apps.deb -O libcamera-apps.deb
sudo apt install ./libcamera-apps.deb

sudo apt update
sudo apt install -y libboost-dev libgnutls28-dev openssl \
	libtiff5-dev qtbase5-dev libqt5core5a libqt5gui5 \
	libqt5widgets5 meson libegl1-mesa-dev cmake \
	libboost-program-options-dev libdrm-dev \
	libexif-dev libegl1-mesa-dev
sudo pip3 install pyyaml ply
sudo pip3 install --upgrade meson

sudo apt update
sudo apt install -y libgstreamer1.0-dev \
	libgstreamer-plugins-base1.0-dev

cd ~
git clone git://linuxtv.org/libcamera.git
cd libcamera
meson build
cd build
meson configure -Dpipelines=raspberrypi -Dtest=false
cd ..
ninja -C build

cd ~
wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/ipa-v0.01/ipa_rpi.tar.xz
tar xvf ipa_rpi.tar.xz
cp ipa_rpi.so libcamera/build/src/ipa/raspberrypi/
cd libcamera && sudo ninja -C build install

cd
git clone https://github.com/anholt/libepoxy.git
cd libepoxy
mkdir _build
cd _build
meson
ninja
sudo ninja install

cd
git clone https://github.com/raspberrypi/libcamera-apps.git
cd libcamera-apps
mkdir build
cd build
cmake ..
make -j4
sudo make install

[all]
dtoverlay=vc4-fkms-v3d

2. Use libcamera to access the camera

2.1. Use libcamera-app to access the camera

2.1.1 Capture image

For example, preview for 5s and save the image named test.jpg.

libcamera-still -t 5000 -o test.jpg

Parameter explanation
-t 5000 means preview for 5 seconds
-o test.jpg means to take a picture after the preview is over and save it as test.jpg

2.1.2 Record video

For example, record a H.264 10s video with the frame size 1920W × 1080H.

libcamera-vid -t 10000 --width 1920 --height 1080 -o test.h264

Parameter explanation
-t 10000 means recording for 10 seconds.
–width 1920 means to set the width to 1920.
–height 1080 means to set the height to 1080.
-o means to output the encoded data stream to the test.h264 file.

Note: H.264 format only supports 1920×1080 and below resolution.

2.2. Create a virtual video node (Access by Python)

2.2.1 Install V4l2loopback

Download kernel source code

cd ~
sudo apt install git bc bison flex libssl-dev
sudo wget https://raw.githubusercontent.com/RPi-Distro/rpi-source/master/rpi-source -O /usr/local/bin/rpi-source && sudo chmod +x /usr/local/bin/rpi-source && /usr/local/bin/rpi-source -q --tag-update
rpi-source -d $(uname -r)

Compile driver

cd ~
git clone https://github.com/umlaeute/v4l2loopback.git
cd v4l2loopback
make clean && make
make && sudo make install
sudo depmod -a

2.2.2 Use v4l2loopback

sudo modprobe v4l2loopback video_nr=3
cd ~/libcamera  # Installation using deb does not need to be executed
export GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer # Installation using deb does not need to be executed
gst-launch-1.0 libcamerasrc ! 'video/x-raw,width=1920,height=1080' ! videoconvert ! tee ! v4l2sink device=/dev/video3

Now you can use the video 3 node to access the camera. (vlc/OpenCV etc. )

Python OpenCV:

import cv2
cap = cv2.VideoCapture(3)
while True:
	ret, frame = cap.read()
	cv2.imshow("Arducam", frame)
	key = cv2.waitKey(1)
	if key == ord('q'):
		break

2.3 Use GStreamer to access the camera

2.3.1 Plugin GStreamer installation

Install GStreamer

sudo apt update
sudo apt install -y gstreamer1.0-tools

Preview

cd ~/libcamera # Installation using deb does not need to be executed
export GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer # Installation using deb does not need to be executed
gst-launch-1.0 libcamerasrc ! 'video/x-raw,width=1920,height=1080' ! videoconvert ! autovideosink

2.3.2 Streaming MJPEG

# server 
libcamera-vid -t 0 --width 1920 --height 1080 --codec mjpeg -n -l -o tcp://0.0.0.0:5001

# client
gst-launch-1.0 -v tcpclientsrc host=xxx.xxx.xxx.xxx port=5001 ! decodebin ! fpsdisplaysink sync=false text-overlay=false

2.3.3 Streaming H.264

# server 
libcamera-vid -t 0 --width 1920 --height 1080 --codec h264 -n -l -o tcp://0.0.0.0:5001

# client
gst-launch-1.0 -v tcpclientsrc host=xxx.xxx.xxx.xxx port=5001 ! decodebin ! fpsdisplaysink sync=false text-overlay=false

Display Images via VLC Media Player

The Raspberry Pi system comes with a VLC madia player.

(The VLC player can only play images that are output in RAW8 format.)

1.Open VLC media plyer

2.Select 【Media】→【Open Capture Device…】

3.【Video device name】→ select “/dev/video0“→ click 【Play】

The image screen may be too large for the display to be complete.
Right-click on the VLC title bar and select 【Maximize】.

4.Adjust the exposure, gain, and frame rate

【Tool】→【Effects and Filters】

Drag the different slider to adjust the corresponding parameter.

Display Images via Mplayer

1. Install mplayer

sudo apt-get install mplayer

2. Display images

mplayer tv:// -tv driver=v4l2:device=/dev/video0

Do not enter this command remotely, otherwise it will be stuck.

The image screen may be too large for the display to be complete. You need to drag the title bar to move the window to see the full image. (The image is pretty smooth.)

Or enter the following command to zoom the image. The image will appear full, but it will become stuck.

mplayer tv:// -tv driver=v4l2:device=/dev/video0 -zoom -x 1280 -y 200

“-x” “- y” denotes the width and height of the scaled image, and it is recommended to modify them in equal proportion.

What’s Next

Here are the things you can do after this quick start:

• Check the Application Note for applications like using Arducam userland MIPI camera drivers.
• Join the discussion in our forum.

Application Note

Depth Mapping on Raspberry Pi

1. Install Driver

Please refer to this page for a driver demonstration.

2. Hardware Setup

3. Reboot RPi

After rebooting RPi, use ls /dev/video* command to see the video device.

4. Software Setup

4.1Download the demo code

git clone https://github.com/ArduCAM/3d-camera.git

cd 3d-camera/stereo-camera/RaspberryPi/stereo_depth_demo/

4.2 Install Dependency for Python3.X

Latest Raspbian version is recommended.

sudo apt-get update && sudo apt-get install -y libhdf5-dev libhdf5-serial-dev libatlas-base-dev libjasper-dev libqtgui4 libqt4-test && sudo pip3 install opencv-python==3.4.6.27

sudo pip3 install stereovision

sudo pip3 install matplotlib

5. Run the Programs

5.1Capture Images

python 1_test.py

Help message:

Example:

5.2 Collect Images for Calibration

python 2_chess_cycle.py

You can see from the video above to see more about this.

30 stereoscopic photos will then be saved in /scenes folder.

5.3 Separate Captured Images

python3 3_pairs_cut.py

5.4 Calibration

python 4_calibration.py

python3 3_pairs_cut.py

5.5 Depth Map Tuning

python3 5_dm_tune.py 

Here is how this looks like:

5.6 Real-Time Depth Map Using Video

python3 6_dm_video.py

The result:

Using on OpenCV

Make sure you have installed the driver.

Download the file here.

import numpy as np
import cv2
import time
def fourcc(a, b, c, d):
    return ord(a) | (ord(b) << 8) | (ord(c) << 16) | (ord(d) << 24)

def pixelformat(string):
    if len(string) != 3 and len(string) != 4:
        msg = "{} is not a pixel format".format(string)
        raise argparse.ArgumentTypeError(msg)
    if len(string) == 3:
        return fourcc(string[0], string[1], string[2], ' ')
    else:
        return fourcc(string[0], string[1], string[2], string[3])
cap = cv2.VideoCapture(0,cv2.CAP_V4L2)
cap.set(cv2.CAP_PROP_FOURCC, pixelformat("GREY"))
cap.set(cv2.CAP_PROP_CONVERT_RGB,0)
now = time.time()
frame_num=0
while(True):
    if time.time() - now > 1:
        now = time.time()
        print(frame_num)
        frame_num=0
    else:
        frame_num = frame_num+1
    # Capture frame-by-frame
    ret, frame = cap.read()
    # Display the resulting frame
    cv2.imshow('frame',frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

Then type the following command:

Generic Highlighting
sudo python ov9281_dulcam_opencv_raw8.py
EnlighterJS Syntax Highlighter