Cameras for Raspberry Pi

  1. Home
  2. Docs
  3. Cameras for Raspberry Pi
  4. Synchronized Stereo Camera HAT
  5. Camarray – Arducam 1MP Stereoscopic Camera Module Bundle Kit

Camarray – Arducam 1MP Stereoscopic Camera Module Bundle Kit

Introduction

This Arducam stereo camera bundle kit consists of two synched OV9281 global shutter cameras and an Arducam Camarray stereo camera HAT. As a stereo bundle system, it enables simultaneously working dual cameras with a variable baseline for better prototyping and evaluation with a single MIPI CSI camera slot: The two image sensors are on separated camera boards, and you can easily move them around for different perspectives while the Camarray HAT handles the rest of the work. This is helpful for proof-of-concept stages when you may want more flexibility than what a fed stereo camera module can offer. You will be able to find the optimal baseline for your application with this bundle, and we welcome you to send us product customization requests after that.

What is Camarray

Camarray is a series of embedded stereo cameras and multiple camera solutions from Arducam. Upgraded from Arducam Sync Stereo Camera HAT, the Camarry can disguise up to 4 synched camera modules as a single camera slot connection to embedded systems like the Raspberry Pi, Jetson nano, and Xavier NX. With Arducam camarray, your camera connectivities are no longer limited to the camera connectors on the motherboard or the carrier board. Along with Arducam-provided camera drivers, more flexibility can be added to your multi-camera applications.

NOTE

1.The Camarray HAT does not support digital pan in half resolution combine and programmable pan speed control.
2.The Camarray HAT is not a crop and a half (halving the horizontal resolution of each camera and halving the field of view) but a compressed half (halving the horizontal resolution of each camera but leaving the field of view unchanged), so it does not need to support scan mode.

Common Specs

Image Sensor

Sensor ModelOV9281
Shutter TypeGlobal Shutter
Active Pixels1280*2×800
Resolution1MP*2
Image Sensor FormatType 1/4″
Pixel Size3μm×3μm
Color Filter ArrayNone(Monochrome)
9281block diagam
OV9281 Block Diagram

Lens Assembly

InterchangeabilityYES
F/NO2.8
Focus TypeManual Focus
Focusing Range30cm to infinity (when focused to infinity)
Effective Focal Length(EFL)2.8 mm
Field of View(FoV)70° Horizontal
Lens MountM12 Lens
IR SensitivityNo IR filter (sensitive to IR light)
2311stero camera lens 3d


Board

HAT Size65×56 mm
Camera Board Size40×40 mm
UC 512.Rev .C DIM 01
OV9281 Camera Board Mechanical DrawingHAT Mechanical Drawing
Pin No.Pin NameTypeDescription
13V3Power3.3V power supply
2FSINInputFrame Sync Input
3STBOutputLED Strobe Output
4GNDGroundGround
53V3Power3.3V power supply
6SDAI/OSCCB serial interface data I/O
7SCLInputSCCB serial interface clock input
8GNDGroundGround

Connector & Cable

Camera HAT Connector InterfaceMIPI CSI-2 22-pin 2-lane
Camera Module Connector InterfaceMIPI CSI-2 22-pin 2-lane
Ribbon Cable Length150mm (22-pin), 300mm(15-22pin), 73mm(15-22pin)
Pin #NameTypeDescription
1GNDGroundPower Ground
2CAM_D0_NOutputPixel Data Lane0 Negative
3CAM_D0_POutputPixel Data Lane0 Positive
4GNDGroundPower Ground
5CAM_D1_NOutputPixel Data Lane1 Negative
6CAM_D1_POutputPixel Data Lane1Positive
7GNDGroundPower Ground
8CAM_CK_NOutputPixel Clock Output Form Sensor Negaitive
9CAM_CK_POutputPixel Clock Output Form Sensor Positive
10GNDGroundPower Ground
11CAM_D2_NOutputPixel Data Lane2 Negative
12CAM_D2_POutputPixel Data Lane2 Positive
13GNDGroundPower Ground
14CAM_D3_NOutputPixel Data Lane3 Negative
15CAM_D3_POutputPixel Data Lane3 Positive
16GNDGroundPower Ground
17POWER-ENInputPower Enable
18LED-ENI/OLED Enable/XCLK
19GNDGroundPower Ground
20SCLInputSCCB serial interface clock input
21SDAI/OSCCB serial interface data I/O
22VCCPower3.3V Power Supply

Driver Performance (With Official V4L2 Driver)

Video FormatGERY(8-bit)/Y10P(10-bit)
Output Interface2-lane MIPI serial output
Output Formats8/10-bit BW RAW
Frame Rates[email protected]*2 × 800(on RPi)
[email protected]*2 × 800(on Jetson)

Quick Start Guide

​​Hardware Setup

image 23
image 24

Note:

1. Perform any hardware operations on the HAT only when the power is off.

2. Make sure the camera boards keep away from the GPIO pins of Raspberry Pi / Jetson boards to avoid any short issues.

Driver Installation

1.Driver note

For the new version of the camera, we recommend this driver for Raspberry Pi:

Pivariety V4L2 driver

Raspberry Pi is transitioning from a legacy camera software stack based on proprietary Broadcom GPU code to an open-source stack based on libcamera. Raspberry Pi OS images from Bullseye onwards will contain only the libcamera-based stack. In order to connect the new system and libcamera, we developed the Pivariety V4L2 driver.

Pivariety V4L2 driver supports using libcamera and arducamstill to fetch images. Therefore, for some customers who still want to use Buster legacy system, we recommend using arducamstill, no need to install OBISP driver.

If you encounter any problem when using the camera, please directly contact [email protected] for help.

Note:

The camera you are using may be the previous version, which looks the same as the new version. If you encounter any problem when you are referring to the following steps, please follow the previous version driver.

2.Install the Driver

Note

The Pivariety V4L2 kernel driver supports the latest version of Raspberry Pi OS. (January 28th,2022 or later releases, Debian version:11(Bullseye)).
Please make sure your Raspberry Pi OS is downloaded from these two official sources:
Raspberry Pi Buster >
Raspberry Pi Bullseye >
If you want to compile the kernel driver by yourself, please follow the chapter on How to build Raspberry Pi Kernel Driver-Pivariety Camera.

2.1. Installation

wget -O install_pivariety_pkgs.sh https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/install_script/install_pivariety_pkgs.sh
chmod +x install_pivariety_pkgs.sh
./install_pivariety_pkgs.sh -p kernel_driver
You can also follow the step to install the driver by yourself
  • Download V4L2 Pivariety driver
wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/Arducam_pivariety_v4l2_v1.0/arducam_pivariety_v4l2.tar.gz
5a5a7620968f410082d21874b64d78583368 1
  • 1.2 Decompress V4L2 Pivariety driver
tar -zxvf arducam_pivariety_v4l2.tar.gz Release/
3d16dd498f56436ebbdb2705d3a6619d3368 2
  • 1.3 Install V4L2 Pivariety driver
cd Release/
./install_driver.sh
948691927c534745b103d84943daba3e3368 1

After pressing y to reboot, driver installation is complete.

Note

Please make sure the camera is connected correctly before rebooting, otherwise, you will not detect the camera.
If you want to upgrade the version of Raspberry Pi(for example you want to upgrade your pi3 or Pi3B+ to pi4 ), please reinstall the driver. Each version of Raspberry Pi when you use needs to be installed V4L2 Pivariety driver once.

2.2. V4L2 Pivariety driver detection

Note

You can not use the official code(vcgencmd get_camera) to detect the camera! Please follow our DOC to detect the camera!

2.2.1 View V4L2 Pivariety driver status

dmesg | grep arducam

The driver is installed successfully if output arducam-pivariety and the camera has been detected if output firmware version.

2 1

Note

[ 31.835619] arducam_vddl: disabling
[ 31.835638] arducam_vdig: disabling
[ 31.835657] arducam_vana: disabling
does not mean your driver installation is failing, please skip it.

By the way, it means that the camera is not detected if the probe failed is displayed, please check the camera connection.
3 1

You can see the following video nodes When everything is normal.

ls /dev/video* -l
4 2

2.2.2 View device node information

v4l2-ctl --list-formats-ext
5 2

2.3. Failed to Install V4L2 Pivariety Driver

3 3

Please check the kernel version, we only provide the driver for the latest official kernel version image when this Pivariety camera is released.

Note

If you want to compile the kernel driver by yourself, please please follow the chapter on How to build Raspberry Pi Kernel Driver-Pivariety Camera.
If you want to switch to the native camera please go here to uninstall the V4L2 Pivariety Driver.

2.4. What is the next?

If you encounter any trouble, please following the DOC to troubleshooting.

3. How to build Raspberry Pi Kernel Driver

We only provide the driver for the latest official kernel version image when the Pivariety camera is released. Please check the kernel version if the driver installation fails. You can choose to compile the kernel yourself if the kernel version used is not provided by us.

There are two compilation methods:

3.1. The first way-Build from the kernel source

Please refer to the official documentation to compile the kernel.

NOTE

The instructions are divided into native builds and cross-compilation.

After obtaining the Linux source code, add the Arducam driver and device tree code through the apply patch.

git clone https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver.git
cd <path>/linux
patch -p1 -i <path>/Arducam-Pivariety-V4L2-Driver/patchs/arducam_driver.patch

At this point, your kernel driver building is complete! You can go to the chapter on the next.

3.2. The second way-Build from the kernel headers

3.2.1 Download the kernel source code

cd ~
sudo apt update && sudo apt install git bc bison flex libssl-dev
sudo wget https://raw.githubusercontent.com/RPi-Distro/rpi-source/master/rpi-source -O /usr/local/bin/rpi-source && sudo chmod +x /usr/local/bin/rpi-source && /usr/local/bin/rpi-source -q --tag-update
mkdir $(uname -r)
rpi-source -d $(uname -r)

3.2.2 Compile the driver and device tree

cd ~
git clone https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver.git
cd Arducam-Pivariety-V4L2-Driver/src

# compile dtbo
patch -p1 -d /lib/modules/$(uname -r)/build -i $(pwd)/../patchs/arducam_device_tree.patch
make -C /lib/modules/$(uname -r)/build  dtbs
sudo cp /lib/modules/$(uname -r)/build/arch/arm/boot/dts/overlays/arducam.dtbo /boot/overlays/arducam.dtbo

# manully add dtoverlay=arducam in /boot/config.txt
# manully add dtparam=i2c_vc=on in /boot/config.txt

# compile arducam.ko
make && sudo make install
sudo depmod
sudo modprobe arducam

At this point, your kernel driver building is complete!

What is the next?

If you encounter any trouble, please follow the DOC to troubleshooting.

How to access the camera by using command line, Python,OpenCV, and GStreamer

1. libcamera-dev and libcamera-apps Installation

wget -O install_pivariety_pkgs.sh https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/install_script/install_pivariety_pkgs.sh
chmod +x install_pivariety_pkgs.sh
./install_pivariety_pkgs.sh -p libcamera_dev
./install_pivariety_pkgs.sh -p libcamera_apps
You can also be following the step to install the driver by yourself

1.1. Use pre-compiled binaries

Note: If you don’t want to compile, you can choose to use our pre-compiled deb package.

1.1.1 Install libcamera

wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/v0.0.2/libcamera-dev.deb -O libcamera-dev.deb
sudo apt install ./libcamera-dev.deb

1.1.2 Install libcamera-apps

wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/libcamera-apps-v0.0.2/libcamera-apps.deb -O libcamera-apps.deb
sudo apt install ./libcamera-apps.deb

If you use the pre-compiled package please go to the chapter of Select the correct graphics driver.

1.2. Build from the source code

1.2.1 Install dependency

If you need to use the plugin GStreamer, please install the GStreamer dependency before compiling libcamera.

sudo apt update
sudo apt install -y libboost-dev libgnutls28-dev openssl \
	libtiff5-dev qtbase5-dev libqt5core5a libqt5gui5 \
	libqt5widgets5 meson libegl1-mesa-dev cmake \
	libboost-program-options-dev libdrm-dev \
	libexif-dev libegl1-mesa-dev
sudo pip3 install pyyaml ply
sudo pip3 install --upgrade meson

Optional dependencies (for GStreamer plugin).

sudo apt update
sudo apt install -y libgstreamer1.0-dev \
	libgstreamer-plugins-base1.0-dev

1.2.2 Download and compile libcamera

Please note that Pivariety can’t be supported by Raspistill, you need to download libcamera to make it work. You can follow the installation steps listed in the code library.

cd ~
git clone git://linuxtv.org/libcamera.git
cd libcamera
meson build
cd build
meson configure -Dpipelines=raspberrypi -Dtest=false
cd ..
ninja -C build

On some lower memory platforms (e.g. 1GB) there have been cases of ninja exhausting all the system memory and aborting (as it will attempt to use all the CPU cores). If this happens, please try replacing ninja -C build by ninja -C build -j 2 this will restrict Ninja to only 2 cores.

After downloading and compiling libcamera, you need to replace the code ipa_rpi.so with.

cd ~
wget https://github.com/ArduCAM/Arducam-Pivariety-V4L2-Driver/releases/download/ipa-v0.01/ipa_rpi.tar.xz
tar xvf ipa_rpi.tar.xz
cp ipa_rpi.so libcamera/build/src/ipa/raspberrypi/
cd libcamera && sudo ninja -C build install

1.2.3 Download and compile libepoxy

libcamera-apps requires libepoxy to be installed.

cd
git clone https://github.com/anholt/libepoxy.git
cd libepoxy
mkdir _build
cd _build
meson
ninja
sudo ninja install

1.2.4 Download and compile libcamera-apps

cd
git clone https://github.com/raspberrypi/libcamera-apps.git
cd libcamera-apps
mkdir build
cd build
cmake ..
make -j4
sudo make install

To check everything is working correctly, type ./libcamera-hello – you should see a preview window displayed for about 5 seconds.

Note for Pi 3 devices

As we saw previously, 1GB devices may need make -j2 instead of make -j4.Also, Pi 3s do not by default use the correct GL driver, so please ensure you have dtoverlay=vc4-fkms-v3d in the [all] (not in the [pi4]) section of your /boot/config.txt file.

1.2.5 Select the correct graphics driver

There are 3 different graphics drivers available on the Raspberry Pi: firmware, FKMS, and KMS.

The firmware graphics driver cannot be used with libcamera-apps. The Raspberry Pi 4 and 400 use the newer FKMS graphics driver by default: this is compatible with libcamera-apps. 

For all other models of Raspberry Pi, you must select the FKMS driver by adding the following line to the /boot/config.txt file(Note: Please add it under the [all] tag ).

[all]
dtoverlay=vc4-fkms-v3d

NOTE: If you want to modify the file of /boot/config.txt, you need to reboot your device after modifying.

Tips: How to add the /boot/config.txt under the [all] tag? 

a. open the file
sudo nano /boot/config.txt #Use the mouse to draw to the bottom #Add dtoverlay=vc4-fkms-v3d under the table of [all]
b. Press the ctrl+o after editing, and press enter to save your file
c. Press the Ctrl+x to exit the editor

You can refer to here to learn more.

2. Use libcamera to access the camera

2.1. Use libcamera-app to access the camera

2.1.1 Capture image

For example, preview for 5s and save the image named test.jpg.

libcamera-still -t 5000 -o test.jpg

Parameter explanation
-t 5000 means preview for 5 seconds
-o test.jpg means to take a picture after the preview is over and save it as test.jpg

2.1.2 Record video

For example, record a H.264 10s video with the frame size 1920W × 1080H.

libcamera-vid -t 10000 --width 1920 --height 1080 -o test.h264

Parameter explanation
-t 10000 means recording for 10 seconds.
–width 1920 means to set the width to 1920.
–height 1080 means to set the height to 1080.
-o means to output the encoded data stream to the test.h264 file.

Note: H.264 format only supports 1920×1080 and below resolution.

2.2. Create a virtual video node (Access by Python)

2.2.1 Install V4l2loopback

Download kernel source code
cd ~
sudo apt install git bc bison flex libssl-dev
sudo wget https://raw.githubusercontent.com/RPi-Distro/rpi-source/master/rpi-source -O /usr/local/bin/rpi-source && sudo chmod +x /usr/local/bin/rpi-source && /usr/local/bin/rpi-source -q --tag-update
rpi-source -d $(uname -r)
Compile driver
cd ~
git clone https://github.com/umlaeute/v4l2loopback.git
cd v4l2loopback
make clean && make
make && sudo make install
sudo depmod -a

2.2.2 Use v4l2loopback

sudo modprobe v4l2loopback video_nr=3
cd ~/libcamera  # Installation using deb does not need to be executed
export GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer # Installation using deb does not need to be executed
gst-launch-1.0 libcamerasrc ! 'video/x-raw,width=1920,height=1080' ! videoconvert ! tee ! v4l2sink device=/dev/video3

Now you can use the video 3 node to access the camera. (vlc/OpenCV etc. )

Python OpenCV:
import cv2
cap = cv2.VideoCapture(3)
while True:
	ret, frame = cap.read()
	cv2.imshow("Arducam", frame)
	key = cv2.waitKey(1)
	if key == ord('q'):
		break

2.3 Use GStreamer to access the camera

2.3.1 Plugin GStreamer installation

Install GStreamer
sudo apt update
sudo apt install -y gstreamer1.0-tools
Preview
cd ~/libcamera # Installation using deb does not need to be executed
export GST_PLUGIN_PATH=$(pwd)/build/src/gstreamer # Installation using deb does not need to be executed
gst-launch-1.0 libcamerasrc ! 'video/x-raw,width=1920,height=1080' ! videoconvert ! autovideosink

2.3.2 Streaming MJPEG

# server 
libcamera-vid -t 0 --width 1920 --height 1080 --codec mjpeg -n -l -o tcp://0.0.0.0:5001
# client
gst-launch-1.0 -v tcpclientsrc host=xxx.xxx.xxx.xxx port=5001 ! decodebin ! fpsdisplaysink sync=false text-overlay=false

2.3.3 Streaming H.264

# server 
libcamera-vid -t 0 --width 1920 --height 1080 --codec h264 -n -l -o tcp://0.0.0.0:5001
# client
gst-launch-1.0 -v tcpclientsrc host=xxx.xxx.xxx.xxx port=5001 ! decodebin ! fpsdisplaysink sync=false text-overlay=false

Display Images via VLC Media Player

The Raspberry Pi system comes with a VLC madia player.

(The VLC player can only play images that are output in RAW8 format.)

1.Open VLC media plyer

1 3

2.Select 【Media】→【Open Capture Device…

Select 【Media】→【Open Capture Device…】b0266

3.【Video device name】→ select “/dev/video0“→ click 【Play

3 1
image 25

The image screen may be too large for the display to be complete.
Right-click on the VLC title bar and select 【Maximize】.

image 26
image 27

4.Adjust the exposuregain, and frame rate

【Tool】→【Effects and Filters】

Adjust the exposure gain and frame rate b0264 1

Drag the different slider to adjust the corresponding parameter.

Adjust the exposure gain and frame rate b0264 2

Display Images via Mplayer

1. Install mplayer

sudo apt-get install mplayer
install mplayer b0267

2. Display images

mplayer tv:// -tv driver=v4l2:device=/dev/video0

Do not enter this command remotely, otherwise it will be stuck.

mplayer command b0267
image 28

The image screen may be too large for the display to be complete. You need to drag the title bar to move the window to see the full image. (The image is pretty smooth.)

image 29
image 30

Or enter the following command to zoom the image. The image will appear full, but it will become stuck.

mplayer tv:// -tv driver=v4l2:device=/dev/video0 -zoom -x 1280 -y 200

“-x” “- y” denotes the width and height of the scaled image, and it is recommended to modify them in equal proportion.

image 31

What’s Next

Here are the things you can do after this quick start:

  • Check the Application Note for applications like using Arducam userland MIPI camera drivers.
  • Join the discussion in our forum.

Application Note

Depth Mapping on Raspberry Pi

  1. Install Driver

Please refer to this page for a driver demonstration.

2. Hardware Setup

image 32
image 33

3. Reboot RPi

After rebooting RPi, use ls /dev/video* command to see the video device.

4. Software Setup

4.1Download the demo code

git clone https://github.com/ArduCAM/3d-camera.git
cd 3d-camera/stereo-camera/RaspberryPi/stereo_depth_demo/

4.2 Install Dependency for Python3.X

Latest Raspbian version is recommended.

sudo apt-get update && sudo apt-get install -y libhdf5-dev libhdf5-serial-dev libatlas-base-dev libjasper-dev libqtgui4 libqt4-test && sudo pip3 install opencv-python==3.4.6.27
sudo pip3 install stereovision
sudo pip3 install matplotlib

5. Run the Programs

5.1Capture Images

python 1_test.py

Help message:

image 34

Example:

image 35

5.2 Collect Images for Calibration

python 2_chess_cycle.py
Collect Images for Calibration
Collect Images for Calibration

You can see from the video above to see more about this.

30 stereoscopic photos will then be saved in /scenes folder.

5.3 Separate Captured Images

python3 3_pairs_cut.py
Separate Captured Images
Separate Captured Images

5.4 Calibration

python 4_calibration.py
python3 3_pairs_cut.py
Separate Captured Images
Separate Captured Images

5.5 Depth Map Tuning

python3 5_dm_tune.py 

Here is how this looks like:

depth map tuning result
depth map tuning result

5.6 Real-Time Depth Map Using Video

python3 6_dm_video.py

The result:

Real-Time Depth Map Using Video Result
Real-Time Depth Map Using Video Result

Using on OpenCV

Make sure you have installed the driver.

Download the file here.

import numpy as np
import cv2
import time
def fourcc(a, b, c, d):
    return ord(a) | (ord(b) << 8) | (ord(c) << 16) | (ord(d) << 24)

def pixelformat(string):
    if len(string) != 3 and len(string) != 4:
        msg = "{} is not a pixel format".format(string)
        raise argparse.ArgumentTypeError(msg)
    if len(string) == 3:
        return fourcc(string[0], string[1], string[2], ' ')
    else:
        return fourcc(string[0], string[1], string[2], string[3])
cap = cv2.VideoCapture(0,cv2.CAP_V4L2)
cap.set(cv2.CAP_PROP_FOURCC, pixelformat("GREY"))
cap.set(cv2.CAP_PROP_CONVERT_RGB,0)
now = time.time()
frame_num=0
while(True):
    if time.time() - now > 1:
        now = time.time()
        print(frame_num)
        frame_num=0
    else:
        frame_num = frame_num+1
    # Capture frame-by-frame
    ret, frame = cap.read()
    # Display the resulting frame
    cv2.imshow('frame',frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
Then type the following command:

Generic Highlighting
sudo python ov9281_dulcam_opencv_raw8.py
EnlighterJS Syntax Highlighter
Was this article helpful to you? Yes No