RS300 Thermal Camera Driver for Raspberry Pi

This driver is for the "mini2" thermal camera module referred to as the rs300 in the original driver.

Some Basic Specs

The "mini2" is a thermal imaging camera that comes in multiple resolutions. For temperature measurements a different module or variant is needed.

The available resolutions are:

1. 256x192 - 25/50hz (25/50hz is available over USB)
2. 384x288 - 30/60hz (30/60Hz is available over USB)
3. 640x512 - 30/60hz (60hz is NOT supported by USB)

NETD: 40mk

Video Output Methods

This module can output two simultaneous video streams. One Digital, One Analog.

USB 2.0

• The 640 module is limited to 30fps over USB.
• The camera is UVC compliant and will work the same as a webcam but camera controls are not exposed. For the 256 module you can even switch the framerate from 25fps to 50fps in standard camera apps.
• The pixel format is YUYV422 but you can change it to a different order like UYVU with the SDK.
• For sending commands to the camera over USB, such as brightness changes, shutter calibration, changing colormaps, you will need to use the provided SDK. The SDK is for windows and linux.

MIPI

• Allows digital 60fps with the 640 module.
• The custom PCB has the 15 pin connector compatible with RPi 4B MIPI CSI-2 camera port. You can use the 15-22 pin adapter cable to connect RPi 5, 0 and CM.
• Uses the rs300 driver to connect to the Pi. Implement camera commands over I2C without the need for any SDK or precompiled binary files. As many commands as you can you could make them V4L2 compliant for easy integration.

CVBS

• Analog Output can be set to NTSC or PAL.
• Can be output simultaneously with USB or MIPI.
• Only White Hot and Black Hot colormaps are displayed. So far all other parameters seem to equally affect the analog video stream like; brightness, contrast, noise reduction, digital detail enhancement (DDE), and digital zoom.
  

Installation

Prerequisites

• Raspberry Pi with Raspbian/Raspberry Pi OS Bookworm (Bullseye not tested)
• Linux headers installed
• DKMS support

Build and Install

Install the needed headers:

sudo apt install linux-headers dkms git

MODIFY DRIVER BEFORE BUILDING. This will be improved soon so it can be set without modifying driver code

static int mode = 2; //0-640; 1-256; 2-384
static int fps = 60; //256: 25/50fps, 384/640: 30/60fps

Run the setup script:

git clone https://github.com/Kodrea/rs300-v4l2-driver.git
cd /rs300-v4l2-driver
# modify driver to select module
sudo nano rs300.c
chmod +x setup.sh
./setup.sh

In your config.txt file add the overlay

• For bookworm
  

sudo nano /boot/firmware/config.txt

Then Add

camera_auto_detect=0
dtoverlay=rs300

Then reboot

sudo reboot

Usage

Viewing Device Information

The camera should now be linked to the Raspberry Pi's Unicam driver and available on /dev/video0 Get basic information about the Unicam bridge driver:

v4l2-ctl --list-devices
v4l2-ctl --all -d /dev/video0

The actual camera is registered as a sub device in the driver

v4l2-ctl -d /dev/v4l-subdev0 --list-ctrls

Testing Video Capture

Set the format resolution for the appropriate module

Mini2-256

v4l2-ctl -d /dev/video0 --set-fmt-video=width=256,height=192,pixelformat=YUYV

Mini2-384

v4l2-ctl -d /dev/video0 --set-fmt-video=width=384,height=288,pixelformat=YUYV

Mini2-640

v4l2-ctl -d /dev/video0 --set-fmt-video=width=640,height=512,pixelformat=YUYV

gstreamer

gstreamer and has been faster than ffmpeg for me first install this

sudo apt install gstreamer1.0-tools

set test-overlay=false to remove fps counter

Mini2-256

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=256,height=192,framerate=50/1 ! videoconvert ! fpsdisplaysink video-sink=autovideosink text-overlay=true

Mini2-384

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=384,height=288,framerate=60/1 ! videoconvert ! fpsdisplaysink video-sink=autovideosink text-overlay=true

Mini2-640

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=512,framerate=60/1 ! videoconvert ! fpsdisplaysink video-sink=autovideosink text-overlay=true

Camera Controls

use v4l2 controls

this triggers the ffc shutter calibration

v4l2-ctl -d /dev/v4l-subdev0 --set-ctrl=ffc_trigger=0

Change colormaps

v4l2-ctl -d /dev/v4l-subdev0 --set-ctrl=colormap=3

Common Issues and Solution

1. Wrong resolution: Make sure to set the stream to your modules resolution. By default the Unicam driver will have it as 640x480

Troubleshooting

If experiencing issues:

1. Check kernel messages: dmesg | grep rs300
2. In a seperate terminal: dmesg wH
3. Verify I2C connection: i2c-detect -y 1
4. Check for device: ls -la /dev/video*
5. I started getting a message that postinst wasn't running, but it's only needed for the first time or if changes to the device tree overlay are made

./setup.sh
sudo sh /usr/src/rs300-0.0.1/dkms.postinst

Various Other Commands I used for troubleshooting

TODO: Cleanup this section

cd rs300-v4l2-driver

• make them executable

chmod +x setup.sh dkms.postinst

Install DKMS if not already installed

sudo apt install linux-headers dkms git

Run the setup script

./setup.sh

Check DKMS status and if module is loaded

dkms status
lsmod | grep rs300
modinfo rs300

check i2c devices

ls /dev/i2c*
i2cdetect -l
i2cdetect -y 10 # right now it's bus 10 for me
i2cdump -f -y 10 0x3c # i2c 10, 0x3c is your device address

check v4l2 devices

ls /dev/video*
v4l2-ctl --list-devices
v4l2-ctl -d /dev/video0 --all
# List supported formats
v4l2-ctl -d /dev/video0 --list-formats-ext
# List supported controls
v4l2-ctl -d /dev/video0 --list-ctrls
# Show current input/output
v4l2-ctl -d /dev/video0 --info

set v4l2 parameters manually; resolution, fps, etc

v4l2-ctl -d /dev/video0 --set-fmt-video width=256,height=192,pixelformat=YUYV
v4l2-ctl -d /dev/video0 --set-fmt-video=width=640,height=512,pixelformat=YUYV
v4l2-ctl --stream-mmap -d /dev/video0 -o test.yuv

ffplay -f v4l2 -input_format yuyv422 -video_size 256x192 -i /dev/video0

ffplay -f v4l2 -input_format yuyv422 -video_size 640x512 -i /dev/video0

Rebuild the module

cd rs300-v4l2-driver
./setup.sh

Check after rebuild and reboot

• check if the module is on i2c
• check if the video node is created
  

i2cdetect -l
i2cdetect -y 10
lsmod | grep rs300
dmesg | grep -i rs300
modinfo rs300

media-ctl -p
ls /dev/v4l-subdev*

v4l2-ctl --list-devices
v4l2-ctl -d /dev/video0 --all

v4l2-ctl -d /dev/video0 --list-ctrls
v4l2-ctl -d /dev/v4l-subdev0 --list-ctrls

v4l2-ctl -d /dev/video0 --list-formats-ext
vcgencmd get_camera


v4l2-ctl -d /dev/v4l-subdev0 --all
sudo cat /dev/kmsg | grep rs300
v4l2-ctl -d /dev/video0 --stream-mmap -o test.yuv
sudo dmesg -wH
ffplay -f video4linux2 -input_format yuyv422 -video_size 256x192 -i /dev/video0
ffplay -f video4linux2 -input_format yuyv422 -video_size 640x512 -i /dev/video0