Auto Tracking Pan Tilt Camera

Project Description

This project is a smart auto tracking Python program for pan and tilt cameras on the Raspberry Pi and other SBCs. This implementation uses the Haar Cascades OpenCV library to do face detection, but this relies on the CPU so you'll need a fast SBC such as the Raspberry Pi 5. Older models likely won't be able to keep up with a 30 FPS video stream, but they might work with a 15 or 20 FPS video stream. The ideal way to do object detection on an SBC is with an NPU, such as the Raspberry Pi AI Kit. The Raspberry Pi AI Camera is also a good choice, but might not work with this project since my example relies on MediaMTX to stream the video over the local network. This makes it easy to record the stream on other devices and applications such as OBS Studio. Because of the various options that available to do object detection, this project is meant to serve as an outline which you can modify to work with the more advanced object detection libraries that are available.

Required Software and Setup Guide

Here are the steps to install the required software for this project. This was done on the official Raspberry Pi OS, but the steps will be more or less the same for other operating systems such as DietPi. These steps are also shown in the video above.

First download and setup MediaMTX for streaming the Pi's camera. This will allow you to view the video stream on any device connected to your local network. MediaMTX Github Repo

Next activate the I2C bus to communicate with the PCA9685 module. Enter the following command:

sudo raspi-config

Navigate to interface options, then select I2C. Select yes to confirm. Now reboot your system before continuing.

Next enter the following command to install Python along with a few other packages required for this project:

sudo apt update

sudo apt install python3-dev python3-venv python3-pip python3-smbus ffmpeg libsm6 libxext6

Now enter the following command to create a Python virtual environment. The system-site-packages argument is required if you want your Python programs to have access to the system's GPIO library:

python3 -m venv venv --system-site-packages

Activate the virtual environment with the following command:

. venv/bin/activate

Next install the packages for the PCA9685 servokit library and sshkeyboard:

pip install adafruit-circuitpython-pca9685 adafruit-circuitpython-servokit sshkeyboard

If you plan to use the Haar Cascades library like I did in the example program, you'll need to install this special version of OpenCV:

pip install opencv-contrib-python

You'll also need to download the OpenCV Github repo, extract it, navigate to the data folder, and copy the haarcascades folder onto your Pi, as I showed in the video. OpenCV Github Repo

Finally, download the example code and place it in the same directory as the haarcascades folder: Download the example code here