Raspberry Pi OpenCV Pan & Tilt Face Tracker

  

Create your own face tracking, pan and tilt camera on the Raspberry Pi!

This tutorial will demonstrate use of the OpenCV (computer vision) library to identify and track faces on the raspberry pi using two servos and a USB webcam. For the interested, I previously covered a more thorough overview of the installation of OpenCV from source here, however, I have found that the apt package is sufficient for all but the most bleeding edge of projects.

This project is based on the OpenCV face tracking example that comes along with the source-based distribution. In short, it performs face detection using haar-like features to analyze the video frame, and locates any faces present within it. In the live video window, identified faces are surrounded with a red rectangle. For additional information about how face detection works as well as the specifics of face detection with OpenCV, I recommend this article by Robin Hewitt.

Using the coordinates of the rectangle vertices, my script calculates the (X,Y) position of the center of the face. If the face is sufficiently on the left side of the screen, the pan servo will progressively rotate leftward, on the right side, rightward. This is likewise performed for the tilt servo as well, if the face is in the upper portion of the screen, it will pan upward, in the lower portion, pan downward. If the face is detected reasonably within the center of the image, no action is performed by the servos. This prevents unnecessary jitter once the camera has locked itself on the face.

 

 

Hardware

Parts needed:

  • 512 MB raspberry pi
  • 2x Hobby servos (Turnigy 9g fom Hobby King)
  • Pan & tilt bracket (from Foxtech FPV)
  • USB webcam (Microsoft LifeCam Show from Amazon)
  • Power supply
  • Hook-up wire
  • Raspberry Pi enclosure (from Built to Spec)

Assembly

Connect the red, power lines of the servos to +5v, the black ground lines to GND, and the yellow signal lines to the desired output pins, GPIO pins 22 and 23 in the example. Here is a diagram of the completed circuit  (created with Fritzing):

raspi_servo_facetracker

And here is how it looks all put together:

 

raspi-facetracker-side

raspi-facetracker-front

Software

Get the source.

The first step of this procedure is to install the required libraries and packages using the Raspberry Pi package manager. Open up terminal shell and run:

sudo apt-get update && sudo apt-get install git python-opencv python-all-dev libopencv-dev

This command will pull down all of the required packages (about 215 MB worth) including the git version control system, as well as the OpenCV development headers and Python bindings. The next step is to configure the Raspberry Pi for use with multiple pulse width modulation (PWM) outputs. Normally, the Raspberry Pi has only one channel of PWM output. However, thanks to the efforts of Richard Hirst, eight channels of PWM can be used through the use of this servoblaster kernel driver.The driver creates a device file, /dev/servoblaster to which commands can be sent to control the servos. The commands take the form “=” with servo number representing the desired servo (0-7 in this case) and servo position representing the pulse width in units of 10 µs. For example, to send servo 3 a pulse width of 120 µs: echo 3=120 > /dev/servoblaster To configure the servoblaster on the Raspberry Pi, first pull down the sources from Richard’s Github repo:

git clone https://github.com/richardghirst/PiBits.git

Then change into the servo blaster directory, and install the module:

cd PiBits/ServoBlaster

make install_autostart

This command also sets up the necessary udev rules for accessing the /dev/servoblaster device. Note: using the ’install_autostart ’ command will set up a Raspberry Pi to load the servoblaster kernel module on every boot. If you don’t want this behavior, execute ‘make install’ instead. In either case, the module will not yet be loaded so go ahead and load it into the kernel using modprobe:

sudo modprobe servoblaster

Now that all the prerequisites have been installed and the servo blaster device configured, it’s time to get the actual face tracking and servo moving code. Clone my repo from Github here:

git clone https://github.com/mitchtech/py_servo_facetracker

Now change directory into the created folder, and run the script like this:

cd py_servo_facetracker

python ./facetracker_servo_gpio.py 0

 

If you have a different servo bracket configuration, the pan and tilt axis may need to be inverted. To do so, invert the sign on the values of panStepSize and tiltStepSize.  Similarly, increasing or decreasing these values will change the sensitivity of the movement, larger numbers corresponding to more degrees moved per face detection frame.

 

20 Comments

  1. In the video you compare the RPi setup with an Arduino style setup, which runs more smoothly. Could you please expand upon the Arduino style setup?

    • JohnOrion

      From my understanding the Arduino is using his computer to do the work and only the Arduino to move the camera. In comparison the RPi is doing both the calculations and the moving of the camera

  2. Do you know how to train a new haar casade object XML for detection?

  3. Jeremy Karnowski

    My RPi does not seem to start up when I have one servo connected to 5V, GRD, and Pin 18 as shown in your diagram. Is there any reason why it should not start up? Thanks.

    • It’s not recommended to power servos with the Raspi 5V because servos draw more current than the Raspi can supply. Power the servos with an external 5V source and share a ground between that power source and the Raspi.

  4. Martin

    Can the RasPi camera module be used in this way to face detect?

  5. Dominic

    Do you know if there is any way to have a higher framerate?

  6. I mounted the raspberry with 2 servos g9 from deal extreme that does not move any servos even putting external power source, which I can do??

    • \Hi Nacho,
      I had same an issues problem. But the problem is how is GPIO activated the pins. I used same servo as ur. I was considering using GPIO.output(pins, boolean).

  7. Hello.
    How can i start-up the program directly with my usb-camera?

  8. benjamin

    Hi, I did exactly the same kind of program a while ago and I had issues with the frame rate in OpenCV.
    I wrote my code in C/C++ with a CVhaardectect and the .xml file provided with the library.
    My code is running slow, like 3 or 4 frame per sec.
    Do you have any trip and/or tricks to speed the program up ??
    Anyway, your blog is amazing… (is it wordpress that you use ?? and on which kind of server ??)

    Thanks for answering in advance …

  9. Hi, thanks for the tutorial.

    I’ve followed it so far, but run into a problem at the ‘make install_autostart’ step. When I execute the command I get the error “make: *** No rule to make target `install_autostart’. Stop.”

    Any guidance or clues would be very much appreciated.

    Kind regards

    Ralf

    • I think I solved my own problem just now. I navigated to ~/PiBits/ServoBlaster/kernel and then ran the ‘make install_autostart’ from there and it worked.

  10. Thank You for the tutorial.

    But I m a newbie. I used 2 days for installing and find out the bugs of my raspi installation.
    Now all things are passing.
    When I do the “sudo modprobe servoblaster” I get an error message. But when I reboot, the servoblaster is activeted (?).

    But if I do the last “python ./facetracker_servo_gpio.py 0″ thing, I get an error:

    “(result:2403): Gtk-WARNING **: cannot open display:”

    Can anyone help me?

    Thanks

    Klaus

  11. bhoobalan

    hello.. i just want to install openCV and not with servo face tracker .. so i gave that first command, but its not working pleas help me.. can u give me some idea regarding this.. thank you..

  12. can;t get servo to work. All i got face recognition working I am using raspberry pi.
    Thank you.

  13. Samigo

    Good day, i am new in raspberry pi projects and language. I want to do the same project but using Raspberry pi camera not the USB cam, how can I archive that?

  14. Leptigsap

    Why is this page still up? It does not work- as several posters have pointed out. If the instructions are followed precisely from beginning to end without the promised result being achieved it’s not much of a “tutorial” and more an elaborate prank by the author to see the number of cumulative hours that can be wasted from gullible individuals diligently trying to learn about new technology and better themselves. It’s only a “contribution” to the community if works and the instructions given are not demonstrably wrong.

    Leaving this online and uncorrected when the author knows it will not work and will waste at least two hours of time of every unfortunate individual who tries it is simply malicious and undermines the spirit of Open Source. Flooding Google with non-working tutorials does nothing to help anyone- except perhaps the authors page rank.

Trackbacks/Pingbacks

  1. 라즈베리파이+, 웹캠 서보 팬/ 틸트 작업중 | Manli000's Blog - […] http://mitchtech.net/raspberry-pi-servo-face-tracker/ […]

Leave a Reply