Posted on by techb

Python Webcam IR Tracking

Python WebCam IR Tracking

This is a tutorial on how to turn your webcam into an infrared tracking device using Python and a simple webcam hack. We will map the IR data to the mouse. This is much like connecting a Wiimote to your computer (via bluetooth) and using PyGlove to emulate the mouse. Except, we’re using python and cutting the $40 Wiimote out of the equation.

First the webcam. We need to filter out all light except infrared. To do this simple place film negative over the lens of the cam. Yes, film; the stuff we used to use before everything turned digital. You may have some old negatives from developed pictures. You don’t need the part if the film that has the images on it, just the black ends. Going to a local camera shop, could find useful. Most shops throw away the exposed negatives away. I layered mine with three layers of film, and taped it to my laptops webcam. To test it and see if it worked, try to view your webcam. If the video feed is black, point a tv remote (or any remote lying around) you should see a white dot appear in the video. You can also use open flame, like a candle or a lighter to test it. After you got that working, its time for the good stuff.

I am using Python 2.6, I’m not sure if it will work on Python 3.x or not. You can download Python at http://www.python.org/ If you’re not familiar with the language, than I suggest you look through the docs.

We also need some third-party library’s to make this work.

  • PIL http://www.pythonware.com/products/pil/
  • VideoCapture http://videocapture.sourceforge.net/

First we import our library’s that we need:

#IRClass.py
from VideoCapture import Device
import Image
from ctypes import *

Next we create a new class and initialize the handle for the webcam.


class IR:
    def __init__(self):
        self.cam = Device()
        self.user = windll.user32 #Handle for the mouse
        rp = self.cam.getImage()
        self.res = (rp.size[0],rp.size[1])

We create a new method to search an image for a IR dot. This is done by using getprojection(). getprojection() returns two lists containing boolean pixel data. If the pixel is black or 0,0,0 it’s a 0. If the pixel > 0,0,0 it’s a 1. That is why we needed the film over the lens so it will be black until an IR and only IR light is present.

def irSingle(self,im):
    imxy = im.getprojection()
    imx = imxy[0] #x coordinates
    imy = imxy[1] #y coordinates
    x = imx.index(1) #try to find a 1
    y = imy.index(1) #which is really our IR dot
    return (x,y)

Now we use the handle to grab an image instance for processing. First we need to resize the image to the resolution of the computer screen to get full range of the mouse. I am using a widescreen laptop, so my resolution is 1440×900.  We put the code inside a try/except statement because irSingle may return None.

def xySinglePos(self, scale=(1440,900)):
    try:
       im = self.cam.getImage()
       im.resize(scale)
       x,y = IR.irSingle(self,im)
       return (x,y)
    except ValueError:
       pass

Finally we set the mouse position with the x,y values we got from our other methods. Note: We are setting the virtual cursor, not the system cursor. This will not work with most video games you buy at the store. Online flash games and a lot of other things it will work with. Most games like Halo and Quake use the system mouse, which is a step below the virtual, making the system mouse the dominate one.

def setPos(self,x,y):
    self.user.SetCursorPos(x,y)

To use this new class:

#IRtest.py
import IRClass

ir = IRClass.IR()
while 1:
    x,y = ir.xySinglePos(scale=(1440,900)
    ir.user.SetCursorPos(x,y)

I hope this was useful to someone. It took me a long time to find everything I needed. And it was a great learning experience. I will be posting on how to incorporate PyGame, so we can see what the camera see’s. Maybe some games using this. I have also used it to “mock” the portal view/frustum correction Johnny Lee did with the Wiimote (http://www.youtube.com/watch?v=Jd3-eiid-Uw). I use Blender3D for the models and Panda3D for the actual game play.

This whole project was inspired by Mr. Lee and what he did with the Wiimote.