In OpenCV/Samples there is  facedetect program.  This program can detect  faces on images and video.  It’s very fun, but its speed leaves much to be desired =(.  Of course  with OpenMP,  it works  faster; on Intel Core Duo 2.7GHZ, it works fast;  but will it work fast on ARM? I have big doubts.  I compiled facedetect without OpenMP and on average it takes 600 ms for 640×480 resolution to find one face.   I wanted to find out, if it’s possible to improve this time by software means or not…  After some investigations, code refactoring and improvements, facedetect started to work 2.5 time faster, even on ARM.  Of course, without big quality loss =)

If you want to generate cascade with OpenCV training tools, you should be ready for waiting plenty of time. For example, on training set: 3000 positive / 5000 negative, it takes about 6 days! to get cascade for face detection.  I wanted to generate many cascades with different training sets, also I added my own features to standart OpenCV’s ones  and refactor algorithms a little bit.  So waiting for 6 days to understand, that your cascade does nothing good =) was really anoying.  To reduce time, I chose paralleling methods.

OpenCV’s standart cascades allow to detect faces and eyes.  I wanted to create cascade in similar way to detect another objects:  pringles or plate for example.  I found some material in Net how to use OpenCV training tools, also I investigated training tool’s source  code myself  to found out, what training parameters can be tuned.

Prepare images.

For training, I needed thousands of images, containing my object with different lightning conditions and perspectives. After trying to find required number of pictures with Google ,  I understood, that it’s really difficult task =).  So I decided to take video with my object,  then I wrote simple program to crop object from video, frame by frame.  In such way,  I generated about 3000 positive samples (cropped images  with my object).  Resolution varied from 50×50 to 100×100.  The advantage of this method – you  get many samples with different reflections, illuminations and backgrounds.  It’s very important, that all these images “features” are various!

Today’s story is about improving performance of OpenCV library on the ARM-based platforms.

As you already know (from here or from here or may be even from here), face detection algorithm implemented by OpenCV library doesn’t work perfectly on ARM processors. Science doesn’t know for certain why this happens. There might be several possible reasons. One of our assumption was missing of hardware support for floating point operations. So we tried to translate Viola-Jones algorithm from floating point to fixed point. And that’s how we did this…

By default OpenCV 1.1 don’t support AXIS ip-cameras. So, this paper describes approach for getting camera interface (cvCaptureFromCAM) from OpenCV when you are using an Axis Ip camera.

This time OpenCV was ported to the Apple iPhone platform.

First of all we need to compile OpenCV library itself so that it can be used on the iPhone. There are two ways here:

1. Use OpenCV as a private framework.
2. Compile OpenCV as a static library.

First approach looks more comfortable for using, though I was not able to make it work properly on the iPhone (it works fine on the simulator, but not on the real hardware).

But anyway, let’s see how both approaches can be followed.

Compiling OpenCV

• Build platform: Windows XP SP3
• Target platform: ARM9 fixed point

Contents

• Download and prepare OpenCV library source code
• Download and prepare compiler
• Create/Modify Makefile
• Compile and link
• Run and Test
• Performance
• Profiling

Here is a brief description of compiling and running OpenCV’s facedetect sample on Pocket PC.

Compiling

I used Visual Studio 2005 Team Suite edition to compile OpenCV 1.1 pre 1. It compiled without any trouble for Win32.

Then I tried to compile for Pocket PC (set Platform to Pocket PC 2003 (ARMV4)). Some compilation errors occured. As I only needed to compile a facedetect sample, I decided to compile the necessary files only.

Compiling OpenCV part 2

Compiling without profiling

This time I compiled OpenCV library on Debian Linux installation.

I used VMWare 6.5 and Debian OS linux distribution.

I setup Virtual Machine to allocate 10GB hard drive space and 512 MB of system memory.

I setup the latest version of CodeSourcery G++ toolchain (version arm-2008q3-72-arm-none-linux-gnueabi) downloaded from http://www.codesourcery.com/sgpp/lite/arm. The installation dir was /opt/crosstool/codesourcery

I played with two ARM9-based single board computers (SBC) recently to investigate how OpenCV would operate on embedded platforms. The SBCs are – TS-7800 and SBC2440-III.

OpenCV uses floating point operations a lot, but not all of the ARM processors have FP coprocessor, so developers should use either some FP library, or as in my case – use Linux kernel FP emulation. There are two types of such emulation OABI and EABI. More details can be found here and here. Kernel release 2.6.16 was the first one to include ARM EABI support.

Unfortunately, CPUs of both SBCs do not support floating point in hardware, but luckily enough, TS-7800 has Debian Linux with 2.6.21 kernel. So I had a chance to compare OpenCV performance for OABI and EABI.

Both SBCs have necessary tool-chains in package – TS-7800 has Linux and Windows(Cygwin) tool-chains, SBC-2440 – only Linux one.  In addition to these I downloaded newer release of Codesourcery ARM tool-chain for Windows, because the one from TS-7800 SW package didn’t work properly with Cygwin.