Today’s story is about improving performance of OpenCV library on the ARM-based platforms.

As you already know (from here or from here or may be even from here), face detection algorithm implemented by OpenCV library doesn’t work perfectly on ARM processors. Science doesn’t know for certain why this happens. There might be several possible reasons. One of our assumption was missing of hardware support for floating point operations. So we tried to translate Viola-Jones algorithm from floating point to fixed point. And that’s how we did this…

First of all, to make life easier it was decided to limit functionality of cvHaarDetectObjects a little:

• Our version will work with one specific cascade for frontal face detection from the bundle distributed with OpenCV
• We will not support Canny pruning
• No image scaling

Actually, it is not necessary to use only frontal face cascade – any cascade without trees (stump based cascade) can be used.

Second. We made few observations:

• Sizes of rectangles, which are used as classifiers, in the frontal face cascade are integer numbers. In OpenCV implementations they are defined as float though.
• Rectangles weights are also integers.
• “Left” and “right” values for rectangles are floating-point numbers in the range (0, 1). Their sum is not bigger than few thousands. Thus, they can be easily converted to integer by multiplying them by some constant. We choose 65536. This provides good enough accuracy (you will still loose some precision).
• The same is valid for stage threshold.

Also we replaced using of sqrt with its integral version. This let us calculate variance in integer numbers.

Thus, with all above it became possible to convert entire detection algorithm to integer numbers. The only floating point calculation, we still have, are calculations of current search window according to the scale factor. But their input in the overall execution time is very insignificant.

So, what are the numbers you’ll ask. And I tell you.

With OpenCV Lena’s face (everybody knows Lena, right?) was found on the iPhone in about 3-16 seconds (depending on the parameters).
With our fixed-point based algorithm the same face was found in 0.5-3 seconds (depends on parameters).

Thus switching from floating- to fixed-point improves detection performance up to 5 times. Search will not be as precise as in the original OpenCV version (it might not find face on some pictures), but it will be accurate enough for the most of the tasks.

Precision problem can be solved if OpenCV implementation is not reworked as radically as in our version – some of floating-point calculations can be left as is. This will keep original accuracy of Viola-Jones, but increase performance (in about 2 times).