Compiling OpenCV

• Build platform: Windows XP SP3
• Target platform: ARM9 fixed point

Contents

• Download and prepare OpenCV library source code
• Download and prepare compiler
• Create/Modify Makefile
• Compile and link
• Run and Test
• Performance
• Profiling

Download and prepare OpenCV library source code

• 1. Download the latest version of OpenCV (http://sourceforge.net/project/showfiles.php?group_id=22870). As build platform is Windows, select Windows version (opencv-win).
• 2. Install OpenCV (e.g. C:\Program Files\OpenCV).
• 3. Copy the complete installation directory into work location (e.g. D:\BH\OpenCV)
• 4. Based on an OpenCV sample (e.g. D:\BH\OpenCV\samples\c\facedetect.c) create a test application which will run on the Target hardware. Save it in its own directory (e.g. D:\BH\OpenCV\fd\src\facedetect.cpp)

Download and prepare cross-compiler

• 1. Download the latest version of Sourcery G++ GNU Toolchain for ARM Linux (http://www.codesourcery.com/sgpp/lite/arm/portal/subscription3057).
• 2. Install it (e.g. D:\gcc)
• 3. Copy short name executables (gcc.exe, g++.exe, ld.exe, ar.exe, etc.) from D:\gcc\arm-none-linux-gnueabi\bin\ to D:\gcc\bin\

Create/Modify Makefiles.

 All paths in all makefiles should be relative to the make directory (D:\BH\OpenCV\_make)

• 1. Use makefile_all_gnu.mak as a basis for the main Makefile. Save is as “Makefile”.
• 1.1. Comment libraries/binaries you are not going to compile/use (like cxcoretest, cvtest, etc.)
• 1.2. Rework “clean” target – remove lines containing *.dll and replace all *.lib with lib*.a; Remove or comment all lines with *.exe and add ..\bin\fd (the name and path of our test application)
• 1.3. Add lines to build application. E.g. @-mkdir ..\_temp\fd$(DR) and @$(MAKE) $(OPT) –directory=../_temp/fd$(DR) –makefile=../../fd/src/makefile.gnu
• 2. Use make_module_gnu.mak as a basis for the module makefile.
• 2.1. You may need to update CXXFLAGS_DBG variable to include necessary flags, like -D”NDEBUG” -O3 -ffast-math -pg -g -fno-bounds-check -c. The “-c” option says compiler to just compile the code (to not link)
• 2.2. All libraries’ makefiles specify BINTYPE=DLL, so update OUTBIN and OUTLIB paths accordingly. For DLL, OUTLIB should generate a library name like ../../lib/lib$(TARGET)$(DBG).a. OUTBIN should be OUTBIN := ../../bin/$(TARGET)$(DBG)
• 2.3. CXX variable should be “arm-none-linux-gnueabi-g++.exe” or just g++.
• 2.4. LINKFLAGS variable should include -Wl,-L../../lib $(LINKFLAGS_DLL)
• 2.5. LIBS variable should include all necessary libraries and should look like “-Wl,-lpthread,-ldl,-lstdc++,$(LIBS)”
• 2.6. AR variable should be equal to “arm-none-linux-gnueabi-ar.exe” or just ar.
• 2.7. LINK variable should be equal to $(CXX).
• 2.8. OUBLIB target $(OUTLIB): $(OBJS) should include @-mkdir ..\..\lib and $(AR) $(ARFLAGS) $@ $^.
• 2.9. OUTBIN target $(OUTBIN): $(OBJS) should include @-mkdir ..\..\bin and $(LINK) $(LINKFLAGS) $^ -g -o $@ $(LIBS).
• 2.10. .o: target should contain command “$(CXX) $(CXXFLAGS) $<“.
• 3. Use a Makefile.gnu from tests\cv\src directory to create Makefile.gnu for fd application. Save it in the fd source code directory (D:\BH\OpenCV\fd\src\)
• 3.1. TARGET should be fd
• 3.2. BINTYPE should be APP
• 3.3. At a minimum, the cxcore and cv includes should be available, so CV_INC and CXCORE_INC variables should be defined (../../cxcore/include and ../../cv/include)
• 3.4. CXXFLAGS should include -D”CVAPI_EXPORTS” and -I”$(CXCORE_INC)” -I”$(CV_INC)” and other include paths as necessary
• 3.5. LIBS should include at least -lcxcore$(DBG),-lcv$(DBG) and other libraries as needed.
• 3.6. After definitions of variables the make module makefile should be included (include ../../_make/make_module_gnu.mak)

Compile and link

• 1. Run Makefile (created on previous section). To do this just type make
• 1.1. Make will first compile all source files (-c compiler option does this) and create corresponding object files at corresponding folders (_temp/cv_Rls, _temp/cxcore_Rls, _temp/fd_Rls, etc.)
• 1.2. After compiling next library (like cxcore or cv, where the corresponding makefile contain BINTYPE := DLL) make will run “ar” command to create static library (libcv.a or libcxcore.a). Static library is just an archive of corresponding object (“*.o”) files. Object files in the static library are not linked, so they may contain errors which may show up later on the linking stage.
• 1.3. When all libraries are created, the application is compiled and linked. Make sure you give the linker all libraries needed to link the application (command should include options like -Wl,-L../../lib,-lcxcore,-lcv). When you specify “-lcxcore” the linker will look for library named libcxcore.a
• 1.4. If everything was right, the bin folder will contain built application fd.

Run and Test

• 1. The only way to run the application is to run it from MMC (Micro SD card). Copy fd to the mmc card then telnet to the device. Or, telnet to the phone and get binary application via ftp. It is recommended to create directory structure for the application like /mmc/mmca1/OpenCV/bin/facedetect (/mmc/mmca1 is a path to the root of MMC card)
• 2. cd to /mmc/mmca1/OpenCV/bin/facedetect and run the application by typing “./fd”.
• 2.1. If everything is correct, the application will display error message about missing cascade classifiers and exit. In this case put xml classifiers from OpenCV installation into the phone (../../data/haarcascades/) and re-run the application.
• 2.2. You may get the “Segmentation Fault” error. Typically, this error is occurred when the application is linked with (or to use) incorrect libraries or incorrect libraries versions (usually dynamically linked).
• 2.3. You may get the following error: “fd: /lib/libpthread.so.0: version ‘GLIBC_2.4’ not found (required by fd)”. It means that the application was linked with the pthread library that is not compatible with the one in the phone Linux installation. To fix this you need to replace corresponding libraries with the ones compatible with the phone. In my case, I had to replace pthread and stdc++ libraries, which caused multiple linker errors and necessity for replacing more libraries. The complete list of libraries I replaced is listed below:
• added gcc\arm-none-linux-gnueabi\libc\lib\ld-2.3.3.so
• replaced gcc\arm-none-linux-gnueabi\libc\lib\ld-2.8.so
• replaced gcc\arm-none-linux-gnueabi\libc\lib\libc-2.8.so
• replaced gcc\arm-none-linux-gnueabi\libc\lib\libc.so.6
• replaced gcc\arm-none-linux-gnueabi\libc\lib\libpthread-2.8.so
• replaced gcc\arm-none-linux-gnueabi\libc\lib\libpthread.so.0
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\crt1.o
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\crti.o
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\crtn.o
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\libc.a
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\libc.so
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\libc_nonshared.a
• added gcc\arm-none-linux-gnueabi\libc\usr\lib\libc_p.a
• added gcc\arm-none-linux-gnueabi\libc\usr\lib\libc_pic.a
• added gcc\arm-none-linux-gnueabi\libc\usr\lib\libc_pic.map
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\libstdc++.a
• added gcc\arm-none-linux-gnueabi\libc\usr\lib\libstdc++.la
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\libstdc++.so
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\libstdc++.so.6
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\libstdc++.so.6.0.10
• added gcc\arm-none-linux-gnueabi\libc\usr\lib\libstdc++.so.6.0.3

Performance

• 1. Only face detection time is displayed. In reality, application takes more time needed to load classifiers, load input file and save resulted output file. All these times are not counted.
• 1.1. It took on average about 6.4 seconds to process file sqa.jpg with resolution of 314×209 pixels.
• 1.2. It took on average about 41 seconds to process file pic.jpg with resolution of 640×476 pixels.
• 2. The application under performance test was compiled with maximum optimization level (-O3) and with the following options (-ffast-math -fno-bounds-check)

Profiling

• 1. To build application with enabled profiling give the “-pg” option to the compiler and linker, and re-build the whole project.
• 2. To make it compile, I had to replace the following libraries with ones compatible with the phone.
• replaced gcc\arm-none-linux-gnueabi\libc\usr\lib\gcrt1.o
• 3. Run application with profiling support on the phone. After completion, the gmon.out file should be created. Copy this file from the phone to the PC.
• 4. Run the gprof.exe with binary (fd) and gmon output (gmon.out) as command line arguments. Redirect standard output to a text file. This file will contain resulted profiling data, including timing (which function took how much time) and call graph.
• 5. In my case, the result was not quite correct – it shown total execution time of about 14 seconds while real execution time was over 45 seconds (about 41 seconds on face detection and other time on loading haar cascades, loading input file and saving output file).