Getting MJPEG stream from Axis Ip-camera (Axis 211M and Axis 214 PTZ) as a camera device in OpenCV with DirectShow
Posted on : 09-04-2009 | By : Aleksey Kodubets | In : OpenCV
47
By default OpenCV 1.1 don’t support AXIS ip-cameras. So, this paper describes approach for getting camera interface (cvCaptureFromCAM) from OpenCV when you are using an Axis Ip camera.
The architecture of OpenCV is set of drivers which allows dynamically get camera interface for currently connected devices to the PC. Particularly, under Windows OpenCV is using VFW (cvcap_vfw.cpp) and DirectShow (cvcap_dshow.cpp) driver.
The main problem is that AXIS camera cannot be added as VFW device to the system, at least AXIS (www.axis.com) said that there is driver which can work with Windows Media Encoder only (it is not true, read further). When I installed the driver and tested it via Windows Media Encoder, I had found that there is new DirectShow video source. Since it is known that OpenCV supports Directshow device, I have tried to connect the device to the OpenCV as is. The result was fail deeply in videoInput lib.
I had downloaded videoInput ( by Theodore Watson theo.watson@gmail.com) which allows to work with DirectShow devices. This library (OpenCV\otherlibs\_graphics\lib\videoInput.lib) is used in OpenCV and thus I had found that actually, frame grabber direct show filter cannot get frames due to incorrect frame buffer size/format.
I used GraphEdit tool for configuring and trace Axis DirectShow devices and filters. Although, part of properties of AXIS MJPEG Capture Filter hidden inside the filter, it can be accessed via GraphEdit and other tools. So, I was able rightly configure filter (set ip address, resolution). With GraphEdit tool I did catch right interfaces and found AXIS MJPEG Capture Filter ({7BB7689B-1635-498D-9214-6AF9C829D052}). Besides, Axis provided Axis MJPEG Decoder DMO filter ({94297043-BD82-4DFD-B0DE-8177739C6D20}) which allows to convert MJPEG stream to RGB24 stream. I modified code of vidoeinput library to use combination of filters (Axis Capture Filter + Axis MJPEG Decoder DMO filter ) instead just querying device by getDevice function. I had to add support of MEDIASUBTYPE_MJPG subtype to videoInput library.
Then I changed highgui library in order to have USB camera priority as first and DirectShow camera as second priority (by default DirectShow was first). Now, it allows to use USB camera and IP camera without rebuilding code.
After this, all works fine. I can use AXIS IP camera with OpenCV as usual USB device.
P.S.
videoInput library has a bug with multithreading, I had to add additional CoInitialize call to setup function in order to work with COM Objects correctly.
I have a question for you.
My project is to inter face an analog camera(Svideo as output) to Opencv and work in real time on the video file.
Can i use a frame grabster?
Is is detected by Opencv?
Can you please answer this..
My email address is harshwardhan.h@gmail.com
I didn’t work with grabster. It look likes usual compact DVR. I assume output of grabster will as IP-camera. If it has driver for Windows OS which could add the mapping ip of grabster to local device as AXIS do in my article. Then first of all, try to see what driver is adding to your system. It could be VWL interface or Direct X Video Source.
If it is VWL device, you can use it with OpenCV as is. E.g. I was able to use AverTV TV Tuner with OpenCV without any code changes.
If it will be Direct X Video Source, then you need to know what output protocol of the grabster, most probably, it is MJPEG then just follow my article.
If it is not possible to add local device (“grabster”) to your system but it is still possible to connect grabster over ip. OpenCV don’t support remote MJPEG streams as camera device. Although, you can find out transmitter component for DirectX, configure your component with grabster IP address and modify videoinput.lib library to add support for your this specific filter.
Hi Aleksey,
I’m doing the same thing but now I have a question that I’m not sure how to solve. What if you want to have multiple Axis IP cameras available and you want a different CvCapture object for each one (I want to process and view video from multiple IP cameras at the same time). It seems that opencv will ask for an IP address and password when I first try to get the Camera Capture from the index that DirectShow was found at but for subsequent calls to cvCaptureFromCAM I can’t put in a differnt IP and password.
Any ideas?
~Elan
Hi Elan,
Axis Video Source windows driver doesn’t allow to use multiple ip-cameras. Besides, I am not sure that it will be possible easy change some settings of Axis DirectX driver from your code. I can recommend to use some component which can get MJPEG stream and convert it to usual opencv BGR frame. So, you need create specific connection for each ip-camera which you want to use. AXIS camera has MJPEG and MPEG 4 interfaces and it is not so big problem to implement such converter. Then you need to modify openCV code and add new type of driver like now opencv has few capture driver (VWL, DirectX etc) and pass ID of camera device to cvCaptureFromCAM function when you need to use it in your code. I didn’t try the same but I tried already to use multiple video files processing (four instances of cvCaptureFromAVI simultaneously ) it works fine.
Aleksey
Hello,
could you give us your modified videoInput.lib and some code exemple ?
Do you think is it possible to use it with a OpenCV csharp wrapper ?
thanks.
Hi Zit,
>>could you give us your modified videoInput.lib and some code exemple ?
[Aleksey] I could be happy to share our code but our company doesn’t allow to do it. You can discuss with our marketing team, see “about” page for details…
>>Do you think is it possible to use it with a OpenCV csharp wrapper ?
[Aleksey] CSharp is just wrapper based on OpenCV which is implemented in C/C++ as dynamic libraries.
My solution is inside OpenCV (interface of cvCaptureFromCAM is not changed.) therefore it doesn’t dependent opencv is used via wrapper or not. So, my solution works under csharp wrapper.
Let me know if you have more question.
Aleksey
Hello I really need what you did i have an ip camera and i need to use it with opencv. I downloaded videoInput and GraphEdit but i have no idea what to do withe them, please can you write me tutorial with more details and steps how to do it? Please help me please
really thanks
Andrej
Hi Andrej,
What ip camera you have?
mobotix Q22 it has also mjpeg stream so i was thinking your solution will be working
Hi Andrej,
See http://www.computer-vision-software.com/blog/2009/08/cross-platform-solution-for-getting-mjpeg-stream-from-axis-ip-camera-axis-211m/
Aleksey
Hi Aleksey,
I have an Axis 211W IP camera, I want to get the decoded video frames into a C program to do some processing with opencv. Can you please suggest me whats the best way to do this ? Also among vivotek and axis cameras which is better IP camera and can easily get the above task done. Plz help me. Thanks in advance.
Jithendra
Hi Jithendra,
>>Can you please suggest me whats the best way to do this ?
[Aleksey] Please follow my article. If you need Windows solution, you need download driver from AXIS site (www.axis.com) and make changes in videoinput library and then rebuild OpenCV and get what you need. If you need solution which has compatibility with both windows/linux , we have own capture module (cross-platform ) for AXIS, ACTI, ARECONT, LOGITECH etc, you could ask our marketing for this (see “about” page for details).
>> Also among vivotek and axis cameras which is better IP camera and can easily get the above task done.
[Aleksey] We tested a lot of cameras with our algorithms. So, we prefer AXIS and Logitech cameras. But we cannot recommend these cameras for all use cases. You need to decide for what use case you need ip camera and then you could create required test cases with your algorithms (use cases) and properly make decision based on test results.
Aleksey
Thank you, I will try to follow your article for fetching video stream but how about audio stream. Is there s way to fetch it also into the C++ program ?
Hi,
I want to know what type of cameras does openCV supports and will it support usb based webcameras. If so what formats of web cam video it supports
thanks
Hi Rajesh Tankasala,
>>I want to know what type of cameras does openCV supports and will it support usb based webcameras. If so what formats of web cam video it supports
[Aleksey] See http://opencv.willowgarage.com/wiki/#Welcome.2BAC8-OS.OSSpecificStuff for your question.
Besides OpenCV doesn’t have any native support for IP cameras.
Aleksey
When reading video frames using openCV it will create its own buffer to store frames. I want to know how much size of buffer it will allocate for the frames
How to find objects (it can be chair, phone, box etc.,) using openCV. i.e., how to find contours or rectangle surrounding objects?? I used findcontours but is there any other approach for the same
Hi Rajesh,
Some background model can be used if background is static and an object is added to the scene… then you will be able to extract contours. As I know, the extraction contours of any object is not resolved task yet. It is better to adapt your algorithm for your task conditions.
hi
i use a sony camera D70 with sensoray 2255 frame grabber. i want to capture from the frame grabber using opencv, the problem is sensoray frame grabber is set to ntsc video format. how can i make opencv to read ntsc video format or make opencv to switch ntsc to pal.
please help!!
Hi Reliah,
NTSC or PAL is out of scope questions. You should configure your grabber and it is outside of OpenCV.
You can connect your grabber to you PC and get following interfaces:
“Our Linux SDK includes a fully functional Video 4 Linux (V4L2) driver for the latest Linux kernel, 2.6.24.” [http://www.sensoray.com/products/2255data.htm].
“1.RGB packed (24 bits/pixel, bitmap compatible)
2.YCrCb packed (16 bits/pixel, YUY2 compatible)
3.YCrCb planar (16 bits/pixel, optimal for image processing)
4.Y8 (8 bits/pixel, monochrome).”
OpenCV supports VFL2 camera interface under Linux , so, it should not be a problem to use this grabber under Linux with OpenCV. Regrading MS Windows, you could download drivers from http://www.sensoray.com and try to understand what windows drivers are. It should be VFW or DirectShow Source Filter, both are supported by OpenCV.
Aleksey
Dear Aleksey,
Thank you for this useful article.
I’ve been busy developing a machine vision software for a long time. It is coded in C++ and using the Borland C++ Builder 6 IDE (too old, but works fine). I’ve also used the OpenCV libraries.
Recently we’ve decided to use Axis IP cameras. What I did initially was to download the Axis Media Control SDK. It provides an ActiveX control which can easily be inserted on a form and used. I tested it in a new fresh application and everything worked like a charm.
The problem is, I can’t use it in the project in which I’m using OpenCV (I’ve tested it and I’m sure that it is because of OpenCV). When I have one instance of this ActiveX on my form, it raises an exception at startup:
“Project Project1.exe raised exception class EOleError with message ‘Could not obtain OLE control window handle’. blah blah blah…”
I guess it is the same bug as you have mentioned at the end of your article (solved using CoInitialize). I didn’t manage to solve it. Could you please help me in it?
(I want to stick with the ActiveX method, if possible!)
Hi Nima,
You can try to remove opencv highgui lib from your project ( or rebuild opencv without highgui) since as I understood you don’t need UI windows from OpenCV. HighGUI lib is using VideoInput.lib which is statically initializing COM. But I am not sure that is reason.
please i want to know if i have an ip camera but it is not axis how can i make the open cv read it or how can i use it please help i need code if it is available thnx
Hi Ahmed,
This is dependent on kind of camera. What camera you have? I may know that and could help quickly.
Aleksey
Hi Aleksey,
yesterday i bought the digitus dn 16050 ip camera to use it with my windows opencv framework. The cam is able to stream mpeg4 and mjpeg but doesn’t have any direct-show driver (or something similar). With the vlc player i am able to receive the streams without a problem but as you know opencv can’t.
Can you help my making it work ?
Thx in advance
Dominic
Hi Dominic,
You could create your own module for getting MJPG frames and convering it to IplImage. See http://www.computer-vision-software.com/blog/2009/08/cross-platform-solution-for-getting-mjpeg-stream-from-axis-ip-camera-axis-211m/
Aleksey
Hi,
I bought a IP cam (IPUX 1310). Is it possible to connect with opencv? Also I want to use the pan/tilt function of the IP cam. Will it able to do this?
Hi Dan,
I didn’t see directshow video source support with this camera in “http://ipux.net/2010_PDF/ICS1300.1310.pdf”. However, you can get ip stream directly via sockets, see http://www.computer-vision-software.com/blog/2009/08/cross-platform-solution-for-getting-mjpeg-stream-from-axis-ip-camera-axis-211m/
Hi,
Thanks for your article. I am trying to do what you have explained for axis camera & opencv. I downloaded the videoinput library and graphedit tool. I am able to fetch and view the video by connecting filters in graphedit; but not to opencv.
I was looking at following function in videoinput – int videoInput::start(int deviceID, videoDevice *VD) specifically at this line hr = getDevice(&VD->pVideoInputFilter, deviceID, VD->wDeviceName, VD->nDeviceName); Please let me know how you modified this line or the ‘sample grabber’ or any other part to get it working.
Thanks in advance for your time and help.
Hi Jamma,
Graphedit tool can generate C++ code which do the same what you created in Graph Edit’s UI. You can look at the auto-generated code and find out difference between it and videoinput library.
Hi Aleksey I have a project can you help me please,
My project:
Let a webcam be connected to your computer
You should write a program satisfying the following scenario: Put an object of your choice in front of the camera, then the program will detect the screen coordinates and display them at the bottom left of the screen. The program should detect the object in real time and display the coordinates in real time while moving the object.
Thank you..
Hi KC,
What help you need? What points of the description are problem for you?
Aleksey
Hi, i am currently doing a project on capturing patient contur and monitoring the patient movement in real time. our clinic bought a Axis 214 PTZ ip camera. my questions are could i using open source project from CGAL Open Source Project (http://www.cgal.org/) and MeshLab (http://meshlab.sourceforge.net/) on the ip camera? could this ip camera model act as camera to capture images??
my knowledge on programming is in entry level…..
Hello Foong,
You should be able to take images from the video captured by Axis 214, but you may need to do tricks described in this blog post (although some commenters said it worked fine for them without any tricks).
We did not work with libraries you mentioned, but from brief look at them, there are no reasons why you can not use them. You also can look at OpenCV as it provides some capabilities for contour extraction and object tracking.
Regards,
Sergey
Aleksey, do you know if the Sony ipela SNC-P1 has directshow video source support?
thanks
Jim
HI,
I successfully retrieve the cam mjpg stream from axis 214 PTZ in this way
CvCapture* capture=cvCaptureFromFile(“http://192.168.1.85/axis-cgi/mjpg/video.cgi?resolution=320×240”);
but this does not work for Axis P1343 and Axis Q1910, why?
I’m using openCV 2.1.
Great blog man!
Sorry Mattia, we did not work with Axis P1343 and Axis Q1910.
Best regards,
Sergey
Hi, I have a wireless sony camera snc-rz30n and I am trying to grab frames using the function cvCreateFileCapture_FFMPEG and cvCaptureFromFile and i did not success, the value these functions returned is NULL. As an input parameter for these function i use: http://username:psw@ip_addr/image
What i am doing wrong ?
thanks,
Hi David,
We didn’t work with sony snc-rz30n. There are a lot of reason why you didn’t have success. E.g. you didn’t provide username and password before getting stream. If this camera provides MJPEG (you can find this info in specification of the camera), you should discuss with Sony or do reverse engineering to understand what specific protocol should be used for this camera.
Aleksey
hi,
I am also using axis ip cam wiith axismjpgcapturefilter with axismjpgdec everything works fine. i got the vdideo buffer and it’s perfectly rendering.
my problem is every time it’s username and password while using
IFileSource_var->Load(L”axmphttp:/ip/axis-cgi/mjpg/video.cgi?resolution=320×240″, &AMT);
to load the url.
even when i set “axmphttp://username:password@ip/axis-cgi/mjpg/video.cgi?resolution=320×240” it asking username and password to load. how can i give the credentials via code.
OpenCV doesn’t have native audio stream support since it Open Computer Vision library 😉 Under Windows OS you can get audio stream via VideoInput.lib (via directX essentially) but you need to make required changes. We don’t have cross-platform solution for getting audio stream from OpenCV but we could develop it for you if it is required (see “about” page for details).
Thanks for your help, magically when I installed that Axis Mjpeg driver , my opencv program is able to fetch the video frames. I was surprised that it worked without changing anything as you described in your post. Even the priority order is taking USB first; if no usb its using the network camera. But still I need to fetch audio packets into the C++ program. Is sending http request to camera server the only way to fetch audio packets or is there any easier/clever way to get the decoded audio. Please suggest me how to use videoinput.lib for audio. Thanks in advance.
Dear JK,
You caught a luck. The DirectShow SampleGrabber in your system has the required support for output format of AXIS direct show source filter. That is why it works.
We have tested it on at least 10 more PCs and my changes were required.
Regarding audio, as I said below, currently we don’t have use audio stream, you could ask our marketing team and we could develop it for you.
Aleksey
Hi All….
I am using Canon VCC4 cameras.
Will it be able to connect my cameras and grab frames using OpenCV?
BTW I am using Picolo card for grabbing video.
I see it is NTSC (380,000 effective pixels), it is not the best camera for OpenCV 😉 However, you need to at least have frame grabber card.
Prabhu,
http://www.euresys.com/Products/picolo/Picolo.asp
Software support: – DirectShow filters
– MultiCam drivers for Microsoft Windows®
Linux
it should work good. OpenCV supports camera as DirectShow Video Source. I cannot give any warranty that you could not find the same problem (as described in this article) with your frame grabber.
Aleksey