Introduction: Intelligent video surveillance technology is a new video surveillance technology based on image processing and pattern recognition. In short, it is to find the moving objects in the image, track and analyze them, find “abnormal†behavior in a timely manner, trigger an alarm and take other measures to intervene.
Intelligent video surveillance technology is a new video surveillance technology based on image processing and pattern recognition. In short, it is to find the moving objects in the image, track and analyze them, find “abnormal†behavior in a timely manner, trigger an alarm and take other measures to intervene.
The development process of intelligent video analysis The intelligent video analysis technology (Video Analytics) integrates multidisciplinary research results. It mainly includes image processing, tracking technology, pattern recognition, software engineering, digital signal processing (DSP) and other fields. After the "911" incident in 2001, the United States greatly strengthened its investment in security research. Many research institutions and researchers have joined the research and development of security technology. Intelligent video analysis is one of the highlights. From the point of view of the number of research papers, there was a clear peak period from 2002 to 2005, which was consistent with the large investment in research funding during this period. At present, research papers in this field of research gradually shift to subdivision problems and directions. This does not mean that intelligent video surveillance has become a problem that has already been solved. On the contrary, even the best business systems at present are far from people's expectations for such technologies. There is no consensus on how to solve the problem. It actually reflects the reduction of theoretical work on originality. The progress of this technology may depend more on the company's own research and development forces in the future.
The development of this market in the country lags behind North America for about three to four years. There are not many domestic companies with independent intellectual property rights and R&D capabilities, mainly Wen An in Beijing, Zhi An Bang, Anwell in Shanghai, and Hong Shi. As the companies are in the early stages of market expansion, there are not many representative projects that have been completed. Typical are nuclear power plants, military projects, and ports.
Main functions The current intelligent video analysis system on the market generally has the following functions:
1, image acquisition / interface. The vast majority of intelligent video analysis algorithms are based on uncompressed image formats such as RGB or YUV. Therefore, the image signal is sent directly to the video analysis unit without being compressed after being captured. Almost all video analysis systems are equipped with an image capture function, usually through the BNC input analog image signal. The image signal in the existing image monitoring system usually exists in the form of a compressed image stream. The image stream can be decompressed and restored to the original image format before analysis.
2, moving object detection. In simple terms, motion detection is the discovery of moving objects in an image. Moving objects can simply be defined as changing parts of an image. Some elementary motion detection algorithms are based on these concepts. The false alarm rate of such methods is too high to be suitable for real-time alarm systems.
Not all changes in the image are moving objects that we are interested in, such as changes introduced by the camera itself, including pixel noise, overall brightness changes caused by the camera's automatic iris control circuit, and high- and low-frequency periodic noise signals introduced in image transmission. , mutations caused by infrared camera cycle calibration. Changes introduced by the external environment include rapid changes in the ground light in cloudy weather, shadows of moving objects, waves or sparkling phenomena on the water surface, swinging of branches on land, halo caused by headlights at night, and rain and snow weather. In addition, when the camera is easily shaken on a windy day, especially on a high light pole, image changes caused by these phenomena should be filtered out. They can be solved by algorithms or other technical means.
From the perspective of the algorithm, it can be simply divided into two categories. One is to establish a background model and find the moving object by comparing it with the background model. The other is through the “optical flow†method to discover moving objects by finding the effect of moving objects on the optical flow field. The other is a method that combines the two or both.
3, multi-object tracking. Tracing is essentially stringing together the same objects found on each frame in chronological order. This area itself is a relatively independent and active research area. The main research direction is to effectively track under complex circumstances such as multiple moving objects, multiple cameras, moving objects blocking each other, disappearing and recurring.
In practical monitoring applications, especially for some intrusion alarm application cases, the requirements for tracking algorithms are relatively low. The existing system does not have a satisfactory tracking effect on "fusion" of moving objects and other complicated application scenes. However, with reference to the pace of technological development in the past, this aspect will soon be perfected.
4, behavioral characteristics analysis. The behavioral feature analysis is to look for an event that meets a predetermined behavioral characteristic from the image. Typical applications on the market today include:
(1) Classification. Judging that the moving object is a person, a vehicle, a ship, an airplane, stopping or suddenly accelerating, for example, the scene of a scene in which the vehicle is anchored in a tunnel or on a highway, and the street is robbed and then run away.
(2) Hey. Such as people in sensitive areas. Pedestrians and vehicles that pass normally are not alerted.
(3) Leftovers. Place explosives at airports, oil depots, etc. and leave.
(4) The item is missing. If the museum’s valuable exhibits discover the disappearance of exhibits, the system will immediately call the police.
(5) Statistics of the number of people. For example, statistics on the number of people entering supermarkets and other places, and the combination of sales data to draw the average consumption curve of the day.
(6) Population density. Such as when the alarm occurs when a large number of people gather, or when the crowd suddenly disperses abnormal conditions, an alarm is issued.
(7) The staff fell to the ground. For example, when the staff suddenly changes from straight to flat.
In general, intelligent video analytics can do a lot of things and require video analytics developers and end users to communicate effectively. As intelligent video analysis is still a relatively new technology, the circle that knows this technology in the country currently only expands to the integrator level. Many application scenarios suitable for video analysis technology have yet to be developed in the market. However, one thing is clear: companies must master core technologies and have independent research and development capabilities. The intelligent video analytics market is composed of many subdivided small markets, and new applications are constantly emerging.
5, set the alarm conditions. The introduction of "smart" in video surveillance has greatly enriched the monitoring content. The currently available alarm elements include areas, time periods, object types, dimensions, direction of movement, speed, behavioral characteristics, and much more.
6, alarm linkage. After the abnormal situation is found in the intelligent video analysis system, it is usually necessary to verify the authenticity of the alarm, such as detailed investigation of the close of the alarm event by another PTZ camera, or promptly notify and remind the monitoring personnel. Common real-time prompt methods include voice, pop-up images, sending text messages, screenshots and other means to prompt monitoring personnel.
Main Product Forms and Features Current intelligent video analysis products are mainly based on general-purpose CPUs such as Intel (server, industrial computer) or DSP. Some products are integrated with DVRs, and some products are made into independent modules that provide an interface and development SDK for integrators. The most integrated product has been integrated with the camera and outputs the intelligent analysis results directly.
A server-based (IPC)-based system is usually suitable for being deployed in the background of a monitoring system. Since its architecture is relatively open, it can be easily integrated with existing monitoring systems. In addition, the CPU processing capability of the server is much higher than that of the DSP, and more complex algorithms can be used. In the past, optimization operations peculiar to DSP processors have also become commonplace on general-purpose processors. Multi-core is the development direction of Intel CPU, and it is very suitable for the needs and development trend of multi-channel image processing, which is helpful for reducing system cost. Intel launches a new product every two years at a much faster rate than Texas Instruments. Server-based system performance can be easily upgraded with Intel product updates. In some high-end intelligent video surveillance systems, the use of servers More. DSP's modular products are usually suitable for being placed at the front end of the monitoring system. It is easy to install and implement, and the protection of intellectual property is also easy to do.
The successful installation and use of intelligent video analysis products should usually be directly supported by experienced integrators or developers in the form of solutions. Nowadays, smart video analysis products on the market have made substantial progress in monitoring efficiency and quality compared with traditional monitoring forms, especially in real-time alarms. But there are still some gaps between the standards that we expect. We can't expect to set up a system just as if we were installing surveillance cameras. Like any new technology, to achieve good results, engineers who understand the characteristics of intelligent video analysis products must have full support from project design to construction and commissioning.
Intelligent video surveillance technology is a new video surveillance technology based on image processing and pattern recognition. In short, it is to find the moving objects in the image, track and analyze them, find “abnormal†behavior in a timely manner, trigger an alarm and take other measures to intervene.
The development process of intelligent video analysis The intelligent video analysis technology (Video Analytics) integrates multidisciplinary research results. It mainly includes image processing, tracking technology, pattern recognition, software engineering, digital signal processing (DSP) and other fields. After the "911" incident in 2001, the United States greatly strengthened its investment in security research. Many research institutions and researchers have joined the research and development of security technology. Intelligent video analysis is one of the highlights. From the point of view of the number of research papers, there was a clear peak period from 2002 to 2005, which was consistent with the large investment in research funding during this period. At present, research papers in this field of research gradually shift to subdivision problems and directions. This does not mean that intelligent video surveillance has become a problem that has already been solved. On the contrary, even the best business systems at present are far from people's expectations for such technologies. There is no consensus on how to solve the problem. It actually reflects the reduction of theoretical work on originality. The progress of this technology may depend more on the company's own research and development forces in the future.
The development of this market in the country lags behind North America for about three to four years. There are not many domestic companies with independent intellectual property rights and R&D capabilities, mainly Wen An in Beijing, Zhi An Bang, Anwell in Shanghai, and Hong Shi. As the companies are in the early stages of market expansion, there are not many representative projects that have been completed. Typical are nuclear power plants, military projects, and ports.
Main functions The current intelligent video analysis system on the market generally has the following functions:
1, image acquisition / interface. The vast majority of intelligent video analysis algorithms are based on uncompressed image formats such as RGB or YUV. Therefore, the image signal is sent directly to the video analysis unit without being compressed after being captured. Almost all video analysis systems are equipped with an image capture function, usually through the BNC input analog image signal. The image signal in the existing image monitoring system usually exists in the form of a compressed image stream. The image stream can be decompressed and restored to the original image format before analysis.
2, moving object detection. In simple terms, motion detection is the discovery of moving objects in an image. Moving objects can simply be defined as changing parts of an image. Some elementary motion detection algorithms are based on these concepts. The false alarm rate of such methods is too high to be suitable for real-time alarm systems.
Not all changes in the image are moving objects that we are interested in, such as changes introduced by the camera itself, including pixel noise, overall brightness changes caused by the camera's automatic iris control circuit, and high- and low-frequency periodic noise signals introduced in image transmission. , mutations caused by infrared camera cycle calibration. Changes introduced by the external environment include rapid changes in the ground light in cloudy weather, shadows of moving objects, waves or sparkling phenomena on the water surface, swinging of branches on land, halo caused by headlights at night, and rain and snow weather. In addition, when the camera is easily shaken on a windy day, especially on a high light pole, image changes caused by these phenomena should be filtered out. They can be solved by algorithms or other technical means.
From the perspective of the algorithm, it can be simply divided into two categories. One is to establish a background model and find the moving object by comparing it with the background model. The other is through the “optical flow†method to discover moving objects by finding the effect of moving objects on the optical flow field. The other is a method that combines the two or both.
3, multi-object tracking. Tracing is essentially stringing together the same objects found on each frame in chronological order. This area itself is a relatively independent and active research area. The main research direction is to effectively track under complex circumstances such as multiple moving objects, multiple cameras, moving objects blocking each other, disappearing and recurring.
In practical monitoring applications, especially for some intrusion alarm application cases, the requirements for tracking algorithms are relatively low. The existing system does not have a satisfactory tracking effect on "fusion" of moving objects and other complicated application scenes. However, with reference to the pace of technological development in the past, this aspect will soon be perfected.
4, behavioral characteristics analysis. The behavioral feature analysis is to look for an event that meets a predetermined behavioral characteristic from the image. Typical applications on the market today include:
(1) Classification. Judging that the moving object is a person, a vehicle, a ship, an airplane, stopping or suddenly accelerating, for example, the scene of a scene in which the vehicle is anchored in a tunnel or on a highway, and the street is robbed and then run away.
(2) Hey. Such as people in sensitive areas. Pedestrians and vehicles that pass normally are not alerted.
(3) Leftovers. Place explosives at airports, oil depots, etc. and leave.
(4) The item is missing. If the museum’s valuable exhibits discover the disappearance of exhibits, the system will immediately call the police.
(5) Statistics of the number of people. For example, statistics on the number of people entering supermarkets and other places, and the combination of sales data to draw the average consumption curve of the day.
(6) Population density. Such as when the alarm occurs when a large number of people gather, or when the crowd suddenly disperses abnormal conditions, an alarm is issued.
(7) The staff fell to the ground. For example, when the staff suddenly changes from straight to flat.
In general, intelligent video analytics can do a lot of things and require video analytics developers and end users to communicate effectively. As intelligent video analysis is still a relatively new technology, the circle that knows this technology in the country currently only expands to the integrator level. Many application scenarios suitable for video analysis technology have yet to be developed in the market. However, one thing is clear: companies must master core technologies and have independent research and development capabilities. The intelligent video analytics market is composed of many subdivided small markets, and new applications are constantly emerging.
5, set the alarm conditions. The introduction of "smart" in video surveillance has greatly enriched the monitoring content. The currently available alarm elements include areas, time periods, object types, dimensions, direction of movement, speed, behavioral characteristics, and much more.
6, alarm linkage. After the abnormal situation is found in the intelligent video analysis system, it is usually necessary to verify the authenticity of the alarm, such as detailed investigation of the close of the alarm event by another PTZ camera, or promptly notify and remind the monitoring personnel. Common real-time prompt methods include voice, pop-up images, sending text messages, screenshots and other means to prompt monitoring personnel.
Main Product Forms and Features Current intelligent video analysis products are mainly based on general-purpose CPUs such as Intel (server, industrial computer) or DSP. Some products are integrated with DVRs, and some products are made into independent modules that provide an interface and development SDK for integrators. The most integrated product has been integrated with the camera and outputs the intelligent analysis results directly.
A server-based (IPC)-based system is usually suitable for being deployed in the background of a monitoring system. Since its architecture is relatively open, it can be easily integrated with existing monitoring systems. In addition, the CPU processing capability of the server is much higher than that of the DSP, and more complex algorithms can be used. In the past, optimization operations peculiar to DSP processors have also become commonplace on general-purpose processors. Multi-core is the development direction of Intel CPU, and it is very suitable for the needs and development trend of multi-channel image processing, which is helpful for reducing system cost. Intel launches a new product every two years at a much faster rate than Texas Instruments. Server-based system performance can be easily upgraded with Intel product updates. In some high-end intelligent video surveillance systems, the use of servers More. DSP's modular products are usually suitable for being placed at the front end of the monitoring system. It is easy to install and implement, and the protection of intellectual property is also easy to do.
The successful installation and use of intelligent video analysis products should usually be directly supported by experienced integrators or developers in the form of solutions. Nowadays, smart video analysis products on the market have made substantial progress in monitoring efficiency and quality compared with traditional monitoring forms, especially in real-time alarms. But there are still some gaps between the standards that we expect. We can't expect to set up a system just as if we were installing surveillance cameras. Like any new technology, to achieve good results, engineers who understand the characteristics of intelligent video analysis products must have full support from project design to construction and commissioning.