Spatial and dynamic models. Classification of types of modeling

There is a class of tasks in which information about the structure and movement of objects of a complex scene is of particular importance (video surveillance in enclosed spaces, in crowded places, control of the movement of robotic complexes, monitoring of the movement of vehicles, etc.). Image sequences are a complex information resource structured in space and time and combining the original information in the form of multidimensional signals, the form of its representation in a computer and physical models dynamic objects, phenomena, processes. New technical capabilities of digital image processing make it possible to partially take into account such specifics of images, while simultaneously using the achievements of the cognitive theory of human perception of visual images.

Analysis of the spatio-temporal volume of data makes it possible to identify not only static, but also dynamic features of objects of observation. In this case, the recognition problem can be defined as a classification of sets of states or as a classification of trajectories, the solution of which cannot be found by classical recognition methods, because temporal transitions can generate image transformations that are not described by known analytical dependencies; Also, along with the task of recognizing dynamic objects, there are tasks of recognizing active actions and events, for example, to detect unauthorized actions in crowded places or to determine the genre of a scene for indexing in multimedia databases. If we consider the task of recognizing objects and events from sequences of images as a single process, then the most appropriate is a hierarchical approach with elements of parallel processing at each level.

Improvement of technical means for collecting and reproducing information in the form of static images (photos) and video sequences requires further development of methods and algorithms for their processing, analysis of situations and recognition of depicted objects. The initial theoretical formulation of the problem of image recognition dates back to 1960-1970. and is reflected in a number of works by well-known authors. The formulation of an image recognition problem can vary from the object recognition problem itself, scene analysis problems to image understanding problems and machine vision problems. At the same time, intelligent decision-making systems based on pattern and image recognition methods use input information of a complex type. It includes both images obtained in a wide wave range of the electromagnetic spectrum (ultraviolet, visible, infrared, etc.), as well as information in the form of sound images and location data. Despite the different physical nature, such information can be represented in the form of real images of objects and specific images. Radiometric data is flat images of a scene presented in perspective or orthogonal projection. They are formed by measuring the intensity of electromagnetic waves of a certain spectral range, reflected or emitted by objects in the scene. Usually, photometric data measured in the visible spectral range are used - monochromatic (brightness) * or color images: Location data are the spatial coordinates of the observed points of the scene. If the coordinates are measured for all points of the scene, then such an array of location data can be called an image of the depth of the scene. There are simplified image models (for example, affine projection models, represented by low-perspective, para-perspective, orthogonal and parallel projections), in which the depth of the scene is considered a constant value, and the location image of the scene does not carry useful information. In this case, sound information has an auxiliary event character.

Photometric data are measured most quickly. Location information, as a rule, is calculated from data obtained from special devices (for example, a laser range finder, radar) or using a stereoscopic method for analyzing brightness images. Due to the difficulties of obtaining location data quickly (especially for scenes with a rapidly changing shape of visual objects), the tasks of describing a scene from a single visual image prevail, i.e. tasks of monocular visual perception of the scene. In the general case, it is impossible to completely determine the scene geometry from a single image. Only under certain restrictions for fairly simple model scenes and the availability of a priori information about the spatial arrangement of objects, it is possible to build a complete three-dimensional description from a single image. One of the ways out of this situation is the processing and analysis of video sequences received from one or more video cameras installed motionless or moving in space.

Thus, images are the main form of representation of information about the real world, and further development of methods for transforming and semantic analysis of both individual images and video sequences is required. One of the most important directions in the development of such intelligent systems is the automation of the choice of methods for describing and transforming images, taking into account their informational nature and recognition goals already at the initial stages of image processing.

The first work of researchers from the USA (Louisiana State University, Carnegie Mellon University, Pittsburgh), Sweden ("Computational Vision and Active Perception Laboratory (CVAP), Department of Numerical Analysis and Computer Science), France (INRIA), Great Britain (University of Leeds) , Germany (University of Karlsruhe), Austria (University of Queensland), Japan, China (School of Computer Science, Fudan University) on image sequence processing and dynamic object recognition were published in the late 1980s Later, similar works began to appear and in Russia: in Moscow (MGU, MAI (STU), MIPT, GosNII AS), St. Petersburg (SPbSU, GUAP, FSUE GOI, LOMO), Ryazan (RGRTU), Samara (SSAU), Voronezh (VSU), Yaroslavl ( YarSU), Kirov (VSU), Taganrog (TTI SFU), Novosibirsk (NSU), Tomsk (TSPU), Irkutsk (IrSU), Ulan-Ude (ESTU) and other cities. in this area, as Academician of the Russian Academy of Sciences, Doctor of Technical Sciences Yu. I. Zhuravlev, Corresponding Member of the Russian Academy of Sciences, Doctor of Technical Sciences V. A. Soifer, Doctor of Technical Sciences N. G. Zagoruiko, Doctor of Technical Sciences L. M. Mestetsky, Doctor of Technical Sciences B. A. Alpatov and others. To date, significant progress has been made in the construction of video surveillance systems, identity authentication systems based on images, etc. However, there are unresolved problems in the recognition of dynamic images due to the complexity and diversity of the behavior of objects in the real world. Thus, this direction needs to improve models, methods and algorithms for recognizing dynamic objects and events from image sequences in different ranges of electromagnetic radiation, which will allow developing video surveillance systems at a qualitatively new level.

The purpose of the dissertation work is to increase the efficiency of recognition of dynamic objects, their active actions and events in complex scenes by image sequences for outdoor and indoor video surveillance systems.

The goal set determined the need to solve the following tasks:

To analyze methods for estimating movement and finding signs of movement of objects from a set of sequential images, methods for segmenting dynamic objects and semantic analysis of complex scenes, as well as approaches to building systems for recognizing and tracking dynamic objects for various purposes.

Develop models for recognition of static and dynamic images based on a hierarchical procedure for processing time series, in particular, image sequences.

To develop a method for estimating the movement of dynamic structures based on spatiotemporal information obtained in different ranges of electromagnetic radiation, which allows choosing segmentation methods depending on the nature of the movement and, thereby, performing adaptive recognition of dynamic images.

Create a model of multilevel movement of dynamic structures in a complex scene, which allows, based on the obtained odometric data, to build trajectories of movement of dynamic structures and put forward hypotheses about the existence of visual objects based on the analysis of the prehistory of movements.

Develop a complex segmentation algorithm that takes into account the set of identified features of dynamic structures for arbitrary directions of movement and overlap of object projections, based on a model of multilevel movement in complex scenes.

Develop a method for recognizing dynamic images presented in terms of a formal grammar and a scene videograph based on the method of collective decision making, as well as methods for recognizing active actions and events in a complex scene using graphs of active actions and events (extending the videograph of a complex scene), and a Bayesian network .

Based on the developed methods and models, design experimental systems for various purposes; designed for processing sequences of images of objects characterized by a fixed and arbitrary set of 2£>-projections, and -recognition of dynamic images c. difficult scenes.

Methods, research. When performing the dissertation work, methods of pattern recognition theory, descriptive theory of image recognition, signal processing theory, methods of vector analysis and tensor calculus, as well as group theory, theory of formal grammars were used.

The scientific novelty of the dissertation work is as follows:

1. A new dynamic image transformation model has been built, which is distinguished by extended hierarchical levels of segmentation (according to local and global motion vectors) and recognition (of objects and their active actions), which allows finding target features for static scenes with moving objects and dynamic scenes based on the concept of maximum dynamic invariant.

2. The descriptive theory of image recognition has been expanded by introducing four new principles: taking into account the recognition goal at the initial stages of analysis, recognition of the behavior of dynamic objects, estimation of prehistory, a variable number of observation objects, which improves the quality of recognition of moving objects by increasing the information content of the initial data.

3. For the first time, an adaptive spatiotemporal method for estimating motion in synchronous sequences of the visible and infrared ranges of electromagnetic radiation has been developed, which makes it possible to extract signs of motion at various hierarchical levels, combining the advantages of both types of image sequences.

4. A new model of multi-level movement has been developed; allowing to decompose the scene into separate levels; not > limited; generally accepted division into foreground and background, which allows for more reliable segmentation of images of objects in; complex perspective scenes.

5: Justified? and built; new; generalized algorithm for segmentation of dynamic objects; with, applying, a set of features^ including behavioral histories; and allows you to track both the dynamics of individual visual objects and the interaction of objects in the scene (overlapping projections; the appearance / disappearance of objects from the field of view of the video sensor) based on group transformations; and the first proposed analysis of the common part of object projections (from two adjacent frames) using integral and invariant estimates.

6. The method of collective decision-making is modified, which differs in finding signs of inter-frame projections of an object and allows taking into account the history of observations for recognizing active actions and events based on the Bayesian network, and also four types of pseudo-distances are proposed to find a measure of similarity v of dynamic images with reference dynamic images in depending on the representation of dynamic features.

Practical significance. The methods and algorithms proposed in the dissertation work are intended for practical application in the "monitoring of vehicles in multi-lane traffic within the framework of the state project "Safe City", in automated control systems for various technological production processes by video sequences, in outdoor video surveillance systems and indoor video surveillance, as well as in systems for identifying objects on aerial photographs and recognizing landscape images.On the basis of dissertation research, software systems for processing and recognizing dynamic objects used in various fields of activity have been developed.

Implementation of work results. The developed programs are registered in the Russian Register of Computer Programs: the program “Image Segmentation handwriting(SegPic)" (Certificate No. 2008614243, Moscow, September 5, 2008); Motion Estimation program (certificate No. 2009611014, Moscow, February 16, 2009); program "Localization of the face (FaceDetection)" (certificate No. 2009611010, Moscow, February 16, 2009); the program "System for imposing visual natural effects on a static image (Natural effects imitation)" (certificate No. 2009612794, Moscow, July 30, 2009); program "Visual smoke detection (SmokeDetection)" (certificate No. 2009612795, Moscow, July 30, 2009); "Program for visual registration of state license plates of vehicles during multi-threaded traffic (FNX CTRAnalyzer)" (certificate No. 2010612795, Moscow, March 23, 2010), program "Nonlinear image enhancement" (certificate No. 2010610658, g. Moscow, March 31, 2010

Acts on the transfer and use of algorithmic and software for recognizing refrigerator cases on an assembly line (JSC KZH Biryusa, Krasnoyarsk), for identifying images of objects on landscape images (Concern for Radio Engineering Vega, JSC KB Luch, Rybinsk, Yaroslavl Region), for forest segmentation of vegetation by a set of successive aerial photographs (OOO Altex Geomatica, Moscow), to detect plates of state license plates of vehicles in video sequences during multi-stream traffic and improve the quality of their display^ (UGIBDD of the Central Internal Affairs Directorate for the Krasnoyarsk Territory, Krasnoyarsk).

The developed algorithms and software are used in the educational process when conducting classes in the disciplines "Intelligent Data Processing", "Computer Technologies in Science and Education", "Theoretical Foundations of Digital Image Processing", "Pattern Recognition", "Neural Networks", "Processing Algorithms images”, “Algorithms for processing video sequences”, “Scene analysis and machine vision” at the Siberian State Aerospace University named after Academician M.F. Reshetnev (SibGAU).

The reliability of the results obtained in the dissertation work is ensured by the correctness of the research methods used, the mathematical rigor of the performed transformations, as well as the correspondence of the formulated provisions and conclusions to the results of their experimental verification.

The main provisions for defense:

1. A model for processing and recognizing dynamic images in complex scenes, significantly expanded by hierarchical levels of segmentation and recognition of not only objects, but also their active actions.

2. Extension of the descriptive theory of image recognition for time series (sequences of images) by increasing the information content of the analyzed data not only in the spatial domain, but also in the time component.

3. Adaptive spatio-temporal method for estimating motion on. on the basis of tensor representations of local IS volumes in synchronous sequences of the visible and infrared ranges of electromagnetic radiation.

4. A model of multi-level movement in complex scenes, which expands the decomposition of perspective scenes into separate levels for a more reliable analysis of object movement trajectories.

5. A generalized segmentation algorithm for dynamic objects that allows, on the basis of group transformations and the proposed integral and invariant estimates, to identify overlapping object projections, the appearance / disappearance of objects from the field of view of the video sensor.

6. Methods for recognizing dynamic images based on a modified method of collective decision making and finding pseudo-distances in metric spaces, as well as active actions and events in complex scenes.

Approbation of work. The main provisions and results of dissertation research were reported and discussed at the 10th international conference "Pattern Recognition and Image Analysis: Modern Information Technologies", (S.-Petersburg, 2010), the international congress "Ultra Modern Telecommunications and Control Systems ICUMT2010" (Moscow, 2010) ; XII International Symposium on Nonparametric Methods in Cybernetics and System Analysis (Krasnoyarsk, 2010), II International Symposium "Intelligent Decision-Technologies - IDT 2010" (Baltimore, 2010), III International Conference. "Automation, Control? and Information Technology - AOIT-ICT"2010" (Novosibirsk, 2010), 10th, 11th and 12th international conferences and exhibitions "Digital signal processing and its applications" (Moscow, 2008 - 2010), X international scientific and technical conference "Theoretical and applied issues of modern information technologies" (Ulan-Ude, 2009), IX international scientific and technical conference "Cybernetics and high technologies of the XXI century" (Voronezh, 2008), all-Russian conference "Models and methods Image Processing” (Krasnoyarsk, 2007), at the X, XI and XIII international scientific conferences “Reshetnev Readings” (Krasnoyarsk, 2006, 2007, 2009), as well as at scientific seminars of the State University of Aerospace Instrumentation (St. Petersburg, 2009), Institute for Computational Modeling of CO

RAS (Krasnoyarsk, 2009), Institute of Image Processing Systems RAS (Samara, 2010).

Publications. Based on the results of the dissertation research, 53 printed works were published, including 1 monograph, 26 articles (of which 14 articles - in publications included in the list of HAC, 2 articles - in publications listed in the Thomson Reuters: Science Citation Index Expanded / Conference Proceedings Citation Index”), 19 abstracts, 7 certificates registered in the Russian Register of Computer Programs, as well as 3 research reports.

Personal contribution. All the main results presented in the dissertation, including the formulation of problems and their mathematical and algorithmic solutions, were obtained by the author personally, or performed under his scientific supervision and with direct participation. Based on the materials of the work, two dissertations were defended for the degree of candidate of technical sciences, during which the author was the official supervisor.

Work structure. The work consists of introduction, six chapters, conclusion, bibliography. The main text of the dissertation contains 326 pages, the presentation is illustrated by 63 figures and 23 tables. The bibliographic list includes 232 titles.

6.7 Chapter Conclusions

In this chapter, the structure and main functions of the experimental software complex "ZROEL", v.1.02, which; performs systemic hierarchical processing of image sequences up to the highest levels of object and event recognition, are considered in detail. automated system, which requires human participation for training and tuning of graphs, networks and classifiers. A number of low-level system modules operate automatically. The structure of the software package is such that it is possible to modify modules without affecting other modules of the system. Functional diagrams of the main modules of the system are presented: a module, preprocessing, a motion estimation module, a segmentation module, an object recognition module, and an active actions recognition module.

Experimental studies based on this software package were carried out on several video sequences and infrared sequences from the OTCBVS^07 test database, on the test video sequences of Hamburg taxi, Rubik cube. "Silent", as well as on their own video material. Five motion estimation methods were tested. It has been experimentally shown that the block matching method and the proposed infrared sequence method show similar values ​​and are the least accurate. The proposed method for the video sequence and the method for tracking point features show close results. At the same time, the developed tensor approach requires a smaller amount of computer calculations compared to the method of tracking point features. Sharing It is expedient to use the synchronized video sequence and the infrared sequence to find the modulus of the velocity vector in conditions of reduced stage illumination.

To recognize visual objects, four types of pseudo-distances were used (Hausdorff, Gromov-Hausdorff, Fréchet pseudo-distances, natural pseudo-distance) to find a measure of similarity of input dynamic images with reference dynamic images (depending on the presentation of a dynamic feature - a set of numerical characteristics, sets of vectors, sets of functions). They have shown their validity for images with admissible morphological transformations. We used integrated normalized estimates of the shape of the contour Kc of the common part of the projection of the object between conditionally adjacent frames and the area of ​​the common part 5e and an invariant estimate - the correlation function of the common parts of the projections Fcor. The use of a modified method of collective decision-making makes it possible to "discard" unsuccessful observations of input images (cases of overlapping projections of objects, scene distortion from lighting sources, etc.) and select the most appropriate observations. Experiments have shown that the use of a modified method of collective decision-making increases the accuracy of recognition by an average of 2.4-2.9%.

Experimental results of motion assessment, segmentation and object recognition were obtained on test sequences of images ("Hamburg taxi", "Rubik cube". "Silent", video sequences and infrared sequences from the test database "OTSVVS" 07). examples from the test databases "PETS", "CAVIAR", "VACE". The nature of the test visual sequence affects the performance. Objects that carry out rotational movement are recognized worse ("Rubik cube"), better - man-made objects of small sizes ("Hamburg taxi", "Video 1"). The best results are shown by recognition by two sequences. Also, the best experimental results were achieved when recognizing periodic active actions of people not in groups (walking, running, raising hands). False positives are due to the presence of shadows in a number of places on the scene .

At the end* of the sixth chapter, such applied projects were considered as “Visual registration of state license plates of vehicles in multi-stream traffic”, “System for identifying models of refrigerator cases from images”, “Algorithms for processing i-segmentation, landscape images. Identification of objects ". Algorithmic and. software was transferred to interested organizations: The results of test operation showed the operability of the software developed on the basis of the models and methods proposed in the dissertation work.

CONCLUSION

In the dissertation work, an important scientific and technical problem of processing spatio-temporal data obtained from sequences of the visible and infrared ranges of electromagnetic radiation and recognizing dynamic images in complex scenes was posed and solved. The system of hierarchical methods for processing and extracting features from spatio-temporal data is a methodological basis for solving applied problems in the field of video surveillance.

The introduction substantiates the relevance of the dissertation work, formulates the goal and sets the research objectives, shows the scientific novelty and practical value of the research performed, and presents the main provisions submitted for defense.

The first chapter shows that visual objects in video sequences are characterized by a more multidimensional feature vector than images in the classical formulation of the static image recognition problem.

A classification of the main types of recognition problems for static images, static scenes with motion elements and image sequences is constructed, which reflects the historical nature of the development of mathematical methods in this area. A detailed analysis of motion estimation methods, segmentation algorithms for moving objects, and methods for interpreting events in complex scenes was carried out.

Existing commercial hardware and software systems in such areas as monitoring vehicles for various purposes, processing sports video materials, security (face recognition, unauthorized entry of people into a protected area) are considered. Research developments for video surveillance systems are also analyzed.

At the end of Chapter 1, the statement of the problem of spatiotemporal processing of image sequences is presented, presented in the form of three levels and five stages of processing and recognition of visual information from image sequences.

In the second chapter of the dissertation, formal models for processing and recognizing objects by their static images and image sequences are developed. Admissible mappings are constructed in the space of images and the space of features for the direct problem and the inverse problem. Rules for constructing invariant decision functions and a generalized maximum dynamical invariant are given. When recognizing, the trajectories of different images in the multidimensional space of features can intersect. When the projections of objects intersect, finding a generalized maximum dynamic invariant becomes even more difficult, and in some cases even impossible.

The basic principles of the descriptive theory of image recognition are considered, which is based on regular methods for selecting and synthesizing algorithmic procedures for processing information in image recognition. Additional principles are proposed that expand the descriptive theory for dynamic images: taking into account the recognition goal at the initial stages of image sequence processing, recognition of behavioral situations of dynamic objects, estimation of the prehistory of dynamic objects, a variable number of observation objects in complex scenes.

The problem of searching for target features for analyzing image sequences depending on the type of shooting (in the case of single-angle shooting), the movement of the video sensor, and the presence of moving objects in the visibility zone is considered in detail. Descriptions of four situations in the feature space are given as the task becomes more complex.

The third chapter formulates the stages of processing sequences of images and recognition of objects, active actions, events and scene genre. The stages reflect the sequential hierarchical nature of visual information processing. The conditions and limitations of hierarchical methods for spatiotemporal processing of image sequences are also presented.

The classification of image dynamic regions is performed by analyzing the eigenvalues ​​31) of the structural tensor, whose eigenvectors are determined from local displacements of the image intensities of neighboring frames and are used to estimate the local orientations of dynamic regions. A new method for estimating motion in the space-time volume of data in the visible and infrared ranges of radiation based on the tensor approach is substantiated. The possibility of using a spatially variable kernel, adaptive to the size and orientation of the point environment, is considered. Adaptation of the environment, which initially has the shape of a circle, and then turns into the shape of an oriented ellipse after 2-3 iterations, improves the assessment of oriented structures in the image. Such a strategy improves gradient estimates in the spatiotemporal dataset.

Estimation of local motion parameters is performed by calculating geometric primitives and singular points of the local region. Thus, the assessment of local signs of the movement of regions is the basis for putting forward subsequent hypotheses that visual objects belong to one or another class. The use of synchronous video sequences and infrared sequences improves the results of segmentation of moving regions in the image and finding local motion vectors.

It is shown that the boundaries in color images can be estimated based on multidimensional gradient methods built in all directions at each point of the boundary, vector methods using order statistics about a color image, as well as using a tensor approach in the framework of multidimensional gradient methods. Ways to refine contour information are essential for regions with an arbitrary number of valid projections.

In the fourth chapter, a multi-level motion model is built based on motion structures, which reflects the dynamics of objects in real scenes and expands the two-level representation of the scene, divided into objects of interest and a stationary background.

Models of motion of objects on a plane based on the theory of compact Lie groups are investigated. Models for projective transformation and varieties of affine transformation models are presented. Such transformations well describe motion structures with a limited number of projections (technogenic objects). The representation of structures with an unlimited number of projections (anthropogenic objects) by affine or projective transformations is accompanied by a number of additional conditions (in particular, the requirement that objects are far from the video sensor, small-sized objects, etc.). Definitions and a theorem proved by L. S. Pontryagin are given, on the basis of which it was possible to find an internal automorphism of group coordinates describing some object up to shifts between neighboring frames. The magnitude of the shifts is determined by the method for estimating the movement of the interframe difference developed in Chapter 3.

An extension of admissible transitions between groups of transformations due to the duality of the nature of 2£)-images (display of changes in the projection of an individual object and visual intersection of several objects: (object interaction)) is constructed. Criteria are found that, when changing groups of transformations, fix active actions and events in the scene, namely, integrated estimates of the shape of the contour Kc of the common part of the projection between conditionally adjacent frames and the area of ​​the common part 5e and invariant estimates - the correlation function of the common parts of the projections Pcog and structural Lie group constants c "g, which allow us to estimate the degree of variability and reveal the nature of the movement of observed objects.

A model of the prehistory of the movement of objects in image sequences was also built, including time series of movement trajectories, changes in the shape of an object when it moves in 3L>-space, as well as changes in the shape of an object associated with the interaction of objects in the scene and the appearance/disappearance of an object from the field of view of the sensor (used to recognize active actions and events in the scene). 1

A generalized algorithm for segmenting objects in complex scenes has been developed that takes into account complex cases of segmentation (overlapping images, the appearance and disappearance of objects from the camera's field of view, movement towards the camera), which includes three sub-stages: pre-segmentation, segmentation and post-segmentation. For each sub-stage, tasks, initial and output data are formulated, flowcharts of algorithms are developed that allow segmenting complex scenes using the advantages of synchronous sequences from different radiation ranges.

The fifth chapter deals with the process of dynamic pattern recognition using a formal grammar, a scene videographer, and a modified method of collective decision making. A dynamic scene with multi-level movement has a time-varying structure, so it is advisable to use structural recognition methods. The proposed three-level contextual grammar for recognizing complex scenes with multilevel movement of objects implements two tasks: the task of parsing a sequence of images and the task of parsing a scene.

A more visual means of semantic description of a scene is a videograph built using the hierarchical grouping method. Based on the complex features of the lower level, local spatial structures that are stable in time, local spatial objects are formed, and a videograph of the scene is built, including recognized spatial objects, a set of their inherent actions, as well as spatio-temporal connections between them.

The modified method of collective decision making is based on a two-level recognition procedure. At the first level, the recognition of the belonging of an image to a particular area of ​​competence is carried out. At the second level, the decision rule comes into force, the competence of which is maximum in a given area. Expressions for pseudo-distances are constructed when finding a measure of similarity of input dynamic images with reference dynamic images, depending on the representation of dynamic features - a set of numerical characteristics, a set of vectors, a set of functions.

When recognizing events, the complex scene videographer is extended to the event videographer: An object-dependent model of a dynamic object is built. As a matching function, the simplest classifiers in the feature space are used (for example, by the ^-means method), since the matching is carried out according to a limited set of templates associated with a previously identified object. The ways of forming templates of projections of visual objects are considered.

The videograph of events is built on the basis of Markov networks. Methods for detecting active actions of agents, as well as the procedure for constructing and cutting an event videograph for recognizing events in a scene, are considered. At the same time, for each event, its own model is built, which is trained on test examples. Event detection is reduced to clustering sequentially executed active actions based on a Bayesian approach. A recursive cutting is performed - the matrix of weight coefficients in the input video sequence and comparison with the reference events obtained at the training stage. This information is* the source for determining the genre of the scene and, if necessary, indexing the video sequence in the database. A scheme for understanding and interpreting images and video materials for indexing in multimedia Internet databases has been developed.

The sixth chapter presents a description of the experimental software complex "SPOER", v.l.02 for processing image sequences and recognizing moving objects and events. It performs systemic hierarchical processing of image sequences up to the highest levels of object and event recognition. It is an automated system that requires human intervention to train and tune graphs, networks, and classifiers. A number of low-level system modules operate automatically.

In experimental studies conducted using the SPOER software package, v.l.02, video sequences and infrared image sequences from the OTCBVS "07 test base", test video sequences "Hamburg taxi", "Rubik cube". "Silent" and our own video materials were used. Five motion estimation methods were tested.The proposed method for the video sequence shows the most accurate results and requires less computer calculations compared to other methods.The combined use of synchronized video sequences and infrared sequences is useful when finding velocity vector modules in low-light scene conditions.

To recognize visual objects with acceptable morphological transformations of projections, we used integrated normalized estimates of the shape of the contour Kc of the common part of the object projection between conditionally adjacent frames and the area of ​​the common part 5e and an invariant estimate - the correlation function of the common parts of the projections Fcor. The use of a modified method of collective decision-making makes it possible to "discard" unsuccessful observations of input images (cases of overlapping projections of objects, visual distortions of the scene from light sources, etc.) and select the most appropriate observations. Experiments have shown that the use of a modified method of collective decision-making increases the accuracy of recognition by an average of 2.4-2.9%.

Experimental score-motion results; segmentation and object recognition were obtained on test sequences of images ("Hamburg taxi", "Rubik cube", "Silent", video sequences and infrared sequences from the test database "OTCBVS * 07"). To recognize the active actions of people, examples from the test databases "PETS", "CAVIAR", "VACE" were used. The best results are shown by recognition by two sequences. Also, the best experimental results were achieved when recognizing periodic active actions of people not in groups (walking, running, raising hands). False positives are caused by backlight and the presence of shadows in a number of places in the scene.

On the basis of the experimental complex "ZROEYA", V. 1.02, video information processing systems for various purposes were developed: "Visual registration of state license plates of vehicles in multi-stream traffic", "Identification system for models of refrigerator cases by images", "Algorithms for processing and segmenting landscape images . Identification of objects". Algorithmic and software were transferred to interested organizations. The results of the test operation showed the operability of the software developed on the basis of the models and methods proposed in the dissertation work.

Thus, the following results were obtained in the dissertation work:

1. Formal models of processing and recognition of spatio-temporal structures are constructed based on an adaptive hierarchical procedure. processing of image sequences, which differ in that they take into account isomorphic and homomorphic transformations and derive generalized functions of static and dynamic invariants. Models for searching for static and dynamic features of objects were also built for four tasks of analyzing sequences of images, depending on the presence of a moving1 video sensor and moving objects in the scene.

2. The main provisions of the descriptive approach to image sequence recognition have been expanded, allowing to take into account the goals of recognition at the initial stages of image sequence processing with subsequent segmentation of areas of interest, build motion trajectories and recognize the behavior of dynamic objects, take into account the history of the movement of objects when crossing their projections, accompany a variable number objects of observation.

3. A hierarchical method for processing and recognizing spatio-temporal structures has been developed, consisting of three levels and five stages and involving the normalization of object projections, which makes it possible to reduce the number of standards for one class when recognizing complex dynamic objects.

4. A method for estimating motion for sequences of images from the visible and infrared ranges of electromagnetic radiation has been developed, which differs in that spatio-temporal data sets are used, presented in the form of 3£> structural tensors and bB tensors. flow, respectively. The resulting motion estimate allows you to choose the most effective method segmentation of dynamic visual objects that differ in the number of valid projections.

5. A model of multilevel movement of image regions based on local velocity vectors has been built, which differs in that it allows dividing the scene not only into foreground and background objects, but also into levels of movement of objects distant from the observer. This is especially true for complex scenes recorded by a moving video sensor, when all objects in the scene are in relative motion.

6. An adaptive segmentation algorithm for dynamic objects has been developed: a) for objects with a limited number of projections, based on the analysis of the prehistory of the movement of local dynamic regions, characterized in that when images overlap, the shape of the region is completed according to the current template and, subject to the application of the Kalman filter, it is predicted current, trajectory; b) for objects with an arbitrary number of projections based on complex analysis, color, texture, statistical, topological and motion features, characterized in that when images overlap, the shape of the region is completed using the active contours method.

7. A method is proposed for constructing a dynamic videograph of a complex scene using the method of hierarchical grouping of lower-level complex features into local spatial structures that are stable in time, and then into local spatial objects. The generated videographer establishes temporal relationships between objects and retains all generalized features for recognizing events in the scene. The two-dimensional grammar of M.I. Schlesinger in the framework of the structural recognition method to a three-level contextual grammar.

8: For the recognition of dynamic objects, the collective decision-making method is modified, which first recognizes that the image belongs to the area of ​​competence, and then chooses the decision rule whose competence is maximum in the given area. Four types of pseudo-distances are constructed to find a measure of similarity between input dynamic images and standards, depending on the representation of dynamic features.

9. A method for recognizing events based on the Bayesian network has been developed, which performs recursive cutting of the matrix of weight coefficients in the input video sequence and comparison with reference events obtained at the training stage. This information is the source for determining the genre of the scene and indexing video sequences in multimedia Internet databases.

10. Practical problems of processing and recognition of sequences of images are solved using the adaptive-hierarchical method of spatio-temporal processing, the efficiency of the method is shown, the effectiveness of the system of hierarchical processing methods is demonstrated, etc. recognition of visual information with the possibility of adaptive selection of features c. problem solving process. The results obtained in the form of designed experimental systems were transferred to interested organizations.

Thus, in this dissertation work, an important scientific and technical problem of information support for video surveillance systems has been solved and a new direction has been developed in the field of spatio-temporal processing and recognition of dynamic images.

1. Automatic analysis of complex images / Ed. EM. Bra-vermana. M.: Mir, 1969. - 309 p. Bongard M.M. Recognition problems. - M.: Nauka, 1967.-320 p.

2. Alpatov, B.A., Detection of a moving object in a sequence of images in the presence of restrictions on the area and speed of the object / B.A. Alpatov, A.A. Kitaev // Digital Image Processing, No. 1, 2007. p. 11-16.

3. Alpatov, B.A., Selection of moving objects under conditions of geometric image distortions / B.A. Alpatov, P.V. Babayan // Digital Signal Processing, No. 45 2004. p. 9-14.

4. Alpatov, B.A., Babayan P.V. Methods of processing and analysis of images" in on-board systems for detecting and tracking objects / B.A. Alpatov, P.V. Babayan // Digital signal processing, No. 2, 2006. 45-51 p.

5. Bolshakov, A.A., Methods for processing multivariate data and time series: Textbook for universities / A.A. Bolshakov, R.I. Karimov / M.: Hotline-Telecom, 2007. 522 p.6: Bongard, M.M. Problems of recognition / M.M. Bongard / M.: Nauka, 1967.-320 p.

6. Bulinsky, A.B. Theory of random processes1 / A.V. Bulinsky, A.N. Shiryaev / M.: FIZMATLIT, 2005. 408 p.

7. Weinzweig, M.N. Architecture of the visual dynamic scene representation system in terms of concepts / M.N. Vaintsvaig, M.N. Polyakov // Sat. tr. 11th All-Russian. conf. "Mathematical methods of pattern recognition (MMRO-11)", M., 2003. pp. 261-263.

8. Vapnik, V.N. The task of learning pattern recognition / V.N. Vapnik / M.: Knowledge, 1970. - 384 p.

9. P.Vapnik, V.N. Theory of Pattern Recognition (Statistical Problems of Learning) / V.N. Vapnik, A.Ya. Chervonenkis / M.: Nauka, 1974. 416 p.

10. Vasiliev, V.I. Recognition of moving bodies / V.I. Vasiliev, A.G. Ivakhnenko, V.E. Reutsky and others // Automation, 1967, No. 6, p. 47-52.

11. Vasiliev, V.I. Recognizing systems / V.I. Vasiliev / Kyiv: Nauk. Dumka, 1969. 292 p.

12. Vasiliev, V.I. Recognizing systems. Directory / V.I. Vasiliev / Kyiv, Nauk, Dumka, 1983. 422 p.

13. Vizilter, Yu.V. Application of the method of analysis of morphological evidence in machine vision problems>/ Yu.V. Vizilter // Bulletin of Computer and Information Technologies, No. 9, 2007 p. 11-18.

14. Vizilter, Yu.V. Projective morphologies based on interpolation / Yu.V. Vizilter // Bulletin of Computer and Information Technologies, No. 4, 2008.-p. 11-18.

15. Vizilter, Yu.V., Projective morphologies and their application in the structural analysis of digital images / Yu.V. Vizilter, S.Yu. Zheltov // Izv. RAN. TiSU, No. 6, 2008. p. 113-128.

16. Vizilter, Yu.V. Investigation of the behavior of autoregressive filters in the problem of detection and analysis of motion on digital video sequences / Yu.V. Vizilter, B.V. Vishnyakov // Bulletin of Computer and Information Technologies, No. 8, 2008. - p. 2-8.

17. Vizilter, Yu.V. Projective image morphologies based on models described by structuring functionals /Yu.V. Vizilter, S.Yu. Zheltov // Bulletin of Computer and Information Technologies, No. 11, 2009.-p. 12-21.

18. Vishnyakov, B.V. Using the modified method of optical flows in the problem of motion detection and interframe tracking.

19. Ganebnykh, S.N. Scene analysis based on the use of tree representations of images / S.N. Ganebnykh, M.M. Lange // Sat. tr. 11th All-Russian. conf. "Mathematical methods of pattern recognition (MMRO-11)", M., 2003.-p. 271-275.

20. Glushkov, V.M. Introduction to cybernetics / V.M. Glushkov / Kyiv: Publishing House of the Academy of Sciences of the Ukrainian SSR, 1964. 324 p.

21. Gonzalez, R., Woods R. Digital image processing. Translation from English. ed. P.A. Chochia / R. Gonzalez, R. Woods / M.: Technosfera, 2006. 1072 p.

22. Goroshkin, A.N., Handwritten text image segmentation (SegPic) / A.N. Goroshkin, M.N. Favorskaya // Certificate No. 2008614243. Registered in the Register of Computer Programs, Moscow, September 5, 2008.

23. Grenander, W. Lectures on the theory of images / W. Grenander / In 3 volumes / Translated from English. Ed. Yu.I. Zhuravleva. Moscow: Mir, 1979-1983. 130 s.

24. Gruzman, I.S. Digital image processing in information systems: Textbook. Allowance / I.S. Gruzman, B.C. Kirichuk, V.P. Kosykh, G.I. Peretyagin, A.A. Spektor / Novosibirsk, NSTU publishing house, 2003. p. 352.

25. Reliable and Plausible Inference in Intelligent Systems, Ed. V.N. Vagina, D.A. Pospelov. 2nd ed., rev. and additional - M.: FIZMATLIT, 2008. - 712 p.

26. Duda, R. Pattern recognition and scene analysis / R. Duda, P. Hart / M.: Mir publishing house, 1978. 512 p.

27. Zhuravlev Yu.I. On the algebraic approach to solving problems of recognition and classification / Yu.I. Zhuravlev // Problems of cybernetics: Sat. st., issue. 33, M.: Nauka, 1978. p. 5-68.

28. Zhuravlev Yu.I. On algebraic correction of information processing (transformation) procedures / Yu.I. Zhuravlev, K.V. Rudakov // Problems of applied mathematics and informatics, M.: Nauka, 1987. p. 187-198.

29. Zhuravlev Yu.I. Pattern recognition and image recognition / Yu.I. Zhuravlev, I.B. Gurevich // Yearbook “Recognition. Classification. Forecast. Mathematical methods and their application”, vol. 2, M.: Nauka, 1989.-72 p.

30. Zhuravlev Yu.I. Pattern recognition and image analysis / Yu.I. Zhuravlev, I.B. Gurevich / Artificial intelligence in 3 books. Book. 2. Models and methods: Handbook / Ed. YES. Pospelova, M .: publishing house "Radio and Communication", 1990. - pp. 149-190.

31. Zagoruiko, N.G. Recognition methods and their application / N.G. Za-goruiko / M.: Sov. radio, 1972. 206 p.

32. Zagoruiko, N.G. Artificial intelligence and empirical prediction / N.G. Zagoruiko / Novosibirsk: ed. NSU, ​​1975. 82 p.

33. Ivakhnenko A.G. On the application of the theory of invariance and combined control to the synthesis and analysis of learning systems / A.G. Ivakhnenko // Automation, 1961, No. 5, p. 11-19.

34. Ivakhnenko, G.I. Self-learning recognition and automatic control systems / A.G. Ivakhnenko / Kyiv: Tekhnika, 1969. 302 p.

35. Kashkin, V.B. Remote sensing of the Earth from space. Digital Image Processing: Textbook / V.B. Kashkin, A.I. Su-hinin / Moscow: Logos, 2001. 264 p.

36. Kobzar, A.I. Applied mathematical statistics. For engineers and scientists / A.I. Kobzar / M.: FIZMATLIT, 2006. 816 p.

37. Kovalevsky, V.A. Correlation method of image recognition / V.A. Kovalevsky // Zhurn. Comput. Mathematics and Mathematical Physics, 1962, 2, no. 4, p. 684-690.

38. Kolmogorov, A.N.: Epsilon-entropy and epsilon-capacity of sets in function spaces / A.N. Kolmogorov, V.M. Tikhomirov // Information Theory and Theory of Algorithms. M.: Nauka, 1987. p. 119-198.

39. Korn, G. Handbook of mathematics for scientists and engineers / G.Korn, T. Korn // M.: Nauka, Ch. ed. Phys.-Math. lit., 1984. 832 p.

40. Kronover, R. Fractals and chaos in dynamical systems / R. Kronover // M.: Tekhnosfera, 2006. 488 p.

41. Lapko, A.B. Non-parametric* and hybrid classification systems for heterogeneous data / A.V. Lapko, BlA. Lapko // Tr. 12th All-Russian. conf. "Mathematical methods and models of pattern recognition" (MMRO-12), M., 2005.-p. 159-162.

42. Levtin, K.E. Visual smoke detection (SmokeDetection) / K.E. Levtin, M.N. Favorskaya // Certificate No. 2009612795. Registered in the Register of Computer Programs, Moscow, July 3, 2009.

43. Lutsiv, V.R. Principles of unification of optical systems of robots / V.R. Lutsiv, M.N. Favorskaya // V-book. "Unification and standardization of industrial robots", Tashkent, 1984. p. 93-94.

44. Lutsiv, V.R. Universal optical system for HAP / V.R. Lutsiv, M.N. Favorskaya // In the book. "Experience in the creation, implementation and use of process control systems in associations and enterprises", L., LDNTP, 1984. p. 44-47.

45. Medvedeva, E.V. Method for estimating motion vectors in video images / E.V. Medvedeva, B.O. Timofeev // In the materials of the 12th international conference and exhibition "Digital signal processing and its application", M.: V 2 vol. T. 2, 2010. p. 158-161.

46. ​​Methods of computer image processing / Ed. V.A. Soifer. 2nd ed., Spanish. - M.: FIZMATLIT, 2003. - 784 p.

47. Methods for automatic detection and tracking of objects. Image processing and control / B.A. Alpatov, P.V. Babayan, O.E. Balashov, A.I. Stepashkin. -M.: Radio engineering, 2008. - 176 p.

48. Methods of computer optics / Ed. V.A. Soifer. M.: FIZMATLIT, 2003. - 688 p.

49. Mudrov, A.E. Numerical methods for PC in BASIC, Fortran and Pascal / A.E. Mudrov / Tomsk: MP "RASKO", 1991. 272 ​​p.

50. Pakhirka, A.I. Face localization (FaceDetection) / A.I. Pakhirka, M.N. Favorskaya // Certificate No. 2009611010. Registered in the Register of Computer Programs, Moscow, February 16, 2009.

51. Pakhirka, A.I. Nonlinear image enhancement / A.I. Pakhirka, M.N. Favorskaya // Certificate No. 2010610658. Registered in the Register of Computer Programs, Moscow, March 31, 2010.

52. Pontryagin L. S. Continuous groups J L. S. Pontryagin // 4th ed., M.: Nauka, 1984.-520 p.

53. Potapov, A.A. Fractals in Radiophysics and Radar: Sampling Topology / A.A. Potapov // Ed. 2nd, revised. and additional - M.: Universitetskaya kniga, 2005. 848 p.

54. Radchenko Yu.S. Investigation of the spectral algorithm for detecting "changes in the video sequence" / Yu.S. Radchenko, A.V. Bulygin, T.A. Radchenko // Izv.

55. Salnikov, I.I. Raster space-time signals in image analysis systems / I.I. Salnikov // M.: FIZMATLIT, 2009. -248 p.

56. Sergunin, S.Yu. Scheme of dynamic construction of a multilevel description of images / S.Yu.Sergunin, K.M.Kvashnin, M.I. Kumskov // Sat. tr. 11th All-Russian. conf: "Mathematical Methods of Pattern Recognition (MMRO-11)", M., 2003. p. 436-439:

57. Slynko Yu.V. Solving the problem of simultaneous tracking and contouring by the maximum likelihood method / Yu.V. Slynko // Digital Signal Processing, No. 4, 2008. p. 7-10

58. Solso, R. Cognitive psychology / R. Solso / St. Petersburg: Peter, 6th ed., 2006. 590 p.

59. Tarasov, I.E. Development of digital devices based on the Xi-linx FPGA using the VHDL language / I.E. Tarasov / M.: Hotline-Telecom, 2005. - 252 p.

60. Favorskaya, M.N. Development of algorithms for digital image recognition in adaptive robotic systems / M.N*. Favorskaya // L!, Leningrad Institute of Aviation. Instrumentation, 1985. Manuscript deposit: in VINITI 23.01.85. No. 659-85 Dep.

61. Favorskaya; M.N. Application of spectral methods for normalization and recognition of images in adaptive robotic complexes / M.N. * Favorskaya // L., Leningradsky, in-t aviation. priborostr., 1985. Manuscript dep. at VINITI on January 23, 1985. No. 660-85 Dep.

62. Favorskaya, M.N. Experience in developing object recognition algorithms for stamping production / M.N. Favorskaya // In the book. "State, experience and directions of work on integrated automation based on GPS, RTK and PR", Penza, 1985. p. 64-66.

63. Favorskaya, M.N. Study of projective properties of groups of objects / M.N. Favorskaya, Yu.B. Kozlova // Bulletin of the Siberian State Aerospace University. Issue. 3, Krasnoyarsk, 2002. - p. 99-105.

64. Favorskaya, M.N. Determination of the affine structure of an object by motion / M.N. Favorskaya // Bulletin of the Siberian State Aerospace University, Vol. 6, Krasnoyarsk, 2005. - p. 86-89.

65. Favorskaya- M.N. General classification of approaches to image recognition / M-.N. Favorskaya // V< материалах X междунар. научн. конф. «Решетневские чтения» СибГАУ, Красноярск, 2006. с. 54-55.

66. Favorskaya M.N. Invariant decision functions in static image recognition problems / M.N. Favorskaya // Bulletin of the Siberian State Aerospace University. Issue. 1 (14), Krasnoyarsk, 2007. p. 65-70.

67. Favorskaya, M.N. Probabilistic methods of video stream segmentation as a problem with missing data / M.N. Favorskaya // Bulletin of the Siberian State Aerospace University. Issue. 3 (16), Krasnoyarsk, 2007. p. 4-8.

68. Favorskaya, M.N. Selection of target informative features in image recognition systems / M.N. Favorskaya // In the materials of the XI Inter-Dunar. scientific conf. "Reshetnev Readings" SibGAU, Krasnoyarsk, 2007 p. 306-307.

69. Favorskaya, M.N. Strategies for segmentation of two-dimensional images / M.N. Favorskaya // In the materials of the All-Russian scientific conference "Models and methods of image processing MMOI-2007", Krasnoyarsk, 2007. p. 136-140.

70. Favorskaya, M.N. Segmentation of landscape images based on the fractal approach / M.N. Favorskaya // In the materials of the 10th international conference and exhibition "Digital signal processing and its application", M., 2008. p. 498-501.

71. Favorskaya, M.N. Handwritten text image recognition model / M.N. Favorskaya, A.N. Goroshkin // Bulletin of the Siberian State Aerospace University. Issue. 2" (19), Krasnoyarsk, 2008. pp. 52-58.

72. Favorskaya, M.N. Algorithms for the implementation of motion estimation in video surveillance systems / M.N. Favorskaya, A.S. Shilov // Control systems information Technology. Advanced Research / IPU RAS; VSTU, No. 3.3(33), M.-Voronezh, 2008. p. 408^12.

73. Favorskaya, M.N. On the question of the use of formal grammars in object recognition in complex scenes // M.N. Favorskaya / In the materials of the XIII international scientific conference. "Reshetnev readings". At 2 h. 4.2, Krasnoyarsk, 2009. p. 540-541.

74. Favorskaya, M.N. Dynamic pattern recognition based on predictive filters / M.N. Favorskaya // Bulletin of the Siberian State Aerospace University. Issue. 1(22) at 2 o'clock 4f. 1, Krasnoyarsk, 20091 p. 64-68.

75. Favorskaya, M.N., Methods, motion search in video sequences / M.N. Favorskaya, A.I. Pahirka, A.C. Shilov; M.V. Damov // Bulletin. Siberian State Aerospace University. Issue. 1 (22) at 2 h. Part 2, Krasnoyarsk, 2009. p. 69-74.

76. Favorskaya, M.N. Finding moving video objects using local 3D structural tensors / M.N. Favorskaya // Bulletin of the Siberian State Aerospace University. Issue. 2 (23), Krasnoyarsk, 2009. p. 141-146.

77. Favorskaya, M.N. Evaluation of the movement of objects in complex scenes based on the tensor approach / M.N. Favorskaya // Digital Signal Processing, No. 1, 2010.-p. 2-9.

78. Favorskaya, M.N. Complex calculation of characteristics of landscape images / M.N. Favorskaya, N.Yu. Petukhov // Optical Journal, 77, 8, 2010.-p. 54-60.

79. Fine, B.C. Image recognition / B.C. Fine // M.: Nauka, 1970.-284 p.

80. Forsythe, D.A. Computer vision. Modern approach / D.A. Forsyth, J. Pons // M.: Williams Publishing House, 2004. 928 p.

81. Fu, K. Sequential methods in pattern recognition and machine learning / K. Fu / M.: Nauka, 1971. 320 p.

82. Fu, K. Structural methods in pattern recognition / K. Fu / M.: Mir, 1977.-320 p.

83. Fukunaga, K. Introduction to the statistical theory of pattern recognition / K. Fukunaga / M.: Nauka, 1979. 368 p.

84. Shelukhin, O.I. Self-similarity and fractals. Telecommunication applications / O.I. Shelukhin, A.V. Osin, S.M. Smolsky / Ed. O.I. Shelukhin. M.: FIZMATLIT, 2008. 368 p.

85. Shilov, A.S. Definition of motion (MotionEstimation) / A.S. Shilov, M.N. Favorskaya // Certificate No. 2009611014. Registered in the Register of Computer Programs, Moscow, February 16, 2009.

86. Sh.Shlezinger, M.I. Correlation method for recognition of image sequences / M.I. Schlesinger / In the book: Reading machines. Kyiv: Nauk.dumka, 1965. p. 62-70.

87. Schlesinger, M.I. Syntactic analysis of two-dimensional visual signals under interference / M.I. Schlesinger // Cybernetics, No. 4, 1976. - pp. 76-82.

88. Stark, G.-G. Application of wavelets for DSP / G.-G. Shtark / Ml: Technosphere, 2007. 192 p.

89. Shup, T. Applied Numerical Methods in Physics and Technology: Per. from English. / T. Shup / Ed. S.P. Merkuriev; M.: Higher. School, 19901 - 255 p.11 "5. Electr, resource: http://www.cse.ohio-state.edu/otcbvs-bench

90. Electr, resource: http://www.textures.forrest.cz/ electronic resource (base of texture images textures library forrest).

91. Electr, resource: http://www.ux.uis.no/~tranden/brodatz.html electronic resource (Brodatz texture image database).

92. Allili M.S., Ziou D. Active contours for video object tracking using region, boundary and shape information // SIViP, Vol. 1, no. 2, 2007.pp. 101-117.

93. Almeida J., Minetto R., Almeida T.A., Da S. Torres R., Leite N.J. Robust estimation of camera motion using optical flow models // Lecture Notes in

94. Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 5875 LNCS (PART 1), 2009. pp. 435-446.

95. Ballan L., Bertini M., Bimbo A. D., Serra G. Video Event Classification using String Kernels // Multimed. Tools Appl., Vol. 48, no. 1, 2009.pp. 6987.

96. Ballan L. Bertini M. Del Bimbo A., Serra G. Action categorization in soccer videos using string kernels // In: Proc. of IEEE Int "l Workshop on Content-Based Multimedia Indexing (CBMI). Chania, Crete, 2009. pp. 13-18.

97. Barnard K., Fan Q. F., Swaminathan R., Hoogs A., Collins R, Rondot P., and Kaufhold J. Evaluation of localized semantics: Data, methodology, and experiments // International Journal of Computer Vision, IJCV 2008, Vol. 77, no. 1-3,2008.-pp. 199-217.

98. Bertini M., Del Bimbo A., Serra G. Learning rules for semantic video event annotation // Lecture Notes In Computer Science; In: Proc. of Int "l Conference on Visual Information Systems (VISUAL), Vol. 5188, 2008. pp. 192-203.

99. Bobick A.F., Davis J.W. The recognition of human-movement using temporal templates // IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, no. 3, 2001.pp. 257-267.

100. Boiman O., Irani M. Detecting irregularities in images and in video // International Journal of Computer Vision, Vol. 74, no. 1, 2007.pp. 17-31.

101. Bresson X., Vandergheynst P., Thiran J.-P. A Variational Model for Object Segmentation Using Boundary Information and Shape Prior Driven4 by the Mumford-Shah Functional // International Journal of Computer Vision, vol. 68, no. 2, 2006.-pp. 145-162.

102. Cavallaro A., Salvador E., Ebrahimi T. Shadow-aware object-based video processing // IEEE Vision; Image and Signal Processing, Vol. 152, no. 4, 2005.-pp. 14-22.

103. Chen J., Ye J. Training SVM with indefinite kernels // In: Proc. of the 25th international conference on Machine learning (ICML), Vol. 307, 2008.pp. 136-143.

104. Cheung S.-M., Moon Y.-S. Detection of Approaching Pedestrians from a Distance Using Temporal Intensity Patterns // MVA2009, Vol. 10, no. 5, 2009.-pp. 354-357.

105. Dalai N., Triggs B., and Schmid G. Human detection using oriented histograms of flow and appearance // In ECCV, vol. II, 2006. pp. 428^141.

106. Dalai N., Triggs B. Histograms of Oriented Gradients for Human Detection // IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. II, 2005-pp. 886-893.

107. Dani A.P., Dixon W.E. Single camera structure and motion estimation // Lecture Notes in Control and Information Sciences, 401, 2010. pp. 209-229.

108. Datta Ri, Joshi D;, Li J., and Wang J. Z1 Image retrieval: Ideas, influences, and trends of the new age // ACM"-Computing Surveys, Vol. 40:, no: 2, 2008. ■ -pp. 1-60.

109. Dikbas S., Arici T., Altunbasak Y. Fast motion estimation with interpolation-free sub-sample accuracy // IEEE Transactions on Circuits and Systems for Video Technology 20(7), 2010. -pp. 1047-1051.

110. Dollar P., Rabaud V., Cottrell G., Belongie S. Behavior recognition via sparse spatio-temporal features // In: Proc. 2nd Joint IEEE International Workshop on Evaluation of Tracking and Surveillance, VS-PETS, 2005. pp. 65-72.

111. Donatini P. and Frosini P. Natural pseudodistances between closed surfaces // Journal of the European Mathematical Society, Vol. 9, no. 2, 2007pp. 231-253.

112. Donatini P. and Frosini P. Natural pseudodistances between closed curves // Forum Mathematicum, Vol. 21, no. 6, 2009.pp. 981-999.

113. Ebadollahi S., L., X., Chang S.F., Smith J.R. Visual event detection using multi-dimensional concept dynamics // In: Proc. of IEEE Int "l Conference on Multimedia and Expo (ICME), 2006. pp. 239-248.

114. Favorskaya M., Zotin A., Danilin I., Smolentcheva S. Realistic 3D-modeling of Forest Growth with Natural Effect // Proceedings of the Second KES International Symposium IDT 2010, Baltimore. USA. Springer-Verlag, Berlin, Heidelberg. 2010.-pp. 191-199.

115. Francois A.R.J., Nevatia R., Hobbs J.R., Bolles R.C. VERL: An ontology framework for representing and annotating video events // IEEE Multimedia, Vol: 12; no. 4, 2005.pp. 76-86.

116. Gao J., Kosaka A:, Kak A.C. A Multi-Kalman Filtering Approach for Video Tracking of Human-Delineated Objects in Cluttered" Environments // IEEE Computer Vision and Image Understanding, 2005, V. 1, no. 1. pp. 1-57.

117 Gui L., Thiran J.-P., Paragios N. Joint Object Segmentation and Behavior Classification in Image Sequences // IEEE Conf. on Computer Vision and Pattern Recognition, 17-22 June 2007. pp. 1-8.

118. Haasdonk B. Feature space interpretation of SVMs with indefinite kernels // IEEE Transactions on Pattern Analysis and Machine Intelligence. Vol. 27, no. 4, 2005.pp. 482-492.

119. Harris C. and Stephens M. A combined corner and edge detector // In Fourth Alvey Vision Conference, Manchester, UK, 1988. pp. 147-151.

120. Haubold A., Naphade M. Classification of video events using 4-dimensional-time-compressed motion features // In CIVR "07: Proceedings of the6th ACM international confcrcnce on Image and video retrieval, NY, USA, 2007. -pp 178-185.

121. Haykin S. Neural Networks: A Comprehensive Introduction. / N.Y.: Prentice-Hall, 1999; .- 658 pi.

122. Hoynck M., Unger M., Wellhausen J. and Ohm J.-R. A Robust Approach to Global Motion Estimation for Content-based Video Analysis // Proceedings of SPIE Vol. 5601, Bellingham, WA, 2004. pp. 36-45.

123. Huang Q., Zhao D., Ma S., Gao W., Sun H. Deinterlacing using hierarchical motion analysis // IEEE Transactions on Circuits and Systems for Video Technology 20(5), 2010. pp. 673-686.

124. Jackins C.L., Tanimoto S.L. Quad-trees, Oct-trees and K-trees: A Generalized Approach to Recursive Decomposition of Euclidean Space // IEEE Transactions onPAMI, Vol. 5, no. 5, 1983.-pp. 533-539.

125. Ke Y., Sukthankar R:, Hebert Mi. Efficient visual event detection using volumetric features // In: Proc. of Int "l Conference on Computer Vision (ICCV), vol. 1, 2005.-pp. 166-173.

126. Klaser A., ​​Marszalek M., and Schmid C.A Spatio-Temporal Descriptor Based on 3D-Gradients // In BMVC, British Machine Vision, Conference, 2008. -pp. 995-1004.

127. Kovashka, A., Grauman, K. Learning a hierarchy of discriminative space-time neighborhood features for human action recognition // Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010. pp.2046-2053.

128. Kumskov M.I. Calculation Scheme of the Image Analysis Controlled by the Models of the Objects to be Recognized // Pattern Recognition and Image Analysis, Vol. 11, no. 2, 2001. p. 446-449:

129. Kwang-Kyu S. Content-based image retrieval by combining genetic algorithm and support vector machine // In ICANN (2), 2007. pp. 537-545.

130. Lai C.-L., Tsai S.-T., Hung Y.-P. A study on the three-dimensional coordinate calibration using fuzzy system // International Symposium on Computer, Communication, Control and Automation 1, 2010. - pp. 358-362.

131. Laptev I. On space-time interest points // International Journal of Computer Vision, Vol. 64, no. 23, 2005.pp. 107-123.

132. Leibe B., Seemann E., Schiele B. Pedestrian Detection in-Crowded* Scenes // IEEE Conference on Computer Vision and "Pattern Recognition, Vol. 1, 2005.-pp. 878-885.

133. Lew M. S., Sebe N., Djeraba C., and Jain R. Content-based multimedia information1 retrieval: State of the art and challenges // ACM Transactions on Multimedia Computing, Communications, and Applications, Vol. 2, no. 1, 2006.pp. 1-19.

134. Li J. and Wang J. Z. Real-time computerized annotation of pictures // IEEE Trans. PAMI, Vol. 30, 2008.pp. 985-1002.

135. Li L., Luo R., Ma R., Huang W., and Leman K. Evaluation of An IVS System for Abandoned Object Detection on PETS 2006 Datasets // Proc. 9 IEEE Intern. Workshop on PETS, New York, 2006. pp. 91-98.

136. Li L., Socher R., and Fei-Fei L. Towards Total Scene Understanding: Classification, Annotation and Segmentation in an Automatic Framework // IEEE Conference on Computer Vision and Pattern Recognition, CVPR, 2009. pp. 2036-2043.

137. Li Q., ​​Wang G., Zhang G.) Chen S. Accurate global motion estimation based on pyramid with mask // Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics, Vol: 21, no . 6, 2009.pp. 758-762.

138. Lindeberg T., Akbarzadeh A. and Laptev I. Galilean-diagonalized spatio-temporal interest operators // Proceedings of the 17th International Conference on Pattern Recognition (ICPR"04), 2004. pp. 1051-1057.

139. Lim J., Barnes, N. Estimation of the epipole using optical flow at antipodal points // Computer Vision and Image Understanding 114, no. 2, 2010.pp. 245-253.

140. Lowe D. G. Distinctive Image Features from Scale-Invariant Keypoints // International Journal of Computer Vision, Vol. 60, no. 2, 2004.pp. 91-110.

141. Lucas B.D., Kanade T. An Iterative Image Registration Technique with an Application to Stereo Vision // International Joint Conference on Artificial Intelligence, 1981. pp. 674-679.

142. Mandelbrot B;B. The Fractal Geometry of Nature / N.Y.: Freeman^ 1982. 468 p.; russ, trans.: Mandelbrot B. Fractal, geometry of nature: Per. from English. / M.: Institute of Computer Research, 202. - 658 p.

143. Mandelbrot V.V., Frame M.L. Fractals, Graphics, and Mathematics Education/N. Y.: Springer-Verlag, 2002. 654 p.

144. Mandelbrot B.B. Fractals and Chaos: The Mandelbrot Set. and Beyond / N.Y.: Springer-Verlag, 2004. 308 p.

145. Memoli F. On the use of Gromov-Hausdorff distances for shape comparison // Proceedings of the Eurographics Symposium on Point-Based Graphics. Prague, Czech Republic, 2007. pp. 81-90.

146. Mercer J. Functions of positive and negative type and their connection with the theory of integral equations // Transactions of the London Philosophical Society (A), vol. 209, 1909. pp. 415-446.

147. Mikolajczyk K. Detection of local features invariant to affine transformations, Ph.D.thesis, Institut National Polytechnique de Grenoble, France. 2002.171 p.

148. Mikolajczyk K. and Schmid G. An Affine Invariant Interest Point Detector // Proceedings of ECCV. Vol. 1. 2002. pp. 128-142.

149. Minhas R., Baradarani A., Seifzadeh S., Jonathan Wu, Q.M. Human action recognition using extreme learning machine based on visual vocabularies // Neurocomputing, Vol. 73 (10-12), 2010. pp. 1906-1917.

150. Mladenic D., Skowron A., eds.: ECML. Vol. 4701 of Lecture Notes in Computer Science, Springer, 2007. pp. 164-175.

151. Moshe Y., Hel-Or H. Video block motion estimation based on gray-code kernels // IEEE Transactions on Image Processing 18(10), 2009. pp. 22432254.

152. Nakada T., Kagami S;, Mizoguchi H. Pedestrian Detection using 3D Optical Flow Sequences for-afMobile Robot // IEEE Sensors, 2008. pp: 116-119:

153. Needleman, S. B:,. Wunsch C.D.; A general method applicable to the search for similarities in the* amino acid sequence of two proteins // Journal "of Molecular Biology Vol. 48, no: 3, 1970. pp. 443-453.

154. Neuhaus M., Bunke H. Edit distance-based kernel functions-for structural pattern classification // Pattern Recognition. Vol. 39, no. 10, 2006. pp: 1852-1863.

155. Nevatia R., Hobbs J., and Bolles B. An ontology for video event representation // In Workshop on Event Detection and Recognition. IEEE, Vol.12, no. 4, 2004.pp. 76-86.

156. Nguyen.N.-T., Laurendeau D:, Branzan-Albu A. A robust method for camera motion estimation in movies based on optical flow // The 6th International