Logo icon

Real-time detection and tracking of human faces

Designed for: NEC (UK) Ltd (a division NEC in the UK), United Kingdom, Cambridge

Purpose: detecting and tracking of human faces on high-definition video in real time

Status: a completed development

The library is designed to detect a face on video in real time. The areas of its use include (but are not limited to) the system of counterterrorism and search for criminals, tracking systems and CCTV systems.

Key features:

  • Detecting frontally oriented (facing the camera) faces in the video stream (the size of the face from 20 × 20 pixels to full-frame, the number of faces detected at the same time - at least 50 guaranteed, in the long term - up to thousands)
  • Tracking the detected faces (preserving the unique identifier of each face from frame to frame) until they disappear from sight or hide behind an obstacle; when possible, the system also tracks a face disappearing briefly (up to 1 second) due to overlap or rotation of the head.
  • The capability to process high-definition video in real time. At the moment the following results are achieved: processing of a 2-megapixel video (1920 × 1080) up to 60 frames per second, a 5-megapixel video (single-GPU GeForce GTX 780 level) up to 20 frames per second, with a delay of 0.1 seconds.
  • Parallel processing of multiple video streams
  • In the future the ability to process a video of higher resolution (up to 6576 × 4384) (the potential to expand functionality and device support)
  • Support for multiple graphics cards
  • Receiving stream from a file or network (RTSP) source, supporting different codecs (including h264)
  • Hardware video decoding using the resources of a camcorder

Viola-Jones object detection framework with basic classifier based on LBP is used for facial detection in the project. OpenCV implementation was modified, asynchronous operations were added, and the use of graphics card in parallel processing multiple frames was improved. The detector was trained on the sample of 4,000 persons. Kanade-Lucas-Tomasi pyramidal feature tracking algorithm as well as OpenCV implementation with the addition of asynchronous operation is used for tracking.


Client: NEC (UK) Ltd (a division NEC in the UK), United Kingdom, Cambridge
Area of use: image processing, face detecting
Type (platform): Library of functions for Windows 64-bit
Technologies and algorithms in use: C++, CUDA, OpenCV, cascade detector Viola-Jones, Kanade-Lucas-Tomasi Pyramidal Feature Tracking

visualization of Large Point Clouds

fast implementation of the direct and inverse Fourier transform (for later use in image processing)