To demonstrate the practical use of EuclidVision technology and its commercial viability, the company has developed the eFLEX codec. Built on a base of H.264 processing, eFLEX incorporates EuclidVision’s advanced algorithms to deliver a processing pipeline that performs conventional compression on a video stream while analyzing the content of the stream and applying the higher level modeling when advantageous.
eFLEX integrates many of the algorithms created in EuclidVision to extend beyond conventional transform coding and macroblock frame segmentation. In particular, eFLEX performs Macroblock caching, Feature Modeling, and Object Modeling to achieve increased compression and to enable transmission of higher quality video over bandwidth constrained networks.
The following diagram is a subset of EuclidVision research areas implemented within eFLEX:
eFLEX works with Windows Media Player (WMP) as a standard codec plug-in for video playback from an eFLEX encoded video file. Installation of the codec can be accomplished on any network attached device running WMP under a modern Microsoft Windows operating system. eFLEX’s compression advantages could be made available to the over 1 billion devices around the world running WMP today. Since the default conventional processing and the higher-level modeling are able to utilize the MPEG-4 part 10 set of algorithmic functionalities as a toolbox, further advances in the MPEG-4 standard and implementation provide additive benefit without conflicting with the higher-level modeling.
The encoding algorithm within the eFLEX codec is based upon the identification of the best encoding technique at the macroblock level. The codec utilizes the EuclidVision algorithms and techniques to compress both individual and sets of macroblocks with the best encoding and likewise decode and re-assemble the macroblocks into frames and subsequently a synthesized video. Features are detected and tracked throughout the video during compression. From those features, models are created that are used within the analysis and compression phase of the macroblock encoding.
The following diagram displays a conceptual simplification of the eFLEX codec:
When the encoding algorithm starts, it analyzes the video stream and identifies the best encoding to use at the macroblock level by determining macroblocks that exhibit salient characteristics. Macroblocks that do not exhibit salient characteristics or have a low encoding cost associated with them are encoded by conventional compression.
When using EuclidVision to encode, the algorithm may choose from a number of methods. One of these methods involves the macroblock cache, which is a set of retained "regions of pels". The determination, of which "regions of pels" to be retained, is dependent upon various criteria. These criteria include the following elements: retain all macroblocks or "regions of pels"; retain "regions of pels" in proximity of some tracked feature or object within the video; retain the "regions of pels" based on their pel-level persistence as groups of pels tend to match other pels in different frames; consider the number of times the particular macroblock has been used to successfully fill a match for another macroblock.
Once the "regions of pels" have been identified then the algorithm considers that a match has been made and determines the effectiveness of the match to model the current macroblock. Depending on these results, additional processing may occur to create a secondary match and so on. These matches along with the primary match are used in combination to identify the best encoding technique and to encode the current macroblock. In the case when more than one match is used to represent the current macroblock, different modeling methods are employed which range from simple interpolation and extrapolation of the matches, to more complex mode modeling methods such as Principal Component Analysis (PCA) and Orthogonal Matching Pursuit (OMP).