Hierarchical Motion Estimation Algorithm
Using Reliable Motion Adoption
K.-S. Choi and S.-J. Ko
Sch. of Electr. Eng., Korea Univ., Seoul, South Korea
IEEE, Electronics Letters
Issue Date : June 10 2010
OUTLINE
•
Hierarchical structure
•
Conventional Hierarchical BMAs (HBMAs)
•
Hierarchical motion correction
•
Experimental results
•
Conclusions
Hierarchical structure
•
A pyramid of multi-level
•
Low resolution images
obtained by down-sampling
or low-pass filter.
• Rough estimation
• Relatively large block
• Relatively large searching
range
• Four child blocks
Conventional Hierarchical BMAs (HBMAs)[3]
•
The estimate of motion vectors at a lower resolution level.
•
Propagate motion vectors from a lower to a higher resolution level.
• Duplication or bilinear interpolation
• Produce consistence motion vector fields
If block B is only included in one moving object
The true motion vectors of the four child blocks are usually very close to V.
Otherwise, if block B covers two or more different objects
The true motion vectors of the child blocks may be significantly different from V.
Child block belongs to the parent block or one of its neighboring blocks.
Smallest motion compensation error is chosen as the motion vector of the child block
[3] Wang, D., and Lauzon, D. : ‘Hybrid algorithm for estimating true motion fields’,
Opt. Eng., 2000, 39, (11), pp. 2876–2881
The actual motion of small objects is hardly predicted either
spatiotemporally or hierarchically! => hierarchical motion correction
l
V FS (l = 0, 1,…, L − 1): L-level MVFs by FS-VBSME
l
V (l=0,1,…,L) : resulting MVFs
Predict from upper level MVF and refinement
L-1
•
Initially, VFS is utilized as the prediction at the first loop ( l = L – 1 )
•
Small refinement search range(RSR)
• To maintain the smoothness of the resulting MVF
• Similarly to conventional HBMAs.
Propagation of MV between adjacent blocks
•
if a block at a higher level has more than one moving object with a different motion,
• an inaccurate MV is predicted for the current block from the parent block.
•
The current block and one of its adjacent blocks having a different parent block may
belong to the same object.
• Replaced with the MV of the adjacent block S i, j.
Adoption of actual motion of objects(1/2)
•
the motion of small objects has not been predicted accurately at the upper levels.
•
VFS(i, j) is utilized as another MV candidate.
•
Since VFS does not only contain meaningful information for any complex motion of
variable sized objects but also random MVs especially at the low levels.
l
l
l
• The reliability of VFS(i, j) should be first examined
Adoption of actual motion of objects(2/2)
•
Reliability measure
l
C1: minimum discontinuity cost imposed on VFS(i, j)
λ: a weighting factor for level l
l
•
MVD represents the cost of discontinuity in the MVF induced when V FS (i, j) is adopted for
the current block.
•
V (i, j) is replaced by VFS(i, j) when following condition is satisfied :
l
l
l
MVD prevents VFS(i, j) from being adopted
with small amount of reduction in the matcherror criterion.
EXPERIMENTAL RESULTS
•
To compare the results of the proposed algorithm objectively and subjectively
•
3DRS
•
Ha’s algorithm
•
HBMA [3]
Stefan sequence
Frame 13
Frame 14
Resulting MVFs
Demonstrate the accuracy of the MVFs objectively
•
three test video sequences
• Even frames have been skipped
• Applied to a motion-compensated frame interpolation (MCFI) method [5] with the
resulting MVF of each algorithm.
[5] Wang, T.-S., Choi, K.-S., Jang, H.-S., and Ko, S.-J.:
‘Enhanced frame rate up-conversion method for UHD Video’.
Proc. IEEE Int. Conf. on Consumer Electronics, January 2010, p. 4.4–3
CONCLUSION
•
Propose a hierarchical true motion estimation algorithm
• Spatially propagates reliable MVs
• Replaces inaccurate MVs with more reliable MVs in the multi-level MVFs
•
Experimental results have shown that
• Produces consistent MVs within each object
• Estimates the different motion of various small objects accurately