1. Using Kalman Predicted Features Location as Input LOC Estimate for
calcOpticalFlowPyrLK
An outdoor natural scene image has lots of similar features. For example, trees in mountains. It is
difficult for a SAD algorithm to find the correct location in this case. By using the KALMAN
filter prediction result as guidance for the optical flow algo, it has a better chance to find the
correct LOC for an input feature point. However, at the beginning a iteration, the Kalman filter
has not gained a good estimate on the features’ depth, and its predictions are therefore very
inaccurate. By giving the optical flow algorithm an incorrect estimate, the optical flow algo finds
a matching point close to the estimated point, and therefore, outputs an incorrect feature
coordinate (due to the similarity texture through out the image. ) For this reason, the input
estimation for the LK can only be given after the EKF establish good estimate on a feature point.
This leads to another question: how to determine if a feature has been accurately estimated by the
EKF? Variables tracked by EKF converge to a stable value, which means that it should show
very little changes in each iteration. The simplest way to determine that tracking validity of an
variable is to calculate the 1st derivative with respect to interations. The other method is to look
at its covariance. The initial variance should be big, and decreases as the variable converge.
2. Feature Quality vs. Iteration
In aerial video, a features can remain in the field of view for a very long time. After some iterations, the
feature quality can decrease as the scene change slowly, lighting condition changes... This impact the
quality of the tracking as a good quality corner feature can become a poor quality one, or even
disappear, which increase the difficulty in SAD matching.
3. Small Error Accumulate
Error in SAD method accumulates over the iterations. Eventually, the coordinate of the feature drift
away from its true coordinate, and the method loses the feature.