1. Using Kalman Predicted Features Location as Input LOC Estimate for calcOpticalFlowPyrLK An outdoor natural scene image has lots of similar features. For example, trees in mountains. It is difficult for a SAD algorithm to find the correct location in this case. By using the KALMAN filter prediction result as guidance for the optical flow algo, it has a better chance to find the correct LOC for an input feature point. However, at the beginning a iteration, the Kalman filter has not gained a good estimate on the features’ depth, and its predictions are therefore very inaccurate. By giving the optical flow algorithm an incorrect estimate, the optical flow algo finds a matching point close to the estimated point, and therefore, outputs an incorrect feature coordinate (due to the similarity texture through out the image. ) For this reason, the input estimation for the LK can only be given after the EKF establish good estimate on a feature point. This leads to another question: how to determine if a feature has been accurately estimated by the EKF? Variables tracked by EKF converge to a stable value, which means that it should show very little changes in each iteration. The simplest way to determine that tracking validity of an variable is to calculate the 1st derivative with respect to interations. The other method is to look at its covariance. The initial variance should be big, and decreases as the variable converge. 2. Feature Quality vs. Iteration In aerial video, a features can remain in the field of view for a very long time. After some iterations, the feature quality can decrease as the scene change slowly, lighting condition changes... This impact the quality of the tracking as a good quality corner feature can become a poor quality one, or even disappear, which increase the difficulty in SAD matching. 3. Small Error Accumulate Error in SAD method accumulates over the iterations. Eventually, the coordinate of the feature drift away from its true coordinate, and the method loses the feature.