Notes from ACB diffim debrief.
Attending: ACB, Scott, Rahul, Russell, Yusra, Mario, Simon
The Problem:
Take two images at different epochs and take the difference (subtract the two).
The PSF will be different.
The background will be different.
Step 1:
Match backgrounds – Additive smooth function
Step 2:
Match the PSFs – Convolution.
Tasks for image matching:
Task 1: Image to Image matching
Task 2: Snap to Snap (assumes PSF matching is small) – ideally this isn’t
needed
Task 3: Model to model – PSF of science image matched to desired output
PSF
Task 4: Doesn’t exist, but could maybe do Image to Model.
Science image is the template convolved with a matching kernel minus a
background function
I = TxK + b
K is regularized by representing it as a linear combination of basis functions
K = Sigma a_i K_i
PSFs and background vary over an image/focalplane.
a_i = a_i(x,y)
bg = bg(x,y)
Basis set for matching kernel:
Default is sum of Gaussians: sigma_1, sigma_2, sigma_3 (Narrow, wider, very wide)
Define sigma_2 = sqrt(sigma_I**2 + sigma_T**2)
sigma_1 = sigma_2/Beta
sigma_3 = Beta*sigma_2
MJ: Can Beta be optimized?
ACB: The literature suggests a value of 2 should be about right
MJ: Is there a reason the scale is the same for sigma_1 and sigma_2?
ACB: Not physically, but this empirically seems to be a good choice.
The code wants a set of images. They can be created in any way you want.
Jim Bosch is using a sum of shapelets
An improvement is that we could have more basis sets.
The symmetric kernels sigma_1, sigma_2, sigma_3 are multiplied by Laguerre
polynomials:
x^ny^m m+n < N_1
x^ny^m m+n < N_2
x^ny^m m+n < N_3
This results in ~30 basis functions.
Delta function basis set:
Low bias, high variance – Can enforce smoothness by binning in outer regions
MJ: Why not enforce smoothness in the fitting.
ACB: The fitter knows nothing about locality, so it would require adding some
machinery.
Caveat: The normalization of the matching kernel assumes that the difference in
zeropoint is constant across the image.
Match by minimizing chi^2
Chi^2 = (I-TxK –b)^2/sigma^2 => b=Ma
b_i = sum over pixels of (C_i*I/sigma^2)
where
C_i = K_i x T
M_ij = sum over pixels in (C_i*C_j/sigma^2)
sigma^2 is the sum of all pixels in the variance plane.
C_i images should be 2*K in size so after convolution there are at least K useful
pixels. Note: Could use whole image but that would miss the positional dependence
because the problem is separated into the modeling at a point and how that model
changes as a function of position.
Spatial model:
a_i is a function of (x,y).
bg is a function of (x,y).
See min 40:00.
M_spatial = element wise sum of M_ij for all test objects.
Regularization must be put in the basis functions or in the way we solve for the
spatial solution.
Implementation:
Registered template and image. This actually caused a bit of a problem and was
worked around and Astrometry.net issue with interpreting a SIP with large XY0.
We should be able to match image to image without going through the sky. This
exists, but should be refactored.
Use spatial cell set to split up the images.
Do image detection for getting postage stamps for matching. Probably should have a
gold standard set of objects: known non-variable, non-moving, etc.
Look for all objects in each cell with no flags on the blacklist.
This becomes the kernel candidate: two postage stamps from template and image
and flag: good, bad, unknown.
Best (bright) kernel candidates per cell are used for the spatial.
MJ: this is not the principled way to do this. You should use all the information you
can by using all good Kernel Candidate.
Flatten kernel candidates into a vector of these. Let a “Visitor” travel over the
vector.
Visitor1: builds local kernel, does QA (is the difference image good enough), sets flag
(good or bad)
Visitor2: kernel sum visitor: aggregates the kernel sums for all Kernel Candidates
and calculates the statistics. Passes through set again to reject outliers.
Visitor1 and Visitor2 are called recursively until there are no rejections (up to a max
number of iterations).
Visitor3: build spatial kernel
Visitor4: assess spatial kernel
Increment nreject
Set BAD.
Iterate.
The good and the bad:
FRE = Full Russell Equiv.
Replace detection This could be removed maybe?
Replace image to eigen homebrew 0.5 week FRE
Homebrewed linear algebra: Jim has a more general solution 0.5 week FRE
Still relies on policy. 1 week FRE
Research work on basis sets. Speed up convolution. Binned delta functions.
MJ: Performance how close is this to optimized
ACB: Spatial convolution and warping of the template image
Gaussian process spatial modeling – 2weeks
Tuning heuristics for AL set.
Dipole measurement needs work. There is a bias in the measured separation.