AIC,BIC and the new CIC
http://omega.albany.edu:8008/CIC
copyMEright by C. Rodriguez. Please copy me!
Carlos Rodriguez
SUNY Albany.
Model: { p(x|q)}
Truth: t(x)
t
q*
Geometrization of Inference
Embedding in Hilbert Space
Fisher Information metric automagically induced on the tangent bundle !
The Volume Form as Prior
A hypothesis space M is said to be regular when (M,g) is a smooth
orientable riemannian manifold. A k-dim regular M has volume form:
In arbitrary (orientation preserving) theta coordinates the volume
of (M,g) is:
Volumes of
Bitnets = dags of bits
Maple Module vTool
http://omega.albany.edu:8008/bitnets
Complete
Bitnets
3
p3
3
t3
t2
p
p1
2
t1
1
Constant
Ricci Scalar
2
x3
x2
x1
3
+
(
Ground 0
Where does it ALL come from?
42
(joke)
110
(funny but no joke)
MDL bold pragmatism
Forget about the data being
generated by a probability
distribution. This is just a
CODING GAME!!
Best model is the one
providing the shortest code for
the observed data.
Data is all there is!
Есть Проблема
The shortest description
length of a sequence is
NON-COMPUTABLE!!
And can only be
approximated with
MODELS.
Data and Theory are Entangled
There is no data in the vacuum.
Data is a logical proposition with truth values only
relative to a given domain of discourse.
A sequence 0110011110… is NOT DATA as the number
2.4 is not data unless is understood as “the result of
such and such experiment is 2.4”.
Data is theory laden.
Theory is data laden.
IMHO
NEED
http://omega.albany.edu:8008/
ignorance
)
Model Selection from Data
Naïve approach
Given Data:
iid
Regular model:
Most likely M is asymptotically given my minimizing the MDL score.
Model Selection Close to the
Vacuum
Choose among Complete bitnets:
For given data sequence:
MLE,MDL,AIC,BIC and…
Worse < BIC < AIC < CIC < Best
% of correct
segmentations v/s N.
Based on 100 reps for
each N. Params at
ramdom each time.
CIC movie
Moving window of 640 bits over sound file.aiff
.jpg
.aiff
.txt
.gz
The Iliad: BOOK I
+
CIC
+
Sing, O goddess, the anger of
Achilles son of Peleus, that brought
countless ills upon the Achaeans.
Many a brave soul did it send
hurrying
down to Hades, and many a hero did
it yield a prey to dogs and vultures,
for so were the counsels of Jove
fulfilled from the day on which the son
of Atreus, king of men, and great
Achilles, first fell out with one
another…………
+
01100…0
.
end