Sculpture of Roman god Janus in Summer Garden, St Petersburg, Russia.
According to Roman mythology, Janus could see both future and past.
Predictive Learning and Advanced Learning Technologies
Vladimir Cherkassky’s Research Group
http://www.ece.umn.edu/users/cherkass/predictive_learning/
Learning is the ability to make inferences from repeated events (observations), in order to predict or
anticipate future events. Two inter-related characteristics of learning, i.e. the ability to explain the past
and predict the future, have been known since ancient times. However, quantitative models for managing
uncertainty and risk have been developed fairly recently in the 20-th century, due to advances in
computer technology and mathematical tools in statistics and machine learning. Prof. Cherkassky’s
research is concerned with methodological, technical and practical aspects of predictive learning.
Most existing methods for learning from data are based on standard inductive-deductive approach,
comprising two distinct steps, induction, when a predictive model is estimated from past data, and
deduction, when an estimated model is used to make predictions with new inputs.
Many challenging new applications of data-driven learning deal with heterogeneous and highdimensional data. For example, in medical diagnostic applications, patients’ features include genetic,
clinical, demographic and imaging data. Such applications may require alternative (non-standard)
learning methodologies. Our current research investigates several emerging non-standard learning
approaches for predictive modeling, including their mathematical formulation, development of practical
strategies for model complexity control (aka model selection) and several real-life medical applications.
Acknowledgements: current research on advanced learning technologies is supported by the National
Science Foundation under Grant No. 0802056, The A. Richard Newton Breakthrough Research Award
from Microsoft Research, and the BICB grant from the University of Minnesota, Rochester.
Philosophical Connections
Predictive learning is concerned with estimating useful models (i.e., extracting knowledge) from
empirical data. It is closely related to the philosophy of science.
Aristotle: All men by nature desire knowledge.
Gottfried Leibniz: Miracles are not to be multiplied beyond necessity.
Immanuel Kant: Experience without theory is blind, but theory without experience is mere intellectual
play.
Karl Popper: Science starts from problems, and not from observations
Non-standard Learning Methods
Many applications involve heterogeneous data, such that labeled training data can be naturally divided
into several groups. Two new methodologies for utilizing such group information include Learning with
Structured Data (aka SVM+) and Multi-Task Learning.
Learning with Structured Data (Vapnik, 2006)
Training Data: originates from several groups
Goal of learning: to estimate a single predictive model.
Multi-Task Learning
Training Data: originates from several groups (tasks)
Goal of learning: to estimate several related predictive models
Universum Learning or Learning through Contradictions (Vapnik, 2006)
How to incorporate a priori information, in the form of Universum data samples, into learning.
References
Vapnik, V., Empirical Inference Science: Afterword of 2008, Springer 2006
Liang, L. and Cherkassky, V. , Connection between SVM+ and Multi-Task Learning,
Proc. IJCNN 2008
Cherkassky, V. , Cai, F. and Liang, L., Predictive Learning with Sparse Heterogeneous
Data, invited paper, Proc. IJCNN 2009 (to appear)
Cai, F., and Cherkassky, V., SVM+ Regression and Multi-Task Learning, Proc. IJCNN
2009 (to appear)
J. Weston, R. Collobert, F. Sinz , L. Bottou and V. Vapnik. Inference with the Universum.
Proceedings of the Twenty-third International Conference on Machine Learning (ICML 2006),
IMLS/ICML, 2006.