Assignment No. -5 (Group -A)
Regularity
(2)
Performance Oral
(5)
(3)
Total Dated
(10) Sign
Aim: Implementation of Naive Bayes to predict the work type for a person with following
Parameters: Age : 30, Qualification : MTech, Experience : 8
Objective:
Student will learn:
i.
The Basic Concepts of Book Recommender system
ii.
General structure of eight puzzle problem.
iii.
Logic of A star implementation for eight puzzle problem.
Theory:
5.1. Introduction:
Naive Bayes is a simple technique for constructing classifiers: models that assign class labels to
problem instances, represented as vectors of feature values, where the class labels are drawn
from some finite set. It is not a single algorithm for training such classifiers, but a family of
algorithms based on a common principle: all naive Bayes classifiers assume that the value of a
particular feature is independent of the value of any other feature, given the class variable. For
example, a fruit may be considered to be an apple if it is red, round, and about 10 cm in diameter.
A naive Bayes classifier considers each of these features to contribute independently to the
probability that this fruit is an apple, regardless of any possible correlations between the color,
roundness and diameter features. For some types of probability models, Naive Bayes classifiers
can be trained very efficiently in a supervised learning setting. In many practical applications,
parameter estimation for naive Bayes models uses the method of
5.2 Naïve bays implementation Steps:
1. Calculate prior Probabilities of class to be predicted
2. Calculate conditional probabilities
3. Calculate posterior probability
4. Highest probability among above is predicted class for query tuple
5.3 Two Naive Bayes Models:
1) Multi-vitiate Bernoulli event model:
All features are binary: the number of times a feature occurs in an instance is ignored.
When calculating p(d | c), all features are used, including the absent features
2) Multinomial event model: “unigram LM”
5.4 Bayes Theorem
Given a hypothesis h and data D which bears on the hypothesis:
P ( D | h) P ( h )
P ( h | D) 
P ( D)
P(h): independent probability of h: prior probability
P(D): independent probability of D
P(D|h): conditional probability of D given h: likelihood
P(h|D): conditional probability of h given D: posterior probability
5.5 Mathematical Module:
Abstractly, naive Bayes is a conditional probability model: given a problem instance to be
classified, represented by a vector
representing some n features
(independent variables), it assigns to this instance probabilities
Using Bayes Theorem the conditional probability can be decomposed as
In plain English, using Bayesian probability terminology, the above equation can be written as
5.6 Naive Bayes solution :
Classify any new datum instance x=(a1,…aT) as:
hNaive Bayes  arg max P(h) P(x | h)  arg max P(h) P(at | h)
h
h
t
To do this based on training examples, we need to estimate the parameters from
the training examples:
For each target value (hypothesis) h
Pˆ (h) : estimate P(h)
For each attribute value at of each datum instance
Pˆ (at | h) : estimate P(at | h)
5.7 Conclusion: Naïve Bayes is linear in the time is takes to scan the data
When we have many terms, the product of probabilities with cause a floating point underflow
Download

A-5.naiveBayes - WordPress.com