Alternative System for
Admission into Engineering
Programmes
Report of the Committee
Presented to the Council of Indian
Institutes of Technology
A proposal for consideration
September 2011
Alternative system
Acknowledgement









Prof Acharya and his Committee members
Dr BK Gairola; Sri V Joshi; Sri H Bhartia; Sri M Tuli
Directors of all IITs
2063 people who participated in opinion poll
Director and Experts from ISI and their students
Colleagues from MHRD, DST and NIC
Chairpersons of school boards and
Chairman and members of the IIT Council
Some Media representatives and all those who
helped in the exercise
September 2011
Alternative system
Decision of the Council in its 41st
meeting




Prof Acharya committee had been commissioned
to study the present system of conducting
examinations for admission into engineering
progammes of the country and suggest
alternatives
The committee presented its interim report in the
41st meeting
There was unanimity that the present system
required a change as proposed
The committee was enlarged to address the issue
of the need to recognize diversity of learning
September 2011
Alternative system
Major findings of Acharya
Committee




Screening based on normalized Board scores at Standard
X and/or Standard XII and Multiple Choice examination
replacing the two stage JEE from 2006.
Entry barrier to be raised to 60% in the +2 examinations.
Factors, other than the Standard XII marks and AIR based
on PCM testing, such as raw intelligence, logical
reasoning, aptitude, comprehension and general
knowledge need to be considered.
Need to factor in school performance more significantly
into the selection process.
September 2011
Alternative system
Major findings of Acharya
Committee





Decision based on one time test needs to be re-examined.
Opportunities to improve must be built in.
Students must be relieved of the pressure of multiple JEEs.
Influence of coaching for JEE needs to be minimized.
Urban-rural and gender bias has to be eliminated or at
least minimized.
The objective type of examination lends itself to undue
influence of coaching. The conventional pen and paper
examination with well designed long and problem solving
oriented questions should be revived by keeping numbers
in any JEE within reasonable limits.
September 2011
Alternative system
Work of the present
committee
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
Study of Acharya committee work
Made Non formal survey among hundreds of school students,
parents, employers, faculty and media person
Met and decided on General approaches
Held with Chairmen and nominees of all school boards
Collected data on scores of school board examinations
Wrote down a philosophy document and placed in public domain
Carried out survey of public opinion poll
Enrolled ISI for carrying out exploring statistical methods for
normalization of school board scores
Met with Directors and Senior Faculties of IITs
Prepared a draft report for comments and advice of the Council
September 2011
Alternative system
Consultation with states
and School boards

First stage



Gained access to data on scores on school board
examinations through formal mechanisms
Enrolled participation in development of NTS
Second Stage


Assistance in harmonization and electronic access to
data
Building trust and development of a process with
safeguards and integrity
September 2011
Alternative system
Public participation
through opinion poll

On-line opinion survey among the people
of India




For multi parametric grading system as
against single test models of JEE
Screening out as against selection strategies
With Responder profile, opinion polls,
suggestions for alternative national test
systems; risk mitigation strategies
Survey time slot open for three weeks
September 2011
Alternative system
Responder profile to the poll

2063 participated



1220 students had participated


59% students; 8% teachers, 5.5% parents, 23.4%
non-teaching professionals, less than 1% coaching
~80% of respondents had taken entrance
examinations in their lives
80% students were from engineering stream and
95% of them had taken entrance examination
~160 teachers had participated

~90% of them are engaged in tertiary education
September 2011
Alternative system
Analysis of the current system
of admission into engineering

960 of 2063 commented on the multiplicity
of entrance examinations with different
sociological implications


715 agreed with the view expressed
947 of 2063 commented on need for
reform

85% voted for reform in admission system
September 2011
Alternative system
Inputs for reforms fo
admission systems
1.
2.
3.
4.
5.
6.
7.
8.
Reduction in the number of examinations to one
Testing knowledge intensity, alignment to the 12th
class syllabus
Reduction of dependency on coaching and
pressure on students
Emphasis on aptitude….. (More than 90%)
Transparency in processes
Removal of negative marking
Online processes
Multiple centers, better scheduling
September 2011
Alternative system
Responses to reform directions

Factor-in performance in school boards

66% in favor and 34% not in favor


620 responded to question on Indian equivalent of SAT
type


73% voted in favor of aptitude type test
646 responded to types of tests


45 of those disfavor fear that board examinations do not
assessing capability and 30% fear non-uniformity
70% prefer a mix of aptitude and advanced type tests
629 responded questions on test features

Dominant support is for high-filter type and SAT type tests
September 2011
Alternative system
On summary




More than 85% supported the concept of a
single entrance test for admission into
engineering programmes and voted for reforms
70 % voted in favor of one test with provisions
for testing both aptitude and advanced
knowledge
66% of people favor factoring in school board
scores
Of 34% People who disfavor fear primarily the
problem of non-uniformity. This could be
addressed.
September 2011
Alternative system
Some Important suggestions for
the committee from opinion poll


National Test in place of multiple competitive
examination is generally welcomed
Concerns expressed about




the process integrity and fairness of testing
methodology
Normalization methodologies across school boards
Multiple chances for candidates for improving
scores
A single screening examination with a mix of
aptitude (like SAT type) and advanced (like JEE
type)
September 2011
Alternative system
Study for normalization of scores
of school board examinations

Committee gained access to some relevant
past data with a view to examine



Stability of scores of the same school board
over time
Potentials for normalization of scores across
various boards
Enrolled ISI into carrying out statistical
studies for normalization of board scores
September 2011
Alternative system
Work carried out Indian
Statistical Institute
For exploring normalization
methodologies for school board
scores
September 2011
Alternative system
Pilot testing

Selected four boards for pilot testing



Variations in Density of population



Central Board, TN Board, WB Board, ISE
Evaluation years 3-4 years for each board
CBSE (5-6lakhs), TN(5.6-7.3 lakhs), WB (3.0 -4.6 lakhs, ICSE
25000-56000
Evaluation of stability of scores over time for the same
board
Potentials for mapping the profiles of several boards onto
one selected board through monotone transformations
September 2011
Alternative system
Models tested


Select a percentile score (P) for all boards and
determine the scores (X1) for P across boards
Mapping
Model 1: Y1= {Xn – X1}/{Xm-X1}
Model 2: Y2 = Xn/X1
where Xn , X1 , Xm are scores obtained by any
candidate, marks corresponding to percentile P,
and maximum scores obtained by any candidate
in each board. Y1 will range from 0 to 1.0; while
Y2 will be ratios in the range from 1.0 to Xm/X1
September 2011
Alternative system
Observed Relationships of
Scores
For the four boards over time
Model 1: Y1= {Xn – X1}/{Xm-X1}
Where aggregate score percentages
are used
September 2011
Alternative system
1.0
Normalized score vs. percentile rank: cutoff 60 %
1.0
Normalized score vs. percentile rank: cutoff 50 %
0.6
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
0.0
0.2
0.4
normalized score
0.6
0.4
0.0
60
70
80
90
100
60
70
80
90
percentile rank
Normalized score vs. percentile rank: cutoff 75 %
Normalized score vs. percentile rank: cutoff 85 %
100
1.0
percentile rank
1.0
50
CBSE 2007
CBSE 2008
CBSE 2009
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
0.0
0.2
0.4
0.6
0.4
0.2
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
0.6
0.8
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
normalized score
0.8
CBSE 2007
CBSE 2008
CBSE 2009
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
0.0
normalized score
CBSE 2007
CBSE 2008
CBSE 2009
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
0.8
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
0.2
normalized score
0.8
CBSE 2007
CBSE 2008
CBSE 2009
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
75
80
85
90
percentile rank
95
100
85
90
95
percentile rank
100
100
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
90
70
percentile rank
80
90
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
CBSE 2007
CBSE 2008
CBSE 2009
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
60
50
70
CBSE 2007
CBSE 2008
CBSE 2009
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
60
percentile rank
Percentile rank vs. aggregate score: cutoff 60 %
80
100
Percentile rank vs. aggregate score: cutoff 50 %
0.5
0.6
0.7
0.8
0.9
1.0
0.6
aggregate score as fraction of max score
100
100
0.9
1.0
Percentile rank vs. aggregate score: cutoff 85 %
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
90
85
90
percentile rank
95
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
95
0.8
aggregate score as fraction of max score
Percentile rank vs. aggregate score: cutoff 75 %
0.6
0.7
0.8
0.9
aggregate score as fraction of max score
CBSE 2007
CBSE 2008
CBSE 2009
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
85
80
CBSE 2007
CBSE 2008
CBSE 2009
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
75
percentile rank
0.7
1.0
0.7
0.8
0.9
aggregate score as fraction of max score
1.0
Normalized score vs. percentile rank for PCMB: cutoff 50 %
1.0
1.0
Normalized score vs. percentile rank for PCMB: cutoff 60 %
CBSE 2007
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
0.6
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
0.0
0.0
0.2
0.4
0.4
0.6
normalized score
0.8
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
0.2
normalized score
0.8
CBSE 2007
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
50
60
70
80
90
100
60
70
80
percentile rank
100
percentile rank
1.0
Normalized score vs. percentile rank for PCMB: cutoff 85 %
1.0
Normalized score vs. percentile rank for PCMB: cutoff 75 %
CBSE 2007
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
0.0
0.2
0.4
0.6
0.4
0.2
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
0.6
0.8
TN 2007
TN 2008
TN 2009
TN 2010
WB 2007
WB 2008
WB 2009
normalized score
0.8
CBSE 2007
ICSE 2007
ICSE 2008
ICSE 2009
ICSE 2010
0.0
normalized score
90
75
80
85
90
percentile rank
95
100
85
90
95
percentile rank
100
Standardized (Normalized)
score
Normalized percentile rank
Percentile rank of student – 75
=
September 2011
100 – 75
Alternative system
X 100.
Some general conclusions
emanating from the ISI study



Percentile scores are relatively stable over the
periods studied for each board when aggregate
percentage scores are analyzed
It is possible to normalize percentile ranks across
boards
Correlations of normalized percentile ranks
against percentile ranks of various boards map
on to common linear relationship
September 2011
Alternative system
Transformations onto one
theoretical board
60
40
0
20
normalized percentile rank
80
100
Normalized percentile rank vs. percentile rank: cutoff 75%
75
80
85
90
95
100
percentile rank
ISI Experts are certain that this transformation will be the same
for all boards for all years
September 2011
Alternative system
It appears that for normalization of
school board scores
A statistical method is feasible after all.
ISI may be encouraged to develop the
methodology further and extend it to all
boards and reconstruct past scenario for
present IIT and NIT students over the last
four years for revalidation of the method
September 2011
Alternative system
A pilot test among a select group
of students: A suggestion

A group of statistically significant number of
some volunteers from current student population
may be enrolled into a pilot test for evaluating
the utility of different models and suggestions.
This pilot test has to be designed with care and
confidentiality based on informed consent of all
involved. The merit or otherwise of the approach
will be discussed within the committee before
decisions are taken
September 2011
Alternative system
Summary of work done so far

Opinion poll reveals support for reforms and favor single
examination


ISI study presents a methodology for scientific
normalization of scores across school boards


with provisions for both aptitude and advanced while making
provisions for factoring-in scholastic performance in school board
examinations
Percentile scores are stable over each board and it is possible to
carry out monotone transform board scores and accomplish
normalization across boards
Weighting options for school and entrance tests’
performance are considered
September 2011
Alternative system
Two approaches considered

Approach 1



weighing consistency of performance in school board
examinations and employ them for testing ability to
write solutions and
One objective screening test with two sections; one for
testing the aptitude and the other advanced
knowledge in domain areas.
Approach 2


weighing consistency of performance in school board
examinations and employ them for testing ability to
write solutions and
one objective aptitude test based on multiple choices
and computer based correction systems
September 2011
Alternative system
Considerations of six different
options

Option 1: Deployment of Scores as criteria based on
class XII performance only




Equal weighting of school board scores A1and A2
Equal weighting of aptitude scores A4 and advanced scores
A5
Normalized score
= {A1 + A 2+A4 +A5 }/4
Option 2: Deployment of Scores as criteria based on
class XII performance only



Equal Weighting of board score A3 which is {A1+ A2}/2
Equal Weighting of Aptitude scores A4 and A5
Normalized score
={A3 +A 4+A5}/3
September 2011
Alternative system
Considerations of six different
options

Option 3: Deployment of Scores as criteria based on consistency of performance
at class X and Class XII levels as well as in National Level Aptitude and
Advanced Tests





Equal weighting for aggregate as well as subject performance at class X and Class
XII levels where ) 0.1X (normalized score at class X in aggregate + normalized score
at class X in subjects of choice + normalized score at class XII + normalized score at
class XII in subjects of choice)
One third weighting of aptitude score 0.3 A4
One third weighting of advanced score A5
Normalized score
=0.1{ Normalized aggregate class X + normalized class X
subject score + Normalized class XII aggregate + Normalized class XII subject
score} + 0.3 A3and 0.3 A5
Option 4: Deployment of School Board Performance as screening but not as
determinant for National ranks




Specify a Cut-off normalized percentile rank score for school performance say as 80
or 85 percentile rank
50% weighting of National Level Aptitude score A4 for candidates passing the cut
off of percentile rank
50% weighting of National Level Advanced Score A5 for candidates passing the
Normalized score
= 0.5 A4 +0.5A5
September 2011
Alternative system
Considerations of six different
options


Option 5: Deployment of School Board performance as
subject score and National Level Aptitude Test as a
combination and avoid the Advanced Testing system
according freedom for the individual institutions to
select mixing proportions within a pre-specified
guideline
Option 6: Equal weighting of School Board
performance as subject score and National Level
Aptitude Test as objective test system where
Normalized score
=0.5 A2+0.5A4
September 2011
Alternative system
Recommended preferences of
the committee




Option 2: Deployment of Scores as criteria based on class XII
performance
Option 6: Equal weighting of School Board performance as
subject score and National Level Aptitude Test as objective test
system
0.5 A2+0.5A4
Option 5: Deployment of School Board performance as subject
score and National Level Aptitude Test as a combination and
avoid the Advanced Testing system with freedom for the
individual institutions to select mixing proportions within a prespecified guideline
Option 4: Deployment of School Board Performance as
screening but not as determinant for National ranks;
Specify a Cut-off normalized percentile rank score for
school performance say as 80 or 85 percentile rank;
and rank by 0.5 A4 +0.5A5
September 2011
Alternative system
A Suggestion



A committee of experts from engineering
institutions could be assigned the task of
interacting with ISI Group for internalization of
methodology for normalization of board scores
IITs could be assigned the task of setting up a
question paper for National Screening Test
based on objective examination models and
conduct the examinations for the year 2012-13
For Aptitude examination like SAT, we may take
the help of NTS or any other global agency
September 2011
Alternative system
Thank you For the patience
and waiting
We recommend
normalization of school board scores for
factoring-in based on ISI inputs
Single National level test to cover aptitude
and advanced or aptitude alone and
Judicious mix of school and national test
performance
September 2011
Alternative system