ConQuest:
preparation of data, work with the
program, the interpretation of
output data
Galina Larina
28-31 of March, 2012
University of Ostrava
About this program
https://shop.acer.edu.au
/acer-shop/group/CON2
• Generalized Item Response Modeling Software
• ConQuest developed by Australian Council for Educational
Research (ACER) and University of California, Berkeley
Advantages
•
•
•
•
•
Works with a big number of
•
models, including Many-Facet
and Multidimensional ones
•
(Rasch)
Reports a confidence interval
for fit statistics
•
Good item analysis
Creates a variable map
Many outputs
Disadvantages
Requires making of control
file
Requires special knowledge
on interpretation outputs in
complex analysis
Doesn’t work with 1PL and
2PL models and their
polytomous extensions
Application
•
•
•
•
•
Performing item analysis
Exploring rater effects
Examining DIF
Estimating latent correlation and testing dimensionality
Fitting a wide variety of item response models:
–
–
–
–
–
–
Rasch’s Model
Rating Scale Model
Partial Credit Model
Multifaceted Models
Multidimensional Item Response Models
etc.
Performing item analysis
(Rasch analysis)
.shw
These tables are for the term item (dichotomous items) and term item*step
(polytomous items), as the first and second terms in the model statement
Performing traditional item
analysis
.itn
This table shows
summary results
These outputs conclude
tables showing classical
difficulty, discrimination and
point-biserial statistics for
each items (dichotomous
and polytomous)
Performing item analysis
Map of latent distributions .shw
This histogram illustrates
the distribution of student’s
achievement. In this
example each ‘X’ means
9.7 cases.
There are two maps in ouputs:
- response model parameter
estimates
- generalized-items tresholds
Items are plotted to
indicate their
difficulty level
These are Thurstonian
thresholdes for each of the
items. The notation x.y is
used to indicate the y-th
thresholds of the x-th item.
ConQuest Plots
Dichotomous item
ConQuest Plots
Polytomous item
ConQuest
Examinees
ID number
Raw score that Maximum
Student’s latent Standard error
student attained possible score
ability
Steps of work
Data
• No missings, recode ones
– Only numerical or letter symbols in matrix data
• Individual file with matrix data
– Without unique ID
– Or with unique ID (in columns 1 through 9)
• Save your data in Notepad and name it like
ex1data.dat
Steps of work
Variable labels
• Individual file with variable labels looks like
• First line of the file is required
===> item
• Amount of spaces doesn’t matter
• In this example the label for item 1 is BSMMA01, the label
for item 2 is BSMMA02, and so on.
• Save your data in Notepad and name it like ex1names.dat
Steps of work
Command File
• Example
• Save your command file in notepad and name it like
ex1run.dat
ConQuest
Commands
• Datafile indicates the name and location of the data
file
• Format statement describes the layout of the data in
the file ex1data.dat. In this example id 1-9 means
unique id is located in columns 1 through 9. And
responses 10-26 means that the responses to the
items are in columns 10 through 26
• Labels indicates the name and location of the file with
variable labels
• Export logfile indicates the name and location of the
logfile
• Codes identifies all valid codes in data file
ConQuest
Commands
• Key statement identifies the correct response for each of multiplechoice item.
– Dichotomous test:
Key 14323487 ! 1;
– Non-dichotomous test:
Key 4111111411231411 ! 1;
Key xxxx22xxxxxx2xx2 ! 2;
• Model specifies the item response model that is to be used in the
estimation.
– model item in case of simple logistic model. We are dealing with
single-faceted dichotomous data
– model item + item*step in case of PCM. We are dealing with
polytomous items or a mixture of dichotomous and polytomous
data
– model item + step in case of RSM. We are dealing with
polytomous items, where the step parameters are the same for all
items
– And so on…
ConQuest
Commands
• Estimate statement initiates the estimation of the item response
model. You can select some special options for your analysis:
– type of method
– maximum number if iterations
– etc.
• Show statement produces a sequence of tables that summarizes the
result of fitting the item response model. The result are redirected to a
file ex1.shw in this example.
• Show cases statement produces a display of the results of a
examinee analysis. The result are redirected to a file ex1_stud.shw in
this example. You can select the type of estimate - it can be eap,
latent, mle or wle.
• Itanal statement produces a display of the results of a traditional
item analysis. The result are redirected to a file ex1.ita in this example.
ConQuest
Run the program
1. File – Open –
Find your command file
2. Run – Run all
ConQuest
Manual
Manual consist of four sections:
– Introduction provides a brief survey of the models
that ConQuest can fit
– Tutorial contains nine samples of ConQuest analysis
and describes how to use the program to address
particular problems without any underlying
methodology
– Technical Matters provides underlying in ConQuest
methodology
– Command Reference contains general information
about the syntax of ConQuest statements
Exploring rater effects
Raters
.shw
Fit statistics for the raters. These ones lap over it’s
confident interval.
Exploring rater effects
Criteria
.shw
Fit statistics for the criteria. These ones lap over it’s
confident interval.
Exploring rater effects
Maps of the parameter estimates.shw
Examinee
Rater
Criteria
Examinee
Rater.Criteria.Step
Exploring rater effects
Plots
Testing dimensionality
Multidimensional model
Control file
Multidimensional model
.shw
Correlations/covariance between dimensions
COVARIANCE/CORRELATION MATRIX
Dimension
-----------------Dimension
1
2
Dim 1
0.553
Dim 2
0.928
------------------------------------------Variance
0.624 0.570
-------------------------------------------
Covariance coefficients
Correlations coefficients
Multidimensional model
Reliability coefficients
Between-Item
RELIABILITY COEFFICIENTS
-----------------------Dimension: (Dim 1)
----------------------MLE Person separation RELIABILITY:
WLE Person separation RELIABILITY:
EAP/PV RELIABILITY:
0.871
-----------------------Dimension: (Dim 2)
----------------------MLE Person separation RELIABILITY:
WLE Person separation RELIABILITY:
EAP/PV RELIABILITY:
0.849
лалала
Unavailable
Unavailable
Unavailable
Unavailable
.shw
Multidimensional model
Examinees
Dimension 1
Dimension 2
.shw