The Development and Application of
the SACMEQ HIV-AIDS Knowledge Test for Pupils
at the Upper-Primary School Level in 15 African Countries
Kenneth N. Ross
1
University of Melbourne, Australia
2 September 2010
1. Sub-Saharan Africa, HIV-AIDS,
and the SACMEQ III Project
2
3
SACMEQ’s Mission
To deliver training and research programmes
for African educational planners and researchers that
will enable them to:
(a) monitor and evaluate the quality of their basic
education systems; and
(b) generate research-based information that can be
used by decision-makers to plan improvements in
the quality of education.
4
Context Information for SACMEQ School Systems
5
SACMEQ Assembly of
Ministers of Education
General Policy
SACMEQ Managing
Committee
Detailed Policy and
Monitoring
Training and
Technical Support
SACMEQ Coordinating Centre
General Administration
and Country-Specific
Technical Issues
6
Scientific &
Measurement
Committees
Co-operation
“External Friends”
Training and
Technical
Support
15 SACMEQ NRCs and
30 D/NRCs
SACMEQ Conceptual Model
School Characteristics:
Type, Location, Enrolment, Resources,
Principal’s Qualification, Parental
Involvement, etc.
Learner Characteristics:
Age, Sex, Attendance, Repetition, SES,
Nutrition, Siblings, Home Help, etc.
Teacher Characteristics:
Age, Sex, Qualifications, Subjects,
Classroom resources, Behaviour,
In-Service Training, etc.
7
Learner Achievement
 Reading
 Maths
 HIV/AIDS
8
The SACMEQ I
Project
The SACMEQ II
Project
The SACMEQ III
Project
1995 - 1998
1999 - 2003
2007 - 2010
7 MoE
20,000 Grade 6
pupils
1,000 schools
14 MoE
40,000 Grade 6
pupils
2,000 schools
Questionnaires
Questionnaires
Questionnaires
Reading Test
for Pupils.
Reading & Maths
Tests for Pupils and
Teachers
Reading, Maths &
HIV/AIDS Knowledge
Tests for Pupils and
Teachers
15 MoE
61,421 Grade 6
pupils
8,045 teachers
2,779 schools
Adults and children estimated to be living
with HIV (UNAIDS/WHO, 2007)
9
Children (< 15 years) estimated to be living
with HIV (UNAIDS/WHO, Dec 2007)
10
Adult HIV/AIDS Prevalence Rate (Aged 15-49)
in SACMEQ Countries (UNAIDS, 2007)
1
2
3
4
5
5
6
7
8
9
11
13
29
50
11
Swaziland
Botswana
Lesotho
South Africa
Namibia
Zimbabwe
Zambia
Mozambique
Malawi
Kenya
Tanzania (United Rep. of)
Uganda
Mauritius
Seychelles
26.1%
23.9%
23.2%
18.1%
15.3%
15.3%
15.2%
12.5%
11.9%
7.1-8.5%
6.2%
5.4%
1.7%
NA
2. SACMEQ HIV-AIDS Knowledge Test
(Known as the HAKT)
12
UNGASS Indicator 13: “Knowledge about HIV Prevention”
Percentage of young people aged 15-24 who provide
correct answers to the following questions about
(a) sexual transmission of HIV, and (b) misconceptions about HIV.
SEXUAL TRANSMISSION OF HIV
1. Can the risk of HIV transmission be reduced by having sex
with only one uninfected partner who has no other partners?
2. Can a person reduce the risk of getting HIV by using a condom
every time they have sex?
MISCONCEPTIONS ABOUT HIV
3. Can a healthy-looking person have HIV?
4. Can a person get HIV from mosquito bites?
13
5. Can a person get HIV by sharing food with someone
who is infected?
Problems with the UNGASS 13 Indicator
1. LOW RELIABILITY (0.28)
2. LOW CONTENT VALIDITY
(a) Poor Curriculum Coverage, and (b) Concentration on “Misconceptions”
3. LINGUISTIC COMPLEXITY (Intersecting questions & qualifiers).
4. INVITATION TO CHANGE INDICATOR QUESTIONS
“The first three questions should not be altered. However,
Questions 4 & 5 ask about local misconceptions and may be replaced by
the most common misconceptions in your country.”
5. WRONG TARGET GROUP (15-24 years is too late).
6. JUSTIFICATIONS GIVEN FOR ITEM SELECTION
For example; “The belief that HIV is transmitted through mosquito bites
can weaken motivation to adopt safer sexual behaviour.”
14
TWO DEFINITIONS OF AIDS
“AIDS, the acquired immunodeficiency syndrome,
is a fatal disease caused by HIV,
the human immunodeficiency virus.”
(UNAIDS, 2007, p. 8).
“AIDS is not a disease. It is a syndrome or
group of illnesses resulting from a weakening
of the immune system.”
(UNESCO, 2007. p. 28).
15
SACMEQ HIV-AIDS Knowledge Test
•Based on Official Curriculum, Teaching Guides & Materials
•Target Group = Grade 6 Pupils (Average Age = 13.5 years)
•Extensive Approval Process: MoEs (at all levels), Ministers of
Education), SACMEQ Scientific Committee, External Experts.
•Standardized Test Administration (Scripted Group
Interview)
•Balance of True/False Items (86)
•Administered to over 60,000 pupils in around 2800 schools
across 15 countries.
16
• Reliability Coefficient = 0.85
Administration of the HAKT
1. Two data collectors manage group administration of the
HAKT for samples of 25 pupils per school.
2. Extensive training of data collectors via role play.
3. Practice questions (eg. “Elephants can fly like birds.”)
4. Presentation (double reading) of test items.
5. Supervision and checking of progress.
6. After the testing session – explanation/discussion.
17
SACMEQ
Knowledge
Topics for
HAKT
Knowledge Topics
Knowledge Sub-Topics
Definitions /
Distinctions
Terminology: HIV
Terminology: AIDS
Terminology: ARV
HIV and AIDS Links: Causal Direction
HIV and AIDS Links: Sickness
Body Fluids: Blood
Body Fluids: Saliva
Body Fluids: Semen
Body Fluids: Sweat
Body Fluids: Vaginal
Hygiene: Food
Hygiene: Water
Hygiene: Smoke/Dirt
Hygiene: Cough/Spit
Mother to Child: Pregnancy and Birth
Mother to Child: Breast Milk
External Sources: Needles
External Sources: Animals/Insects
External Sources: Bites
Sexual Conduct: Abstain (“A”)
Sexual Conduct: Be Faithful (“B”)
Sexual Conduct: Condoms (“C”)
Sexual Conduct: Delay
Sexual Conduct: Appearance Healthy
Sexual Conduct: Uninfected Person
Sexual Conduct: Additional Risks
Social Contact: Proximity
Social Contact: Touch
Social Contact: Sharing Food
Social Contact: Sharing Objects
Social Contact: Additional Risks
Diagnosis : Medical Test
Diagnosis : Virus Detection
Diagnosis : Opportunistic Infections
Diagnosis : Appearance (HIV)
Diagnosis : Appearance (AIDS)
Treatment: ARV Medication
Treatment: Doctor’s Medicine
Treatment: Cure
Myths: Healing
Myths: Witches
Misconceptions: Avoidance Posssibility
Misconceptions: Certainties
Total=
(10 items)
Transmission
Mechanisms
(28 items)
Avoidance Behaviours
(24 items)
Diagnosis and
Treatement
(16 items)
18
Myths and
Misconceptions
(8 items)
Items
N
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
86
Selected Items from the HAKT
19
Three Pupil & Teacher Performance Scores:
Transformed Scores – scaled HAKT scores transformed to a
SACMEQ pupil mean of 500 and standard deviation of 100.
Minimal Knowledge Scores – dichotomous scores that
indicated whether or not respondents had mastered at least
50% of the official curriculum.
(Expectation=100% of pupils reach this level)
Desirable Knowledge Scores – dichotomous scores that
indicated whether or not respondents had mastered at least
75% of the official curriculum.
(Expectation=at least half of the pupils reach this level)
Question 1
What knowledge do Grade 6 pupils
and their teachers have about HIVAIDS?
21
School
System
Mauritius
Lesotho
Zimbabwe
Seychelles
Zambia
Uganda
Botswana
SACMEQ III
Zanzibar
Namibia
South Africa
Mozambique
Kenya
Malawi
Swaziland
Tanzania
22
Transformed
score
Mean
SE
453
5
465
4
477
5
488
2
488
4
489
4
499
4
500
4
501
3
502
3
503
4
507
6
509
4
512
5
531
3
576
4
PUPILS
Reaching
minimum
level
%
SE
17
2
19
1
30
2
25
1
35
2
33
2
32
2
36
2
38
1
36
2
35
2
40
2
39
2
43
2
52
2
70
2
TEACHERS
Reaching
Reaching
desirable Transformed
minimum
level
score
level
%
SE Mean
SE
%
SE
2
1
698
6
98
1
5
1
751
8
99
1
4
1
785
7
99
0
3
0
789
3
99
0
4
1
744
7
98
1
4
1
708
9
98
1
7
1
782
6
100
0
7
1
746
7
99
1
4
0
657
5
94
1
6
1
764
6
100
1
8
1
781
6
100
0
8
2
741
7
99
1
7
1
793
8
100
0
9
1
714
9
99
1
4
1
759
7
100
0
24
1
724
7
99
1
Reaching
desirable
level
%
SE
63
3
82
3
93
2
95
0
86
2
72
3
93
2
82
2
45
3
87
2
93
2
81
3
95
2
72
4
89
2
82
3
TEACHERS
PUPILS
MINIMUM LEVEL
23
DESIRABLE LEVEL
Question 2
What are the within-country
regional variations in pupil
knowledge about HIV-AIDS?
24
Mozambique
PUPILS
Reaching
Province
Transformed minimum
score
level
Mean
SE
%
SE
Tete
448
9
20
4
Cabo Delgado
452
10
22
4
Niassa
473
9
27
4
Sofala
487
8
37
3
Gaza
Inhambane
503
12
39
7
Sofala
Zambesia
506
10
42
5
MOZAMBIQUE
507
6
40
2
Manica
507
12
41
6
Nampula
509
23
42
6
Maputo
City
522
18
46
7
Inhambane
Maputo Province 537
21
50
6
Sofala
Maputo
Gaza City
579
34
59
7
25
Reaching Transforme
desirable
d
level
score
%
SE Mean
SE
2
1
736
16
2
1
704
20
1
1
734
26
4
1
790
24
4
2
744
26
4
1
721
21
8
2
741
7
8
2
796
23
12
6
708
22
9
5
725
14
11
6
755
17
24
11
754
17
TEACHERS
Reaching
minimum
level
%
SE
100
0
100
0
100
0
100
0
100
0
100
0
99
1
100
0
94
6
100
0
98
2
100
0
Reaching
desirable
level
%
SE
86
8
66
11
73
12
87
6
77
10
77
9
81
3
91
5
75
10
83
8
86
6
88
5
26
HAKT
Scores by
regions
27
Question 3
Are there gender differences among
Grade 6 pupils in knowledge about
HIV-AIDS?
28
PUPILS
School
System
Botswana
Kenya
Lesotho
Malawi
Mauritius
Mozambique
Namibia
Seychelles
South Africa
Swaziland
Tanzania
Uganda
Zambia
Zanzibar
Zimbabwe
SACMEQ III
29
Transformed score
Boys
Mean
SE
491
514
462
523
446
514
500
478
493
531
581
495
496
503
476
500
5
4
4
6
5
6
3
3
4
3
5
5
5
4
6
5
Reaching minimum level
Girls
Mean
SE
507
505
467
500
460
498
504
498
512
530
571
483
479
499
479
499
4
5
4
6
4
7
3
3
4
3
5
4
5
3
5
4
Reaching desirable level
Boys
%
SE
Girls
%
SE
Boys
%
SE
Girls
%
SE
30
42
18
48
16
43
35
22
31
53
72
36
36
39
30
37
34
37
20
39
18
37
37
29
38
51
69
31
34
38
29
36
7
8
5
11
2
8
6
2
6
4
26
5
5
4
5
7
8
6
5
7
2
8
5
4
10
4
23
4
3
3
4
6
2
2
2
3
2
2
2
2
2
2
2
2
2
2
3
2
2
2
2
3
2
2
2
2
2
2
2
2
2
2
2
2
1
1
1
1
1
1
1
1
1
1
2
1
1
1
1
1
1
1
1
1
1
2
1
1
1
1
2
1
1
1
1
1
Question 4
What was the variation in Grade 6 pupil
knowledge about HIV-AIDS – between
and within schools?
30
Average Pupil
HAKT Score
Between-School Variation
29
Tanzania (576)
90
14
Swaziland (531)
31
Kenya (85)
54
51
Mozambique (507)
Namibia (79)
49
30
64
11
Zanzibar (501)
SACMEQIII (500)
31
Botswana (499)
30
Seychelles (62)
Zimbabwe (112)
75
77
17
Lesotho (465)
Lesotho (94)
53
25
40
Zambia (97)
58
37
60
Uganda (97)
72
4
80
Botswana (97)
59
Seychelles (488)
Maurice (453)
SACMEQ III (100)
67
25
Zambia (488)
Zanzibar (75)
69
38
Zimbabwe (477)
South Africa (102)
55
Namibia (502)
Uganda (489)
Mozambique (129)
78
47
South Africa (503)
31
Malawi (112)
62
Kenya (509)
20
0
20
40
Tanzania (119)
Swaziland (48)
34
50
Malawi (512)
100
Total Variation
in HAKT Scores
Within-School Variation
60
Maurice (78)
80
100
Question 5
How did Grade 6 teachers rate their
own risk of becoming infected with
HIV?
32
Teacher Assessment of Personal Risk of Infection with HIV
33
Question 6
What percentage of Grade 6 pupils
attended lessons on HIV-AIDS?
34
35
Question 7
What did Grade 6 pupils nominate as
“the best source of information
about HIV and AIDS”?
36
Best Source of Information on HIV-AIDS
Medium
Electronic (Radio, TV, Video)
Performance (Play, Drama, Concert)
Printed (Books, Magazines, Newpapers)
Computer/Internet
Person
Teacher
Health Professional
Family
Friend or Peer
Person with HIV
Religious
TOTAL
Percent
42.3
7.7
6.8
1.7
17.7
13.0
4.4
3.8
1.6
0.9
100.0
Question 8
What were the “Curriculum Gaps”?
38
List of the 25 best understood questions
(pupils)
List of the 25 worst understood questions
(pupils)
List of the 25 best understood questions
(teacher)
List of the 25 worst understood questions
(teachers)
Question 9
What were the country rankings on
the HAKT compared with the
UNGASS 13 indicator?
43
Comparison of National Rankings on UNGASS and HAKT Indicators
FUTURE DIRECTIONS
 IIEP Policy brochures (2010)
 SACMEQ III Data Archive (2010-11)
 UBW research programme (2010-11): designing and validating
different language versions of the HAKT
 SACMEQ IV Project
FURTHER INFORMATION
Kenneth N. Ross
University of Melbourne
k.ross@unimelb.edu.au
 WWW.SACMEQ.ORG
45