ND
BASIC IRRS TRAINING 2
EDITION AND REFRESHER
COURSE CONCEPT
FURTHER DEVELOPING THE BIT CONCEPT
VERSION 3, MARCH 2014
1
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
Table of Contents
I.
II.
BACKGROUND ............................................................................................................................. 2
I.1.
INTRODUCTION ................................................................................................................... 2
I.2.
BIT COURSE 1st EDITION ...................................................................................................... 2
I.3.
FEEDBACK ON THE 1st EDITION ........................................................................................... 3
I.4.
EVALUATION OF THE ANSWERS TO THE TEST QUESTIONNAIRES ...................................... 5
ISSUES TO CONSIDER FOR THE 2ND EDITION ............................................................................... 9
II.1.
GENERAL ISSUES................................................................................................................. 9
II.2.
SPECIFIC ISSUES .................................................................................................................. 9
III. CONCEPT OF A REFRESHER COURSE ......................................................................................... 10
III.1.
GENERAL .......................................................................................................................... 10
III.2.
INITIAL MISSION REFRESHER LECTURES ........................................................................... 10
III.3.
FOLLOW-UP MISSION REFRESHING LECTURE ................................................................... 11
IV. IMPLEMENTATION PLAN........................................................................................................... 12
V.
IV.1.
FINALIZATION OF THE CONCEPT ....................................................................................... 12
IV.2.
REVISION OF TRAINING MATERIAL ................................................................................... 12
IV.3.
ORGANIZATION OF BIT COURSE 2ND EDITION (6-9 OCTOBER 2014) ................................. 12
IV.4.
DEVELOPING THE REFRESHING TRAINING MATERIAL ...................................................... 12
IRRS MODULES .......................................................................................................................... 13
2
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
This project is funded
by the European Union.
Basic IRRS Training (BIT) 2nd
Edition and Refresher Course Concept
I.
BACKGROUND
I.1.
INTRODUCTION
The IAEA Integrated Regulatory Review Service (IRRS) was established by the International Atomic
Energy Agency to strengthen and enhance the effectiveness of national regulatory infrastructure for
nuclear, radiation, radioactive waste and transport safety and the security of radioactive sources
whilst recognizing the ultimate responsibility of each State to ensure safety in these areas.
IRRS team members are recruited from IAEA Member States, the staff of the IAEA Secretariat, and
consultants from outside the IAEA Secretariat. Those recruited from Member States are experienced
regulators, expert in specific topics.
In order to train potential new team members for upcoming Missions, the first IAEA Basic IRRS
Training Course was implemented in October 2013. The Training Course provided essential
information on preparing and conducting IRRS missions.
The 1st BIT Course was held with the participation of 62 experts of 38 Member States. Contents of
the Course are given in the next section.
This Concept is meant to summarize the lessons learned from the 1 st BIT Course and formulate the
issues and actions needed for preparing the 2nd course on one hand and for a refresher course on
the other hand. The intended audience of this Concept is the team taking part in the development of
the course lectures and test questionnaires.
I.2.
BIT COURSE 1st EDITION
Over four working days the course provided 16 training lectures and three information lectures in
the form of PowerPoint presentations developed by experienced IRRS reviewers. The effectiveness
of the presentations was measured using test questionnaires completed by the participants at the
end of each day. Evaluation of the questionnaire responses is detailed in the next section. The
presentations given in the first BIT course are listed below (authors in parentheses):
Training presentations
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
Objectives and the Process (A. Stritar)
IAEA Safety Standards as the Bases (I. Lux and L. Baekelandt)
The Preparatory Phase (G. Schwarz)
Responsibilities and Roles (G. Schwarz)
Methodology of the review, Parts I and II (E. Jende)
Evaluation and Documentation of Observations (I. Lux)
Report writing (I. Lux)
Scheduling IRRS missions (I. Lux)
Reviewing IRRS Modules 1, 2 and 3 (A. Stritar)
Reviewing IRRS Modules 4 and 12 (E. Jende)
Reviewing IRRS Modules 5, 7, and 8 (A. Stritar)
3
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
12.
13.
15.
16.
17.
Reviewing IRRS Module 6 (J. Misak)
Reviewing IRRS Module 9 (L. Baekelandt)
Reviewing IRRS Module 10 (A. Baciu)
Reviewing IRRS Module 11 (V. Kamenopoulou, M-L. Perrin, and L. Baekelandt)
Follow-up mission (I. Lux)
Information presentations
1. History and Background of IRRS (I. Lux)
2. Host country preparations (S. Evans)
3. Evaluation of a mission (I. Lux)
Lectures and Test Questionnaires of the 1st Edition are available from the Share Point area Basic IRRS
Training Documents
I.3.
FEEDBACK ON THE 1st EDITION
At the end of the first BIT course participants were invited to give feedback on the quality and
effectiveness of the course by giving marks as well as comments in reply to seven questions. The
questions and the average rating by participants are given in the table below. (5 = highest
satisfaction).
Evaluation of the BIT Course by participants
Question
Mark
How useful you consider the training in your possible future IRRS activity?
4.64
How do you rate the training course presentations from the point of view of
the information therein and their completeness
4.45
their clarity
4.30
their format
4.29
How do you rate the training course lecturers from the point of view of
their competence in the field
4.69
the way they held the presentations
4.26
What is your opinion on the course-related organization and arrangements by
IAEA?
4.69
Average:
4.47
Number of answers:
56
The numerical results express high participant satisfaction. Important comments offered by
participants are summarized below.
I.3.1. General and participation
 The course should be held regularly
 Because this is basic training, it is hoped it will be followed by more or advance training
 All reviewers of IRRS missions should participate in such a course
 The course would be useful also for liaison officers and other representatives of IRRS host
countries
 The term “module” was used in several confusing meaning
 Participants should have a basic knowledge of the IAEA safety standards prior to training
 Longer time for discussions would be appreciated
4
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
I.3.2. Materials to distribute
 Some material (e.g. GSR Part 1, IRRS Guidelines) should be distributed prior to the course
 Hand-outs with the presentations, with space to take notes should be distributed
 Presentations should be distributed in advance
 Presentations should be distributed on memory sticks instead of CD’s
I.3.3. Quality of the presentations
 Some slides were hard to read (with too small letters)
 Some pictures had too much information
 Presentations should have more pictures, diagrams and flowcharts to be more interactive
and attractive
 Too much wording on some slides
I.3.4. Quality of presenting
 Some lecturers do not add their own experiences; the more experiences to illustrate, the
more interesting the presentation
 Very good lecturer from Slovenia, others too formal
 Mr Stritar was the most interesting because of his field experience. Also the introduction in
legislation was very interesting (Baekelandt, Lux).
 Lecturers are very competent and give the presentations in a professional manner
 Some read mainly from the slides, the best ones offered insight, example
 Excellent preparedness of all lecturers involved in this training course
 Very professional lecturers with a lot of practical experiences
 They are good and professional in their fields
 There is still an opportunity for presenters to more improve their vocal and body language
characteristics for better delivery of presentations
 Some presentations were too fast
I.3.5. Contents of the presentations
 An overall introduction to describe the scope of presentations to come would be needed
 Slides on "Recommendations of experienced IRRS members" were particular interesting
 More practical examples would make the presentations more useful
 Too much information given in a short time, difficult to absorb and remember
 There were overlaps among speakers
 Some presentations were too theoretical
 Some practical examples would be beneficial
 The presentations should go beyond what is written on the slides
 There is no need to go through all requirements of GSR Part 1 in a separate presentation, all
relevant requirements were handled in different modules
 TM5 is actually a fairly general auditing techniques (not specific to IRRS) - many attendants
might already know that
 The presentation on evaluation of missions should be shorter
 Review of IRRS Module 6 (Review and Assessment) should be treated in more detail
 Generally it was easy to understand, TM12 Module 6 was hard to understand
 The additional areas were not covered thoroughly enough
1.3.6. Extending the scope of the course
 Some practical sessions and some movies for IRRS reviewers are recommended
 A short review should be simulated during the training course to practice review of
documents
5
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept

Clearer presentation of the preliminary work to be done by the IRRS reviewer and more
examples on the questions/ data to be included in the interviews would be needed
Maybe some in-depth modules on the content (modules of IRRS) would be appropriate,
divided into subgroups where specialists for specific IRRS modules would get deeper
knowledge on common understanding of requirements/guides
Modular workshops could be organized
It would be great to hold courses on different modules separately



1.3.7. Test Questions
 The questions should be more related to the presentations
 The questions in the test were sometime confusing - the meaning of a possible answer was
not clear enough
I.4.
EVALUATION OF THE ANSWERS TO THE TEST QUESTIONNAIRES
Participants of the course were invited to answer test questionnaires related to all training modules.
The questionnaires contained questions on important information in the presentations and offered
various possible answers. The participants had to select the right answer(s) for each question.
Consistently wrong answers to the test questionnaires may indicate either


possible weakness in the content or presentation of the training lecture, or
ambiguity in the test question
The results of the test questionnaires are summarized below based on responses from 55 to 60
participants. Numbers in the shaded fields refer to the training lectures, the numbers of the test
questions are given in black; the blue numbers are the absolute values and the red numbers give the
fraction of the right answers relative the total number of answers (which is given after the cells
denoted by No). The average fractions are shown in the right lowest cells.
31
2
38
3
38
4
43
5
38
6
45
7
38
8
33
9
40
10 34
11 48
12 47
13 48
14 25
15 13
16 34
No. 60
1
48
2
34
3
34
4
41
5
44
6
43
7
55
8
42
9
57
10 40
11 46
12 42
13 49
14 24
15 52
No. 57
1
Pres
Q

ratio
1
47
50
47
54
55
50
48
44
48
51
39
55
0.85
2
3
4
5
1
6
7
8
9
10
11
No.
0.91
0.85
0.98
1.00
0.91
0.87
0.80
0.87
0.93
0.71
0.88
2
0.84
0.60
0.60
0.72
0.77
0.75
0.96
0.74
1.00
0.70
0.81
0.74
0.86
0.42
0.91
0.76
38
2
49
3
48
4
56
5
49
6
32
7
46
8
50
No. 57
1
3
0.67
0.86
0.84
0.98
0.86
0.56
0.81
0.88
0.81
45
2
39
3
55
4
30
5
50
6
39
7
51
8
24
No. 56
1
4
0.80
0.70
0.98
0.54
0.89
0.70
0.91
0.43
0.74
5
Number and ratio of right answers to the test questionnaires for the training lectures
0.55
0.68
0.68
0.77
0.68
0.80
0.68
0.59
0.71
0.61
0.86
0.84
0.86
0.45
0.23
0.61
0.66
6
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
57
2
48
3
53
4
58
5
58
6
56
7
54
8
50
9
33
10 56
11 54
12 55
13 58
14 55
15 55
16 58
17 54
No. 58
0.98
37
2
12
3
42
4
37
5
49
16 6 37
7
40
8
28
9
42
10 39
No. 55
0.67
1
54
2
50
3
43
4
46
5
29
6
45
7
39
8
41
9
50
No. 57
1
6
0.95
9
0.88
51
2
54
3
52
4
51
5
49
11
6
55
7
43
8
55
9
50
No. 55
0.78
1
57
51
3
51
13
4
35
5
16
No. 57
1
47
2
34
3
33
4
40
5
23
15 6 23
7
45
8
41
9
23
10 45
No. 55
1.00
2
0.89
1
0.81
0.79
0.68
0.72
0.88
0.77
48
2
47
3
51
4
48
5
51
6
58
No. 60
1
0.51
7
42
2
50
3
56
4
58
5
57
12 6 57
7
30
8
55
9
38
10 58
No. 59
1
1
47
45
52
41
43
53
57
60
0.75
0.93
0.98
0.95
0.93
0.89
1.00
0.78
1.00
0.91
0.93
0.80
2
0.78
3
0.85
0.80
8
4
5
0.85
6
0.97
7
0.82
No.
0.71
0.85
0.95
0.98
0.97
0.97
0.51
0.93
0.64
0.98
0.85
0.75
0.87
0.68
0.72
0.88
0.95
0.80
0.83
0.91
1.00
1.00
0.97
0.93
0.86
0.57
0.97
0.93
0.95
1.00
0.95
0.95
1.00
0.93
0.92
37
2
37
3
53
4
50
10 5 36
6
49
7
24
8
54
No. 58
0.64
1
54
53
3
36
4
44
17 5 42
6
43
7
24
8
38
No. 55
0.98
2
0.96
1
0.64
0.91
0.86
0.62
0.84
0.41
0.93
0.85
0.89
0.61
0.28
0.74
0.85
0.62
0.60
0.73
0.42
0.42
0.82
0.75
0.42
0.82
0.64
1
0.22
0.76
0.67
0.89
0.67
0.73
0.51
0.76
0.71
0.66
Number and ratio of right answers to the test questions for the training lectures (cont.)
0.65
0.80
0.76
0.78
0.44
0.69
0.76
7
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
The results summarized in the tables above are also displayed in the diagrams below. The red line
marks 75% success in answering.
8
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
9
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
The average presentation-wise solution ratios are given in the figure below:
II. ISSUES TO CONSIDER FOR THE 2ND EDITION
When preparing the second edition of the BIT course, consideration should be given to changes
necessary in the structure and contents of the course in general and in the various presentations in
particular. This chapter summarizes the general and specific issues to be taken into account.
II.1.
GENERAL ISSUES
The questions below need consideration.
Is it necessary to make any change in the
1.
2.
3.
4.
5.
6.
7.
8.
scope/contents of the course?
duration of the course?
organization of the course?
schedule of the course?
general format of the presentations?
mode(s) of presenting?
participation in the course?
other attributes/character of the course?
II.2.
SPECIFIC ISSUES
The individual training and information presentations may need review and revision. In order to
initiate and perform this, the following questions/issues need attention for each presentation:
A.
B.
C.
D.
E.
F.
G.
H.
I.
Correction of errors recognised in the presentation
Any topics missing from the presentation?
Any topics need more detail in the presentation?
Any topics need more explanation (i.e. for poor test questionnaire responses)?
Any details that are unnecessary in the presentation?
Correction of errors recognised in the test questions
Does the test questionnaire need more questions on certain topics?
Does the test questionnaire need correction (i.e. for poor responses)?
Any other specific issues.
10
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
III. CONCEPT OF A REFRESHER COURSE
III.1. GENERAL
Purpose: A second set of lectures is proposed, as a refresher training tool just prior to each IRRS
mission, in conjunction with the initial team meeting in the host country.
Scope: The refresher would cover the most important information, issues and tasks related to
successful conduct of the mission. Basic definitions used in the IRRS process; techniques to be
applied in interviews; data collection and reporting; tools to be used etc.
Format: Presentations would have a similar format to the BIT course, although with an abridged
content. Learning objectives would be listed initially then the main messages highlighted in a
manner that makes everything easy to remember. The last slides of each presentation would
summarize the topic with examples and illustrations of significant issues.
Certain parts of refresher lectures may need to refer to specifics of the current mission. In these
cases either generic contents may be included or space left for the specific content (suitably labelled
to ensure it is completed by the presenter prior to the mission).
III.2. INITIAL MISSION REFRESHER LECTURES
The following lectures are foreseen for a pre-mission refresher course (expected to take about half a
day):
Refresher Lecture 1 – Overview of the IRRS Process
A short, condensed presentation of the main attributes of the IRRS process to bring every team
member to a common understanding. The lecture includes:




Goals and objectives of the IRRS process
Structure, scope, modules, policy discussion
IAEA safety standards as bases for the various modules with particular emphasis on the
latest changes
Scope of the mission, team assignments, groups, cooperation and cross contribution among
groups
Refresher Lecture 2 – Methodology of review
Principles, tools, review techniques. Substantially, this is a summarized version of Training Lecture
No. 5.
Refresher Lecture 3 – Evaluation and Documentation of Observations
A summary of all issues relating to recommendations, suggestions and good practices. The lecture
includes:




Definitions of the various observations
Detailed characterization of the observations
Formulation of observations
Recommendations from experienced reviewers
Refresher Lecture 4 – Report writing
On effective contribution to the writing of the mission report. The lecture includes



Detailed presentation of the Standard Mission Report Template
Techniques and schedule of drafting
Roles, responsibilities and teamwork in producing the report
11
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
Refresher Lecture 5 – IRRS Modules to review
A brief summary of the IRRS Modules in the scope of the mission. One slide for each Module
containing:



IAEA safety standard requirements related to the module
Important subjects, issues to cover in the review
Reporting requirements
Refresher Lecture 6 – Mission Schedule
 Time schedule of the actual mission is presented based on the Standard Mission Schedule.
 Important deadlines are stressed and explained
III.3. FOLLOW-UP MISSION REFRESHING LECTURE
A single lecture is foreseen to summarize the refresher training material for a follow-up mission.
Refreshing Lecture 7 – Follow-up Mission
The specifics of a follow-up mission including:





Objectives, goals and scope specific to the follow-up mission
Evaluation and documentation of observations in the follow-up mission (status of initial
observations, possible new ones)
Report Template for the mission
Team assignments and cooperation during the mission
Time schedule of the mission
12
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
IV. IMPLEMENTATION PLAN
IV.1.
No
1
2
3
IV.2.
No
1
2
3
4
5
6
7
8
IV.3.
No
1
2
3
4
5
6
7
8
IV.4.
No
1
2
3
4
5
6
7
FINALIZATION OF THE CONCEPT
Activity
Commenting the concept by external experts
Commenting the concept by IAEA experts
Preparing the final form of the concept
Deadline
February 21
February 21
February 28
Responsible
RAS
IEC, RAS, RIT
RAS
Deadline
April 11
Responsible
RAS
April 11
IEC, RAS, RIT
April 25
May 16
28-30 May
June 6
June 6
August 31
RAS
Authors
RAS
RAS
RAS
Authors
REVISION OF TRAINING MATERIAL
Activity
Suggestions by external experts on issues 1 through 8 in
II.1 and A. through I. in II.2 taking into account the
feedback in I.3 and the evaluation in I.4
Suggestions by IAEA experts on issues 1 through 8 in II.1
and A. through I. in II.2 taking into account the feedback
in I.3 and the evaluation in I.4
Collecting and summarizing the expert suggestions
Planning revision of lectures and test questions
CS on discussing suggestions and deciding on actions
Action plan for revision of lectures and test questions
Correcting errors identified during the 1st Edition
Developing revised versions of lectures and TQs
ORGANIZATION OF BIT COURSE 2ND EDITION (6-9 OCTOBER 2014)
Activity
Providing location in VIC for the training course
Preparation of prospectus and invitation
Finalization of prospectus and invitation
Dissemination of the prospectus and invitation
Nomination of participants
Replies to nominations
Recruiting lecturers
Logistic support of the training course
Deadline
January 31
March 15
March 31
April 30
July 30
August 30
August 30
October 6
Responsible
RAS
RAS
IEC, RAS, RIT
RAS
Member States
RAS
RAS, RIT
RAS
DEVELOPING THE REFRESHING TRAINING MATERIAL
Activity
Comments by external experts on the planned contents
Comments by IAEA experts on the planned contents
Collecting and summarizing the expert suggestions
Planning development of lectures and test questions
CS on discussing suggestions and deciding on actions
Action Plan for development of lectures
Developing refreshing training lectures
Deadline
April 11
April 11
April 25
May 16
28-30 May
June 6
September 30
Responsible
RAS
IEC, RAS, RIT
RAS
Authors
IEC, RAS, RIT
RAS
Authors
13
Basic IRRS Training (BIT) 2nd Edition and Refresher Course Concept
V. IRRS MODULES