LSC: CCD ANALYST – APPLICATION LEVEL QUESTIONS
INSTRUCTOR: Starting Instructions
Lesson Set-Up
Set up a full virtual learning environment platform that utilizes videoconference and classroom
functionality. (The Google classroom platform provides the required functionality, though you may use
alternative software platforms as needed)
Notify learners by email prior to starting to have Google classroom accessible through their computers.
Have learners bring accessible MLPs from their workplace. This is imperative for the success of the
workshop activity with work-related MLPs.
On the Google classroom classwork section page, the learning objectives and topics to be covered shall
be presented in large lettering and prominent in bullet form as a post:
1.
2.
3.
4.
5.
6.
What is an application-level question?
What triggers their development?
How do learners’ benefit from answering these questions?
Why is this type of question necessary when confirming retention?
Refer to Bloom’s taxonomy of questions and review the six levels of comprehension.
What are the different types of questions can you identify that you have developed as a CCD
analyst?
7. Developing application-level questions (practice exercises)
8. Conclusion
Materials:
1. Infographics on Bloom’s taxonomy.
2. Overview of the cognitive and knowledge dimensions.
3. QSP documentation (retrieved from students’ divisions)
4. Lesson plans derived from QSP (retrieved from students’ divisions)
5. Courseware (e.g. PowerPoint presentations, workbooks, course texts)
6. Visual aid overview for developing application-level questions (step-by-step graphical chart)
7. Visual Aids for demonstrating workshop subject matter (download accessible from references)
8. Laptops with a (civilian) internet connection
9. Headset and microphone
10. Virtual learning software platform (e.g. Google classroom)
11. Microsoft PowerPoint, Word software
12. Question development checklist
The following instructions will assist the pre-instruction and instructional periods.
1. Workshop conducted online
a. Ensure that all students can access the software platform used NLT 48 hours before the
workshop.
b. A shared screen may represent the ‘whiteboard’ for all participants. Students may
provide answers by holding up papers to their cameras if the software fails.
c. Ensure all workshop resources are available and accessible online.
2. Workshop in person (if conducting F2F)
1/6
National Training Development Centre (Pacific)
a. Ensure that all workshop resources are available:
i. Written material – Handouts, study guides, manuals, instructional manipulatives,
models, QSP, lesson plans, courseware (material may be provided electronically
if available or through a hard copy if electronic copies are not available).
ii. Visual – Projected (presentations, slides, videos, multimedia, etc.) & NonProjected (Real Objects – training aids, Cue Cards, Map, Pictures, Illustration,
Cartoon, Chart, Flashcards, Models, Posters, etc.)
iii. Performance Support – Infographics, Process Maps, Checklists, Templates,
Quick Reference/Step-by-step guides, FAQs, PDFs, etc.
3. Timings
a. Timings within this lesson are flexible and may be adjusted depending on the
comprehension level and performance of the learners.
4. Class size
a. This workshop may accommodate class sizes varying from 3 to 30 students. Larger
classes may utilize groups of 2 or 3 students dependent on student proficiency and
comprehension.
References/Resources and Tools
1. Moore, C 2019, How to rewrite a quiz question as scenario-based training, viewed 16 Jun 2020,
<https://blog.cathy-moore.com/2012/05/scenarios-what-are-they-good-for/>.
2. Anderson and Krathwohl; A Taxonomy for Learning, and Assessing: A Revision of Bloom’s
Taxonomy of Educational Objectives, New York: Longman, 2001
<https://www.uky.edu/~rsand1/china2018/texts/Anderson-Krathwohl%20%20A%20taxonomy%20for%20learning%20teaching%20and%20assessing.pdf>
3. Canadian Forces Training Development Centre (CFTDC), AIT Learner Manual, June 2019
4. Clark, Ruth, Accelerating Expertise With Scenario-Based Learning, T+D, 2009
<http://www.clarktraining.com/content/articles/ScenarioBasedLearning.pdf>
5. Application (Transferring) Question Checklist (Appendix A)
6. RCN USQ AB Machinery Maintainer Master Lesson Plan (MLP), PO 011 – Maintain Ancillary
Systems, EO 001.01 – Maintain Gas Turbine Water Wash Systems, 2019
7. Buffa, EOD Explosives Marking Skill Lesson Plan, May 2020
Lesson Objectives
Course Control Document (CCD) analysts will be able to create and develop application-level
confirmation questions.
CCD analysts will be able to perform quality assurance checks for application-level confirmation questions
with job-aids.
Learners will be able to:
● Define what an application-level question is.
● Develop application-level questions from lesson plans.
● Utilize quality control tools for application-level questions.
2/6
National Training Development Centre (Pacific)
Introduction 15 mins
(Ice breaker warm-up exercise)
1. Ice-breaker: “Think, pair, share exercise.”
Ask the students to take a moment to think about what they want to get from this workshop.
Group into pairs, find out what the partners’ expectations are from the session. Afterwards, have
students ask each other what they have and identify others with similar expectations:
a. What do you want to get out of this workshop?
b. How are you going to achieve in getting the most out of this workshop?
c. Write down what their responses, share.
Lesson 130 mins
Note: A “mind is on station” is a whiteboard presented on-screen set up with several headers. Students
rotate through each idea and jot down all of their ideas within a given time frame. The instructor reviews
the concepts provided under each subheading and connects to other purposes not mentioned. These will
help the instructor assess where students are at, further develop ideas, fill in the gaps, and judge the
pace of the lesson. (This may be presented as a simple blank sheet on the screen where students may
annotate and perform additions as the discussion progresses).
1. Mind on Station (Define Application-level question) – 15 mins
a. Mind’s on: What is an application-level question?
b. With the world Application prominently displayed on the screen. Pair students within a
private voice or text chat to discuss what they think the word application means, and write
on the shared class document and post under Application-level.
c. Review answers, clarify descriptions.
d. Look for these anticipated responses (ARs):
i. Application-level questions use a procedure to perform exercises or solve
problems.
ii. Application-level questions using key verbs: apply, execute, carry out, use,
implement, demonstrate.
iii. Application-level questions are formulated with the following characteristics:
1. How would you solve;
2. How would you do;
3. What would you say to; and
4. How would you work a case of?
iv. Application-level questions use activities to demonstrate confirmation of
understanding by using the following actions:
1. Solve a problem;
2. Write a response to a case study; and
3. Perform an experiment (or other activity).
v. Application-level questions focus on ‘doing’ something (or following procedures)
IOT achieve an expected result.
vi. Application-level questions confirmation questions are essential in proving that
one is capable of performing procedures.
2. Mind on Station (Define application-level triggers) – 10 mins
a. Mind’s on: What do you think triggers the need for application-level questions?
b. Write the word Trigger up on the shared page, Pair students to discuss what they think
triggers development, and post under triggers.
c. Review answers, clarify descriptions.
3/6
National Training Development Centre (Pacific)
d. Look for these answers:
i. Processes
ii. The complexity of the problem
iii. Problem-solving
iv. Scenarios
v. Required to perform a function through specified sequences
vi. How to implement actions to solve a problem
vii. Responding to a case study or scenario
viii. Proving you understand what you are doing and demonstrate it
3. Mind on Station (Developing application-level questions) 10 mins
a. Think, Pair, Share Ask: What application-level questions have you developed?
b. Have students post a list of questions on a shared page.
c. Keep the list of questions for use later.
d. Look for these responses:
i. Lesson Plans – confirmation questions
ii. EC evaluation questions
iii. PC evaluation questions
iv. Within checklists when evaluating performance
v. Within scenarios or case study problems
e. Debrief students: Were there any components to formulating application-level questions
previously covered that you found hard to understand?
f. What were you most successful at?
g. What was the most difficult?
4. Creating application questions from a lesson plan (simple lesson plan example) – 25 mins
a. Provide an example of a lesson plan (provided an example of EOD Explosives Marking
Skill Lesson Plan). If other lesson plan content is more suitable for the learners, switch
lesson plan.
b. Refer to the introduction and body stages.
c. Divide class into groups (1 group for each step).
d. Assign each group a task from a lesson plan to examine and post on a blank virtual
classroom page provided by the instructor. Instruct each group to identify the purpose of
the task and the processes associated with completing it.
e. Review each process, instructor to correct where applicable.
i. What is the task instructed?
ii. Are there any processes associated with completing this task?
iii. How did you come up with that order?
iv. What may influence the order/steps/the process? Why?
f. Identify keywords that could apply to your task derived from defining application-level
questions covered earlier.
g. Each group shall identify key terms that enable them to develop application-level
questions.
h. This instructor shall show and provide a detailed image (visual aid or slide) of the
different stages of the EOD tasks used within the lesson plan stages, highlighting key
terms that are used to develop application-level questions.
i. The instructor shall show one slide per action (e.g. stage 1 - conducting a vehicle search,
stage 2 – choosing a suitable marking tool)
j. The instructor shall provide examples of application-level questions attribute that
represents application-level confirmation questions.
5. Application questions from RCN MLP – 25 mins
4/6
National Training Development Centre (Pacific)
a. Emphasize to students the fundamental approach may be applied to more complex MLPs
with unfamiliar subject matter.
b. Provide an example of an RCN MLP (provided example RCN USQ AB Machinery
Maintainer, PO 011, EO 010.01).
c. Identify the performance and standard stipulated within EO 010.01.
d. Divide class into groups (1 group for each process)
e. Have each group examine teaching point 2 from EO 010.01.
f. Review each process, instructor to correct where applicable.
i. What are the teaching points?
ii. Look for predictable responses which comprise of:
1. Useful verbs.
2. Questions that may apply to different circumstances.
3. Instruction for potential activities or products.
iii. Are there any processes associated with completing them?
iv. In what order of steps do you need to follow to complete this task?
v. How did you come up with that order?
vi. What may influence the order/steps/the process? Why?
g. Provide students with time to reflect and consider their answers.
h. If a student is familiar with the subject matter instructed, allow the student to provide
examples and insight as to how they came to his/her group’s answer. Allow the student
the opportunity to demonstrate their insight and thought processes when identifying
actions that may require application-level questions. Provide recognition and
confirmation for their experience and contribution.
i. Identify key terms that would apply to your task derived from defining application-level
questions covered earlier.
j. Each group identifies and organizes the key terms they have defined.
k. Show one slide per teaching point with key action words highlighted. Ask the group what
the action words entail and what they believe would be required to perform it.
l. Provide examples of application-level questions attribute that represent application-level
confirmation questions for the EOs provided to the class.
6. Application questions from work-related MLPs – 25 mins
a. Have learners use one MLP from their workplace IOT develop application-level
questions. If other subject content is more suitable for the learners, switch content
material.
b. Identify the performance and standard stipulated within the MLP.
c. Divide class into virtual breakout groups (1 group for each process) by:
i. Creating further hangouts which run at the same time as the regular room
meeting. These will become the breakout rooms.
ii. Ask students to join their assigned breakout rooms after providing them the invite
link.
iii. Remind students if they get lost, they may return to the main room and start
again.
d. Have each group examine one teaching point from their own MLPs IOT identify potential
application-level questions.
e. Review each process, instructor to correct where applicable.
i. What are the teaching points?
ii. Are there any processes associated with completing them?
iii. How did you come up with that order?
iv. What may influence the order/steps/the process? Why?
f. If a student is familiar with the subject matter instructed, allow the student to provide
examples and insight as to how they came to his/her group’s answer. Allow the student
5/6
National Training Development Centre (Pacific)
the opportunity to demonstrate their insight and thought processes when identifying
actions that may require application-level questions. Provide recognition and
confirmation for their experience and contribution.
g. Identify key terms applicable to your task derived from defining application-level
questions covered earlier.
h. Each group identifies and organizes the key terms they have chosen.
i. Have learners explain their teaching points with key action words highlighted. Ask the
group what the action words entail and what they believe would be required to perform it.
j. Provide students provide examples of application-level questions attribute that represent
application-level confirmation questions for their teaching points within their MLPs.
7. Applying Confirmation Checklist – 20 mins
a. Introduce the application-level question checklist tool to students. (See Annex A)
b. Read checklist instructions to groups.
c. Provide each group approximately 10 minutes to use the checklist to compare to their
previously developed questions.
d. Have each group-swap questions with another IOT identify any questions not matching
the checklist, and how they would revise them using the breakout room process as cited
in Para 6(f).
e. Provide feedback, guidance and correction for groups as needed.
f. Have members devise a plan on how they can implement the lessons learned within this
workshop for continuous improvement.
8. Debriefing Activity – 10 mins
a. What had happened during the activities within the workshop that stuck in your mind?
b. How useful do you feel the activities were in helping you understand and develop
application-level questions (opinions, reflections, instincts)?
c. How closely did this activity reflected your actual tasks while working on the job. What
was your biggest key take-away from the activity? Why was this your ‘biggest’ take
away?
d. Do you think this activity would change how you ensure appropriately worded applicationlevel questions?
Instructors shall provide feedback and encouragement.
9. Confirmation Activity – 3 mins
a. The instructor will ask the class a few rapid-fire questions IOT re-affirm their confidence
in the subject matter.
i. What useful verbs are found in application-level questions? AR verbs as
presented here.
ii. How would you phrase an application-level question? AR sample questions as
provided here.
iii. How can you express an application-level question to an activity or product?
Look for AR similar to examples provided here.
Consolidation & Conclusion 5 mins
1. Review the main points of the application-level question, checklist and practical application.
a. Application-level questions emphasize verbiage on actions that demonstrate the ‘doing’
of a task.
6/6
National Training Development Centre (Pacific)
b. Application-level questions may also have a student demonstrate the application of
knowledge by describing the steps needed to complete a task while using personal
circumstances.
c. The checklist provided is a tool to assist students in reminding them of the criteria needed
IOT develop application-level questions.
2. Discuss the importance of the development of application-level questions which allow a student to
demonstrate their comprehension an application of knowledge within evaluations. Being, if
students are able to demonstrate an ability, they are deemed to be capable of performing in realworld circumstances. If students are trained with theoretical information only, it is no guarantee
that a student will be able to perform to the required standard in real-world situations.
3. Questions or concerns at this point?
_____________________________________________
Post-workshop meeting with instructors and instructional staff.
1.
2.
3.
4.
Review: Consolidate lessons learned from debriefing activity.
Identify: Potential changes for continuous improvement for the workshop.
Consider: Impact of changes.
Apply: Changes agreed upon within meeting (if any).
7/6
National Training Development Centre (Pacific)
Appendix A
Application (Transferring) Question Checklist Tool
Instructions: Compare your application questions.
If an item checks “No”, revise your question to match the checklist criteria.
Points to consider
Use of appropriate verbs
Does the question utilize useful verbiage
such as:
Solve
Show
Use
Illustrate
Construct
Complete
Examine
Classify
Yes
Some
No
If ‘No’
Change question verbs
Applying or transferring knowledge
Does the question encourage students to
apply or transfer knowledge to their own
life or within a different context? (examples
provided)
__________________________
Alternative circumstances:
Describe another instance where…
Alternative possibility:
How could this have happened in…
What factors would you change if…
Knowledge application:
Describe a group characteristics such as…
How would you apply the methods used to
your personal experiences…
Rephrase questions to reflect
the application or transfer of
knowledge
Troubleshooting application:
What do you do if…
Development application:
From info provided, develop a set of
instructions about it…
Open application:
How would this info be useful if you had
(insert circumstance)… why?
1/2
National Training Development Centre (Pacific)
Appendix A
Can the question be answered with a
simple yes/no?
Do not use as an applicationlevel question
Learners ‘doing’ or ‘creating’ something.
Does the question require a student to do
a potential activity or create a product?
(i.e. are you required to apply knowledge
while “doing” something?)
_____________________________
Practical application:
Examine a model to demonstrate how it
will works…
Rephrase question to require the
student to ‘do’ something IOT
demonstrate an application
Determine your current position…
Take notes during your briefing…
Design a COA for ship operations…
Write a briefing note…
2/2
National Training Development Centre (Pacific)