Multichannel Learning System (MLS)
Collaboration Working Group
Ms Janie Glover
Deputy Director, Technical Director
Naval Education & Training Security Assistance Field Activity (NETSAFA)
MLS Project Manager
1
AGENDA – DAY 1
2
AGENDA – DAY 2
3
MLS PROJECT
US STAKEHOLDERS
4
NETSAFA
NAVAL EDUCATION AND TRAINING SECURITY ASSISTANCE FIELD ACTIVITY
NETSAFA MISSION & RELATIONSHIPS
SECNAV
ASN RD&A
NAVY IPO
SECDEF
DSCA
INTERNATIONAL
POLICY
MARITIME
USCG/IA
USMC
MCSCG
ENGAGEMENT
MANAGEMENT
Develop, execute and manage
the Department of Navy’s
Security Assistance and Security
Cooperation training and
education programs that support
the U.S. security strategy in
building partner capacity to build
and strengthen enduring
international partnerships.
NETSAFA
NAVAL EDUCATION AND TRAINING SECURITY ASSISTANCE FIELD ACTIVITY
NETSAFA’S ROLE
Training Providers
Requirements / Priorities
Material Cases
DEFENSE INSTITUTE OF SECURITY
ASSISTANCE MANAGEMENT (DISAM)
Department
Of Defense (DoD)
Defense Security
Cooperation Agency
(DSCA)
Defense Institute
of Security Assistance
Management (DISAM)
Defense Institute
of International
Legal Studies (DIILS)
DISAM MISSION STATEMENT
• The Defense Institute of Security Assistance
Management provides professional education, research,
and support to advance U.S. foreign policy through
Security Assistance and Cooperation.
Tasks
• Resident, non-resident, and online courses
• Seminars and tutorials
• Research, consultation, studies
• Information dissemination
IMS PRE-DEPARTURE BRIEFING
International Military Student Pre-Departure Briefing
(IMSPDB)
• For all International Military Student (IMS) selected to
attend training in U.S.
• Security Cooperation Offices (SCOs) tasked with ensuring
IMS are briefed
• Minimum content spelled out in Joint Security
Cooperation Education and Training regulation (JSCET)
JOINT KNOWLEDGE ONLINE (JKO)
On behalf of the Joint Staff J7, JKO develops and delivers online joint
training to prepare and assist individuals to support joint and coalition
operations.
• Provide 24/7 global access to JKO Portals to deliver cost effective, and efficient
training using state-of-the-art technology.
• Develop and deliver Web-based joint training products, courses, tailored
curricula, small group staff training and immersive mission scenario culture and
language training per the requirements of the Secretary of Defense, the Joint
Chiefs of Staff, the Combatant Commands, and other JKO Stakeholders.
• Track, document and report DoD-wide individual joint training completed in
order to meet DoD/Combatant Command mandatory training requirements, and
Service retirement / promotion or joint credit criteria.
10
JKO TRAINING PLATFORM
Mobile Learning
Tier III – Expand Access
Interactive
Desktop
Simulation
Virtual Cultural Awareness Trainer
Small Group Scenario Trainer
Tier II – Enhance Learning with Advanced Technology
JKO Content Management Architecture
First Level
Webbased
Courses
Learning Content
Management
System
Web-based
Courseware Authoring
JKO
Tool
Portals
JKO
Portals
Tier I – Deliver Global, Distributed Learning Capability
11
Continuous Learning (711HPW/RHAS)
Air Force Research Laboratory
(AFMC)
711 Human Performance Wing
Human Effectiveness (RH)
Warfighter Readiness Research
Division (RHA)
• Develop mission
relevant content for
learning
• Measure training
outcomes in complex
mission environments
• Accelerate
development of
mission critical
competencies
• Optimize training value
of live, virtual, and
constructive
environments
Where We Are Going:
Game Changing Learning S&T
Metrics and scenarios
• Tailored needs assessment/gap analyses
• Cross-domain readiness assessment
• Proficiency tracking and feedback
Proficiency-based training
• Evidence-based fidelity assessments
• Cross domain after-action reviews
• Team and team-of-teams assessment
• Family of complementary trainers
Adaptive, distributed content and assessment
• Scenario authoring, scripting, and metrics definition
standards
• Common data exchange, control, management
• Managed learning/feedback across LVC contexts
Globally distributed learning
• Seamless integration of LVC environments and players
• Adaptive training and operational environments
• Portable, personal learning and occupational records
• Predictive performance models anticipate gaps/refreshers
Approach for Continuous Learning
Warehousing
 What was learned
 Where
 How much
 How often
Adapt Conditions
Based on Performance
& Diagnosis
Readiness/Proficiency Monitoring
Competencies




Knowledge
Skills
Experiences
Gaps …
Adaptation
Diagnosis
Learning
Opportunities
AAR
FEEDBACK
Record
Match Opportunities to
Objectives
Scenarios Instructional Events 
Structure & Triggers 
Strategies & Principles
Support resources
Measure
Data

What/How to Measure
Match Measures to
Objectives & Conditions
Individual/Team/Team of Teams
Objective and Subjective
OTHER US ORGANIZATIONS
• ADL Co-Lab
• Office of Naval Research Global (ONRG)
• Office of Naval Research (ONR) Reserve Component
(RC)
• U.S. Naval Space & Warfare Systems Command Pacific
(SPAWAR Pacific)
15
MLS PROJECT
GOALS, OBJECTIVES & DELIVERABLES
16
MLS VISION
17
MLS GOALS
• Evaluate the effectiveness of using multiple learning
technologies (i.e., web-based e-learning, mobile apps,
etc.) to support learning objectives
• Assess the interoperability between U.S. and foreign
partners learning management systems and their ability to
meet Joint Security Cooperation Education and Training
regulation (JSCET) training, information sharing,
collaboration and coalition interoperability requirements
18
MLS OBJECTIVES
• Evaluate the learning effectiveness of using learning
technologies (web-based e-Learning, mobile applications,
etc.) to support joint, interagency, intergovernmental and
multinational (JIIM) education and training requirements.
• Assess the interoperability of the US and foreign partners
learning management systems to meet US/USG training,
information sharing, and collaboration, and coalition
interoperability requirements
• Ensure that international partners have access to courses
via DoD e-learning systems in accordance with Security
Cooperation requirements
19
MLS DELIVERABLES
• Provide e-learning courses in both the web-based and
mobile app formats
• The infrastructure to permit informed decision on the best
delivery methodology for international partners
• The capability of tracking courseware related content via
partner LMS’ will be available via web services from the
existing Joint Knowledge Online (JKO) coalition training
capability.
• A legally compliant and integrated tracking with partner
Learning Management Systems (LMS) and JKO Mobile
courses/content required to support US and its partners
that comply with Security Cooperation requirements
20
MLS WORKING GROUPS
21
LEARNING CONTENT WORKING GROUP
• Ensure the learning courses are developed according to the
optimum format (e.g., audio course, e-course, apps, etc.) to
meet the learning objectives
• Identify the delivery formats to support just-in-time learning
complements traditional training and education systems as
well as more of a “personal assistance for learning”
approach
22
TECHNOLOGY & INTEROPERABILITY
• Develop content, based on US-customer needs, to support a
multichannel learning approach (e.g., web-based, mobile
applications, etc.) content.
• Demonstrate the capability to incorporated a multichannel,
multinational distributed learning architecture that uses the
Joint Knowledge Online Internet Public (JKO-IP) for both
web-based and mobile learning applications.
• Make recommendations on how the lessons learned may be
used to support future DoD and other USG International
Military Student Predeparture Training Requirements.
23
TESTING & EVALUATION
• Conduct an evaluation of a multichannel learning approach
(e.g., web-based, mobile applications, etc.) to evaluate the
effectiveness of providing a ‘personal assistance to learning
(PAL)’ approach.
• Evaluate the interoperability between US/International
Learning Management Systems to support training and
education needs and requirements.
• Make recommendations on how the lessons learned may be
used to support DoD and other USG International Military
Student Predeparture Training Requirements.
24
MLS RESEARCH PROTOCOL
DISCUSSION
25
HUMAN RESEARCH PROTOCOL GUIDING
PRINCIPALS
• US Directives :
•
•
•
•
•
32 CFR 219
DoD Directive 5400.11
DoD Directive 3216.2
DoD Instruction 3210.7
SECNAVINST 3900.39D
• UK Ministry of Defence (MoD) Ethics JSP 536
• EU Directives:
•
•
•
Ethical Review in Seventh Framework Programme (FP7)
EU Directive 95/46/EC
Guidance for Informed Consent
• Canada Tri-council Policy Ethical Conduct for Research involving
Humans
26
PRIVACY OF INDIVIDUALS
DoD Directive 5400.11 (DoD Privacy Program)
•The privacy of an individual is a personal and fundamental right that
shall be respected and protected
DoD Directive 3216.2 (Protection of Human Subjects in DoDSupported Research)
•…specifies authority for protecting the rights and welfare of human
beings “used as subjects of study” in DoD-supported RDT&E
•…identifies the requirement that US DoD-support research efforts must
also meet the requirements of other countries when the research is
conducted outside the US
27
PROTECTION OF HUMAN SUBJECTS
32 CFR 219 (Sec 219.101):
Unless otherwise required by department or agency heads,
research activities in which the only involvement of human
subjects will be in one or more of the following categories
are exempt from this policy:
(1) Research conducted in established or commonly
accepted educational settings, involving normal educational
practices, such as (i) research on regular and special
educational instructional strategies, or (ii) research on the
effectiveness of or the comparison among instructional
techniques, curricula, or classroom management methods
28
“INFORMED CONSENT IS NOT JUST
ABOUT PATIENTS”
EU Data Protection Requirements:
“From a data protection and privacy issues points of view, all
study participants present in a research project need to be
informed about the planned research use of the collected
data independently of they type of data collected
If a survey is planned within a project, participant need not
only to be informed of how their personal data is planned to
be handled, but also provide appropriate authorization. The
design of the survey must guarantee that only data
specifically required for the purpose of the research project
will be gathered (unless clearly stated otherwise).”
29
PARTICIPANT TOTALS (MoLE PROJECT)
•
•
•
Launched: 268
Started Proof of Concept: 177 (66.4%)
Completed Proof of Concept: 137 (51.12%)
•
•
•
Original sample design: 30 per country
Minimum number per country: 28
Minimum sample design: 234
30
PROJECT INTERACTION
31
ASIA PACIFIC AREA NETWORK (APAN)
https://community.pan.org
32
LINKEDIN COLLABORATION
33
WORKSHOP EXPECTATIONS
34
WORKSHOP EXPECTATIONS
• Ensure each participant understands the ‘new focus’ since
it is no longer a CWP project
• Obtain each organization’s requirements/capabilities in lieu
of the aforementioned change
• Collaboration on how to ‘effectively’ evaluate multichannel
learning (i.e., web based and mobile apps)
• Develop a shared understanding of participant expectations
for the MLS Proof of Concept (PoC)
• Determine if/how many partners will remain with the project
to provide the best delivery methods/channels for the
IMSPDB content
35
PROJECTED MLS SCHEDULE
• May 2013: Requirements & Capabilities Workshop
• Oct/Nov 2013: MLS Kick-off Workshop
• Apr 2014: MLS Proof of Concept (PoC) Workshop
• May/Jun 2014: MLS Proof of Concept
• Aug 2014: MLS Final Presentation of Results
36
QUESTIONS/COMMENTS
37