MLS Management Teleconference
14 February 2014
Roll Call
Janie Glover
Dr. Mark Ahles
Douglas Howard
Dwayne Eldridge
Julie Howell
Dr. Kristen Barrera
Tink L’Ecuyer
Jacob Hodges
Subhashini Ganapathy
NETSAFA/Project Manager
DISAM/Deputy Project Manager
Marine Corps Security Cooperation Group (MCSCG)
DISAM/Learning Content & Concept Evaluation
SSC Pacific /Research Protocol
711th HPW/RHAS /Testing & Evaluation
LMITT Management Lead
LMITT Working Group Coordinator/Administrative Lead
Wright State University (WSU)
Opening Comments
Janie Glover
Dr. Mark Ahles
Discussion:
-
-
-
-
-
The Agenda:
o Introduction/Comments – Janie Glover/Mark Ahles
o Research Protocol Process – Julie Howell
o Testing & Evaluation – Kristen Barrera
Janie Glover opened by welcoming everyone and acknowledging Douglas Howard who
was attending his first meeting.
Janie Glover asked everyone if they had reviewed the Plan of Action & Milestones
(POA&M) that was sent out and if there were any changes/updates required. No
comments were noted.
Janie Glover asked everyone if they had accessed the MLS Collaboration Portal since it
has been moved from the APAN to JKO. Several Management Team Members did not
have access to the portal. Jacob Hodges will coordinate their registration and access.
Janie Glover said she would like the Calendar in the Collaboration Portal to reflect key
meetings and events. Jacob Hodges will update the Calendar to reflect project activities,
meetings and other information events.
Dr. Mark Ahles stated that in talking with the International Military Schools, DISAM has
the authority to conduct evaluations at the U.S. Military Schools under the authority of
DoD Directive 5105.65 (Defense Security Cooperation Agency) and the Defense Security
Cooperation Agency (DSCA) Manual 5105.38-M (Security Assistance Management
Manual (SAMM).
Research Protocol
Julie Howell
Discussion:
-
-
-
-
-
Julie Howell discussed the Research Protocol (RP) Process that had been developed by
her and Jacob Hodges. Her discussion was focused on RP activities leading up to the
Security Cooperation Office (SCO) Interim Evaluation.
o During the discussions on the fourth slide, Mark Ahles noted the title Security
Cooperation Offices (SCO) was not an accurate naming of the evaluation since the
leads for obtaining volunteers is the International Military Student Officer (IMSO).
Both Janie Glover and Mark Ahles agreed to change the name from SCO to IMSO.
On the second slide, Julie Howell briefly discussed the US Directives/Instructions
reviews; Letters of Support that were needed to submit the Research Protocol Request,
and Learning Content informational requirements.
o In following-up on the information provided by Mark Ahles regarding the “Authority
to Conduct Evaluations,” Julie Howell mentioned that if DISAM provided a Letter of
Support, it would have to specifically identify the schools which would participate in
the evaluation
On the third slide, Julie Howell briefly discussed the next steps of activities. Specifically,
Testing & Evaluation informational requirements, identifying the IMSO Schools,
obtaining Letters of Support, identifying IMSO volunteers before developing the SSC
Pacific RP Request. Once these steps have been successful, the email addresses of IMSO
volunteers would be needed.
o It was noted there are several processes that need to occur. The first being the
identification of the schools who will participate in the IMSO Evaluation. Second,
contacting the schools to find out their support toward the MLS Evaluation and the
process needed to support the data collection efforts. Third, to determine if the school
uses a 2nd or 3rd party Institutional Review Board (IRB). And last, but not least, the
schedule of when it will be best to conduct the IMSO Evaluation so that it will be
aligned with the MLS Timeline.
On the fourth slide, Julie Howell completed her discussion by briefly mentioning that once
the questions surrounding the aforementioned activities are determined, an email will be
sent to each IMSO volunteer so they can participate. After the evaluation has completed
and the Testing & Evaluation Team has conducted the analysis, the RP Interim Report
will be prepared.
o Several options were discussed with regard to email addresses and the volunteers
(e.g., provide access details to the IMSO, send emails directly to volunteers, etc.).
On the last slide, Julie Howell said this slide was meant to show action items before the
RP Request can be submitted to SSC Pacific.
o Janie Glover said that she’d like to be a broader selection of countries than those
participating in the MLS Concept Evaluation. She also mentioned that we should use
the ‘Poland National Defence University (NDU)’ model, which brings everyone into
o
o
the classroom. They are provided with a briefing on the process, review the Informed
Consent and then take the evaluation.
The action to identify what countries will participate in the IMSO Evaluation will be
identified by LM (i.e., Tink L’Ecuyer) once a data pull has been conducted on
selecting the schoolhouses that will participate.
The decision on who will sign the Letter of Approval will be decided once the
schoolhouses have been identified and we know if they have additional IRB
requirements.
Testing & Evaluation
Kristen Barrera
Discussion:
-
-
-
-
-
Kristen Barrera and Subhashini Ganapathy discussed two documents that were provided
prior to the Management Meeting (i.e., T&E Update Presentation and Data Matrix &
Analysis Plan (DMAP)).
On the second slide, they mentioned the DMAP was approximately 95% complete and
that it was being cleaned-up to identify the data collection types (i.e., subjective vs.
transparent data) as well as a description of each collection type. Once the DMAP has
been finalized, it will be posted on Google Docs for the international partners to review.
On the third slide, they discussed the initial data collection worksheet that has been placed
on Google Docs which seeks inputs from the international partners on evaluation
questions on how to evaluate ‘the best methods for providing distance education’. Inputs
from the Google Docs spreadsheets are due by 7 March 2014.
On the fourth slide, Subhashini Ganapathy outlined the Testing Architecture (i.e., secure
log-in, Informed Consent, Pre-Survey/Demographics Survey, Access Content, and Post
Survey questionnaire) via a SQL Database.
On the last slide, both Kristen Barrera and Subhashini Ganapathy confirmed that the data
collection efforts were consistent with the RP process. Subhashini Ganapathy stated that
the initial email to evaluation volunteers would include a ‘pre-assigned’ content type (i.e.,
video, e-publication, mobile app, or web-based) approach with an additional selection
being available.
o Dwayne Eldridge asked how the log-in would work and would volunteers be
provided a ‘content of choice’ once they had completed the initial format they were
initially assigned? Subhashini Ganapathy said that after volunteers had completed
their initially-assigned format, each volunteer would be asked to select another
format to evaluate.
o As part of the post evaluation questions, each volunteer would be asked their opinion
on the type of contents they evaluated, what was their opinion about the content’s
ability to meet the goals of the content, and would their selection of content type
make a difference based on the length of the content.
o
Dwayne Eldridge mentioned that the e-publication, video and web-based format are
95% done and that he is reviewing them. However, the mobile application has not
been started yet and it will take some time since there are others on his team that will
need to be training on how to develop mobile applications.
Action Items
Action items:
POC:
Deadline:
Upload Management Meeting documents to MLS
Collaboration Portal
Jacob Hodges
Completed
Douglas Howard and Subhashini Ganapathy will be added to
the MLS Collaboration Portal.
Jacob Hodges
25 Feb 2015
Update the calendar in the MLS Collaboration Portal to
reflect key meetings
Jacob Hodges
28 Feb 2014
Provide Jacob Hodges with changes to the MLS Timeline and
POA&M
All
28 Feb 2014
Update the Research Protocol Strategy Documents and
Processes to reflect DoD Directive 5105.65 (Defense Security
Cooperation Agency) and the Defense Security Cooperation
Agency (DSCA) Manual 5105.38-M (Security Assistance
Management Manual (SAMM)
Jacob Hodges
28 Feb 2014
Modify all MLS documents to replace “SCO Interim
Evaluation” to “International Military Student Officer
(IMSO) Evaluation.”
Jacob Hodges
7 Mar 2014
Develop an RP Flowchart Process specifically related to the
details needed in order to conduct an IMSO Evaluation
Jacob Hodges
12 Mar 2014
Conduct a pull of the IMSO Database in order to identify
which schools/countries should participate in the IMSO
Evaluation based on the MLS Timeline schedule (late June –
early July 2014)
Tink
L’Ecuyer
14 Mar 2014
Obtain IMSO Evaluation Points-of –Contact (POC) in order
to collaborate on the RP/Evaluation Requirement
Tink
L’Ecuyer
14 Mar 2014