Metrics that Matter ®
MTM Session
For Novell Partners
Agenda
 Introductions
 Metrics that Matter Overview
 Benefits of MTM
 MTM and Evaluation Walk Through
 Questions/Comments?
13 April 2015
© 2010, KnowledgeAdvisors
2
MTM Overview
Monitor customer satisfaction
Metrics that MatterTM (MTM) is a
web-based learning evaluation
system that allows Novell and
Measure training impact
Novell Partners to costeffectively
Improve performance
13 April 2015
© 2010, KnowledgeAdvisors
3
MTM Benefits
Benefits
Easily implement and
administer a
technology-based
measurement and
benchmarking
solution
13 April 2015
Gain the quantifiable
knowledge required
to improve and
monitor performance
of learning programs
on an ongoing basis
© 2010, KnowledgeAdvisors
Obtain customer
satisfaction and job
impact data
4
MTM Benefits Overview
Partners
Novell
13 April 2015
Monitor customer
satisfaction of Novell
courses
Measure and improve
training programs
Improve the products and services
provided to the customers through
Novell channel
Validate the effectiveness of
learning programs
Enable Novell training centers to
more efficiently demonstrate their
own value
Improve return on learning
investment
© 2010, KnowledgeAdvisors
5
Metrics that Matter ®
Agenda
 MTM Overview
13 April 2015
© 2010, KnowledgeAdvisors
6
MTM Process Overview:
1. Class Setup:
A class is scheduled
in MTM, which
triggers creation of
an evaluation link
2. Evaluation
Distribution:
The post event link is
distributed to
students
4. Reporting:
Data is available in
MTM for reporting
and analysis
3. Data Collection:
Students access the
evaluation link and
submit data
13-Apr-15
© 2009, KnowledgeAdvisors
7
MTM Evaluation Process
 Collect evaluations
 Tag evaluations with class information
(instructor, date, learning methodology)
 Populate reports
Schedule
Classes
13 April 2015
Collect
Evaluations
© 2010, KnowledgeAdvisors
View
Reports
8
MTM Evaluation Process
Choose Novell
Courses
Choose Your
Locations
Choose Novell
Instructors
Schedule
Classes
13 April 2015
Collect
Evaluations
© 2010, KnowledgeAdvisors
View
Reports
9
MTM Evaluation Process
Schedule
Classes
13 April 2015
Collect
Evaluations
© 2010, KnowledgeAdvisors
View
Reports
10
Approaches to Collect Evaluations
Approach
Alias URL
Email
Paper
13 April 2015
Description
Simple URL that
displays available
evaluations
Automatic emails
invite learners to
complete
evaluations
Collect paper
evaluations and
manually enter
Best for
In-class evaluations
(requires computers)
Classes without
computers, high
volume training,
eLearning
Students without email
addresses and no
computers in class
© 2010, KnowledgeAdvisors
11
MTM Evaluation Process
Schedule
Classes
13 April 2015
Collect
Evaluations
© 2010, KnowledgeAdvisors
View
Reports
12
Metrics that Matter ®
Agenda
 Evaluation Overview:
What’s on the survey?
13 April 2015
© 2010, KnowledgeAdvisors
13
5 Leaning Levels
Quality control
(Level 1)
Ensure
effectiveness
(Level 2)
Improve
application to job
(Level 3)
Create Value
(Level 5)
Continually Improve
14
Impact the right
business results
(Level 4)
Novell Evaluation
The Novell evaluation:
 Collect metrics across the 5 Learning Levels
 Satisfaction, Learning Effectiveness, Job
Impact, Business Results, ROI
 Collects demographic information:
 Industry
 Collects responses indicating a desire for
additional training needs
 Collects an “overall satisfaction” metric
13 April 2015
© 2010, KnowledgeAdvisors
15
Communicating the Value of Training
Account Executives and Marketing team should have the
following metrics based on MTM data:
 % New knowledge/skills learned
 % Participants with significant impact on key
results:
 Quality, Satisfaction, Cycle Time, Productivity
 % Impact training on job performance
improvement
16
Associate Certified Courses; Add
Instructors and Locations
Associate
Courses
Agenda
Add
Instructors
Objectives
 Associate Certified Courses
 Add Instructors and Locations
Add
Locations
Schedule
Classes
13 April 2015
Collect
Evaluations
© 2008, KnowledgeAdvisors
View
Reports
17
Associate Certified Courses
 Associates official courses to your account
 Makes ratings visible to Content Providers
 Assigns proper evaluation
Novell
13 April 2015
© 2008, KnowledgeAdvisors
18
Add Locations
 Most organizations
use cities as
location names
 Alias can be used
in a URL to access
evaluation
 Use a single word
(no spaces)
 Explained further
later today
13 April 2015
© 2008, KnowledgeAdvisors
19
Add Instructors
 Creates instructor
that can be
scheduled in
classes
 Creates instructor
user account for
reporting
 Can only see their
own class data
13 April 2015
Uncheck Email to User if
the instructor should not
access MTM
© 2008, KnowledgeAdvisors
20
Add Admin and Reporting Users
MTM Admin
Capabilities
MTM
Reporting
Capabilities
# Available
Admin
Full
Reporting
None
Instructor
None
Full
Full
Limited to
own classes
See contract
See contract
Unlimited
• Select appropriate group on Add User
page
13 April 2015
© 2008, KnowledgeAdvisors
21
Schedule Classes
Choose
Courses
Agenda
Choose
Locations
Objectives
 Add Classes
 Search for Classes
 Edit Classes
Choose
Instructors
Schedule
Classes
13 April 2015
Collect
Evaluations
© 2008, KnowledgeAdvisors
View
Reports
22
Add Classes
 Evaluations must be
attached to a class
 Standard Learning
Methodologies:
 Instructor Led
Training
 Add an instructor
Uncheck if adding
only 1 class
13 April 2015
© 2008, KnowledgeAdvisors
23
Search for Classes
Required
Optional
13 April 2015
•
•
•
Select dates of class
Enter additional information to filter search
Use Begins With / Contains
© 2008, KnowledgeAdvisors
24
Edit Classes
Action
Edit
Survey
Summary
Audit
Copy
Multi-Add
13 April 2015
Description
Change course, instructor, dates,
etc.
Change surveys, survey open dates,
email forms
Obtain URLs for each survey
View record of changes to class
Create a duplicate class
Create multiple duplicate classes on
different dates
© 2008, KnowledgeAdvisors
25
Wrap-Up
Agenda
 Summary
 Q&A
13 April 2015
© 2010, KnowledgeAdvisors
26
Summary
Metrics that Matter Overview
Benefits of MTM
MTM and Evaluation Walk Through
13 April 2015
© 2010, KnowledgeAdvisors
27
Q&A
 Any questions, comments, or
suggestions?
?
13 April 2015
© 2010, KnowledgeAdvisors
28