Providing better workforce preparation through experiential
learning classes that promote collaboration and consulting
A. John Bailer
baileraj@MiamiOH.edu
Abstract
Genuine statistical collaboration and consulting experiences in the curriculum provide
service learning opportunities where students can integrate community service into
course content. They are also experiential learning opportunities where students
engage with direct experience to increase knowledge and develop skills. Finally, these
experience can provide a natural capstone to a degree program in statistics. In this
talk, I will describe why these experiences are important, how this was implemented
in the past, what are present and evolving practices and reflect on the value of this
experience for students. I will provide a context for promoting these experiences
based on recent revised guidelines for undergraduate statistics degrees and learning
outcomes for masters degrees in statistics. I will illustrate the implementation of these
ideas with data practicum and data visualization courses at Miami University.
Miami University? Oxford, Ohio?
University of Miami
http://www.contacttopuniversities.com/blog/mainmenu/University%20Articles/top-universities-for-physical-therapy/umEntrance.jpg
Miami University
Miami University
Outline
In my remarks, I will consider the general question of
developing genuine statistical collaboration and
consulting experiences in the curriculum
• Why is this important? Context for remarks
• Past patterns
• Present (& evolving) practice
• Reflection
Why is Important? Context of remarks
ASA Workgroup on Master’s degrees in statistics
(http://magazine.amstat.org/wp-content/uploads/2013an/masterworkgroup.pdf )
Recommendations (based on survey of recent grads and employers):
1.
Solid foundation in statistical theory and methods.
2.
Programming skills critical and should be infused throughout the
graduate student experience.
3. Communication skills critical and should be developed and practiced
throughout graduate programs.
Context of remarks – graduate programs
ASA Workgroup on Master’s degrees in statistics
(continued)
4.
Collaboration, teamwork, and leadership development should be part
of graduate education.
5. Encounter non-routine, real problems throughout their graduate
education.
6. Internships, co-ops or other significant immersive work experiences
should be integrated into graduate education.
Context of remarks - undergraduate
ASA Undergraduate Guidelines Workgroup
http://www.amstat.org/education/pdfs/guidelines2014-11-15.pdf
1. Increased importance of data science
a.
b.
c.
Access and manipulate data
Perform algorithmic problem solving
Higher-level programming & database systems
2. Real applications
a.
b.
Concepts and approaches for working with complex data
Experience designing studies and analyzing non-text book data
Context of remarks - undergraduate
ASA Undergraduate Guidelines Workgroup (ctd.)
3. More diverse models and approaches
a.
b.
c.
Predictive & explanatory models
Model building & assessment
Apply theoretical foundation to sound analysis of data
4. Ability to communicate
a.
b.
c.
Communicate complex methods to managers and other audiences
Understand ethical standards
Opportunities to practice and refine skills during studies
Context of remarks – workforce prep. prof. societies
INGenIOuS (Investing in the Next Generation through Innovative and
Outstanding Strategies) project (AMS,MAA,SIAM,ASA
http://www.maa.org/programs/faculty-and-departments/ingenious )
1.
Bridge gaps between business, industry, and government (BIG)
and academia.
2. Improve students’ preparation for non-academic careers - better
preparation will increase the number of graduates who enter the workforce well
equipped with skills and expertise in mathematics and statistics. Change is needed both in
curricula and in some faculty members’ perceptions of BIG careers for their students.
3. Increase public awareness of the role of mathematics and
statistics in both STEM and non-STEM careers.
Context of remarks
INGenIOuS (continued)
4. Diversify incentives, rewards, and methods of recognition in
academia - A well-balanced mathematical sciences program offering a bachelor’s
degree or above should include faculty with a variety of interests: discovery research
(in pure and applied mathematics and statistics and mathematics education); work in
applied, collaborative, and interdisciplinary areas; and teaching and
preparation for careers both within and outside of academia.
5. Develop alternative curricular pathways.
6. Build and sustain professional communities.
Context of remarks – local concerns
Miami University graduation requirements require:
1. Experiential Learning
http://www.units.miamioh.edu/oir/2020Plan/Experiential_Learning.pdf
process of making meaning from direct experience in a real world or an “out of
the traditional classroom” context
2. Capstone course
http://www.miamioh.edu/liberal-ed/current-students/sr-cap.html
completed near the end of baccalaureate studies, integrates liberal learning with
specialized knowledge. Each Capstone emphasizes sharing of ideas, synthesis, and
critical, informed reflection as significant precursors to action, and each includes
student initiative in defining and investigating problems or projects.
What do employers want?
Quick review of postings on ‘linkedin.com’ [ 17mar2015 ]
Data Mining / Predictive Analytics Consultant in São Paulo
Job description- Experience required:
• Experience in implementations of Business Analytics solutions, focused
on Statistics and Data Mining;
• Previous experience in at least one of these industries: Financial
Services, Insurance, Banking;
• Previous experience with IBM Business Analytics software (SPSS
Statistics, SPSS Modeler, SPSS Analytical Decision Management);
• Previous experience with Data Mining and Statistics software from any
other vendor.
What do employers want?
Skills required:
• Excellent communication, interpersonal, and presentation skills;
• Academic background in Statistics and Data Mining;
• Deep knowledge on implementation and use of Statistics and Data
Mining solutions.
Principal responsibilities:
• As a Consultant within the Professional Services team your primary
responsibility will be to assist IBM’s high profile customers to achieve
successful implementations of the SPSS/IBM Predictive Analytics
Solutions.
Past Patterns for data practicum classes
Initially, common features of early versions of data
practicum classes included:
•
Problems described and motivated by the instructor using artificially
clean preprocessed data
•
Labs were previously analyzed and a particular solution is likely
•
Students alternated presenting by all students submitted reports of
each analysis (often initial + final report)
•
Stat instructor provided all feedback to oral/written reports
Past Patterns for data practicum classes
A company was thought to be polluting a local lake by discharging its manufacturing waste into
the lake without pre-treatment. To investigate whether the lake was polluted, the EPA took five
samples from the lake receiving the discharge (Lake #2) and five samples from a nearby
unpolluted lake (Lake #1). Strontium measurements were recorded for each of the samples.
Data:
Lake #1: 27.2 29.1 33.2 31.4 32.8
Lake #2: 37.4 35.0 41.2 40.6 36.2
Goal:
Determine whether the strontium concentrations are different for the two lakes.
Requirements:
Provide both graphical and numerical summaries as part of your analysis. All reports must be
typed. Line printer plots are NOT acceptable.
Past Patterns for data practicum classes
Strengths of historical structure:
1. Labs could be designed to span a breadth of
statistical methods
2. Class was usually small (3-8) and students had lots
of chances to present.
3. Opportunity to present ideas not formally covered in
other classes.
4. Relatively homogeneous student population
(teams made up of students with similar majors)
Past Patterns for data practicum classes
Weaknesses of historical structure:
1. Problems were already well formulated by a
statistician – no need to translate problem from a
client
2. Data were preprocessed and relatively easy to mold
into an analysis data set
3. Relatively homogeneous student population
Present (& evolving) practice
Current context –
Moved undergraduate data practicum and graduate data
practicum courses to client focused
Added data visualization class with multidisciplinary teams
working on projects
Present (& evolving) practice
STA 475 Data Analysis Practicum (3) MPC
The use of statistical data analysis to solve a variety of projects. Emphasis on integrating a broad spectrum of
statistical methodology, presentation of results both oral and written, use of statistical computing packages to analyze
and display data, and an introduction to the statistical literature. A term project involving student teams combines
elements of all of the above. CAS-QL. Prerequisite: STA 463/563 or 363; or ISA 291.
STA 660 Practicum in Data Analysis (3)
Supervised practice in consulting and statistical data analysis including use of computer programs. Maximum of six
hours may be applied toward a degree in mathematics or statistics. Offered credit/no-credit basis only.
STA 404/504 Advanced Data Visualization (3)
Communicating clearly, efficiently, and in a visually compelling manner using data displays. Identifying appropriate
displays based on various data characteristics/complexity, audiences, and goals. Using software to produce data
displays. Integrating narratives and data displays. Critiquing visualizations based on design principles, statistical
characteristics, and narrative quality. CAS-QL. Prerequisite: at least one of the following: STA 261, 301, 368, 671; IMS
261; ISA 205; or by permission of instructor. Cross-listed with IMS/JRN.
Present (& evolving) practice
Desire:
1. Direct engagement in wrestling with client-defined
tasks
2. writing outcomes
3. group work
4. service learning
Present (& evolving) practice
Challenges and implementation
1. Getting Clients?
2. Projects
3. Reflection
Getting Clients?
Need to actively recruit clients and screen projects
Targeted email …
SUBJ: An invitation to propose projects for data analysis capstone / practicum
class
Greetings,
Have you or your office collected data that you haven't had the chance to analyze? Are
you planning for future studies and would like some assistance determining how many
observations you might need? Do you like working with motivated students? If you can
answer "yes" to any/all of these questions, then I invite you to put my students to work
…
Getting Clients? (continued)
If you have a project(s) where statistical assistance might be valued, then let me know.
Please send me a short description of the project including:
(on the email subject, please use the convention - SUBJECT: STA 475 project: your name project title)
1. Short descriptive project title
2. Goal of the analysis (e.g. design phase - project planning; data analysis, etc.)
3. Data to be analyzed (e.g. Excel data sheets; still to be collected; ...)
4. Type of statistical analysis anticipated (e.g. logistic regression, anova models, etc.)
5. Timeline for analysis (when are results needed)
Getting Clients? Data Visualization
Happy New Year. I hope that you are doing well. We are teaching the data visualization class
(IMS/STA 404-504) for a second time this spring. This course will be populated by
undergraduates and graduate students representing a diverse set of backgrounds including
design, finance, psychology and statistics. Our course will focus on the construction of well
designed data displays that tell accessible stories from data. A major component of this class
is a project that will be conducted for an external client, and we thought you might be
interested in participating as a client.
Do you or your office have data that would benefit from better analysis and visual display?
Do you have a complicated story involving numerical summaries in which visualization might
lead to insight? Do you have data that you haven’t fully investigated but you believe might
contain the nugget of an interesting story? Do you have reports that are overly dense with
tables and basic graphics that would benefit from redesign? If you are interested in help
addressing these issues, we invite you to submit a project idea for consideration.
Getting Clients? Data Visualization
As an external client, you would meet with the class early in the semester (mid-February) to
outline the project and desired outcomes, in the middle of the semester (late-March) to
review interim results and prototypes of visual displays and at the end of the semester (earlyMay) to receive and review the final project. Ideally, a client would be available to meet on
campus, but we could also arrange WebEx/Skype as well. We will be supervising the students
as they work on these projects.
If you have a project that you think might be appropriate, please let us know
(baileraj@MiamiOH.edu; kuhnar@MiamiOH.edu ).
Getting Clients? Data Visualization
Send us a short description of the project, including
1. A short title;
2. Goal of the analysis (e.g. dashboard displaying important data; website with interactive
visualization; a story for possible print/web publication);
a. If possible, provide at least one or two specific questions to be answered by the
analysis
3. Data to be analyzed? Source of data? Format? (e.g. Excel spreadsheets, CSV files);
4. Publications or report that have previously presented the data
5. Type of statistical analysis anticipated
Getting Clients? (continued)
Observations:
Do a good job once and clients will return in future.
Current repeat clients in my data practicum class:
Gerontologists
Exercise Physiologist
Current repeat clients in data visualization class:
Local paper – Cincinnati Business Courier
Research center – Scripps Gerontology Center
Projects
I am the first client (probably the worst they will have)
Project: compare dissolved oxygen-depth relationship between two lakes
Very general guidance on first draft report
Extensive commenting on first draft to be addressed with revision (at least one
revision)
Projects (continued)
Ideas to convey early …
1. Revision and critical reading of reports key skill and learning outcome
2. Better graphical displays lead to easier writing and communicating with
clients
3. Reporting effect estimates often richer than exclusively reporting the results
of hypothesis testing { indicates what students are taking away from our
classes }
4. Writing a structured report is a skill
Projects (continued)
Later projects – teams; charter-traditional schools; CELTUA; OMA; development)
• Project life course
• Client meeting and discussion with class
• Team work
• Interim reporting of analyses
• Draft report produced (class wiki for other teams to comment)
• Client presentation of final analysis
• Final report and presentation sent to client
Projects – Example OPTAB
Task: conduct an occupancy study of the various parking areas in town so that the
Oxford Parking & Transportation Advisory Board (OPTAB) could make data-based
decisions regarding meter rates, hours, fines, etc.
Client: board which represents nearly every constituency is town: City Council,
Miami University, Chamber of Commerce, Oxford Landlords, Mile Square Residents,
Year Round Residents, ASG, Talawanda Schools and McCullough-Hyde. Ex- officio
members include Chief of Police and the City Manager
Projects – Example OPTAB
Issues:
When a block face, i.e., the collection of parking spaces on a street for a city block,
exceeds 85% occupancy it appears full.
If the remaining 15% of block face spaces are vacant, then this would lead to a loss of
earnings (if these are metered spaces).
City parking garage has low occupancy.
Projects – Example OPTAB
Conduct an analysis of a sample of the 790 metered spots in Oxford as well as the
parking garage.
Goal: investigate different rates based on location to spread occupancy from “hot
spots” to outer locations as well as into the unoccupied city garage. In addition,
‘meter feeding’ could be removing spaces from circulation for extended periods of
time.
Question: extending meter hours past the current 6 p.m. deadline which could force
many vehicles to park away from the high and main intersection as well as produce
additional income for the City of Oxford.
Projects – Example OPTAB
Solution:
Developed a data collection plan that was implemented in March of 2010. Four days
(Monday, Thursday, Friday and Saturday) were sampled hourly (11 a.m.-3 p.m.; 4
p.m.-8 p.m.) - restricted attentions to 27 block faces and the parking garage for a
total of 377 parking spaces.
Logistics of the data collection:
order that spaces were checked;
data collection sheets;
decisions about what to record such as license numbers)
appropriate documentation and emergency numbers.
Projects – Example OPTAB
Solution:
Developed a data collection plan that was implemented in March of 2010. Four days
(Monday, Thursday, Friday and Saturday) were sampled hourly (11 a.m.-3 p.m.; 4
p.m.-8 p.m.) - restricted attentions to 27 block faces and the parking garage for a
total of 377 parking spaces.
Logistics of the data collection:
order that spaces were checked;
data collection sheets;
decisions about what to record such as license numbers)
appropriate documentation and emergency numbers.
Projects – Example OPTAB
For each metered spot, occupancy was recorded and whether the meter was in
violation. Summaries of the collected data included:
Occupancy/Availability rate per time, day and block on street parking
Occupancy /Availability rate per time, day in garage
Violation rate per time, day and block on street parking
Violation rate per time, day in garage
Meter feeding: less than 2hrs, occupied over 3hrs including location
Results reported to the OPTAB clients included:
Satellite Map with highlighted lines
Heat map
Projects – Example OPTAB
Example of Heat Map of occupancy for one sampling time
Projects – Example OPTAB
Occupancy of block face over time on different days
Projects – Example OPTAB
Impact
Students presented the results of this data collection and analysis effort to the
OPTAB. The board was very impressed with the depth, quality and insight provided
by this project. The board mentioned that these data would be relevant for setting
parking meter rates that might include differential rates for underutilized spaces.
Finally, the OPTAB members commented that this level of work may have exceeded
the value of a previous report that cost over $20000 to conduct. At this point, the
students were left speechless.
Projects – Example OPTAB
Follow-up
Experience of working with a local government organization and contributing to
information needed to support decision making
Students had to coordinate an extensive data collection effort, process the data into
an analysis data set, construct displays and summaries that gave insight and develop
a presentation and report that captured all of this work.
Comment about the cost of previous parking studies led to interesting class exercise
of developing a cost estimate of how much billable work was reflected in the
analysis, and what they would charge if they did this as a consulting company
Projects – Example Data Visualization
Advanced Data Visualization class (team taught by graphic designer and statistician)
Final Client Project (25% of grade).
Students create a visualization integrated with a brief narrative using data provided
by an outside client to solve a problem or answer a question (“Describe clients
receiving emergency services”, “Display budget of city”, etc.). Students will work
collaboratively with faculty and clients in the development of the solution. Client and
faculty will collaborate on final grade. Students will conduct peer reviews, which will
factor into final grade as well.
http://dataviz.miamioh.edu/UnitedWay/
http://dataviz.miamioh.edu/CBC-Cincy-Budget/
Projects – Example Data Visualization – peer review
404/504 Peer Feedback for Final Client Project
Peer Name (required)
Ratings: (Terrible) 1 … 2 … 3 … 4 (Great)
• Attendance at meetings held outside of class by the team:
• Attendance at meetings held outside of class by the team:
• Amount of contribution outside of team meetings to the team's sprint tasks:
• Quality of contribution outside of team meetings to the team's sprint tasks:
• Effectiveness in working collaboratively (e.g. sharing decision-making with
others, soliciting other's ideas, encouraging, energizing):
{ Client assessment given in Appendix of extra slides }
Projects – Example Data Visualization – peer review
404/504 Peer Feedback for Final Client Project
Peer Name (required)
Open response
• Relative to other team members, how would you describe this person's
contribution?
• What are this person's major strengths? What have they brought to the table
for your team?
• How could this person improve his/her effectiveness as a member of your
team?
1-10 (10 being highest), how likely would you be to recommend this person to
another team? [1=‘Do not work with them’ … 10=‘Work with them!’
Reflection
Assessments:
Learning outcomes (BS STA degree)
1. Students shall be able to analyze and interpret data critically using statistical
models and programming skills { also part of practicum }
2. Students shall demonstrate understanding of the mathematical basis and
theoretical foundations of statistics
3. Students shall be able to effectively communicate, both orally and in written
form, results of statistical analyses to both the expert and layperson { data
practicum class critical data for evaluating this LO }
Reflection (continued)
Reflection - STA 475 Data Practicum Portfolio and Self evaluation
“Each Capstone emphasizes sharing of ideas, synthesis, and critical, informed reflection as significant precursors to
action, and each includes student initiative in defining and investigating problems or projects”
(http://www.units.muohio.edu/led/Capstone). Your final exam is a paper reflecting on the following questions derived
from the reflections associated with service learning courses.
What?
What projects did you work on this semester?
What was your role on project teams? Note that this may change on different projects.
What did you observe?
What did you like/dislike about working on these projects?
What missing knowledge or skills would have made you a more effective contributor to the work on these teams?
Reflection (continued)
So What?
What stuck out about the experience in this capstone?
What was the best/worst thing that happened?
What have you learned about yourself?
How does this experience compare to others you’ve had?
Now What?
What have you learned about working as a statistical collaborator/consultant?
How did this experience challenge you?
Do you believe that this course prepared you for future collaborative work? If so, how? If not, why?
Summary
What grade would you assign to your efforts this semester? Why?
Describe the work or contribution of particular individuals to project teams that you thought was noteworthy.
Reflection (continued)
Reported from students … during interviews with potential employers
Employers expect graduate will have the stat and math background but
look for the experience with client (often messy) problems, where they
worked in teams and had to produce written and oral reports for the
clients.
Experiences such as described above provide this and I encourage you to
develop (as academics) and to support (as potential clients)
Thank you
Rubrics - presentations
Rubric for assessing MS Statistics, Learning Outcome 3: Students shall be able to effectively and independently communicate, both orally and in
written form, the results of statistical analyses. [Applied to the Master’s oral exam and accompanying written report of the Final Project for the MS in
Statistics.]
Category
Written report
Does Not Meet Expectations
Description of project/questions is inaccurate or insufficient. Results of
statistical analyses poorly described. Conclusions inadequately address
questions. Assumptions used in analysis are unstated or invalid.
Organization is awkward. Literature citations are missing or inappropriate.
Graphical and tabular displays are missing or uninformative. Errors in
language mechanics (grammar, spelling) are frequent or serious.
Meets Expectations
Project is introduced accurately. Principal questions, methods and results
of analyses (including assumptions), and conclusions addressing client
objectives are all clearly stated. Report is organized sensibly and includes
appropriate graphical and tabular displays and literature citations.
Occasional errors in language mechanics might be present but aren’t
distracting.
Exceeds Expectations
Meets expectations and…
Writing is fluid and succinct. Description of methods, analyses, results and
recommendations are presented authoritatively and convincingly. Validity
of analysis has been evaluated and confirmed. Graphical and tabular
displays are well-chosen, carefully formatted and correctly referenced.
Errors in language mechanics are rare and minor.
Communicating
results (applies to
both written and
oral
presentations)
Conclusions are lacking or unclear. Descriptions of statistical procedures
are overly technical or missing. Analysis is inadequately connected to the
project’s background and needs. Graphical and tabular displays are
difficult to read and fail to provide insight into the problem or its solution.
Statement of statistical techniques is omitted or described in vague terms.
No connection is given to statistical literature. Terminology is used
incorrectly.
Conclusions are stated clearly and are connected to the project's original
questions. Descriptions of statistical procedures are described at an
appropriate technical level. Analyses connect directly to the project’s
objectives. Appropriate citations to statistical literature are given.
Terminology is used correctly. Graphical and tabular displays emphasize
insight regarding conclusions.
Meets expectations and…
Analyses and descriptions enhance and extend the understanding of the
project and its goals. Provides comprehensive, elegant, and well-sourced
presentation. Terminology is used precisely. Relevant diagnostics are
discussed and interpreted correctly.
Oral present-ation Order of slides is illogical or unintuitive. Many slides are too dense or
incomprehensible. Speaker is difficult to understand, seems unprepared,
or answers questions awkwardly. Font sizes and colors make slides difficult
to read. Talk is overly detailed and too long, or is underdeveloped and too
short. Speaker struggles to respond to questions.
Slides are logically ordered and mostly uncluttered. Each slide either
provides a summary or focuses on a single point of discussion. Font sizes
and colors are appropriate for presentation media and venue.
Presentation is well paced and of appropriate length. Speaker is
understandable, well-prepared, and answers questions comfortably.
Speaker discusses each slide, but does not read from slides.
Meets expectations and…
Slides provide a strong introduction, smooth and logical flow throughout,
and clear statement of conclusions. Presentation is engaging, polished,
and notably enhanced by careful choice of displays and formatting.
Speaker is able to respond to questions aptly and concisely.
Rubrics – presentations to clients
STA/IMS 404 Client Project - Client Evaluation
Evaluator Name:
Client:
Students:
{ all questions rated from “low” to “high” }
• How would you rate the project’s underlying concept (interesting story, display answers an interesting question, etc.)?
• How would you rate the project’s technical level?
• How would you rate the analysis done to execute the project (is the display accurate and appropriate, is there evidence of
deeper analytical thinking and insight)?
• How would you rate the aesthetic qualities of the project (is it aesthetically pleasing, are colors and typography
appropriate and consistent, etc.)?
• How likely would you be to actually implement some of the ideas presented by the students?
• How does the project align with what your expectations were?
• Based on this project, how likely would you be to recommend these students to a colleague faced with a similar problem?
{ open ended }
Final thoughts?