2015 QLP Workshop
QUANTITATIVE LITERACY
PROGRAM
RAJ BOPPANA
MARY DIXSON
KIM MASSARO
GAIL PIZZOLA
KIMBERLY WARD
Agenda
9:30 – 9:50am
Introduction & Purpose of QLP
Rajendra Boppana
9:50 – 10:10 am
QLCDS – Data Submission Process
Kim Massaro
10:10 – 10:20 am
Q-course Logistics
Kimberly Ward
10:20 – 10:40 am
Incorporating QL into MAT 1043
Jonathan Brucks
10:40 – 11:00 am
Incorporating QL into WRC 1013/1023
Gail Pizzola
11:00 – 11:30 am
Analysis of Q-course Results
Rajendra Boppana
11:30 – 12:45 am
Lunch Break
12:45 – 1:05pm
Incorporating QL into CRJ 3013
Rob Tillyer
1:05 – 1:25 pm
Incorporating QL into KIN 3323
Sakiko Oyama
1:30 – 2:30 pm
Discussion with the Provost
Dr. Frederick
2:30 – 3:00 pm
Wrap Up and Questions
QLP Team
Location: JPL Faculty Center, Assembly Room
QLP Team
Dr. Rajendra Boppana
Kim Massaro
Kimberly Ward
Project Director of QLP
QLP Program Coordinator
QLP Program Coordinator
Dr. Gail Pizzola
QLP Implementation/
Training Coordinator
Dr. Mary Dixson
Robin Schulze
QLP Implementation/ QLP Coordinator & Analyst
Training Coordinator
Prajan Pradhan
QLP Data Specialist
Quantitative Literacy Program (QLP) Program Goals
 Develop quantitative skills in
undergraduate students
 Implement effective teaching pedagogies
and assessments to support the
development of an exemplary
quantitative scholarship program
at the undergraduate level
 Provide the organizational framework
and resources for an institutional
transformation to graduate a
quantitatively informed citizenry
Q-Course
Current
Course
QLP
Data
+ Q. Methods
+ Redesign
Q-Course
• Learn Quantitative
Skills
• Think critically
• Interpret and use data
that naturally exist in
the subject area
• Make informed
decisions
• Makes the course
more engaging
Student Participation in QLP
Students
QLAT
Advising
Q-courses
QLAT
Graduation
 Completion of one or more Q-courses is a
graduation requirement
 Started with core courses, expanded to major-
required upper division courses
Faculty Participation in QLP
 167 faculty members, 150+ TAs/graders
participated in QLP since Fall ‘11
QLP Timeline
 Year 1




First cohort of Q-faculty and students; 10 Q-courses
Program website is created
Every incoming freshman takes the entrance QLAT
Course data is collected
 Year 2




Developed online QLAT entrance exam
Workshop (QLW) is created to address core complete transfer
students
Individual faculty and overall Q-course reports are developed
Surveys of students begin
QLP Timeline (contd.)
 Year 3






QLP maximized its enhancement of core courses
QLP invites upper division courses for redesign
Surveys to faculty, department chairs, and advisors begin
Exit QLAT is administered to compare to the baseline
Develops online version of QLW Workshop for transfer students
QLP awards first Faculty Excellence Award
 Year 4




Data Collection process is streamlined
8 upper division courses are enhanced with QL
Surveys to employers and alumni begin
QLP awards second Faculty Excellence Award
QLP Growth
Year One
(2011-12)
20 faculty
10 Q-Courses
113 Sections
Year Four
(2014-15)
100 faculty
27 Q-Courses
556 Sections
6,845 enrollments
26,599 enrollments
2015-16 Q-course List
ANT 2033
ANT 2043
ARC 4183
ARC 4283
BIO 1233
BIO 1404
COM 3073
CRJ 3013
ECO 2003
ECO 2013
ECO 2023
ENG 2413
ES 2013
27 Q-courses
19 core, 8 upper division
HIS 2123
HIS 2133
KIN 3323
MAT 1043
MDS 4983
PHI 1043
POL 1013
POL 1113
SOC 1013
SOC 3323
SPE 3603
STA 1053
WRC 1013
WRC 1023
Student Coverage: First-time, Full-time Students
Completion of a Q-course by year by freshmen cohorts
Each colored segment in a bar represents one year
Student Coverage: Transfer Students
Completion of a Q-course by transfer student cohorts
Each colored segment in a bar represents one year
Student Coverage: Graduating Students
QLC: Completed at least one Q-course
QLW: Completed a 3-hour workshop instead of a Q-course
QLE: Exempted based on major; Q-course not completed
Student Enrollments in Q-Courses
Fall 2015 enrollments are based on Aug 13 2015 data
Q-Course Performance Analysis (Fall 2014)
 12 out of 19 (63%) of core level Q-Courses showed
significant increase from pre to post-test

Out of those 12, 11 courses reported an average score greater
than 70 on post-test.
 5 out of 8 (63%) of upper-division Q-Courses showed
significant increase from pre to post-test

Out of those 5, 4 courses reported an average score greater
than 70 on post-test.
Q-Course Performance Analysis (Spring 2015)
 13 out of 18 (72%) of core level Q-Courses showed
significant increase from pre to post-test

Out of those 13, 10 courses reported an average score greater
than 70 on post-test.
 6 out of 8 (75%) of upper-division Q-Courses showed
significant increase from pre to post-test

Out of those 6, all 6 courses reported an average score greater
than 70 on post-test.
QLP Data Submission Process
Presenter: Kim Massaro
Pre/Post Test
 Give the Pre-test before any Q-material is taught
 Grade the Express Question
 Bubble in the score for the Express question on the
Parscore for each student
 Take Parscores to Testing Services
 Make sure to include a note: give permission to
upload data to the QLP drive
How to Bubble Express
Large Form: SUBJ Score
Small Form: Exam #
5
5
QLCDS Website
 qlcds.it.utsa.edu (open using Mozilla Firefox)
 Log on with your abc123 and password
 Download your courses’ template
 Course coordinators will create the templates at the beginning
of the semester
 Upload the item level data
Course Coordinators
• Create pre-test template
– Upload Pre-test document with SLO’s and taxonomy
– Upload rubric for the Express Question with Answer Key
– Upload a dummy file to generate SLO’s
• Create Homework template
– Upload Q-assignment with SLO’s, taxonomy, and answers
• Create post-test template
Best Practices
 For Course Coordinators:
 Create a course specific “primer” for faculty teaching

Ex: Materials handbook, Blackboard Learn shell, packet, etc…

Meet with new faculty prior to beginning of semester

Create QLCDS templates during first month of semester

When uploading materials, include SLOs, taxonomies, and
correct answers clearly marked on document/rubric.
Best Practices (contd)
 For New Faculty teaching Q for first time:
 Attend recommended trainings with the QLP team

Meet with course coordinator and other “Q” team members

Contact Testing Services for ParScore training (optional)

Visit QLP program website (http://qlp.utsa.edu) for more
information on the program, workshop materials, video
presentations, tutorials, and technical reports.
Best Practices (contd)
 For All Q-Faculty


Contact the Course Coordinator at beginning of each semester
Bookmark the following websites:






http://qlp.utsa.edu QLP program website
http://qlcds.it.utsa.edu Data Collection Website (very important!)
http://qlp.utsa.edu/faculty (Resources include workshop materials,
technical reports, and data collection tutorials)
https://medialibrary.utsa.edu/Brwose/Category/57 (video presentations
from QLP staff and other Q-faculty)
Make sure TA/Grader attends QLP Training Workshop (8/28) or
schedules one-on-one training
Verify that TA/Grader has all documents, rubrics, and understands
the needs of the course.
Q-Course Logistics
Presenter: Kimberly Ward
Q-Course Logistics
Beginning of the semester
1.
Course Coordinator creates course template for pre-test, assignment,
and post-test on QLCDS website
2.
Faculty register pre/post-test with Parscore at Testing Services
3.
Faculty meet with TA/Grader and provide documents and rubrics for
grading Q materials.
4.
Faculty give Blackboard gradebook access to TA/Grader (optional)
5.
All Q-Faculty submit pre-test data by Wednesday Sept. 9th

Must state “Give permission for Testing Services to send results to QLP”
Q-Course Logistics (contd.)
Middle/End of the semester
1.
Faculty or TA/Grader downloads “Homework” Excel template from
QLCDS website
2.
Faculty or TA/Grader enters student roster information and itemized
student scores in Excel template columns.
3.
Faculty or TA/Grader uploads the completed Excel template and
indicates scoring criterion for each question.
4.
“Q” Homework data is submitted to QLCDS by Friday Dec. 4th
5.
Post-test ParScore forms are dropped off at Testing Services by
Friday Dec. 4th
1.
Unless on Final Exam, then due Tuesday Dec. 15th (Grades Due Deadline)
Q-Course Logistics (contd)
 Should there be problems/missing data after submission,
the QLP will email the faculty member to resolve the issue.

1st Email – Individual Faculty

2nd Email –Individual Faculty and Course Coordinator

3rd Email—Individual Faculty, Course Coordinator, and Department
Chair
Murphy’s Law and QLP
When things go wrong/errors happen:
 TA/Grader  Assigned Faculty
 Faculty  Course Coordinator
 Course Coordinator  QLP Team
Email the QLP team at qlp@utsa.edu
NOT individual team member emails
QLP Data Submission Process & Due Dates
 Pre-test Data (via Parscore)
 Due September 9th (one week after census)
 Homework Data (for Upper-division courses only)
 Due December 4th (study days)
 Post-test Data (via Parscore)
 Due December 4th (study days)
Incorporating Quantitative Literacy
into MAT 1043
Introduction to Mathematics
Presenter: Jonathan Brucks
Incorporating Quantitative Literacy
into WRC 1013/1023
Freshman Composition I and II
Presenter: Gail Pizzola
Analysis of Q-course Results
Presenter: Rajendra Boppana
Break for Lunch
Will resume at 12:45pm
Agenda
12:45 – 1:05pm
Incorporating QL into
CRJ 3013
Rob Tillyer
1:05 – 1:25 pm
Incorporating QL into
KIN 3323
Sakiko Oyama
1:30 – 2:30 pm
Provost’s Discussion
Dr. Frederick
2:30 – 3:00 pm
Wrap Up and Questions
QLP Team
Incorporating Quantitative Literacy
into CRJ 3013
Research Design and Analysis
Presenter: Rob Tillyer
Incorporating Quantitative
Literacy into KIN 3323
Biomechanics
Presenter: Sakiko Oyama
Round Table Discussion
With Dr. John Frederick
UTSA Provost and Vice President for Academic Affairs
Thank you for your participation.