2012-13
Research & Institutional Effectiveness Worksheet
Planning for Institutional Effectiveness
For 2012-13, all department and unit PIE documentation will be recorded on this form. While this work should be done by each department,
how the department/division decides to organize its efforts is up to each area to decide.
Please remember that all outcomes assessment work for courses, services, programs and general education outcomes must be recorded in
TracDat (http://tracdat.mtsac.edu/tracdat/)
Institutional Planning Framework
Institutional Mission
The campus is unified through its demonstrated connection to the mission. Driven by the California Master Plan for Higher Education, revised
by the President’s Advisory Council, and approved by the Board of Trustees, it informs all planning and assessment.
The mission of Mt. San Antonio College is to welcome all students and to support them in achieving
their personal, educational, and career goals in an environment of academic excellence.
Last Updated: July 30, 2013
Page 1
College Themes and Goals
College themes and goals allow the campus to focus on critical issues. Articulated by the President’s Advisory Council and approved by the
Board of Trustees, they guide institutional planning and assessment processes.
To Advance Academic Excellence and Student Achievement
 Prepare students for success through the development and support of exemplary programs and services. (Goal #2)
 Improve career/vocational training opportunities to help students maintain professional currency and achieve individual goals. (Goal #3)
 Utilize assessment data to guide planning, curriculum design, pedagogy, and/or decision-making at the department/unit and institutional
levels. (Goal #14)
To Support Student Access and Success
 Increase access for students by strengthening recruitment opportunities for full participation in college programs and services. (Goal #7)
 Ensure that basic skills development support services as well as success and progression through basic skills courses are college priorities.
(Goal #10)
 Engage students in activities and programs designed to increase their term-to-term enrollment (i.e. persistence). (Goal #12)
 Ensure that curricular, articulation, and counseling efforts are aligned to maximize students' successful university transfer. (Goal #13)
To Secure Human, Technological, and Financial Resources to Enhance Learning and Student Achievement
 Secure funding that supports exemplary programs and services. (Goal #1)
 Utilize and support appropriate technology to enhance educational programs and services. (Goal #5)
 Provide opportunities for increased diversity and equity for all across campus. (Goal #6)
 Encourage and support participation in professional development to strengthen programs and services. (Goal #8)
 Provide facilities and infrastructure that support exemplary programs and consider the health and safety of the campus community. (Goal #9)
 Utilize existing resources and improve operational processes to maximize efficiency of existing resources and to maintain necessary services
and programs. (Goal #15)
 Ensure appropriate staffing to maintain necessary services and support critical functions to implement the college mission. (Goal #16)
To Foster an Atmosphere of Cooperation and Collaboration
 Improve the quality of its partnerships with business and industry, the community, and other educational institutions. (Goal #4)
 Improve effectiveness and consistency of dialogue between and among departments, committees, teams, and employee groups across the
campus. (Goal #11)
Last Updated: July 30, 2013
Page 2
Contact Person: Barbara McNeice-Stallard (research@mtsac.edu)
Department: Research and Institutional Effectiveness (RIE)
Phone Ext.: 4109
Note: This PIE was done is collaboration with all members of the RIE team.
I.
ACCOMPLISHMENTS
Identify accomplishments from the current academic year and the college goal to which they relate.
College
Goal
11
Previous Year Accomplishment(s)
For the fourth year, RIE provides below a listing of its department-level goals along with the link to the current College and Instruction Team
Goals. Under each department-level goal are the listings of major tasks accomplished. This listing provides a plethora of information regarding
the variety of activities the department accomplished for the college from July 1, 2012 to June 30, 2013.
1. DIALOGUE: RIE will promote and facilitate purposeful college engagement in inclusive, informed, reflective and intentional dialogue about
institutional quality and improvement based on reliable information and provide clear inquiry, meaningful interpretation, and broad
communication. (Linked to College Goal 11 and Instruction Team Goal 1.)
-- Accountability Report for Community Colleges (AB1417/ARCC): worked with President's Advisory Council on reviewing the draft documents
from the Chancellor's Office and creating the self-assessment; presented report to the Board of Trustees.
-- RIE Retreat to provide more team work, collaboration and quality time for reflecting
-- Presented papers on outcomes assessment, predictive analytics, computer programming (SQL), mentoring, and learning communities at the
Association of Institutional Research annual meeting, the RP Group, and the Strengthening Student Success conferences.
-- TracDat/PIE presentations/sessions/emails/website to numerous Mt. SAC employees (including department chairs) and consulted with
colleges across the US.
-- External Surveys: College Board Expense Update, Peterson's Interim External report, Wintergreen Orchard House Survey, AIR, ACT, IOWA
general education. External reports were completed on time and reported to the external groups.
-- Meeting with Basic Skills project managers and discussing the intricacies of their projects insofar as it relates to development of Student
Learning Outcomes and Administrative Unit Objectives.
-- Basic Skills Initiative Year-End update/recommendations 2012-13 and preparation for 2013-14 (see
http://www.mtsac.edu/governance/committees/bscc/annualreport.html)
-- Committee work (OC, BSCC, IEC, Matriculation & Assessment, ASAG): Active participation in encouraging meaningful dialogue about
pertinent issues.
-- RIE Data Requirements in BANNER/Argos (updating SQL knowledge, learning table structures, creating DataBlocks)
-- Gave a presentation at the Perkins Beginning of Year Meeting to familiarize program managers with research support options for improving
their programs.
-- Provided data about the sequence of student course taking patterns in Child Development for departmental discussion about the linkages
between courses.
-Completed an enrollment and success progression study for the arts division and implemented a discussion between IT, research and the Arts
Division. The progression study helped to reduce the amount of transcript studies being done by hand.
Last Updated: July 30, 2013
Page 3
College
Goal
15
Previous Year Accomplishment(s)
-Completed a math study to show the effectiveness of Math 96 and student’s overall success rate (classes at prior and post class level).
--Participated in Arise Program staff retreat, providing the voice of “meeting grant objectives” when planning year’s activities.
2. INSTITUTIONAL EFFECTIVENESS: RIE will promote and facilitate timely, integrated, ongoing and systematic research, planning and
evaluation of the college/s mission, goals and institutional effectiveness based on reliable information and provide clear inquiry, meaningful
interpretation, and broad communication. (Linked to College Goals 15 and Instruction Team Goals 8)
-- 2013 Accreditation Midterm Report and Substantive Change:
* Worked with college on creating template for gathering data for midterm report data
* Discussed Substantive Change issues with Instruction and President. Added to PAC their annual review for Substantive Change
-- Discussions weekly/daily with IT regarding RIE access to Argos/Banner. Training on the systems.
-- Reviewed the IPEDS survey, worked with managers as needed, and locked surveys to allow college to be in compliance
-- Reviewing proposals for research outside the normal day-to-day operations of the college (Institutional Review Board) and working with new
IRB to get policies and procedures in place. Requested CITI online training for all employees, students, and outside researchers and obtained in
on a one-time only basis. Will need to request it as an ongoing budget item. It is critical to have this training as it ensures that we are all aligning
with and using the same rules of standard, safe research methods and procedures. CITI training protects the college, its employees, its
students, and the researcher(s) conducting the study.
-- Finalized Executive Summary of Annual Summary of PIE for IEC and provided support for updating processes and documents
-- Coordinated the Employee Survey by working with the vendor, Administration, PAC, IEC, and employees
-- Worked as an RIE Team to complete PIE
-- Working with a sub-group of PAC on the new strategic objectives and processes for a revised strategic plan.
-- Provided data for grant-funded projects (NSF-STEP, STEM Scholars, TEST-UP STEM) to fulfill grant obligation on annual data submission.
-- Completed ACCJC Annual Report; providing guidance for college on completing proficiency and midterm report
-- Reviewed numerous external research projects through the Research Ethics procedure
-- AAC&U Roadmap – provided consultation to the Mt. SAC Roadmap team (provided draft of questions to committee)
-- Carnegie Statway – attending meetings/workshops, provided large multi-year data extracts, provided input for workshop activities for IR,
helped create and clarify standards for data submission in coordination with their project manager.
-- Completed CCCCO Basic Skills Assessment Survey using pre-developed DataBlock.
-- Provided data and report writing for numerous ad hoc requests to support programs/units with outcome measurements (Honors Program
Transfer Tracking study, AWE Multiple Measure Success Rate study, Disproportionate Impact Studies on AWE, DRP, CCDT)
-- Provided data for federal grant Title III&V Eligibility, a status must be met for these federal grant applications; Title V grant was funded
-- Assisted with survey form preparation ( TPI Student Survey, Graduate Survey)
-- Provided data to support program accreditation (Continuing Education WASC accreditation, Histologic Tech Program, Paralegal Program)
-- Provided 2010 Bridge Cohort focus group report and discussions of the reported results (consultation with VP, counseling faculty, and Dean
of Counseling.
-- Provided data to support inquiry regarding Mt.SAC matriculation & Assessment (DRP Multiple Measures data, AWE 3-YR data, MIS data
related to matriculation) to assist the committee in making evidence based decision making in planning and improving practices.
-- Created and are using an online research request form
-- Provided research support for Perkins’ Funded projects such as (John – please add bullets with your projects listed)
-- Produced two evaluation reports for the EOP&S CTE Counselor to aid in understanding the students using the service and the effect being
had on the students.
-- Produced an evaluation report for the Tech Prep program to aid in understanding the students using the program and subsequent academic
Last Updated: July 30, 2013
Page 4
College
Goal
11
Previous Year Accomplishment(s)
progress of those students.
-- Provided data on the sequence of course taking behavior by students to evaluate the recent redesign of the Interior Design program.
--Worked closely with AANAPISI director to complete year one report to the Department of Education.
-Completed grant data requirements (SPEAKS: The Garrett Lee Smith Memorial Youth Suicide Prevention and Early Intervention Program, and
College Affordability and Transparency Explanation Form (CATEF)
-Coordinated data collection of alumni survey, database tracking of survey completers, recruitment, regular updates on response rates, and data
analysis reports. Student contact data college from 3 different groups. Survey templates created for scannable paper version surveys and
online format. Email Surveys conducted in April and May 2013. US mail surveys conducted in early May 2013. Phone Surveys conducted in
mid June. This will help meet the employment outcomes accreditation requirements.
--Provided High School Outreach program with over 40 progress reports for each of their participating High Schools for their counselors and
administrators. As a result of these reports, program changes were implemented to better serve the needs of our incoming High School
students.
3. MODELS: RIE will model and implement best practices in research to support the development of a campus-wide culture of inquiry. (Linked to
College Goals 11 and Instruction Team Goals 8.)
-- Worked with staff and program managers to further develop predictive modeling for Basic Skills students via the Student Athlete project on
using EEASY (extreme early alert System).
-- Doing walk around meetings to showcase Graduate Survey Results and other projects and encouraging interactions with the data from the
group and using their thoughts as the conclusion/recommendation section – in essence, trying to apply inquiry in group settings to further their
discussions
- Assisted Asian American, Pacific Islander (AANAPISI – now called ARISE) in developing measureable outcomes for this new grant.
-- Described research options to the Perkins managers at the Beginning of Year meeting.
-- Modeled inquiry into the validity of assessment reports (Perkins Core Indicator reports) to Perkins program managers to encourage greater
accuracy in the data used to evaluate programs.
14
4. ASSESSMENT: RIE will support and facilitate the development of planning and assessment processes including student learning outcomes
(SLOs), general education outcomes (GEOs), and administrative unit objectives (AUOs). (Linked to College Goals 14 and Instruction Team
Goals 8)
-- Worked with the Outcomes Coordinator and Outcomes Committee to help support and direct their work based on current college needs (e.g.,
internal and external – accreditation) (add to manager rpt)
-- Provided updates to the Instruction Team and asked for assistance as needed
-- Conducted Assessment Pulse open dialogue on SLOs/GEOs and faculty engagement
-- Showcased the winners of the President’s awards on outcomes at Flex Day
-- Submitted work for publication in Community College Review on the analyses of a multi-college mentoring program
-- Continued improvement of outcomes web page
-- Used TracDat SQL Server thereby allowing the design, development, and implementation of TracDat reports targeting deans and chairs; the
reports provide clarity on the progression toward Competency as defined by the ACCJC SLO Rubric.
-- Attended department and Instruction Team meetings to provide information and answer questions
-- Implemented the National Outcomes Assessment Project to research the exemplary methods used by selected colleges to reach and maintain
sustainable outcomes assessment. A summary report and resource binder were produced. The report and binder have served as resources to
Last Updated: July 30, 2013
Page 5
College
Goal
Previous Year Accomplishment(s)
generate new ideas on outcomes assessment with regard to transforming from proficiency to sustainability status.
-- Produced a summary of the best practices proposed by leading experts on the assessment of general education programs. The summary is a
resource to increase knowledge of the issue, help produce new ideas, and plan for the future.
-Established highly successful department chair training efforts.
-Lead campus-wide conversations about assessment that included the VPI and President
-Hosts roundtable discussions for areas to discuss and use results to improve the program (GEO Area E).
Started process for qualitative review of course outcomes and implemented an automated process to pull the most up to date information into
the new SLO Checklist.
Hands on workshop hosted in a computer lab to answer questions related to the Planning for Institutional Effectiveness process (PIE).
11
5. DATA INTEGRITY: RIE will collect/maintain/provide useful, objective, reliable and valid data in an understandable format to college
stakeholders with extensive measures in data integrity. (Linked to College Goals 11 and Instruction Team Goals 8)
-- Worked with CTE Deans on Alumni Survey, current student survey, and current graduating student survey to assess their employment
-- Scanned and entered responses to the Alumni Survey. Contacted alumni to increase the survey response rate as the Alumni Survey included
the documentation of job, information crucial for accreditation purposes and funding.
-- RIE-IT have bi-weekly meetings on BANNER/Argos/Research DW to discuss data issues and resolve them
-- Working with Instruction office and deans on general education coding issues in Banner versus catalogue
-- Worked with the Educational Paraprofessional program to validate program assessment reports (“Core Indicator” reports)
-- Developed materials to aid the Perkins program managers in assessing the validity of the coding of their awards and courses resulting in
greater accuracy of data used to evaluate programs.
-Implemented some documentation for SQL data blocks 2012-13 basic skills projects. This will help future researchers re-run the similar
outcome data and produce similar results.
-Provide assistance and updated outcome data to all divisions in instruction meeting and the website.
Provide assistance to divisions to understand how to develop degree outcomes (for example Humanities and Social Sciences).
--By developing a staff-user database for the ANNAPISI grant, the accuracy and efficiency of generating data for the grant has improved. Upon
networking with other ANNAPISI institutions, it was discovered that many of them do not know who their students are. Due to the data tracking
methods implemented by our department, the grant director can quickly and easily contact, count, etc. her students.
-- Review ARGOS reports, investigation on ‘unknown’ ethnicity data, investigation on non-credit enrollment data. With these efforts, data
integrity is being enhanced and practices streamlined for efficiency and consistency.
11
6. ROLE: -- RIE will strengthen its role on campus through research, evaluation, and planning services to a variety of campus entities. (Linked to
College Goals 11 and Instruction Team Goals 8)
-- Provided Data for ANNAPISI (ARISE) grant
-- provided direction and research assistance for the outcomes assessment processes
-- provided direction and assistance for accreditation aspects including the 2013 Midterm Report and the Substantive Change proposal needed
for fall 2013
-- Coordinated with a variety of entities within and outside Mt. SAC to verify the documentation of Research Ethics research projects conducted
on campus. Created a database of innovative research completed and currently underway on campus. This information is crucial for the
Last Updated: July 30, 2013
Page 6
College
Goal
Previous Year Accomplishment(s)
protection of students, staff, and faculty who serve as potential human subjects. Also, the database can help raise awareness of completed
research on campus that can provide learning and growing opportunities.
--Became member of Arise Program (AANAPISI Grant) Operational Advisory Committee, the purpose of this committee is to help navigate the
direction of the two million dollar AANAPISI grant. Among some of its duties this committee is charged with approving requests for funding (RFP)
for projects. The researcher on this committee serves to focus and evaluate the direction of the grant from the objectives stated within the grant
proposal.
8
7. PROFESSIONAL DEVELOPMENT: RIE will participate as learners in a continuing education process. (Linked to College Goal 8 and
Instruction Team Goals 4)
-- RP Group activities:
* Provided input and leadership for the Leading from the Middle initiative and Strengthening Student Success conference
*Participated as a member of the Leading from the Middle initiative (2 off campus meetings, webinar, coaching, and meeting preps) team.
*Received Basic Skills Funding for travel to the Strengthening Student Success Conference in October 2013.
Received funding to attend the Association of Institutional Research annual meeting in long beach, ca.
Attended the California Association of Institutional Research annual meeting in Anaheim, CA and learned about institutional research activities
across the segments of California higher education.
-- Professional Development Opportunities
* Lead the coordination and operation of CAMP Research regional meetings with institutional researchers from other colleges and used the
CAMP Listserv to actively engage researchers in dialogue, regarding updates, etc.
* Edited numerous papers for the national JARCC publication
* Reviewed papers for and presented at the Association for Institutional Research (AIR) conference on predictive analytics
- Participated in discussion and presented research topics at CAMP Research meetings
-Participated in Access Curriculum Classes to further development of database topics and improve knowledge for campus’ research interests.
Self-learning about SQL programming language in order to replicate past basic skills funding reports.
-Participated in the APAHE (Asian Pacific Americans in Higher Education) conference, resulting in greater understanding of the issues facing
this population. This conference also provided the opportunity to develop relationships with other institutions and organizations which have lead
to offers of collaborative nationwide research opportunities.
- Provided cross-training for new and part-time researchers to familiarize them with DW system and structure
- Attended RIE retreat on office review, goals, plan, and workloads to discuss RIE functions and productivity
- Provided data coaching, consultation to A&R and Assessment Office staff to enhance their local datasets and practices
- Attended Student Success Conference Part II to learn from research studies, students’ voices, and the impact of state movement
- Participated in webinars on various mandatory data requirements to update knowledge about current issues
2
8. INNOVATION: RIE will continually seek innovative solutions to meet the research needs of the college, as well as improve office efficiency.
(Linked to College Goal 2 and Instruction Team Goal 8.)
--Implemented new on-line research request form for staff, faculty and managers to utilize for submitting research requests to our office. Upon
submitting the form, the request is immediately sent to the director’s inbox, which allows her to quickly forward requests to her staff or ask
questions of the requester. This request is also housed in a database for future reference.
Last Updated: July 30, 2013
Page 7
College
Goal
16
Previous Year Accomplishment(s)
--Developed a database which allows the two million dollar AANAPISI grant to track and monitor students and objectives in order to efficiently
report the information back to the Department of Education.
--Implemented a report system to allow for easy tracking of students through a sequence of courses to allow for evaluating groups of courses.
9. RESOURCES: RIE will maintain adequate resources to ensure ongoing operational needs and achievement of goals. (Linked to College
Goals 16 and Instruction Team Goals 3)
-- Budget: Acquired some money to help with outcomes assessment for professional experts to offset D. Lamoree being re-assigned to IT
-- Hired A. Medina as the new Educational Research Assessment Analyst
II.
INTERNAL AND EXTERNAL CONDITIONS
Consideration of internal and external conditions is the basis of department planning and assessment processes.
Internal Conditions - Identify internal conditions that have influenced the department goal-setting process. Please note those with the most significant
influence. Consider providing, at the beginning of the condition, a one- or two-word theme that relates to this condition.
(Internal conditions can include results of previous SLOs/AUOs assessment, IT data, changes in technology, changes in budget, staffing, resources,
enrollment issues, facilities issues, etc.)
-
Increasing participation in SLOs/GEOs on campus
Increasing emphasis on planning and evaluation and its more solid link to resource requests
Institutional surveys
Changing technological demands requiring changes to staff skill set (Banner/Argos and RDW)
First Year Experience Project, a campus-wide new initiative (Sociology)
Part-Time Permanent Classified rules that prevented an efficient workload distribution during full-time RIE staff reassignment.
full-time RIE staff reassignment impact on workload and training issues for other researchers
External Conditions – Identify external conditions that have influenced the department goal-setting process. Please note those with the most significant
influence. Consider providing, at the beginning of the condition, a one- or two-word theme that relates to this condition.
(External conditions can include disciplinary or regulatory changes, changes in technology or legislative changes, accreditation recommendations,
enrollment issues, advisory committee input, etc. Some thematic areas might be regulatory, technology, legislative, accreditation, or advisory committee.)
Last Updated: July 30, 2013
Page 8
- California Budget is a challenge because of its fluidity
- Accreditation changes/mandates (SLO proficiency; midterm report; annual report) – to maintain high standards and to adhere to all
accreditation requirements more staff is needed (professional expert, faculty coordinator, secretarial support). Faculty coordinator position is
going through the faculty contract process. Professional expert is being obtained by the CE department, but may change when we decide
not to do a joint accreditation. Secretarial support is being addressed by VPI for her department, but some support for RIE is also
warranted.
- Institutionalization of outcomes and assessment as required by accreditation
- Grant requirements for research support for writing research grants and for conducting research for funded grants
- Institutional directives based on external requirements (e.g., President Obama’s directive to improve educational outcomes)
- State/Federal mandates especially with the Student Success Scorecard and Student Success Act of 2012 (Senate Bill 1456, Lowenthal)
Critical Decisions - Describe any critical decisions your department made this year and the reasons for those decisions.
1. Increase Research support for outcomes assessment to align with SLO Proficiency: The hiring of another Educational Research Assessment
Analyst allowed the department to provide more support to outcomes assessment. Working collaboratively with the Outcomes Coordinator and
Committee, the RIE department provided more guided support for general education outcomes (GEOs). Using the work of a professional expert,
the department also provided a National view of the best practices of colleges in institutionalizing outcomes assessment. All information is being
used to re-think and add more structure to Mt. SAC’s outcomes assessment process to achieve sustainability.
2. Additional support and meetings for student services. Had initial meeting with the VP of Student Services and managers to receive input
on how to improve the process for student services outcomes. Had expanded meeting with other support services (Library, tutoring)
to discuss how the outcomes process could be made more inclusive.
3. Maximizing Technology: Several critical decisions were made to improve RIE’s use of technology.
a. Argos/RDW: RIE continued to align itself with the IT department in order to gain the necessary training on these systems. Day-to-day
access to one IT expert continues along with periodic meetings every few weeks. RIE strategically prioritizes projects and aligns those with
the training needs. A cross-training of team members continues with members working collaboratively on larger projects.
b. TracDat: The team maximized its use of information in Banner/Argos to linked to that in ePIE (TracDat) so that the two are more closely
aligned. RIE also created better aligned reports that could be used more easily by the Deans and Chairs.
c. DegreeWorks. RIE continues to work with IT to provide feedback on the degreeworks reports. Researchers attended training sessions.
d. Omni Update. RIE members attended training sessions and began additional use of Omni Update to manage website.
e. Data Element Dictionary: The RIE Team has tried many times to push into its work the further refinement of the data element
dictionary for the data warehouse. There continues, however, to be more pressing issues to the extent that each team member
has become both an expert on certain data elements as well as a generalist on others. The team will work more strategically on
creating working sessions to allow for these experts to create the data dictionary while also giving their knowledge to novices.
Last Updated: July 30, 2013
Page 9
Internal Conditions - Identify internal conditions that have influenced the department goal-setting process. Please note those with the most significant
influence. Consider providing, at the beginning of the condition, a one- or two-word theme that relates to this condition.
(Internal conditions can include results of previous SLOs/AUOs assessment, IT data, changes in technology, changes in budget, staffing, resources,
enrollment issues, facilities issues, etc.)
f. SARS: The RIE team now has better access to SARS data (i.e., counseling appointments) which allows the team to be able to
better answer related counseling questions on a cohort basis.
III.
INFORMATION ANALYSIS
Departments should bolster their planning efforts with information, conduct appropriate analyses, and make supportable conclusions.
Report the trends you are seeing in your department, what information you used to determine those trends, and the impact of the
trends on your department.
Trends (e.g., In XXX Department, course success Information source(s) used (e.g., Success and
increased by 2.4% and retention rates slipped by Retention Rates of Sp. 2011-Sp. 2012.)
3.5%.)
Impact (e.g., The Department faculty retreat will
address issue of increased “W” grades.)
Data Integrity
As MIS data is used more for external accountability
and Report/Score Cards, the need for accurate data
is not only more pronounced, but also more evident
if inaccurate data is used because the public sees it
and the College is graded on it.
The College will not be represented correctly to its
constituency groups and to the Federal and State
Governments which could impact its budget and
reputation. Thus, the College should have a body
that manages the integrity of its MIS data. (see
Team Goals, IN-11 Progress/Challenges) (add to
manager’s report)
Data Inquiry
We have gone from a campus of employees saying
“we have no data” to a campus of employees
requesting data to be part of the inquiry process.
The number of and depth of research requests has
increased exponentially to the point where we have
more complex research studies (e.g., multi-year
cohort tracking projects that require the researcher
to cross between 3+ different types of data systems
(e.g., ICCISDW, Banner/Argos, RDW) to go
backward in time by 10 years).
Employees are tending to use data/information
more for decision-making and/or to validate
already made decisions. Employees still need
guidance on how to re-think their work. Some are
focused on the resource that they need and they
make that resource central to their issue. What
should be central to their issue, however, is what
they are trying to achieve (e.g., improved student
success) and how the resource (e.g., more fulltime faculty) will allow them to achieve their
outcome.
Last Updated: July 30, 2013
Page 10
More Federal and State Mandates are becoming
part of the RIE office’s duties to oversee or
coordinate (e.g., IPEDS; MIS coding).
Data Mandates
IV.
RIE is required to put more of its time into these
mandates while its budget shrinks and its
personnel are being re-arranged (e.g., movement
of an employee to IT). The department is trying to
balance the importance of these mandates with
the importance of having valid data with its
mission which is to conduct research to improve
the College’s programs and services. Some nonmandated projects will not get completed due to
this competing interest.
OUTCOMES ASSESSMENT (SLO / PLO / GEO / AUO)
The outcomes assessment process is a formal process that seeks to identify how well our students are learning and to use that data to
improve curricula, teaching, services, and student achievement.
o Student Learning Outcomes (SLOs) are a means to determine what students know, think, feel or do as a result of a given
learning experience.
o Program-Level Outcomes (PLOs) are a means to determine what students know, think, feel or do as a result of progress
towards a degree or certificate.
o General Education Outcomes (GEOs) are statements that define the knowledge, skills, and perspectives acquired by
students who satisfy our general education requirements.
o Administrative Unit Objective (AUOs) are statements that concern the fundamental functions of an administrative unit and the
resulting services provided to clients.)
The RIE Department supports the role of outcomes assessment on campus with two full-time classified employees devoted to it (D. Lamoree
(now in IT until Dec 15, 2013) and A. Medina) along with the director (B. McNeice-Stallard) who helps to oversee the Outcomes Coordinator
faculty member (J. Chevalier) and works collaboratively with the Outcomes Committee. In 2012-13, RIE also had one external consultant (L.
Umdenstock) provide national-level perspectives on outcomes assessment.
Along with outcomes assessment work, the RIE Department had some strategic actions that it worked on this past year. Progress toward the
outcomes desired are listed below.
Dept/Unit goal # Strategic Actions (SAs)
8: Innovation
Done: The team created an On-line research request form for the web such that a request comes to the director
6: Role
Ongoing: The team will create an internal only database of research reports such that all teams can access it and can
Last Updated: July 30, 2013
Page 11
Dept/Unit goal # Strategic Actions (SAs)
easily search it and final ALL research reports so that any thematic review of areas of research could be accomplished.
6: Role
Not Yet Started: There will be an on-line listing of research projects (based on database without the full reports) so
that the college can have a better idea of the breadth of research done by the team. (Status: awaiting database
creation)
6: Role
Done: Updated the RIE’s home page to better articulate the breadth and depth of research and evaluation work done
by the team and its immense impact on the College. http://www.mtsac.edu/administration/research/index.html
6: Role
Done: Produced the National Outcomes Assessment Project summary report and binder after researching
information from across the nation on the exemplary methods used by colleges to reach and maintain sustainable
outcomes assessment. The report has allowed the REI staff, the Outcomes Committee, and Outcomes Coordinator to
get new ideas on transforming from proficiency to sustainability status. The binder, a resource made available to Mt.
SAC’s faculty and staff, has allowed the Outcomes Coordinator and REI staff to easily access information on what and
how other institutions are doing with regard to outcomes assessment.
6: Role
Done: Produced a summary of the best practices proposed by leading experts with regard to the assessment of
general education programs. The summary was used by REI staff and the Outcomes Coordinator to get new ideas
and for planning purposes. The summary will be shared with faculty responsible for assessing GEOs.
6: Role
In Progress: Provide support to department chairs, faculty, and administrators on the learning outcomes process
(methods and data collection). Additional support is provided to input learning outcome data on electronic
repository (tracdat). Development of useful tools (for example surveys, rubrics, samples from other colleges).
Done: Provide an on-line method to measure SLOs for the several Writing Center Workshops as well as AmLa’s Verb
Boot Camp.
6: Role
V.
RESOURCES NEEDED TO ACHIEVE GOALS
Resources requested should support the achievement of one or more college, team, or department goals. Resource type may be in the
following categories:




Rate Driven Increase
Instructional Equipment
Staffing
Facilities Modification
Last Updated: July 30, 2013




Technology: Equipment, Software, or Support
Professional Development Training
Research Support
Marketing
Page 12
Note: Any department with an “immediate need” for resources resulting from shortfalls in funding that, unless filled
immediately, could cause the program to cease to function should request needed funding using the Immediate Need
Request Form following the process identified in the college’s Budget Review and Development Process.
To justify the resource request, please provide supporting information under “Justification of Need,” such as relationship to college
and/or department goal(s), outcomes assessment data, or advisory committee input. Please organize requests by resource type and
prioritize the requests within each category.
Staffing
Priority #
Resource
(Be specific, but not dollar amount needed)
Justification of need
1
Need permanent part-time researchers attached to
grants.
Accreditation is focusing more on the outcomes assessment piece. As Yes
such, it is RIE’s desire to push assessment into many different aspects
of the college’s operations including the research and evaluation of
grants as well as the normal assessment tasks required of the College.
The impact of obtaining these resources would be improved
efficiencies and increased, dramatically, productivity.
No
1
Faculty accreditation coordinator for accreditation
Yes
To maintain high standards and to adhere to all accreditation
requirements more staff is needed (professional expert, faculty
coordinator, secretarial support). Faculty coordinator position is going
through the faculty contract process. This position would bridge the
gap between academics, student support services, administration, and
accreditation.
No
1
Professional expert for accreditation
To maintain high standards and to adhere to all accreditation
requirements more staff is needed (professional expert, faculty
coordinator, secretarial support. Professional expert is being obtained
by the CE department, but may change when we decide not to do a
joint accreditation. This position would bridge the gap between credit
and noncredit in accreditation matters and provide high-level support
for the process.
Yes
No
1
Secretarial support for accreditation
Yes
To maintain high standards and to adhere to all accreditation
requirements more staff is needed (professional expert, faculty
coordinator, secretarial support). Secretarial support is being
addressed by VPI for her department, but part-time support for RIE is
also warranted. RIE does not have any secretarial support. The impact
No
Last Updated: July 30, 2013
New request
Page 13
Staffing
Priority #
Resource
(Be specific, but not dollar amount needed)
Justification of need
New request
of this work would be improved efficiencies, more engaging materials,
and wider readability of materials.
2
Permanent part-time Classified Employee temporary
increase in hours for intensive projects
Each year, the college provides in-depth data reports to its district high Yes
schools. The High School Outreach Annual Reports require intensive
data work from the RIE office; this work is beyond that regularly
available from the full-time employees who are working on other
mandatory data projects. In 2012-13, we temporarily increased the
hours of one of the permanent part-timers to provide support for this
project (per the VPSS request); however the increase in hours
becomes an issue for the college’s benefits budget and needs to be
watched carefully. It is unclear as to how this process can be done to
leverage the expert skills of a permanent part-timer for a short period
of time in order to improve efficiencies without creating a long-term
financial issue.
1
Overtime for full-time Classified Employees and
Professional Expert Hours (already achieved for
August-Dec, 2013)
With the reassignment of Daniel Lamoree to IT for the decision support
system, there was a need for replacement staffing and overtime to
keep his work level at a minimum to support the needs of the college
especially as it relates to basic skills and learning outcomes.
No
Technology: Equipment, Software, Support
Priority #
Resource
(Be specific, but not dollar amount needed)
Justification of need
1
Institutional Review Board training for whole campus
including employees and students as well as off
campus researchers who wish to conduct their
research at Mt. SAC
CITI online training is required so that all research ethics trained
Yes
persons understand the correct methods in conducting research at Mt.
SAC and understand the possible issues in dealing with vulnerable
populations and how to mitigate the possibility of an adverse event
happening and what to do when it does happen. Training is for all
employees, students, and outside researchers. In 2012-13, one-time
funding was received from the VP of Instruction’s discretionary money.
RIE needs to request CITI funding as an ongoing budget line item. It is
critical to have this training as it ensures that we are all aligning with
and using the same rules of standard, safe research methods and
procedures. CITI training protects the college, its employees, its
students, and the researchers conducting the study. It is a necessary
training expense such that without it all research should be
discontinued. Annual, unlimited use cost is $2,500
Last Updated: July 30, 2013
New request
No
Page 14
1
IT and RIE need periodic trainings and updates on
Banner, Argos, and RDW. Without this training, the
RIE team’s job would be very difficult.
The College relies on its data from its transactional system to provide
its overall view of its effectiveness. Thus, the RIE team must
understand it and be able to access it accurately to help the College
with its inquiry process.
Yes
No
3
Software licensing and updates. At some point in the
future, the licenses for some software will expire. We
will make do with it for now, but in the long term we
will need the updates and there are sometimes
unforeseen software needs.
In order to continue with the predictive analytics projects and to
continue to service the needs of the programs and services, the RIE
team will need some monies in the next few years for software
licensing and updates.
Yes
No
New request
Professional Development Training
Priority #
Resource
(Be specific, but not dollar amount needed)
Justification of need
1
SQL training (from IT)
Improved cohort based data collection and report writing (data blocks). Yes
Overall, to increase efficiencies.
No
3
Association of Institutional Research Conference,
Outcomes Conferences (Assessment Conference at
Cal State Conference, Assessment Institute Indiana
University, Strengthening Student Success
Conference by RP Group) (Conference and Travel)
(Conference and Travel)
In order to keep moving ahead with outcomes assessment it is
important to continue momentum on ideas that are working well, to
improve ideas that need improving, and to gain new insights on how
others are dealing with these issues across the nation. These
conferences would allow RIE staff (and Outcomes Coordinator) indepth information about current research and assessment efforts
across the nation in higher education and how they link to
accreditation and sustainability of our outcomes efforts.
Yes
No
2
ACCJC Site Reviewer (ACCJC)
Additional information for accreditation procedures and site visit.
Yes
No
VI.
FUTURE PLANS
Please describe your department’s plans for the next 5 years given the current conditions. Please try to list only three to five major
planning themes (e.g., staffing, facilities, equipment, etc.) along with a short description of each and the connected department and
team goals.
Future Plans
Team Goal # Department Goal (state the goal)
Improve Efficiency in Data Access and Understanding: RIE
will work with IT to continue to get the necessary training for
Banner/Argos/RDW.
IN-11
RIE-3. Models: RIE will model and implement best practices in research to
support the development of a campus-wide culture of inquiry. (Linked to
College Goals 11 and Instruction Team Goals 8.)
Improve Use of Information for Decision Making: RIE is not
IN-11
RIE-3. Models: RIE will model and implement best practices in research to
Last Updated: July 30, 2013
Page 15
Future Plans
Team Goal # Department Goal (state the goal)
always clear how our work is being used (i.e., what is the
impact of our work). We have a standard statement that
goes on the front page of all major internally produced
documents that provides guidance to the recipient as to all
the team members inside and outside of the RIE Offices who
contributed toward the study. The description provides
direction to the reader to note these groups when they quote
the information within it. A review of the RP Group’s Inquiry
Guides will be done done to improve RIE Team members’
understanding of how to provide data to groups and how to
allow them time to reflect on it, embrace it, and use it for
improvement (or asking further research questions).
support the development of a campus-wide culture of inquiry. (Linked to
College Goals 11 and Instruction Team Goals 8.)
IN-11
Improve data documentation: There are two documentation
issues. First, the RIE Team needs to improve its internal data
documentation/coding. For example, when SQL queries are
created it is sometimes difficult for a second researcher to
use it when there is limited narrative describing what each of
the 2000 lines of code means.
RIE-5. Data Integrity: RIE will collect/maintain/provide useful, objective,
reliable and valid data in an understandable format to college stakeholders
with extensive measures in data integrity. (Linked to College Goals 11 and
Instruction Team Goals 8)
IN-11
Improve Cohort Tracking Efficiency: The college’s
implementation of Banner allows different programs to create
cohorts within Banner. These cohorts are being used by IT
and RIE when they create Argos reports and run queries
against the data. Those who create the cohorts, however,
are not always understanding of the structure behind the
screens they use. For example, novices are first putting
students in to cohorts during their first semester, as should
be done, but then once the semester is over, they are
removing them from the cohort. The process is sometimes
done incorrectly which makes retrospective views of the data
impossible because the cohorts were removed. Luckily,
researchers have maintained some backup databases which
have been used to correct the situation. The RIE team will
work closely with IT during their regular meetings to
understand this issue further and create a quick how to guide
for creating and maintaining cohort groups in Banner.
RIE-3. Models: RIE will model and implement best practices in research to
support the development of a campus-wide culture of inquiry. (Linked to
College Goals 11 and Instruction Team Goals 8.)
VII.
EVALUATION OF PLANNING PROCESS
Last Updated: July 30, 2013
Page 16
The Institutional Effectiveness Committee is interested in assessing the forms & processes that have been established for the college.
What suggestions would you offer to improve the planning process for your department?
Perhaps ask for departments to include a follow-up to the Previous Year’s Future Plans. They should, of course, be included in their accomplishments
section, but just in case…
What additional information should the College provide to assist your department?
None.
THIS REPORT IS DUE TO YOUR DEAN / DIRECTOR BY JUNE 28, 2013
June 28, 2013 Department documentation is completed on this form (remember TracDat must be updated for all SLOs/GEOs for courses and
programs [degrees/certificates]); departments notify and share report with the division office or appropriate manager and with all
members of the department. Please email a copy to your dean/director and retain a copy for department records.
August 2, 2013 Deans/managers prepare a manager’s summary of PIE, submit to appropriate Vice President, and share with department members.
September 6, 2013 Vice Presidents prepare a summary of PIE input at the team level, submit to IEC, and share with members of the team.
Fall 2013 IEC reviews all submitted Vice Presidents’ summaries and other related documents, prepares a year-end report to President’s
Advisory Council on progress made in meeting College goals and recommendations for improvement, and communication is given to
the campus that the relevant documents are placed on the web and the next cycle is to begin. President’s Advisory Council will
review the report and make suggestions and approve recommended changes.
If you have questions about PIE, please direct them to Jason Chevalier (jchevalier@mtsac.edu) or research@mtsac.edu
VIII. ADDENDUM
Instruction Team Goals
The following noted Instruction Team goals are ones that the RIE Department may be best able to address in its daily work.
Instruction Team
** IN-1. Enhance communication among and between internal and external stakeholders, including students, faculty, advisory committee members, industry representatives
and the general public.
IN-2. Provide and maintain state-of-the-art instructional technology, equipment, facilities and infrastructure for safety, currency, and effectiveness and to accommodate
growth.
Last Updated: July 30, 2013
Page 17
**IN-3. Address staffing needs to maintain and enhance delivery of instruction and instructional services (including replacement, growth, and contribution to technical or
disciplinary currency.)
**IN-4. Encourage and support participation of instructional personnel in ongoing professional development to improve instruction and service to students.
**IN-5. Update curriculum and expand successful modes of delivery for currency and to improve effectiveness.
IN-6. Support and expand opportunities for academic enrichment, including provision of guest lecturers, visiting artists, as well as student participation in regional, state, and
national competitions and events.
IN-7. Secure funding to support ongoing operational needs of programs (supplies, accreditation, transportation, travel, etc.).
**IN-8. Encourage and support unit-level participation in planning and evaluation processes including PIE, SLOs, GEOs, and accreditation self-study (to establish a culture of
meaningful assessment and documentation for both internal and accreditation purposes).
**IN-9. Expand opportunities for external funding and acquisition of other supporting resources through pursuit of grants and partnerships.
**IN-10. Increase support for basic skills activities that benefit an increasing number of students across the College.
**IN-11. Strengthen the ability to access data on student success and achievement through the development and maintenance of effective instructional support activities and
course delivery models.
http://www.mtsac.edu/governance/committees/iec/form
Last Updated: July 30, 2013
Page 18