Chabot College Administrative Services Program Review Report 2014-­‐15 Year One of Program Review Cycle Office of Institutional Research Submitted on November 1, 2013 Contact: Carolyn Arnold Final Forms, 1/18/13 Table of Contents _ X__ Year 1 Section 1: Who We Are ............................................................. 2 Section 2: Where We Are Now .................................................. 3 Section 3: The Difference We Hope to Make .............................. 4 Required Appendices: A: Budget History .......................................................................................... 7 E: New Initiatives .......................................................................................... 1 F2: Classified Staffing Requests ..................................................................... 3 F5: Supplies and Services Requests ............................................................... 6 F6: Conference/Travel Requests ................................................................... 8 F7: Technology and Other Equipment Requests ........................................... 9 F8: Facilities Requests ................................................................................ 10 i Chabot College Office of Institutional Research: Program Review, Year One: November, 2013 Updated 11/1/13
1. Who We Are
Mission
The Office of Institutional Research (OIR) provides data that makes a difference! The mission of the OIR
is to produce meaningful information for the college community for the purposes of planning, evaluation,
accountability, and compliance. Research plays a crucial role in the cycle of planning, assessment and
improvement of the college by determining whether we are meeting our college goals for our students.
Responsibilities
The Office of Institutional Research (OIR) is responsible for collecting, organizing, producing, analyzing,
integrating and disseminating information on Chabot students, staff, programs, and the local community.
This research is used by the campus community for planning, evaluation, improvement, and
accountability. The OIR develops and maintains student and staff databases, updates public student
characteristics, success, and outcomes data, develops and maintains accountability measures and reports,
provides data for state and federal compliance reports, conducts periodic student and staff surveys such as
student satisfaction and accreditation surveys, provides grant development and evaluation research for
new and existing college and program-level grants, and assists faculty with course- and program-based
research. Through periodic environmental scans and targeted research presentations, OIR integrates
research into the strategic and program planning processes of the college.
Goals
The detailed annual OIR Research Agenda encompasses these four major overarching and ongoing goals:
1) Collect, compile, and maintain accurate student, staff, program, and community data;
2) Produce and analyze student, staff, program, and community research;
3) Integrate research into the college cycle of planning, assessment, and continuous improvement;
4) Disseminate student, staff, and program research and analyses to the campus and local community.
Support of other Units
The Office of Institutional Research is part of the President’s Office and the Coordinator of OIR reports
to the President. OIR works very closely with the Grant Development Office to provide research and data
for use in grant proposals and grant evaluations. The OIR supports all administrative, academic and
student services units by providing student, staff, enrollment, and survey data, analyses, and consultations
for program review and evaluation purposes. The OIR staff work with faculty, administrators, and staff in
clarifying, conducting, and interpreting research results. OIR staff infuse research and research
perspectives throughout the college by serving on PRBC and presenting relevant research at other
council, committee, division and program meetings. The OIR also supports the District Office by working
with the LPC Office of Research and Planning to provide consistent college-level data for the District.
Impact on Student Learning
The Office of Institutional Research supports student learning and success by monitoring student progress
and success for the college as a whole and for faculty, disciplines, programs, and committees, so that all
areas receive feedback about the effectiveness of their courses and programs for students.
Staffing
The OIR is staffed by a full-time faculty Coordinator, a part-time programmer (professional specialist),
and 1-3 student assistants. From 2000 to 2011, the OIR also had a full-time Research Analyst and a parttime temporary Clerk III, but both were lost in the budget cuts of 2011. Since Aug 2012, the professional
specialist programming position has been granted for only 3 months at a time on an emergency basis,
while a permanent Research Analyst/programmer is waiting for approval from the District. If the position
is only funded as a half-time job, it will only replace the programmer. This level of staffing will be
inadequate to meet college research needs.
22 Chabot College Office of Institutional Research: Program Review, Year One: November, 2013 Updated 11/1/13
2. Where We Are Now
As you enter a new Program Review cycle, reflect on your achievements over the last few years.
Our achievements are bittersweet. In 2009, we received a commendation from the Accreditation Visiting Team for
being so responsive to faculty research and strategic planning. Now, we struggle to achieve that with half the
professional staff that made it possible—1 FTE was lost in Fall 2011 to an internal transfer, leaving 1 FTE. Despite
this loss, I am proud of how much research I have produced in the last 2-3 years to answer important planning
questions about basic skills students, bottlenecks, equity, and identifying types of students to monitor progress. We
have also been able to provide major grant evaluation research for the Title III, Basic Skills, Hayward Promise
Neighborhood, CCAMPIS, and FIPSE-TECS grants, as well as grant development research for the Title V HSI grant.
What did you want to accomplish?
Overall, I wanted to continue to provide targeted research for faculty research and strategic planning. On top of a
full research agenda, I wanted to obtain our transfer data from the national database, monitor achievement of the
Strategic Plan through the student educational goal groups, produce ongoing environmental scans, continue
measuring our College-wide Learning Goals (CWLG), provide answers to routine requests, and make the IR
research more visible and used by the college community.
What were your Service Area Outcomes (SAOs)?
According to our SAOs of staff satisfaction and number of research requests, we did well. Our satisfaction rating
was 97% on the recent Admin Services Survey. Our number of research/programming requests was 127 in 2011-12
and 206 for 2012-13. The first year is comparable to previous fully staffed years, so we kept our productivity despite
the losses. The increase in 2012-13 reflects the increased expertise of the programmer—the reason to stabilize that
position. This research was high in quality as well, as virtually all requests were strategic planning-related.
Describe how changes in resources provided to your office have impacted your achievements.
The ability of the IR Office to serve the college has been compromised due to the losses of the fulltime
classified Research Analyst and the temporary part time Clerk III. Each year I have trained a staff of 3 very parttime students who work miracles but cannot make up for the expertise and person hours we lost. I have made up for
some of it by working way more than full-time on a regular basis for the last two years. Faculty research requests
have dropped, and important ongoing work such as monitoring the Nursing program, Distant Ed, tutoring, and the
environmental scan have been put off. A programming review research priority in Spring 13–the Administrative
Services Survey–offended classified professionals. This was a direct result of pushing myself to do something on top
of everything else and not being able to take the time to collect and provide the information in a more sensitive way.
I have been able to keep up with the research needs for college-level accountability, program review for all
disciplines, PRBC, and grant development and evaluation. However, I have done little else. There has been no time
to obtain data to track our four-year transfers, conduct more CWLG research, update staff data, automate procedures
for routine reports, or send out periodic announcements of new research so staff knows what is available.
What are you most proud of, and what do you want to continue to improve?
I am proud of how much meaningful research I have been able to produce with only 3 part-time students, especially
the just-in-time research that addresses college questions. My development of the PRBC educational goal student
groups was a breakthrough research strategy that will provide ways to meaningfully understand groups of students,
target services to them, and monitor their progress. This has provided a crucial new way to measure progress on our
strategic plan and has been used as the basis for the forthcoming Title V Hispanic-Serving Institutions grant. This
method is also getting statewide recognition as a way to measure progress towards required institutional outcomes.
Discuss important trends that will have a significant impact on your unit over the next three years.
Accountability brings increased scrutiny of our college-level measures, and a need for more within-college analysis to
determine how to improve them. The demand for more explanation and interpretation of data will increase. Computer
technology improvements should allow easier access to routine data for more users. While many of our state-wide
indicators have become automated, the need for interpretation and presentation remains. In addition, more grants bring
more evaluation and scrutiny of college programs.
What opportunities and challenges do your foresee in the next three years?
The need for more trusted college data at the program level to interpret and be able to respond to college-wide
accountability measures, as well as the need to provide more with less are both challenges and opportunities.
33 Chabot College Office of Institutional Research: Program Review, Year One: November, 2013 Updated 11/1/13
3. The Difference We Hope to Make
What initiatives would you like to undertake to improve or strengthen your unit or to address
demand for your services?
A fully-staffed Office of Institutional Research is required to strengthen this unit and address
demand for our services. Going into an Accreditation self-study year with research that has been left
undone when demands for research substantially increase during this time compromises our accreditation.
Specifically, I would like to complete the current tasks plus the backload of undone tasks on the
OIR Research Agenda (see below), plus the additional accreditation research and writing requirements
during a self study year.
Overall, I would like to continue to fulfill the mission of the OIR – provide research to address
current college planning needs in a way that makes a difference. First, I will need to provide required
Accreditation research services and participation for the Accreditation self-study. I will also continue to
work with PRBC and Basic Skills and Grants leaders to identify major obstacles to student success,
services that help, and progress by students. I also want to automate obtaining and displaying routine
results and responding to routine requests. In addition, I would like to make the IR research much more
visible and used by the college community.
What initiatives are underway in your area, or could you begin, that would support the
achievement of our new Strategic Plan goal and strategies and/or improve student
learning?
The recent research development and analysis of the new student educational groups directly
addresses the strategic plan goal and strategy of measuring and monitoring how we make a difference for
specific groups. In addition, recent research on equity and basic skills monitors progress on those fronts.
However, the entire OIR Research Agenda supports the Strategic plan in one way or another.
Identify specific, measurable goals and the resources needed to achieve them.
Below is the research agenda of the OIR during the accreditation self-study year of 2013-14.
Most of it will continue through 2014-15. It requires a fully-staffed Office of Institutional Research.
Color Code:
The blue color shows the research that was done or partially done in 2012-13.
The red shows additional research that the OIR needs to do during 2013-14.
Both blue and red research tasks need to be completed in 2013-14, starting with the Accreditation Self
Study. Most will continue through 2014-15.
Institutional Research Agenda 2013-14 (and 2014-15)
ACCREDITATION SELF STUDY [NOT needed in 2012-13]
ACCREDITATION STANDARDS CHAIR AND/OR STEERING COMMITTEE
a. Provide leadership or consultation about institutional accountability standards
b. Provide baseline and ongoing updates of accountability measures
c. Organize and lead the writing of one of the standards.
FALL 13 and SPRING 14 SURVEYS
ON-CAMPUS STUDENTS [NOT needed in 2012-13]
a. Develop and conduct Fall 13 Accreditation year survey with standards committees
CHABOT STAFF [NOT needed in 2012-13]
b. Develop and conduct Accreditation year survey with standards committees
ONLINE STUDENTS [NOT done in 2012-13]
c. Conduct 2nd biennial satisfaction survey in online classes (work with COOL)
ENVIRONMENTAL SCAN [NOT done in 2012-13]
a. Provide latest environmental scan data for Accred. Self-Study and Ed. Master Plan
44 Chabot College Office of Institutional Research: Program Review, Year One: November, 2013 Updated 11/1/13
STRATEGIC PLANNING
ENVIRONMENTAL SCAN [NOT done in 2012-13]
b. Update, display, and present latest internal and external planning data for PRBC
STRATEGIC PLAN GOAL [Partially done in 2012-13]
a. Refine new student educational goal groups that allow measurement of strategic plan goal.
b. Provide baseline and ongoing measures of student groups reaching their educational goal.
c. Conduct detailed analyses on the status, characteristics, or progress of each group.
STUDENT PATHWAYS ANALYSIS [NOT done in 2012-13]
a. Identify and clarify major student pathways or interest groups.
b. Identify capacity of each pathway/interest group based on student educational goals
c. Analyze student characteristics and outcomes in pathways/interest groups
PROGRAM REVIEW DATA and ANALYSIS [Done in 2012-13]
a. Produce latest course success tables for program review by late September
b. Produce latest selected course sequence tables for program review by late September
c. Update progress benchmarks for new student cohorts through Spring 2013
STUDENT LEARNING AND OTHER OUTCOMES [Partially done in 2012-13]
c. Identify and provide college-wide learning goals of current students using Fall 13 survey
d. Document plans and Chabot experiences of Spring 13 degree and certificate graduates
e. Document College-wide Learning Goals of Graduates and transfer students
BASIC SKILLS/TITLE III PROJECTS
BASIC SKILLS COMMITTEE RESEARCH [Partially done in 2012-13]
a. Update overall Basic Skills and Student Equity data
b. Identify new questions/produced new data needed by committee
TITLE III GRANT OUTCOMES – Final year reporting [Done in 2012-13]
a. Identify Spring 13 Title III/Basic Skills FIGs and track persistence into Fall 2013
b. Produce Title III success and engagement outcomes for Spring 13
c. Produce college-wide Title III outcomes for 2012-13
d. Identify Fall 2013 Title III/Basic Skills FIGs and Learning Assistant sections
e. Conduct & analyze student engagement survey for Fall 13
LEARNING CONNECTION RESEARCH [Partially done in 2012-13]
a. Analyze student engagement in sections with learning assistants [Done in 2012-13]
b. Analyze course success for tutored students [NOT done in 2012-13]
ASSESSMENT PLACEMENT TEST VALIDATION
MATH, ENGLISH, and CHEMISTRY TESTS [NOT done in 2012-13]
a. Conduct Math and English disproportionate impact study since cut score changes
b. Conduct re-validation study of Chemistry test
MANDATED REPORTING REQUIREMENTS
State accountability system data (ARCC) [Done in 2012-13]
a. Conduct in-depth analyses of 2013 data
b. Prepare Presentation of ARCC 2013 data for Fall Board meeting
c. Review preliminary ARCC 2014 data in December
Federal Gainful Employment Act [Done in 2012-13]
a. Update program information on CTE programs on Chabot Degree/Certificates website.
CAL Grants SB270 reporting [NOT needed in 2012-13]
Integrated Postsecondary Education System (IPEDS) [Done in 2012-13]
a. Check and provide data for October deadline
b. Check and provide data for December /Feb deadline
55 Chabot College Office of Institutional Research: Program Review, Year One: November, 2013 Updated 11/1/13
GRANT DEVELOPMENT AND EVALUATION RESEARCH
Grant Development research [Done in 2012-13]
a. Develop or implement evaluation of Hispanic-serving Institution (HSI) Title V grant
b. Provide research analyses for any new grants in development
GRANT DEVELOPMENT AND EVALUATION RESEARCH (CONTINUED)
Grant Assessment and Evaluation research
a. CCAMPIS: Provide final year tracking evaluation [Done in 2012-13]
b. Hayward Promise Neighborhood: Provide data and tracking [Done in 2012-13]
c. BRIDGES – provide cohorts of potential students. [Done in 2012-13]
d. MESA – Identify and track cohorts of students in program [Partially done in 2012-13]
e. Nursing grants – track nursing cohorts [NOT done in 2012-13]
OTHER MAJOR RESEARCH PROJECTS
Provide latest research results in useable formats to college/decision-makers
a. Provide Board Report on success & outcomes for both colleges [NOT done in 2012-13]
b. Provide latest Basic Skills Research – Assessment, Success, persistence [Partially done]
c. Upload all new research information onto IR website [Done in 2012-13]
d. Update staff on new research available. [NOT done in 2012-13]
Monitor Distance Learning [Partially done in 2012-13]
e. Monitor success and retention rates of distance education courses.
f. Determine 2013-14 DE courses in preparation for running success rates
g. Conduct analyses of online students [NOT done in 2012-13]
Conduct other ongoing and ad-hoc research and programming projects [Partially done 2012-13]
a. Conduct ad-hoc programming requests from programs, disciplines, staff.
Obtain and analyze Chabot student Transfer data routinely [NOT done in 2012-13]
a. Implement use of National Clearinghouse tranfer files
b. Continue use of state First Files
c. Identify ways to use Cal-PASS data and encourage UC/CSU partners to join
INSTITUTIONAL RESEARCH INFRASTRUCTURE
Maintain and Enhance Institutional Research Database [Partially done 2012-13]
a. Work with ITS to add needed data elements to the IR database:
i. Race-ethnicity changes; parent’s education, dislocated workers
b. Develop programs to use new data [NOT done in 2012-­‐13]
Maintain and Develop IR/Grants Staff/Professional Development [Partially done in 2012-13]
a. Re-hire Student assistants and programmer [Not needed in 2012-13]
b. Hire full or half-time replacement of Research Analyst [Not needed in 2012-13]
c. Coordinator attend Student Success/Research Conferences [Done in 2012-13]
Streamline IR Office/maintain IR business [Partially done in 2012-13]
a. Update and enhance IR website with latest data and reports
b. Continue to update IR data/definitions documentation for programming
c. Automate routine reports and requests [NOT done in 2012-­‐13]
COLLEGE RESPONSIBILITIES/COMMITTEES of COORDINATOR [DONE IN 2012-13]
Accreditation Standard Chair and Research Consultant
Planning, Review, and Budget Council (PRBC)
Chabot Enrollment Management Committee (CEMC)
Title III à Title V Hispanic Serving Institution (HSI) Leadership Teams
Basic Skills Committee
OUTSIDE RESPONSIBILITIES of COORDINATOR [DONE IN 2012-13]
Board Member of Research and Planning Group (RP Group), Northern California representative
66 Chabot College Office of Institutional Research: Program Review, Year One: November, 2013 Updated 11/1/13
Appendix A: Budget History and Impact Audience: Budget Committee, PRBC, and Administrators Purpose: This analysis describes your history of budget requests from the previous two years and the impacts of funds received and needs that were not met. This history of documented need can both support your narrative in Section A and provide additional information for Budget Committee recommendations. Instructions: Please provide the requested information, & fully explain the impact of the budget decisions. Category
Classified Staffing (# of positions)
Full-time Research Analyst (40 hrs)
Part-time Temporary Clerk (20 hrs)
Prof Spec Programmer (15-20 hrs)
Student Assistants (2@ 15hrs )
Supplies & Services (software)
Technology/Equipment
Other: Membership in RP Group
TOTAL
2012-13
Budget
Requested
0.50
0.50
$5,000
$2,085**
$825
0
$350
$
2012-13
Budget
Received
0
0
$5,000*
$2,085
$825
0
$350
2013-14
Budget
Requested
1.00
0.50
$5,000
$2,085**
$825
0
$350
2013-14
Budget
Received
?
0
$5,000*
$2,085
$825
0
$350
*These funds will go into Research Analyst position if it is funded; **Cut drastically from previous years
1. How has your investment of the budget monies you did receive improved your Service Area Outcomes? When you requested the funding, you provided a rationale. In this section, assess if the anticipated positive impacts you projected have, in fact, been realized. This office could not function at all without a dedicated programmer and 20 hours or more of research
assistance. The funds I did receive –—$5,000 for a professional expert programmer and $2,085 for
student assistants —have made it possible to fulfill major college research responsibilities such as federal
and state reporting, program review, PRBC, and grant evaluations and development research. This was
the rationale for maintaining these positions. However, the table above does not show the extent of the
under-funding—these amounts are decreases from previous years. At least $8,000 is needed for a
programmer, and $8,000 is needed for research assistants, so I have supplemented these funds with
grants, including FWS, Matriculation, Title III, HPN, Bridges and Nursing funds. None of the grants are
permanent, but they are sustaining these crucial positions along with the college funds.
Our Service Area Outcomes – the number of Programming/Research Requests completed —went up this
year to 206 in 2012-13 from 127 for all of 2011-12. I credit the increased expertise of the programmer and
the student assistants, who continued from last year, as well as my almost complete dedication to
coordinating their work in a way that addressed the most important college needs and used the growing
expertise of this little band of students. Because I had a programmer who is an advanced Computer
Science student at CSUEB, we were able to do more complex programming and create the PRBC student
educational goal groups, which we are now using for everything. (see accomplishments)
Ironically, while I had adequate funds for a programmer and assistants due to grants, we were prohibited
from continuing the long-time positions of temporary Classified and professional specialists, so I have
turned to Student Assistants to do lower level research assistant work. While I love working with
students, this made the work much less professional and less consistent. The professional specialist
position is being continued temporarily on a 3-month at-a-time basis while we await approval of a
permanent Research Analyst position. This will stabilize this needed position, but if only half time, it will
only replace the current programming position, rather than restore any professional research assistance.
77 Chabot College Office of Institutional Research: Program Review, Year One: November, 2013 Updated 11/1/13
2. How has your investment improved your Service Area Outcomes?
Assess the anticipated positive impacts you projected have, in fact, been realized.
The supplies, software, and membership accounts, as predicted, have kept the office functioning at a
minimum level of paper, pens and print cartridges, online survey and survey analysis software, and
membership in the research organization that has provided access to much of the latest transfer and basic
skills data and research. These were adequate for this year, but will not be adequate during an accreditation
survey year of 2013-14, when more survey supplies are needed. 3. What has been the impact of not receiving some of your requested funding? How has your ability to
serve the college and our students been impacted, or safety compromised?
The IR Office has been most heavily impacted by losing the full-time Classified Research Analyst
position in Fall 11. The effects of that are many and continue.
Some major IR Office functions that no longer happen or are more limited due to this loss are:
1) providing customized research requests for faculty for program review, and the time to explain and
interpret the results
2) updating and producing our standard student characteristics and outcomes reports for the Board;
3) conducting careful programming based on years of experience and discovering anomalies in our
District and state data that leads to corrections that can change our state accountability measures.
4) conducting in-depth analyses for grant proposals that allowed us to go beyond our usual statistics to
focus on the target group of that grant, whether middle school students or low-income students at Chabot.
These analyses were crucial in winning several of our federal grants, such as TRIO and Title III.
5) becoming adept at obtaining and quickly using transfer data on our students to state and national 4-year
colleges. The data are available. The time to train in this process is not.
6) Conducting a thorough Environmental scan for PRBC. We are relying on last year’s data.
7) The Research Analyst’s presence in the IR Office allowed the Coordinator to play major roles in
PRBC, CEMC, Basic Skills, the Student Learning Outcomes Assessment Committee, Faculty Senate, and
Accreditation, which brought research expertise and organization to these groups, making them more
effective. Due to the loss of the Research Analyst, the Coordinator has not been able to play a major role
in these groups, and in some cases is not at all involved.
Funds for programming and research assistance were also cut. As mentioned above, funds for the
programmer professional specialist and student assistants were cut; the temporary part-time Clerk position
was lost. To fund the programmer and students, I supplemented the budget with temporary grants.
The office continued to function, but the effect of the decreased funds for the programmer was to become
more dependent on grants, and less able to provide consistent programming services over the long run.
The effect of losing the part-time Classified Clerk position is that I could not develop the expertise of a
student or non-student who could stay in the position longer. Consequently, the students and programmer
whom I carefully trained in the last 2 academic years all graduated and/or transferred and I had to train
new students in all of these positions in Fall 2013. While I enjoy that, it greatly delays and decreases the
amount of new research that can be turned out by the office. We were buffered in 2012-13 by the grants
and the students who continued from the previous year. In 2013-14, we still have grants, but not for long.
The ability of the IR Office to serve the college has been compromised. The most important research
requests have gotten done, but not the next most important things. In 2012-13, major research that was
put off was a survey of online students, the engagement survey in Spr 13, an update of the environmental
scan, and monitoring of the success and persistence of nursing and distant education students. The impact
on the Coordinator has been personal wear and tear from working much more than full-time during the
last two years— in order to provide some of the lost services as well as maintain the regular research
functions. However, this is not sustainable. For 2013-14 and 2014-15 a stable staff is needed.
88