The Research Agenda of the
Ohio Articulation and Transfer Network (OATN)
0
I. Introduction:
The research agenda of the Ohio Articulation and Transfer Network
(OATN) includes specific questions on the following four broad
themes:
1. The
quantification of transfer movements among
institutions in the University System of Ohio (USO).
2. The extent of cost-savings facilitated by the transfer of
credit.
3. The comparison of academic performance between
transfers and continuing students.
4. The evaluation of recent OATN initiatives such as the
Transfer Assurance Guides (TAGs) and the Advanced
Placement (AP) test policy on the academic
performance of transfer students.
A regular updating of the complete matrix of transfer movements
among the 61 USO campuses addresses the first set of questions as
it provides both the volume and the directions of annual transfer
movements in the system.i An aggregation of the differences in the
costs – between the source and the receiving institutions – of
transferred credit hours provides estimates of annual transfer
facilitated savings. The estimates of selected academic outcome
indicators such as graduation rates or course completion rates
across transfers and continuing students provide comparisons
between the two groups. Finally, evaluations of recently
implemented OATN initiatives –TAGs and the AP test policy – reveal
their impacts on academic performance.
Credit transfers allow undergraduates to earn credit from one
institution and use it at a different institution in a subsequent
period. Dual high school and college enrollments or the
standardized Advanced Placement (AP) tests, on the other hand,
allow high school students earn college credit. Regardless of the
specifics of how individuals attempt college credit at one institution
with a view to using the hours in a different institution, they could
be required to repeat previously completed courses if the transfer
or the granting of credit is subject to the details of the courses or
the characteristics of credit granting and receiving institutions. In
recent years, OATN has implemented specific initiatives – TAGs and
the AP policy – to streamline the transfer of credit.
TAGs were implemented in 2005-06; they provide a guaranteed
pathway to baccalaureate degree completion. TAGs serve as an
advising tool, specifying the courses leading to baccalaureate
degrees in each of the 40 different disciplines that have been
approved to date by higher education faculty panels. Importantly,
credits earned in the approved equivalent courses transfer and
apply to meeting pre and beginning major requirements in any
institution in the USO.
The Advanced Placement test policy, on the other hand,
standardizes how colleges grant credit for AP tests. Prior to the
current policy directive issued in 2009-10, institutions granted credit
for AP tests but exercised their discretionary choices. The current
policy requires colleges to identify equivalent courses for each AP
subject, and to grant credit in corresponding courses for a minimum
AP test score of three.ii
The OATN initiatives are geared to the goal of rendering credit
earned at one institution equivalent to credit from any other
institution. Accordingly, the initiatives reduce the need for course
repetition and are likely to shorten graduation time. The initiatives
also enable students to save resources by earning credit at lowercost institutions and using it at more expensive institutions, or by
earning college credit as high school students at a fraction of the
cost. These initiatives also create greater degree completion
opportunities as students are able to get on a lower-cost pathway
1
to receiving a baccalaureate degree. The benefits of credit transfer,
therefore, can be partially measured with estimates of aggregate
annual savings received by students.
Although cost-savings made possible by the ‘equivalency of credit’ is
a tangible benefit, the question remains whether credits are indeed
equivalent across institutions. On a specific note, are students
similarly prepared for related higher level courses as a result of
completing courses with the same expected learning outcomes at a
community college or at a 4-year university? By the same token, do
students receive the same academic preparation for future related
work by taking an AP test (with a minimum score of three) or by
completing the corresponding course at a 4-year university?
The OATN is committed to implementing a comprehensive policy
that upholds academic standards while ensuring that credits do
count. The validation of the equivalency of credit, therefore,
commands paramount importance in the OATN research agenda.
Validation efforts are split into two components. The first focuses
on the comparison of selected academic outcomes between
transfers and continuing students while the second component
carries out evaluations of recent OATN initiatives such as TAGS and
the AP test policy.
Because the equivalency of credit across institutions is likely to have
the largest bearing when students transfer to 4-year university main
campuses, academic outcomes are compared between transfer
students to 4-year university main campuses and continuing
students in those same campuses. The same two comparison
groups are used also in the statistical evaluations of the impacts of
TAGs and the AP test policy.
methodologies, as well as the anticipated completion date of each
investigation.
II. Quantification of Transfer Movements: Volumes and
Directions
i.
How many students transfer among institutions in the USO
in a given year?
ii. What are the directions of the transfers (among specific
institutions and sectors)?
Methodology: Quantification of Transfer Movements
The flow of transfer students within the University System of Ohio is
an important indicator of the effectiveness of various transfer
related policies. A trend of transfer volumes illustrates the growth
of transfer numbers while the ratio of transfer to enrollment, also
over time, informs on the growth of the proportion of students who
transfer in a given year. Moreover, the directions of transfers –
among broad sectors – help identify the major flows, informing on
the current and future needs of transfer students in the system.
The most important methodological question centers on how to
define a transfer student. While the question is at the core of
investigation, obtaining a consistent and workable definition is
complicated by a number of observations. In any academic year,
students in the University System of Ohio attend multiple
institutions or even multiple campuses of the same institution. The
multiplicity of enrollment is often observed in the same term of the
year. Since ‘transfer’ involves the movement of students or of
credits, it is necessary to define the ‘college of attendance’ before
defining the transfer movement.
In the following, brief descriptions of the questions and the relevant
methodologies used in addressing those questions are provided.
This research agenda also includes a table that lists the broad
themes, associated questions, specifics of the data and the
2
The college of attendance is defined as the one where the student
attempted the maximum credit hours in the year. A student is
defined as a transfer if she makes a clean break from her college of
attendance in an academic year where a clean break in turn is
defined to have occurred in any of the following situations:
i.
The student formally transfers credits from her college of
attendance to another institution.
ii. The student does not transfer credit to another institution
but attends a single institution of higher education that
is different from her original college of attendance.
iii. The student does not transfer credit to another institution
but attends multiple institutions and attempts the
maximum number of credit hours in an institution other
than her original college of attendance.
The definition includes students who ‘formally’ transfer credit to
other colleges as well as those who change college without
transferring credits. The definition, however, does not include
transients – students who attend an institution for a few credit
hours over a brief period before returning to their original colleges.
Summer-term enrollments at colleges close to parental homes are
typical examples of transient movements. The definition and
observed patterns of transfer movements can be found at
http://www.ohiohighered.org/transfer/research.
Armed with the definition, a complete matrix of transfer
movements – the number of students moving among the campuses
in the University System of Ohio is prepared for the academic year.
Subsequently, the numbers are aggregated to obtain transfer
movements among three broadly defined sectors – community
colleges, regional campuses of 4-year universities, and main
campuses of 4-year universities. The annual transfer matrix from
2002 through 2010 – in its original and abbreviated forms – is
available at http://www.ohiohighered.org/transfer/research.
III
Transfer Facilitated Savings
What are the aggregate annual amounts of transfer facilitated
cost-savings in the USO?
Transfer movements, of students and credits, enable
undergraduates to save resources as they take courses at less
expensive institutions and use them at more expensive institutions.
The low-cost pathway, facilitated by transfer movements, creates
greater opportunities for degree completion for a large number of
students.
The investigation considers two specific sources of transferfacilitated cost-savings:
i. Inter-institution transfer of college credit
ii. College credit received through dual high school and college
enrollment.
In the calculation of savings from either of the above sources, only
credits earned in general and baccalaureate level courses are
included. Credits earned in courses classified as technical education
are excluded because those courses are often occupation specific
and may not be accepted by 4-year universities. Remedial education
hours are excluded because they do not lead to credit
accumulation.
III.A
Methodology: Savings from Inter-Institution Transfer of
Credit
Savings from inter-institution transfers of college credit are
generated from the following activities:
-
Transfer of students from less expensive institutions to
more expensive institutions; it is maintained that savings
are generated only if students transfer in an upward
direction, i.e., from a community college to a regional or the
main campus of a 4-year university, or from the regional
3
-
campus of a 4-year university to the main campus of a 4year university.
Transfer of credit, but not of students, from less expensive
to more expensive institutions; in this case, students
continue attendance in a single institution in two successive
academic years, but earn credit simultaneously from a less
expensive institution.
Savings are calculated as a product of the following two
components.
a. The differences in the expenditure per credit between main
campuses of 4-year universities and community colleges,
between the main and the regional campuses of 4-year
universities, and between regional campuses of 4-year
universities and community colleges.
b. The aggregate number of credit hours completed at less
expensive institutions and intended for use at more
expensive institutions. Specifically, this includes hours
completed at community colleges – to be used at the
regional or the main campus of a 4-year university, and
hours completed at a regional campus of a 4-year university
to be used at a main campus of the same.
The calculated savings reflect the difference in the total costs of
those hours – the sum of student, institution and state expenditures
– between the more and the less expensive sectors. However,
because state outlays for similar courses differ little between
sectors, estimated cost savings accrue mostly to students.
III.B
Methodology: Savings from High School College Dual
Enrollments
Similar to the estimation for inter-institution transfer of credit, costsavings resulting from high school and college dual enrollment is
also estimated as a product of the following components:
i.
The aggregate number of college credit completed by high
school students (through dual enrollment).
ii. The difference in the per credit cost; the difference refers to
the amount relevant for undergraduates less the amount
relevant for high school students (through dual enrollment).
Because high school students do not pay for college credits
attempted through dual enrollments, associated cost
differences are the costs of the credit hours.
The number of aggregated completed hours is obtained by starting
with all undergraduates in a given academic year who are 20 or
younger. These students are classified as eligible undergraduates at
community colleges, regional campuses of 4-year universities, and
main campuses of 4-year universities. Subsequently, the number of
aggregate completed hours for each group of is obtained from dual
high school and college enrollment records from the previous two
years. For each group of students, the number of general and
baccalaureate level hours is obtained separately. Finally, per credit
costs of baccalaureate and general education hours are used to
obtain estimates of overall cost-savings.
IV.
Comparison of Academic Performance: Continuing
Students vs. Transfers
How do continuing students and transfers compare with respect to
academic performance?
The comparison requires the selection of outcome indicators and
the selection of separate groups of continuing and transfer students
(for comparison).
The choice of outcome indicator depends on the target parameters,
and also on the interest of specific audiences. In the current
context, cumulative graduation rates satisfy both concerns. First, an
improvement of graduation rates of transfer students is an obvious
target for transfer policy initiatives. Graduation rates are also an
easy to understand indicator of academic performance, satisfying a
4
broad range of audiences interested in the performance of the
system as a whole. In future efforts, the choice of the performance
indicator can be expanded.
Regarding the choice of comparison groups, current efforts consider
students transferring to 4-year university main campuses in a given
year and students continuing in those universities in the same year.
The focus on 4-year university main campuses acknowledges that
various transfer initiatives are likely to have the maximum bearing
for students who transfer to 4-year university main campuses.
Both transfer and continuing students, however, differ in important
characteristics. Transfer students to 4-year university main
campuses, for example, transfer at different points in their higher
education paths, as observed from their differential ranks at the
destination campuses. Second, the intensity of attendance also
differs as they attend destination campuses as part-time or full-time
students. Similarly, continuing students include students with
differential ranks and attendance intensities. The research
methodology, therefore, compares students – incoming transfers
and continuing students at 4-year university main campuses – by
rank and by attendance intensity; sophomore (juniors) full-time
transfer students are compared to sophomore (junior) full-time
continuing students.
Conditioning on rank and attendance intensity helps create
appropriate comparison groups. However, large underlying
differences still exist with respect to key demographic, economic
and social characteristics. Since those factors are also related to
academic performances, graduation rates are subsequently
compared by conditioning on selected individual and family
characteristics as well.
V.
Evaluation of impacts of TAGs on graduation rates
What are the impacts of TAGs on graduation rates?
TAGs provide a guaranteed pathway to baccalaureate degrees.
Serving as an advising tool, TAGs specify the courses that students
need to take to graduate in each of the 40 different disciplines that
have been approved to date by higher education faculty panels to
date. Under TAGs, credits earned in approved equivalent courses
transfer across institutions in the USO, and meet pre and beginning
major requirements. Since the initiative provides pathways leading
to graduation and also renders credits equivalent (in approved
courses) across institutions, it is important to know how it
influences the actual graduation outcome of transfer students. The
current effort answers the question by using comparisons between
transfer students to 4-year university main campuses and
continuing students in those campuses.
Methodology: Impact of TAGs
The identification of the impacts of TAGs is obtained as follows.
First, the change in the rate of graduation, before and after the
implementation of TAGs, is obtained for continuing students.
Second, the change in the rate of graduation, before and after the
implementation of TAGs, is obtained for the transfer students.
Finally, a comparison of the change in graduation rates between the
two groups reveals the impact of TAGs. The estimation
methodology also accounts for differences in individual and family
characteristics between groups and over time.iii
VI.
Evaluation of Advanced Placement (AP) test policy
What are the impacts of AP test policy (on academic outcomes)?
The AP test policy requires colleges to specify equivalent courses for
each of the AP subject tests and to grant students college credit in
an equivalent course for a minimum AP test score of three.iv Prior to
5
the AP policy directive, colleges used discretionary choices in
granting credit for AP tests.
In the evaluation of the AP test policy, it is necessary to choose
appropriate comparison groups and the outcome indicator.
Although comparison groups are drawn from 4-year university main
campus students with and without AP tests, data availability
considerations have a significant bearing on the choice of the
comparison groups as well as the choice of the outcome indicators.
Methodology: AP Test Policy
The directive for the AP policy went to institutions in the summer of
2009 (the first term of the 2009-10 academic year). Consequently,
the effect of the new policy can be investigated using available data
only on freshman students from the 2009-10 and 2010-11 academic
years. Given the lack of data covering an extended period, the
evaluation effort cannot consider graduation rates as the outcome
indicator. Instead, the effort focuses on course completion rates
(ratio of completed to attempted hours), cumulative GPA, and firstyear retention rates.
Moreover, colleges are interested in knowing whether receiving
college credit for AP tests encourages students to attempt
additional credit in related topics. In light of the particular concern,
the AP test policy evaluation effort considers whether credits for AP
tests in a given subject are complementary in nature or whether
students use such credit as a pure substitute and divert from the
discipline(s) in question in future work.
The identification for the AP test policy is also based on a beforeand-after comparison of groups of students who did and did not
take the AP test. The pre-implementation differences in the
outcome indicator are obtained from before 2009-10 and the postimplementation comparison refers to 2009-10 and 2010-11
academic years. Because beneficiaries had chosen to take the AP
tests, the comparison can be applied to more narrowly matched
groups of program beneficiaries and students who did not benefit
from it; the matching is conditional on the similarity of estimated
probabilities of taking the AP test.v
VII.
Conclusion
The research agenda of the Ohio Articulation and Transfer Network
(OATN) lists questions related to four broad themes: the
quantification of transfer movements, transfer facilitated costsavings, transfers vs. continuing student academic performance,
and impacts of OATN initiatives such as TAGs and the AP test policy.
The agenda describes the various benefits of credit transfers while
emphasizing the need for rigorous assessments of academic
performances of transfer students. The agenda provides brief
descriptions of methodologies used in the estimation of transfer
facilitated benefits and in evaluating the impact of OATN initiatives
on academic outcome indicators.
i
Transfer movements are calculated for 61 campuses in the University
System of Ohio. Of those, 14 are 4-year university main campuses; transfer
movements involving the Northeastern Ohio University Colleges of
Medicine and Pharmacy are included in the accounting. Of the remaining
47 campuses, 24 are regional campuses of 4-year universities, and 23 are
community colleges.
ii
The specification of equivalent courses is subject to the availability of
relevant courses. A detailed description of the directive is available at
http://www.ohiohighered.org/transfer/advancedplacement
iii
The estimation methodology is known as the difference-in-difference
(DID) estimator in the program evaluation literature. Specific assumptions
regarding the applicability of the DID estimators will be discussed in the
report.
iv
Same as endnote ii.
v
Similar to the evaluation of the impact of TAGs, the evaluation of the AP
test policy applies the DID estimator. However, given that students choose
to take the AP test, the DID estimator is applied to matched comparison
groups of students who did and did not take the AP test.
6