BS Natural Sciences Educational Effectiveness Assessment Plan Page 1 of 13

ABLE OF
ONTENTS
Introduction _________________________________________________________________ 3
Program Objectives ___________________________________________________________ 6
Program Outcomes ___________________________________________________________ 7
Assessment Tools _____________________________________________________________ 8
Assessment Implementation & Analysis for Program Improvement ____________________ 11
General Implementation Strategy _________________________________________________________ 10
Method of Data Analysis and Formulation of Recommendations for Program Improvement ___________ 10
Modification of the Assessment Plan ______________________________________________________ 11
Appendix A: ETS Praxis II Subject Test in Biology ________________________________ 12
Appendix B: ePortfolio Academic Assessment Proposal ____________________________ 15
BS Natural Sciences Educational Effectiveness Assessment Plan Page 2 of 13

The baccalaureate program in Natural Sciences is based on a curriculum that emphasizes the interrelationships among scientific disciplines. The Natural Sciences degree was significantly revised and strengthened in AY 2011. This newly modified program, which just completed its first year deployment in AY12, requires students to: 1) select one of three degree options; 2) complete all courses required within their elected option; and 3) also complete sufficient science elective courses to meet minimum unit requirements for graduation.
Students enrolled in this flexible degree program must select one of three options, including the:
1) General Sciences Option, which is designed for students who interested in understanding the interrelationships among various scientific fields, or in teaching science at the secondary level; 2)
Pre-Health Professions Option intended to meet the admission requirements of specific professional schools in medicine, dentistry, and veterinary medicine; and 3) Environmental
Sciences Option, crafted to prepare students for graduate school or for employment in the private or public sector.
Past efforts to assess the BS Natural Sciences program were ineffective for several reasons.
First, the purpose of the degree was diffusely defined. Although the degree was originally designed to assist students in the military that tended to transfer credit from other institutions, it soon became a popular degree among non-military students seeking to avoid academically challenging courses such as organic chemistry and calculus. Secondly, the number and kinds of courses approved to satisfy degree requirements proliferated. While the Department was actively involved in the approval process, it had little appreciation of the long term consequences as it pertained to assessment. As such, students could effectively earn degrees by taking diverse combinations of classes such as anthropology and psychology without having to take additional courses in Biological Sciences besides BIOL A115-A116 (Fundamentals of Biology). All three
Natural Sciences degree options now limit such courses to those that complement the
Environmental and Pre-Health Professionals options. Thirdly, there was no effective way to contact Natural Sciences students in other CAS departments other than Biological Sciences. For example, we lacked such tools as DegreeWorks, and generally interacted with most students only when they completed degree paperwork required for graduation. Lastly, it seemed that we were precluded from using a single assessment tool to provide meaningful insights to the effectiveness of this degree program. For instance, the ETS Major Field Test has been employed successfully to assessment the BA and BS Biological Sciences degree programs: we have evaluated nearly all of these students. In contrast, data used to gauge the BS Natural Sciences degree program are based approximately on a paltry 27 graduating seniors who completed the ETS Major Field Test over the past decade. Paradoxically, over 100 BS Natural Sciences students actually graduated from UAA during this same period.
Given the department’s recent curricular revision, this document reiterates the educational objectives and expected outcomes for the Bachelor of Sciences in Natural Sciences, and presents a revised blueprint of a plan originally adopted by the department for assessing the achievement of the stated objectives and outcomes. I anticipate future departmental discussions and potential reconsideration of existing program objectives for this degree.
BS Natural Sciences Educational Effectiveness Assessment Plan Page 3 of 13

Present degree program objectives are designed to ensure that candidates for this degree are: (1)
Knowledgeable of the central conceptual models used in the major thematic areas of the natural sciences and (2) Identify problems, devise solutions and communicate solutions effectively .
Major thematic areas of Natural Sciences include biological sciences, chemistry, geology and physics. The level to which students in the BS Natural Sciences can focus and specialize in a particular thematic area well exceeds that available to majors in BA and BS degree tracks of
Biological Sciences. We therefore anticipate difficulties in creating a single assessment instrument primarily on select thematic areas for seniors graduating with BS Natural Sciences degrees. Given the commonality of 100-level coursework in the above four thematic areas, we propose the following:
1.
Administer the standardized ETS Praxis II Series Subject Test to all seniors who graduate from UAA with a BS degree in Natural Sciences.
As noted above, we currently use Educational Testing Service’s Major Field Test in
Biology as a means of assessing the outcomes of the BA and BS Biological Sciences degree programs. While this strategy captures most of BA and BS majors, we have never managed to effective census our BS Natural Sciences majors.
Other institutions offering the Praxis II Series Subject Test tests use the results as one criterion to accept applicants into teacher education programs, or also use them as part of the teacher licensing process. The Praxis tests address a wide range of topics, and may allow us to customize those sections for which students will be assessed. For example, all Natural Science majors must take BIOL A115-A116 (Fundamentals of Biology I-II) and CHEM A105-A105L and A106-A106L (General Chemistry I-II). However, students in the Environmental Sciences option must also take courses in Geology and
Environmental Sciences, but are not required to take Physics (even though they may elect to do so). Similarly, students in the Pre-Health Professionals option are required to take
Physics, but not Geology, while those in the General Sciences option are obliged to take courses in Geology and Physics.
Preliminary discussions with ETS personnel lead us to conclude that the Praxis II Series
Subject Test exam may be suitable for use as an assessment tool. The committee has concerns about the limited usefulness of data generated by this exam. Focusing on lower-level courses shared by all students seems an appropriate step towards effective academic assessment of the Natural Sciences baccalaureate program. We are continuing our discussions with ETS personnel to address these issues.
2.
Modify Major Requirements of the course catalogue (chapter 10, p 120) to include a statement that the ETS Praxis II Series Subject Test exam requirement must be taken during the semester of graduation.
Decisions to modify the curriculum must be made by the departmental curriculum committee and faculty. In light of recent misunderstandings we’ve encountered over the
“requirement” that the ETS Major Field Test must be administered to BA and BS
Biological Sciences majors, it may be necessary to submit all such changes as PARs for
BS Natural Sciences Educational Effectiveness Assessment Plan Page 4 of 13

major degree requirements and a CAR-CCG in FY15. Such a catalogue modification will clarify expectations of academic assessment. Issues such as the logistics, costs and timing will be addressed by the department early in the coming academic year. Final approval to change BS Natural Sciences degree requirements rests with the department.
BS Natural Sciences Educational Effectiveness Assessment Plan Page 5 of 13

1.
It is expected that graduates of the Natural Sciences program will:
A.
Demonstrate their knowledge of the central conceptual models used in the major thematic areas of natural sciences, and
B.
Identify problems, devise solutions and communicate solutions effectively.
BS Natural Sciences Educational Effectiveness Assessment Plan Page 6 of 13

The specific educational outcomes that support the program objectives are to produce graduates who are:
1.
knowledgeable of the central conceptual models used in the major thematic areas of natural sciences, including:
A.
Biology
B.
Chemistry
C.
Physics
D.
Geology
This outcome is measurable and comparable across time and among thematic areas of biology, chemistry, geology and physics programs nationwide through the administration of standardized Praxis tests. We plan to administer the ETS Praxis II
Subject Test either in Biology and General Science (which includes chemistry, geology and physics), Biology Content Knowledge or Chemistry Content
Knowledge.
2.
Identify problems, devise solutions and communicate solutions effectively.
This outcome is measurable and comparable across time in the single thematic area of biology since all BS Natural Sciences majors are required to take BIOL A108. This program outcome will be assessed through the use of ePortfolios, which will be implemented for BS and BA Biology majors beginning the fall term of AY16. Details of our assessment plan involving ePortfolios for biology majors are detailed in the recently submitted Biology_BA-BS_Academic_Assessment_Plan_AY16 and appended herewith.
BS Natural Sciences Educational Effectiveness Assessment Plan Page 7 of 13

A description of the tools used in the assessment of the program objectives and their implementation are summarized in Table 1. The tools and their relationships to the program objectives are listed in Table 2.
There is a separate appendix for each tool that describes the factors that affect the results and give examples of the tools and how they will be implemented.
Table 1
Program Objectives Assessment Tools and Administration
Tool
Tool #1
Tool #2
Description
ETS Praxis II Subject Test Score ePortfolios
Start Date/
Frequency
Fall/ AY16
2x/Yr
Fall/ AY16
2x/Yr
Collection
Method
Exam ePortfolios
Administered by
Department
Faculty
Department
Faculty
BS Natural Sciences Educational Effectiveness Assessment Plan Page 8 of 13

Table 2
Association of Assessment Tools to Program Objectives
Objective 1A - Biology
Objective 1B – Chemistry
Objective 1C – Physics
Objective 1D – Geology
┼
┼
┼
┼
Objective 2 – ePortfolio
┼ Tool is used to measure the associated objective.
┼
BS Natural Sciences Educational Effectiveness Assessment Plan Page 9 of 13

General Implementation Strategy
Although the present assessment process in Natural Sciences involving the ETS Praxis II Subject
Test was approved in AY12, questions about how best to administer this exam were unresolved at the time. The department decided to require natural science students take the ETS Praxis II
Subject Test no later than the end of the last or second-to-last-semester before graduation. This test is designed as a one-part, two-hour exam which can be administered on-line. Although performance results will never be used to assess a student’s performance, participation in the exam will be a requirement for passing the class. Exam result will also be included in each student’s ePortfolio.
As noted above, our assessment process in Biological Sciences was initiated in the fall semester,
2002. To date, the ETS Major Field Test has been given 23 semesters to 407 students in the
Biological Sciences and Natural Sciences programs; only around 35 of these students were BS
Natural Sciences majors. We have under-assessed this degree program over the past decade since we presently have approximately 200 BS-Natural Sciences majors in our department.
In addition to the ETS Praxis II Subject Test, we will assess our academic program through the use of ePortfolios in BIOL A108, commencing the fall semester of AY16. It would be useful to extend our assessment to include other courses. However, BIOL A108 is the only common course in the degree program that all Natural Science majors are required to take. Details of our assessment plans involving ePortfolios are available in our Biology_BA-
BS_Academic_Assessment_Plan_AY16, which is included in this report.
Method of Data Analysis and Formulation of Recommendations for Program
Improvement
The analysis of data from the ETS Praxis II Subject Test is straight-forward. Scores for each student are obtained from ETS, including a comparison of our student’s performance to those of students in other programs across the country. The departmental assessment coordinator for the
BS Natural Sciences program will summarize these results at the end of each academic year
(ETS data become available within 2-3 weeks once completed Praxis II exams are returned to
ETS). The results are reviewed by the departmental assessment committee, incorporated into an
UAA Educational Effectiveness Assessment Report for Academic Affairs in June, and presented at a faculty meeting in August-September. Faculty will discuss the results, discuss whether changes or modifications to either the assessment tools or the curriculum should be made, and prepare recommendations for amendments, if necessary, by October.
The departmental Assessment Committee is tasked with compiling results of ePortfolios for each of the fall and spring semesters at the close of an academic year. Faculty will be discussing the organization and implementation of ePortfolios during the spring 2015 semester
BS Natural Sciences Educational Effectiveness Assessment Plan Page 10 of 13

Modification of the Assessment Plan
Departmental faculty will ultimately decide whether to alter the assessment process based on their review of assessment data. Changes may be made to any component of the plan, including the objectives, outcomes, assessment tools, or any other aspect of the plan. The assessment program coordinator will oversee the documentation of these changes, if any, and submit the revisions to the faculty for approval. The modified assessment plan will then be forwarded to the dean/director’s office and the Office of Academic Affairs.
BS Natural Sciences Educational Effectiveness Assessment Plan Page 11 of 13

PPENDIX
RAXIS
UBJECT
EST IN
IOLOGY
Tool Description:
The ETS Praxis II Subject Test is a standardized test prepared and scored by the Educational
Testing Service, Princeton, NJ. The tests in Biology & General Science and Chemistry
Comprehension each requires two hours to complete, whereas Biology Comprehension takes one hour. We favor the Biology & General Sciences test since questions from all four major thematic areas of Natural Sciences are included. It also appears that ETS categorizes the scores of these tests into subcategories of these same thematic areas for comparison.
Factors that affect the collected data:
Factors affecting the validity and usefulness of the data collected using this method includes the following.
1.
Historically, Natural Sciences students who have taken in BIOL A492 were required to take the ETS Major Field Exam in Biology. Although these results were used to assess our Natural Science program, the approach is flawed in at least two important respects.
First, the class has never been required for majors in the Natural Sciences; only 27
Natural Sciences participants have taken this exam over the past decade. Hence, most of the majors are not being assessed at the present time. Secondly, the Major Field Exam tests students only in the area of Biological Sciences, and does not examine their knowledge and competency in the other sciences. Although most natural science students focus in biology, this is not always the case, and we therefore miss the opportunity to examine their breadth of knowledge across the sciences
2.
We seek the Department’s approval early in the fall semester, 2012, requiring BS Natural
Science students to take at least one credit of BIOL A492 (Undergraduate Seminar). Such a change will require the submission of a PAR to modify this degree’s major requirements and a CAR-CCG to amend undergraduate seminar.
3.
Students entering BIOL A492 class may or may not be close to graduation. However, we believe most students will have completed sufficient coursework necessary to perform well on the Praxis II exam if they will have completed their 24 to 32 credits of required100-level coursework prior to taking a required 400-level senior seminar course.
Ideally, it will be advantageous to instead administer this test to students immediately prior to graduation since the results would be more representative of program effectiveness.
4.
There are no incentives to motivate a student to perform well on the test. While exam performance is not a formal element of an individual student’s assessment in our program, some students may choose not to perform to their best. Although the faculty will not change this policy, we do strongly advise our students to “do their best.” We believe most students will take this test seriously; many who have completed the ETS
BS Natural Sciences Educational Effectiveness Assessment Plan Page 12 of 13

Major Field Test as part of the BS-BA Biological Sciences program come in to find out how they did on their exams. We continue to emphasize to students that the exams are good practice for exams for graduate school and for professional programs (e.g., the GRE and MCAT exams).
How to interpret the data:
ETS Praxis II Subject Test results are reported as a single overall score, range of scores possible on the particular test, and the average score (50%) of all examinees for the previous three years.
Interpretation of the test results is less straight-forward. Apart from indicating how students are performing overall, we will apparently not be able to parse results into the 4 major thematic areas of biology, chemistry, geology and physics. It may be possible to compare scores on the basis of percentiles for students taking the same test nationwide. This option will be discussed with ETS officials this summer.
Overall, results of the ETS Major Field Test in Biology for the 27 BS-Natural Sciences students who have taken the test to date reveal an average total score of 157, resulting in relative national ranking (for these seniors majoring in Biology) in the 57 th
percentile. The following results reflect are anecdotal. For example, the 27 BS-Natural Science students who took the ETS exam scored at or above the 60 th
percentile in all subcategories except Organismal Biology (15 th percentile). It would appear that we need to focus on such course offerings for our biology program in future. By contrast, BS-Natural Science students scored highest in the
Ecology/Evolution subcategory (90 th
percentile).
BS Natural Sciences Educational Effectiveness Assessment Plan Page 13 of 13