Assessment Methods
The Advanced GIS Project senior experience course along with curriculum mapping is our current approach to assessment of the GIS Concentration within the Land Use Program. First the GIS
Concentration goals were mapped to student learning objectives (SLOs) for required GIS courses. Then student success in completion of the senior experience was evaluated. Areas of strength as well as deficiencies were determined.
Course Mapping
Our GIS Program is comprehensive with seven goals and numerous courses addressing those goals. For most goals more than one course reinforces those goals . Repetition and elaboration on key concepts and skills help students gain a good deal of proficiency in GIS in the areas of data models, cartography, remote sensing, data management, data analysis, and the application of scientific methodology (Table 1).
Table 1: GIS Concentration Goals Relative to SLOs of GIS Course Requirements
GIS Concentration Goals
GIS
2250
GIS
3250
GIS
2710
GIS
4840
GIS
4850
GIS
4960
1. Understand GIS principles including raster and vector data models and formats, map projections, and analysis functions and demonstrate proficiency in their application to geospatial problems x x x x x 2. Create quality cartographic products x
3. Understand remote sensing principles, photogrammetry and image interpretation, and image processing and demonstrate proficiency in their application to geospatial problems.
4. Create spatial data and metadata, determine analysis needs, and conduct spatial analysis x x x x x 5. Model spatial processes x
6. Design and troubleshoot spatial databases to meet enterprise objectives.
7. Initiate, manage, and evaluate GIS projects using scientific methods x
GIS
4870
GIS
4880
GIS
4890 x x x x x x x x x x x x x x
Senior Experience
The senior experience is an important course in a student’s academic training. Many of the duties specified for GIS students could apply to other concentrations within the Land Use Program. The development of a uniform set of core duties and proficiencies related to scientific inquiry is a critical

component of this experience in addition to communication and demonstration of discipline-specific skills.
Senior experience student expectations are to:
A.
Demonstrate the ability to focus a scientific research topic, organize information, design a study, analyze data, and evaluate a unique study or application
B.
Demonstrate communication skills through writing (scientific format with correct citations), speech (well-designed presentation with clear delivery), and graphics (maps and other visualizations).
C.
Reinforce and demonstrate the knowledge, skills, and abilities learned within a program of study
Student proficiencies were evaluated using a rubric based on scientific reports, writing, and visualization objectives (Table 2). Other GIS skills were evaluated on an individual basis with creativity, effort, analytic reasoning, and project difficulty being evaluation elements. As Table 2 indicates most students were successful at framing a research problem. They improved since 2013 in framing their objectives and developing methods sequences to achieve those objectives. They also improved in describing their results in writing as well as through cartographic visualization. General writing skills and paragraph construction was excellent within this group of students.
Table 3 shows the spring 2014 rubric that is was more specific as well as more instructive for students although the same key categories were evaluated. Students were given a more detailed rubric early on in the semester which they could consult as a guide to developing their research paper. However, they fell short evaluating their project results in the discussion section. This may be due in part to the more specific language about what information the discussion should provide so that my evaluation was using a higher expectation standard for the discussion section than the previous year. Failing to cite figures and tables properly is a constant problem that seems to persist regardless of instruction.
Table 2: Specific Scientific Paper Content Objectives (from
2013)
Introduction section explains context of problem and relevant studies
Objectives clearly state of the purpose of a well-focused project
Category
Introduction and
Background
Topic, Question,
Problem
Mean percent of possible points
2013
Mean percent of possible points
2014
0.88
0.73
0.87
0.94
Methods are very clear so another person could duplicate them
Results include verbal descriptions of the spatial characteristics of features on the maps as well as information about the features
Methods
Results
0.77
0.68
0.96
0.91
Discussion of results and conclusions covers usefulness and limitations of methods to achieve purpose
Discussion of project
Conclusions
Discussion
0.83
0.83
0.57
0.75

Specific Writing and Construction Objectives
Sentences and paragraphs are well-constructed with consistent tense
Category
Paragraph
Construction
No spelling errors Mechanics
Methods are written in narrative form (in paragraphs not bulleted) Structure
Figure headings below figures and table headings above tables
Figures and tables properly cited in text
Tables summarize data when appropriate
Structure
Structure
Structure
0.90
0.99
0.95
0.95
0.96
0.96
Specific Cartographic Product Objectives
At least 2 maps are included in the project with elements below
Map Elements
Good choice of symbols and/or colors to show spatial variances
Category
Visualization
Visualization
Content
0.98
1.00
0.96
NA
NA
0.94
Good overall composition Visualization 0.89 0.90
Table 3: Scientific Research Grading
Rubric spring 2014
Content structure Content Excellent (7)
Acknowledgements
Abstract section is a one-paragraph summary of the project
Objectives clearly state of the purpose of a well-focused project
Introduction section explains context of problem and relevant studies
Acknowl edgemen ts
Abstract Abstract summarizes project well
Topic,
Question
,
Problem
Introduc tion
Section
Topic, question, and problem are clear and wellfocused.
Well-written and interesting, making the reader eager to continue reading
Very Good(6)
Abstract summarizes project adequately
The TPQ is clear, but could be refined or narrowed down.
Adequately written , but does not completely hold the reader's interest
Satisfactory (5)
Abstract summarizes project somewhat
The TPQ is somewhat clear and could be better refined.
Adequately written, but does not hold the reader's interest
Poor (2) do not exist
(0)
Abstract summarize s project poorly
The TPQ is not clear and too broad.
Is poorly written and does not entice the reader to continue
0.92
1.00
NA
NA
0.82
0.91

Data description includes overview of the datasets and includes metadata including the name, format, description, and source of each dataset (table format recommended)
Methods are very clear so another person could duplicate them
Correct flowcharts track all process that include input datasets, functions, and output datasets and coordinate with written methods narratives
Results include verbal descriptions of the spatial characteristics of features on the maps as well as information about the features
Conclusions covers usefulness and limitations of methods to achieve purpose as supported by the results
Data descripti on
Data description includes thorough overview of the datasets and metadata
Data description includes adequate overview of the datasets and metadata
Data description includes information about some datasets and metadata
Data description is inadequate or missing
Methods narrative
Well-defined and clearly explained methods with appropriate sampling, measures, and/or analysis sequences
Flowcha rt
Flowchart is well constructed and complete without errors
Mostly welldefined and clearly explained methods with mostly appropriate sampling, measures, and/or analysis sequences
Flowchart is mostly well constructed and complete without many errors
Somewhat welldefined and clearly explained methods with somewhat appropriate sampling, measures, and/or analysis sequences
Flowchart is somewhat well constructed and complete
Poorly defined and clearly explained methods with inappropria te sampling, measures, and/or analysis sequences
Flowchart is poorly constructed or missing
Results
Conclusi ons
Description of results are very comprehensive and easy to understand relative to objectives
Description of results are mostly comprehensive and easy to understand relative to objectives
Description of results are somewhat comprehensive and easy to understand relative to objectives
Summarize the research and its importance relative to hypotheses and/or objectives well
Generally summarize the research and its importance relative to hypotheses and/or objectives well
Summarize the research and its importance relative to hypotheses and/or objectives adequately
Description of results are notcompre hensive and easy to understand relative to objectives
Summarize the research and its importance relative to hypotheses and/or objectives poorly

Discussion of project evaluates the project objectives and methods. It looks for sources of error, evaluates assumptions and data sources, and proposes additonal studies. It looks for alternative explanations for the results.
Content communication
Headings and subheadings guide the reader through the project progression from one section to another
Section transitions are smooth
Sentences and paragraphs are wellconstructed with consistent tense
No spelling errors
Discussi on
Alternative explanations evaluated and addressed well
Alternative explanations generally evaluated and addressed well
Alternative explanations evaluated and addressed minimally
Alternative explanation s evaluated and addressed poorly
Content Excellent (7) Very Good(6) Satisfactory (5) Poor (2)
Headings and Subheadings
Structure
Paragrap h
Constructi on
Mechanic s
Paper sections are well defined for abstract, introduction, methods, results, conclusions, and discussion
All paragraphs within a section flow well from one to the next
All paragraphs include introductory sentence, explanations or details, and concluding sentence.
No grammatical, spelling or punctuation errors.
Paper sections are mostly well defined for abstract, introduction, methods, results, conclusions, and discussion
All paragraphs within a section generally flow well from one to the next
Most paragraphs include introductory sentence, explanations or details, and concluding sentence.
Almost no grammatical, spelling or punctuation errors
Paper sections aresomewhat well defined for abstract, introduction, methods, results, conclusions, and discussion
All paragraphs within a section flow somewhat from one to the next
Paper sections are poorly defined
All paragraphs within a section flow poorly
Paragraphs included related information but were typically not constructed well.
A few grammatical spelling, or punctuation errors.
Paragraphi ng structure was not clear and sentences were not typically related within the paragraphs.
Many grammatica l spelling, or punctuation errors.

Tables are always cited in the text before they are inserted in the paper, with table headings above the table.
Figures are always cited in the text with figure captions below the figure. Maps are figures.
Content visualization
Table and
Figure
Citations
Tables and figures are always cited properly
Content Excellent (7)
Tables and figures are usually cited properly
Tables and figures are sometimes cited properly
Tables and figures are not cited properly
Very Good(6) Satisfactory (5) Poor (2)
At least 2 maps are included in the project with elements below
Content
Visualizati on
Map or graphic contains important information that is easier to understand in a visual context.
Well-constructed visualizations
Map or graphic contains mostly important information that is easier to understand in a visual context.
Somewhat wellconstructed visualizations
Map or graphic contains somewhat important information that is easier to understand in a visual context.
Adequatelyconstructed visualizations
Graphic content is poor
Poorlyconstructed visualizatio ns
Maps include appropriately-sized figure, descriptive title, legend with unitsand easily interpreted symbols, metadata, north arrow, scale bar, and good composition, in a large enough paper size to read all text and symbols.
Tables are used to summarize results and/or methods when appropriate
Written Report Total
Tables Tables are used when needed, and are wellconstructed.
Tables are used sometimes and are wellconstructed.
Tables are seldom used and are somewhat wellconstructed.
Tables are not used or are poorly constructed
Peer Assessment: Since 2010 Senior Experience students have assessed one another’s proposals that include their literature review as well as their objectives, proposed methods, and expected results. Each student is required to assess 3-4 peers with grading being directed at participating in the assessment process not based on the student paper being assessed. This process helps to reinforce good scientific paper protocol and helps students share their research experiences.
Peer Feedback on Proposal. Each student will provide an assessment that includes the following information:


Sections/Topics that were clearly developed/interesting/creative/or intriguing

Sections/Topics that were difficult to understand/missing information/ or vague

Specific Questions

Specific Suggestions

Specific Concerns from Proposal Checklist
Summary of Senior Experience Assessment : Overall students were excellent in meeting the student learning objectives for the Program as well as performing the duties and tasks associated with their senior experience project. As for the paper content most students were adept at determining and executing methods and examining results. They were able to instigate a project of their own imagination, find the appropriate data and literature, develop methods and execute them. Their cartographic products were mostly excellent.
Additional Assessments: The same assessment rubric is now being applied to GIS 4860 student projects so that students become familiar with the scientific report format and correct documentation procedures prior to the GIS 4890 course and individual improvements can be tracked. This helps to demystify the senior experience expectations and give students more confidence that they can become involved in undergraduate research of their own creation.