Document 13261414

advertisement
This document provides faculty with a
short introduction to “TracDat”, the on-line
system that helps units to manage,
organize and report on student learning
assessment activities.
Reporting on
Student
Learning
Assessment
Plans: An
Introduction to
TracDat
The Office of Institutional Research and Assessment
Introduction
For almost ten years, AU departments have developed learning outcomes for their programs, and
have assessed how well the learning outcomes have been met. AU has documented these efforts in
annual assessment reports using templates in Microsoft Word. While templates have changed over
the years to better meet faculty needs, some challenges remained. Sometimes it has been difficult to
know who has the latest official copy of the plan. Compiling support documents, updating the plan
during the year, and sharing results throughout the year have not been easy.
In order to make the assessment process as clear and efficient as possible, AU has utilizing a new
online data management tool. The new software, called TracDat, is configured specifically to facilitate
assessment reporting and program improvement. It enables departments to:






Update, change or archive student learning outcomes on-line, at any time;
Document assessment plans, including any supporting documentation (such as rubrics, survey
instruments, or evaluation notes);
Tie assessment results to actions taken by the department;
Develop and print reports at any time;
Have a centralized location for the department to see feedback that has been given by the
Learning Outcomes and Assessment Team; and
Submit the annual report electronically.
The purpose of this document is to provide academic units with a basic introduction to TracDat so
that you can submit your annual assessment report electronically. Throughout the upcoming year,
more information about the many capabilities of the software will be available.
1
Basic Instructions for the Annual Report: FAQs
How do I access TracDat?
The Office of Institutional Research and Assessment will provide assessment representatives in each
department access TracDat. Simply go to http://american.TracDat.com, enter the first part of your
email address (before the @) as the ID and your AU four-digit telephone extension as the password. If
you are unsure if OIRA has your assessment representative contact information or if you need help
logging in please contact Karen Froslid Jones at kfroslid@american.edu or
AULearning@american.edu.
How do I change my password?
To change the password used to login, click the “Home” tab in the top bar, then click the “Profile”
sub-tab. Finally, click the “Change Password” button at the bottom of the page.
Who can see my report and access my assessment results?
Currently, you and any other people you designate for assessment from your department, along with
representatives of the Senate Committee on Learning Outcomes and Assessment, have access to your
TracDat reports. Some schools/colleges also designate administrators as a resource as you
enter/submit your assessment reports. Deans do not have access to TracDat at this time.
Will others be able to view the results?
All AU assessment plans (consisting of learning outcomes and assessment methods) are posted on
AU’s assessment website so that they are available to the public. (This is considered a best practice by
accreditors.) TracDat has reporting capability that will help us prepare a PDF of your plan. Currently,
the actual results of your assessments are only available to you and the Committee. You may decide
how you share it with others.
Are assessment plans already entered into TracDat?
Your unit’s learning outcomes and assessment methods have been entered into TracDat. We have
not entered your past assessment results. You are not required to fill in TracDat with past
assessment results but you may do so if you wish.
Can I see what I reported in the past, including the assessments that I have done? Is there a place
where I can see my past report/submission?
Yes, a copy of your past submission is included in the “Documents” tab of TracDat. Here you will find
document folders for “Assessment Reports” (to deposit current and past reports), “Supporting
Documents” (to deposit any supporting documentation you wish to submit) by year, and
2
“Feedback/Communication from COLA” (see any relevant communication between your department
and the Committee on Learning Assessment).
How is the reporting different than what we have done using the Microsoft Word template?
The 2010 Word template organized your assessment plan around each assessment method and
TracDat is organized around the learning outcomes. TracDat will require you to tie each assessment
result explicitly to one or more learning outcomes. Units who have reported on assessment results
more generally in the past may need to be more systematic about how they make connections
between the assessments and their learning outcomes. The expectation that assessment results be
tied to learning outcomes is not new. TracDat facilitates this connection, enabling you to provide a
clearer picture of how one’s learning outcomes are being met and the steps taken to improve one’s
program.
The other big difference is that TracDat allows you to attach documents to your report. It may be
possible to write a shorter report and include supporting documentation for those who want more indepth information.
If TracDat is organized by learning outcomes, how do I enter assessment results about things not
tied to learning outcomes?
In the past, some departments have used the annual assessment report as an opportunity to report
on assessments more broadly, such as assessments of student satisfaction or evaluations of course
offerings. These assessments are an important part of the department’s overall improvement efforts
but they are not the focus of the Senate Committee on Learning Assessment (COLA). Such
information will not be collected as part of the annual assessment report. At a later point, we will
expand the use of TracDat to enable you to add non-learning outcome goals, such as student
satisfaction. We will share more information about how to add this information after the initial
student learning assessment data collection is complete. In the meantime, if you would like to report
on your assessments in other areas, please feel free to attach a summary of such assessments in a
folder under the document tab of TracDat.
What is the reasoning behind attaching ‘documentation’?
The ability to attach documents (or reference websites) has several advantages.


It provides you with a place to house or store the rubrics or assessment instructions you use. That
way, they’re easy to find next time you want to do the assessment.
Documentation enables the COLA to better understand your assessment processes, thereby
enabling it to give you useful feedback. The more you can document your assessment processes,
the better.
3

Documentation enables units to attach more lengthy assessment results (such as survey results)
so that it has a back-up for the description of findings. There is no need to write any long text for
your report. Simply upload the documentation that your department has already collected or
produced.
How do I submit the report?
There is no need to print out a copy of your report and submit a paper version. On the “Assessment
Unit” tab, “General” subtab is the question “Is 2011 Report Ready for Submission?” You can pick “yes”.
Simply email AULearning@american.edu with the message that your latest information has been
entered into TracDat and is ready for review.
How do I get help with TracDat?
The Office of Institutional Research and Assessment will provide you with support as you transition to
assessment reporting in TracDat. Please email us at AULearning@american.edu with any questions or
concerns or phone Karen Froslid Jones at 202-885-6155.
In addition, COLA members are trained on TracDat, as well as some key administrators, especially in
CAS.
4
STEP ONE: Log on to TracDat
Go to: http://american.TracDat.com.
Your user name is the first part of your email address. The password is your AU telephone extension
(a four digit number).
STEP TWO: Choose the Assessment Plan You Would Like to Update
Choose from the drop-down menu the assessment plan that you would like to update (see blue arrow
below). You should be able to see all assessment plans associated with your department. If you need
to enter an entirely new plan not listed, please contact an AU TracDat administrator listed at the end
of this training document. He or she will give you the information you need to enter a new plan.
5
STEP THREE: Update the Plan’s Contact Information.
Once you have chosen a program to update, you’ll notice that TracDat has a series of blue tabs and
each tab has subtabs. We’ll be going through each of these tabs to update information.
Click on the “Assessment Unit” Tab (see blue arrow above) and the “General” subtab to enter the
name and contact information of the person who completes the report. This will enable members of
the Committee on Learning Assessment to know who to contact if they have any questions. If your
program has a “mission statement” you may enter it here. This isn’t required.
You’ll notice that COLA has added some other questions/information here that is intended to help
with the review of your report. One of these is a question about whether the report is ready for
review. Please check “yes” when the report is ready for submission.
The “Personnel” subtab located on this page lists the people who have access to your assessment
plan. Usually, this will include some members of the Committee on Learning Assessment. If you
would like more people to be granted access to the plan, contact Karen Froslid Jones at
kfroslid@american.edu.
6
The “Courses” subtab lists courses that are associated with your program in the AU Catalog. For now,
you do not need to do anything with this information. This will be a resource for you should you later
like to link specific courses with specific program learning outcomes.
STEP FOUR: Review Your Learning Outcomes
Next, click on the tab marked “Plan” (see blue arrow below) and make sure that the “Learning
Outcomes” subtab (seen below in yellow) is open. Review your learning outcomes to make sure that
they are still accurate. If no changes are needed, move to step five.
To add a new learning
outcome to the list, go
here.
Go here to change the
wording of a learning
outcome or to archive
a learning outcome.
(Don’t delete learning
outcomes!)
Note:


You’ll notice that all learning outcomes have been given a quick name that can be used to identify
the learning outcome (this is in the left column.) You may change this, if you would like, by
choosing “edit” next to the “outcome status” column.
The assumption is that all learning outcomes listed in your 2010 report are still active. If you have
a learning outcome listed that you no longer which to have included, archive it by going into ‘edit’
7
next to the “outcome status” column and change the outcome status to archived. Please don’t
delete the outcome, since we would like to keep a record of the changes made over the years.
If you go in to edit a learning outcome, it will look like this:
Hold down the Control
Key on your computer,
you can highlight more
than one year
The start date for most learning
outcomes is defaulted to the date
of last year’s report. You may
change this if you know when the
outcome was first approved by
your department.
You only need to provide an
“End Date” when you archive an old
learning outcome. For new learning
outcomes you may leave this field blank.
STEP FIVE: Review Your Assessment Methods
Click on the “Plan” tab (blue arrow below) and the “Assessment Plan” subtab to see the assessment
methods associated with each of your learning outcomes. You’ll notice that this is one place where
TracDat is a little different than AU’s old paper reporting system in that it more directly ties each
assessment method to each learning outcome.
8
At the top of the page you’ll notice that there is a drop down menu for each of your learning
outcomes, using the short name that was given to the learning outcome. Any assessment methods
associated with the learning outcome will be listed below, along with the target that you set, and
whether it is still active.
To add an entirely new
assessment method go
here. Remember to make
sure that the correct
learning outcome is listed
(above).
The assessment
methods, along with
the target or standard
you’ve set for success,
are here.
Assessment Methods Used for More Than One Learning Outcome:
Many programs might use the same assessment method (such as a survey or a review of a student
paper) for more than one learning outcome. How is this reflected in TracDat? TracDat allows you to
copy the assessment method to other learning outcomes using the “copy” function on the right-side
of this page. AU has already done this for you if your assessment plan included an assessment
method used for multiple learning outcomes. (Please note: to avoid duplication when using the copy
function, you need to first unlink the assessment method from the original learning outcome and
then select those outcomes you wish to link.)
To make the best use of TracDat, consider writing/revising your assessment methods and targets to
more specifically address the learning outcome. For example, instead of “student survey” as the
assessment measure, you can edit it to mention the specific question on the survey you’ll use to
assess the learning outcome. Instead of saying you’ll use a rubric to review a paper, you might be able
to say what, specifically, you’re looking for in the student work.
9
Tips for Adding/Modifying/Changing Your Assessments
AU has tried to
categorize your
assessment method for
you. You can change
this at any time.
Suggestions for new
categories are
welcome.
How often will the
assessments take
place? Be sure that the
schedule for this
assessment method is
accurate.
Do you have assessments that are not tied directly to a specific learning outcome?
The goal of the Learning Outcomes and Assessment Committee is to focus specifically on assessment
of learning outcomes. (This is the only reporting required in the annual report.) The TracDat system
enables one to track other outcomes (such as student satisfaction) in addition to your program’s
learning outcomes. Lessons on doing this will be available after the initial student learning
assessment reports are submitted.
10
STEP SIX: Report Your Assessment Results
For programs that have made no changes to their assessment plan, and only want to report on the
assessments they have done, this is the main place for you to enter information. Choose the
“Assessment Results” Tab and the “By Unit” Subtab. You’ll see a quick summary of each of your
learning outcomes listed here.

Click on “Add Assessment Results” at the bottom of the page.
As you add assessment
results, you will be able to
see them here.
TracDat does not currently
include any of your past
assessment results but you
may add them in if you
would like.
The button to add
assessment results is here

Next, select the learning outcome associated with the assessment you completed. If your
assessment provides you with insight into more than on learning outcome, you will want to
enter results in each learning outcome.
Select the learning outcome
for which you would like to
report assessment activity
here.
11

A pop up screen will then ask you to pick the assessment method you want to report on. If
you do not see the assessment method you need, it is because it is not associated with the
learning outcome as part as your plan. Simply go back to “Step Four” and either add the
assessment method (if it isn’t in the system at all) or copy an assessment method so that it
applies to the correct learning outcome.
Select the assessment
method on which you
would like to report
here.



Entering Result Data:
Report on what you learned as a result of your assessments in the “Assessment Result”
section (see blue arrow below). Please be specific about how the result relates to your target.
(This information was usually entered in the second column of "Section Three" in the former
template.)
TracDat requires that you categorize your result under “Summary of Results” by whether the
results demonstrate that you met your target. This information is a tool to help you track your
progress and to do broad university-wide summaries of overall university progress. Finding a
gap between your expectations and assessment results is quite common, so there is no need
for concern should the assessment findings be that you have not meet your goal.
TracDat also requires you to indicate whether the results are leading to further action. This is
in the section “Observation Status”. Don’t forget to save your changes.
12

After you have entered all your data, click “add Use of Finding” to report on how you intend
to use your findings (see red arrow). Please follow the instructions for Step Six below.
These items
are preprepopulate
d based on
the earlier
steps
completed.
Put the date that
you actually did the
assessment. You
can use this later to
organize reports.
This drop-down menu asks you to categorize
your intended follow-up actions. You may
choose between: Action Plan Being
Implemented; Action Plan Completed; No
Action Plan Necessary.
STEP SIX: Report How You Plan to Use the Results
As you’ll note in the screen shot above, there is a place at the very bottom of the assessment report
to document how you plan to use the results (see red arrow on previous page). This part asks you to
explain how you are using your findings to improve your curriculum or program. You can provide
examples of changes you have made or are making. (Note: If you are familiar with the Microsoft
Word template, this information was previously in the third column of "Section Three" of the
template.)
13
Report on how
you plan to use
findings here
TIPS FOR THIS SECTION:
When you report on how you plan to use the findings be as specific as possible, including adding
information on whether someone from your department has been tasked with follow up, whether a
course is being revised, etc. Feel free to reference how this relates to the work you are doing to
improve other learning outcomes, if you making broad changes that impact more than one expected
outcome.
14
STEP SEVEN: Deposit Assessment Documents
TracDat allows you to deposit and archive documents relevant to your assessment activity in one
central location. Simply go to the “Document” tab (see blue arrow below) and the “Document
Repository” subtab and select one of the following folders to store your documents:


“Assessment Reports” (here you may store older assessment reports or new ones generated
by TracDat)
“Supporting Documents” by academic year (here you may store a wide variety of
documentation relevant to your program’s assessment activity, including surveys, rubrics or
other tools you use for assessment. You may want to save statistics or summaries that
document your findings or notes on departmental discussions or resolutions regarding
assessment).
First choose the appropriate folder, then click “Add New Document” at the bottom of the page and a
pop-up window will prompt you to name, describe, and upload your file.
Choose from these folders to
deposit your assessment
documents.
Click “Add New Document” and upload
and name your file
Note: You will also find a folder titled “Feedback/Communication from Cola” here. This is the place in
which the Committee on Learning and Assessment will store any relevant communication with your
department.
15
STEP Eight: Run Assessment Reports
TracDat enables you to automatically generate comprehensive assessment reports. Under the
“Reports” tab you’ll find a series of report options. Try running the “Program Assessment Plan” report
to see your assessment plan without results. Try running the “Program Assessment Report –Four
Column” to see a report that includes your assessment results. Reports can be run in different
formats, such as PDF and Excel.
Try these options to generate assessment
reports.
STEP NINE: Report Completion of Your Report to the Learning Outcomes and
Assessment Team
Once you have entered your latest assessment results into TracDat and you consider your report
complete, please go to the Assessment Unit Tab, “General” subtab and check “yes” to the question
“Is 2011 Report Ready for Submission?” Email AULearning@american.edu and let us know. The
Learning Outcomes and Assessment Team will be able to access your report directly from TracDat
and will contact you if they have any questions.
THANK YOU!
16
Download