This document provides faculty with a short introduction to “TracDat”, the on-line system that helps units to manage, organize and report on student learning assessment activities. Reporting on Student Learning Assessment Plans: An Introduction to TracDat The Office of Institutional Research and Assessment Introduction For almost ten years, AU departments have developed learning outcomes for their programs, and have assessed how well the learning outcomes have been met. AU has documented these efforts in annual assessment reports using templates in Microsoft Word. While templates have changed over the years to better meet faculty needs, some challenges remained. Sometimes it has been difficult to know who has the latest official copy of the plan. Compiling support documents, updating the plan during the year, and sharing results throughout the year have not been easy. In order to make the assessment process as clear and efficient as possible, AU has utilizing a new online data management tool. The new software, called TracDat, is configured specifically to facilitate assessment reporting and program improvement. It enables departments to: Update, change or archive student learning outcomes on-line, at any time; Document assessment plans, including any supporting documentation (such as rubrics, survey instruments, or evaluation notes); Tie assessment results to actions taken by the department; Develop and print reports at any time; Have a centralized location for the department to see feedback that has been given by the Learning Outcomes and Assessment Team; and Submit the annual report electronically. The purpose of this document is to provide academic units with a basic introduction to TracDat so that you can submit your annual assessment report electronically. Throughout the upcoming year, more information about the many capabilities of the software will be available. 1 Basic Instructions for the Annual Report: FAQs How do I access TracDat? The Office of Institutional Research and Assessment will provide assessment representatives in each department access TracDat. Simply go to http://american.TracDat.com, enter the first part of your email address (before the @) as the ID and your AU four-digit telephone extension as the password. If you are unsure if OIRA has your assessment representative contact information or if you need help logging in please contact Karen Froslid Jones at kfroslid@american.edu or AULearning@american.edu. How do I change my password? To change the password used to login, click the “Home” tab in the top bar, then click the “Profile” sub-tab. Finally, click the “Change Password” button at the bottom of the page. Who can see my report and access my assessment results? Currently, you and any other people you designate for assessment from your department, along with representatives of the Senate Committee on Learning Outcomes and Assessment, have access to your TracDat reports. Some schools/colleges also designate administrators as a resource as you enter/submit your assessment reports. Deans do not have access to TracDat at this time. Will others be able to view the results? All AU assessment plans (consisting of learning outcomes and assessment methods) are posted on AU’s assessment website so that they are available to the public. (This is considered a best practice by accreditors.) TracDat has reporting capability that will help us prepare a PDF of your plan. Currently, the actual results of your assessments are only available to you and the Committee. You may decide how you share it with others. Are assessment plans already entered into TracDat? Your unit’s learning outcomes and assessment methods have been entered into TracDat. We have not entered your past assessment results. You are not required to fill in TracDat with past assessment results but you may do so if you wish. Can I see what I reported in the past, including the assessments that I have done? Is there a place where I can see my past report/submission? Yes, a copy of your past submission is included in the “Documents” tab of TracDat. Here you will find document folders for “Assessment Reports” (to deposit current and past reports), “Supporting Documents” (to deposit any supporting documentation you wish to submit) by year, and 2 “Feedback/Communication from COLA” (see any relevant communication between your department and the Committee on Learning Assessment). How is the reporting different than what we have done using the Microsoft Word template? The 2010 Word template organized your assessment plan around each assessment method and TracDat is organized around the learning outcomes. TracDat will require you to tie each assessment result explicitly to one or more learning outcomes. Units who have reported on assessment results more generally in the past may need to be more systematic about how they make connections between the assessments and their learning outcomes. The expectation that assessment results be tied to learning outcomes is not new. TracDat facilitates this connection, enabling you to provide a clearer picture of how one’s learning outcomes are being met and the steps taken to improve one’s program. The other big difference is that TracDat allows you to attach documents to your report. It may be possible to write a shorter report and include supporting documentation for those who want more indepth information. If TracDat is organized by learning outcomes, how do I enter assessment results about things not tied to learning outcomes? In the past, some departments have used the annual assessment report as an opportunity to report on assessments more broadly, such as assessments of student satisfaction or evaluations of course offerings. These assessments are an important part of the department’s overall improvement efforts but they are not the focus of the Senate Committee on Learning Assessment (COLA). Such information will not be collected as part of the annual assessment report. At a later point, we will expand the use of TracDat to enable you to add non-learning outcome goals, such as student satisfaction. We will share more information about how to add this information after the initial student learning assessment data collection is complete. In the meantime, if you would like to report on your assessments in other areas, please feel free to attach a summary of such assessments in a folder under the document tab of TracDat. What is the reasoning behind attaching ‘documentation’? The ability to attach documents (or reference websites) has several advantages. It provides you with a place to house or store the rubrics or assessment instructions you use. That way, they’re easy to find next time you want to do the assessment. Documentation enables the COLA to better understand your assessment processes, thereby enabling it to give you useful feedback. The more you can document your assessment processes, the better. 3 Documentation enables units to attach more lengthy assessment results (such as survey results) so that it has a back-up for the description of findings. There is no need to write any long text for your report. Simply upload the documentation that your department has already collected or produced. How do I submit the report? There is no need to print out a copy of your report and submit a paper version. On the “Assessment Unit” tab, “General” subtab is the question “Is 2011 Report Ready for Submission?” You can pick “yes”. Simply email AULearning@american.edu with the message that your latest information has been entered into TracDat and is ready for review. How do I get help with TracDat? The Office of Institutional Research and Assessment will provide you with support as you transition to assessment reporting in TracDat. Please email us at AULearning@american.edu with any questions or concerns or phone Karen Froslid Jones at 202-885-6155. In addition, COLA members are trained on TracDat, as well as some key administrators, especially in CAS. 4 STEP ONE: Log on to TracDat Go to: http://american.TracDat.com. Your user name is the first part of your email address. The password is your AU telephone extension (a four digit number). STEP TWO: Choose the Assessment Plan You Would Like to Update Choose from the drop-down menu the assessment plan that you would like to update (see blue arrow below). You should be able to see all assessment plans associated with your department. If you need to enter an entirely new plan not listed, please contact an AU TracDat administrator listed at the end of this training document. He or she will give you the information you need to enter a new plan. 5 STEP THREE: Update the Plan’s Contact Information. Once you have chosen a program to update, you’ll notice that TracDat has a series of blue tabs and each tab has subtabs. We’ll be going through each of these tabs to update information. Click on the “Assessment Unit” Tab (see blue arrow above) and the “General” subtab to enter the name and contact information of the person who completes the report. This will enable members of the Committee on Learning Assessment to know who to contact if they have any questions. If your program has a “mission statement” you may enter it here. This isn’t required. You’ll notice that COLA has added some other questions/information here that is intended to help with the review of your report. One of these is a question about whether the report is ready for review. Please check “yes” when the report is ready for submission. The “Personnel” subtab located on this page lists the people who have access to your assessment plan. Usually, this will include some members of the Committee on Learning Assessment. If you would like more people to be granted access to the plan, contact Karen Froslid Jones at kfroslid@american.edu. 6 The “Courses” subtab lists courses that are associated with your program in the AU Catalog. For now, you do not need to do anything with this information. This will be a resource for you should you later like to link specific courses with specific program learning outcomes. STEP FOUR: Review Your Learning Outcomes Next, click on the tab marked “Plan” (see blue arrow below) and make sure that the “Learning Outcomes” subtab (seen below in yellow) is open. Review your learning outcomes to make sure that they are still accurate. If no changes are needed, move to step five. To add a new learning outcome to the list, go here. Go here to change the wording of a learning outcome or to archive a learning outcome. (Don’t delete learning outcomes!) Note: You’ll notice that all learning outcomes have been given a quick name that can be used to identify the learning outcome (this is in the left column.) You may change this, if you would like, by choosing “edit” next to the “outcome status” column. The assumption is that all learning outcomes listed in your 2010 report are still active. If you have a learning outcome listed that you no longer which to have included, archive it by going into ‘edit’ 7 next to the “outcome status” column and change the outcome status to archived. Please don’t delete the outcome, since we would like to keep a record of the changes made over the years. If you go in to edit a learning outcome, it will look like this: Hold down the Control Key on your computer, you can highlight more than one year The start date for most learning outcomes is defaulted to the date of last year’s report. You may change this if you know when the outcome was first approved by your department. You only need to provide an “End Date” when you archive an old learning outcome. For new learning outcomes you may leave this field blank. STEP FIVE: Review Your Assessment Methods Click on the “Plan” tab (blue arrow below) and the “Assessment Plan” subtab to see the assessment methods associated with each of your learning outcomes. You’ll notice that this is one place where TracDat is a little different than AU’s old paper reporting system in that it more directly ties each assessment method to each learning outcome. 8 At the top of the page you’ll notice that there is a drop down menu for each of your learning outcomes, using the short name that was given to the learning outcome. Any assessment methods associated with the learning outcome will be listed below, along with the target that you set, and whether it is still active. To add an entirely new assessment method go here. Remember to make sure that the correct learning outcome is listed (above). The assessment methods, along with the target or standard you’ve set for success, are here. Assessment Methods Used for More Than One Learning Outcome: Many programs might use the same assessment method (such as a survey or a review of a student paper) for more than one learning outcome. How is this reflected in TracDat? TracDat allows you to copy the assessment method to other learning outcomes using the “copy” function on the right-side of this page. AU has already done this for you if your assessment plan included an assessment method used for multiple learning outcomes. (Please note: to avoid duplication when using the copy function, you need to first unlink the assessment method from the original learning outcome and then select those outcomes you wish to link.) To make the best use of TracDat, consider writing/revising your assessment methods and targets to more specifically address the learning outcome. For example, instead of “student survey” as the assessment measure, you can edit it to mention the specific question on the survey you’ll use to assess the learning outcome. Instead of saying you’ll use a rubric to review a paper, you might be able to say what, specifically, you’re looking for in the student work. 9 Tips for Adding/Modifying/Changing Your Assessments AU has tried to categorize your assessment method for you. You can change this at any time. Suggestions for new categories are welcome. How often will the assessments take place? Be sure that the schedule for this assessment method is accurate. Do you have assessments that are not tied directly to a specific learning outcome? The goal of the Learning Outcomes and Assessment Committee is to focus specifically on assessment of learning outcomes. (This is the only reporting required in the annual report.) The TracDat system enables one to track other outcomes (such as student satisfaction) in addition to your program’s learning outcomes. Lessons on doing this will be available after the initial student learning assessment reports are submitted. 10 STEP SIX: Report Your Assessment Results For programs that have made no changes to their assessment plan, and only want to report on the assessments they have done, this is the main place for you to enter information. Choose the “Assessment Results” Tab and the “By Unit” Subtab. You’ll see a quick summary of each of your learning outcomes listed here. Click on “Add Assessment Results” at the bottom of the page. As you add assessment results, you will be able to see them here. TracDat does not currently include any of your past assessment results but you may add them in if you would like. The button to add assessment results is here Next, select the learning outcome associated with the assessment you completed. If your assessment provides you with insight into more than on learning outcome, you will want to enter results in each learning outcome. Select the learning outcome for which you would like to report assessment activity here. 11 A pop up screen will then ask you to pick the assessment method you want to report on. If you do not see the assessment method you need, it is because it is not associated with the learning outcome as part as your plan. Simply go back to “Step Four” and either add the assessment method (if it isn’t in the system at all) or copy an assessment method so that it applies to the correct learning outcome. Select the assessment method on which you would like to report here. Entering Result Data: Report on what you learned as a result of your assessments in the “Assessment Result” section (see blue arrow below). Please be specific about how the result relates to your target. (This information was usually entered in the second column of "Section Three" in the former template.) TracDat requires that you categorize your result under “Summary of Results” by whether the results demonstrate that you met your target. This information is a tool to help you track your progress and to do broad university-wide summaries of overall university progress. Finding a gap between your expectations and assessment results is quite common, so there is no need for concern should the assessment findings be that you have not meet your goal. TracDat also requires you to indicate whether the results are leading to further action. This is in the section “Observation Status”. Don’t forget to save your changes. 12 After you have entered all your data, click “add Use of Finding” to report on how you intend to use your findings (see red arrow). Please follow the instructions for Step Six below. These items are preprepopulate d based on the earlier steps completed. Put the date that you actually did the assessment. You can use this later to organize reports. This drop-down menu asks you to categorize your intended follow-up actions. You may choose between: Action Plan Being Implemented; Action Plan Completed; No Action Plan Necessary. STEP SIX: Report How You Plan to Use the Results As you’ll note in the screen shot above, there is a place at the very bottom of the assessment report to document how you plan to use the results (see red arrow on previous page). This part asks you to explain how you are using your findings to improve your curriculum or program. You can provide examples of changes you have made or are making. (Note: If you are familiar with the Microsoft Word template, this information was previously in the third column of "Section Three" of the template.) 13 Report on how you plan to use findings here TIPS FOR THIS SECTION: When you report on how you plan to use the findings be as specific as possible, including adding information on whether someone from your department has been tasked with follow up, whether a course is being revised, etc. Feel free to reference how this relates to the work you are doing to improve other learning outcomes, if you making broad changes that impact more than one expected outcome. 14 STEP SEVEN: Deposit Assessment Documents TracDat allows you to deposit and archive documents relevant to your assessment activity in one central location. Simply go to the “Document” tab (see blue arrow below) and the “Document Repository” subtab and select one of the following folders to store your documents: “Assessment Reports” (here you may store older assessment reports or new ones generated by TracDat) “Supporting Documents” by academic year (here you may store a wide variety of documentation relevant to your program’s assessment activity, including surveys, rubrics or other tools you use for assessment. You may want to save statistics or summaries that document your findings or notes on departmental discussions or resolutions regarding assessment). First choose the appropriate folder, then click “Add New Document” at the bottom of the page and a pop-up window will prompt you to name, describe, and upload your file. Choose from these folders to deposit your assessment documents. Click “Add New Document” and upload and name your file Note: You will also find a folder titled “Feedback/Communication from Cola” here. This is the place in which the Committee on Learning and Assessment will store any relevant communication with your department. 15 STEP Eight: Run Assessment Reports TracDat enables you to automatically generate comprehensive assessment reports. Under the “Reports” tab you’ll find a series of report options. Try running the “Program Assessment Plan” report to see your assessment plan without results. Try running the “Program Assessment Report –Four Column” to see a report that includes your assessment results. Reports can be run in different formats, such as PDF and Excel. Try these options to generate assessment reports. STEP NINE: Report Completion of Your Report to the Learning Outcomes and Assessment Team Once you have entered your latest assessment results into TracDat and you consider your report complete, please go to the Assessment Unit Tab, “General” subtab and check “yes” to the question “Is 2011 Report Ready for Submission?” Email AULearning@american.edu and let us know. The Learning Outcomes and Assessment Team will be able to access your report directly from TracDat and will contact you if they have any questions. THANK YOU! 16