In Blackboard Learn, you can use tests to measure student knowledge, gauge progress, and gather information from students. In this workshop, the main focus is on creating tests and questions, choosing appropriate settings, and adding tests to course areas. Along the way, you will learn some best practices for using tests as teaching tools and encouraging academic honesty. The Big Picture Create a Test © 2014 Blackboard Inc. 1 Table of Contents ACCESS TESTS .............................................................................................................................. 4 LEARN the LINGO ....................................................................................................................... 6 STEP ONE: CREATE the TEST ....................................................................................................... 7 ACCESS the TESTS TOOL ............................................................................................................ 8 ADD a TEST .................................................................................................................................. 9 SPECIFY QUESTION SETTINGS ................................................................................................... 10 OFFER PARTIAL CREDIT............................................................................................................. 10 USE NEGATIVE POINTS .............................................................................................................. 11 AWARD EXTRA CREDIT ............................................................................................................. 12 ADD QUESTIONS ....................................................................................................................... 14 QUESTION TYPES ....................................................................................................................... 16 ASSIGN POINTS ......................................................................................................................... 18 BEST PRACTICE ......................................................................................................................... 19 ORDER the QUESTIONS ............................................................................................................ 20 TRY IT ................................................................................................................................. 21 © 2014 Blackboard Inc. 2 BEST PRACTICE ......................................................................................................................... 22 STEP TWO: DEPLOY the TEST .................................................................................................... 24 ADD TESTS to CONTENT AREAS ............................................................................................... 25 TEST OPTIONS PAGE ................................................................................................................. 26 BEST PRACTICE ......................................................................................................................... 27 TRY IT ................................................................................................................................. 28 REUSE QUESTIONS ..................................................................................................................... 29 QUESTION POOLS ..................................................................................................................... 33 TRY IT ................................................................................................................................. 34 FREQUENTLY ASKED .................................................................................................................. 34 SPOTLIGHT on YOUR COURSE ................................................................................................. 35 GOING FURTHER ....................................................................................................................... 36 © 2014 Blackboard Inc. 3 ACCESS TESTS Students typically access tests from a content area and click the title of the test to access it. You can choose to provide information following the test title. Next, students click Begin and the test appears. If the test is timed, the timer starts when Begin is clicked. © 2014 Blackboard Inc. 4 Take Test Page Along with the information you write in the Description and Instructions area when creating the test, students are given information about the test options you chose, such as whether they can take the test multiple times, how long the time limit is, and whether they can continue beyond the specified time. Students see a collapsible status bar above the test questions, showing which questions they answered. If you set a timer for the test, it also appears in the status bar. Student View: Taking a Test © 2014 Blackboard Inc. 5 LEARN the LINGO Deploy a test Adding a test to a course area, such as a content area, learning module, lesson plan, or folder where students click its title to access it. Extra credit An extra credit correct answer results in adding the points listed in the question's Points box to the points earned for the test. An incorrect answer does not result in a point deduction. Questions designated as extra credit are removed from the total points calculation for the purpose of grading. Negative points You can penalize students' incorrect answers with negative points. This feature is most often used in Multiple Choice tests to discourage guessing. Partial credit When you enable partial credit, a portion of the total points for a question is awarded when an answer is not entirely correct. Question pool A collection of test questions that you can include in tests. Question set A collection of test questions retrieved from selected tests and pools. From this set, you specify how many questions to display. Question Settings After creating a test, you can determine the options available for creating test questions on the Question Settings page. For example, you can set options for scoring, feedback, images, metadata, and how questions are displayed to students. Random block A set of test questions retrieved at random from one or more question pools to be presented each time the test is taken. Test Canvas As you create a test, first you add the test information. Then, the Test Canvas appears, displaying the test name, description, and instructions. From this page, you create or find questions, select question settings, determine point values, and reorder questions. © 2014 Blackboard Inc. 6 STEP ONE: CREATE the TEST The first step in a test’s lifecycle is to create the test. When creating a test, you need to do the following actions: Add the test. Specify the question settings. Add the questions. Order the questions. © 2014 Blackboard Inc. 7 ACCESS the TESTS TOOL On the Control Panel, expand the Course Tools section and select Tests, Surveys, and Pools. On the Tests, Surveys, and Pools page, click Tests. The Tests page displays all existing tests. The Deployed column lists where you added the test in your course. You can also build new tests. © 2014 Blackboard Inc. 8 Quick Steps ADD a TEST The Facts When you add a test, you can include the following basic information: Name of the test. Description, which appears under the title in a content area and at the top of the test after it is started. Instructions, which appear at the top of the test after it is started. Do It 1. On the Tests page, click Build Test on the action bar. 2. On the Test Information page, type a Name, Description, and Instructions. You can use the content editor functions to format the text and include files, images, web links, multimedia, and mashups. 3. Click Submit. NOTE: A description and instructions are optional, but recommended. After adding the test information, the Test Canvas appears, displaying the test name, description, and instructions. The next step in the test creation process is selecting the question settings. Help Create Tests Test Canvas © 2014 Blackboard Inc. 9 Quick Steps SPECIFY QUESTION SETTINGS The Facts The second step in creating a test is to specify the test’s question settings. Question settings control the options available when creating test questions. Your selections include whether to provide the following information when creating questions: Feedback for individual answers Images, files, and web links in feedback and answers Question metadata, such as categories, topics, levels of difficulty, and keywords Scoring defaults, such as a default point value, or partial or negative credit for answers Specifics for how answers are displayed, such as random ordering of answers Do It 1. On the Control Panel, expand the Course Tools section and click Tests, Surveys, and Pools. 2. On the Tests, Surveys, and Pools page, click Tests. 3. On the Tests page, access the test's contextual menu and click Edit. 4. On the Test Canvas, click Question Settings on the action bar. 5. On the Test Question Settings page, make your selections. 6. Click Submit. Help Question Settings Question Metadata OFFER PARTIAL CREDIT © 2014 Blackboard Inc. 10 Quick Steps The When you enable partial credit, a Facts a question is awarded when an answer is not entirely correct. portion of the total points for You can enable partial credit during question creation or when editing a question. Partial credit is available for the following question types: Calculated Formula Fill in Multiple Blanks Jumbled Sentence Matching Multiple Answer Multiple Choice Opinion Scale/Likert Question Ordering Quiz Bowl For example, you include the following Multiple Choice question with four answers: "What is on a beach?" If the correct answer, "sand," is awarded 5 points, but another choice is "the ocean," you can set partial credit to 40%, awarding 2 points for that answer. If partial credit is enabled, but negative points is not, valid percentage values for a question are 0.0 to 100.0. Do It 1. On the Test Question Settings page, select Specify partial credit options for answers and click Submit. 2. For each appropriate question, select the check box for Allow Partial Credit. 3. For each incorrect answer, type a percentage in the Partial Credit % box and click Submit. Help Question Settings USE NEGATIVE POINTS © 2014 Blackboard Inc. 11 Quick Steps The Facts You can penalize students' incorrect answers with negative points. This feature is most often used in Multiple Choice tests to discourage guessing. For example, you might use negative points when creating a test in the medical field where a student’s incorrect answer could result in patient injury or death. You can enable the negative points option during question creation or when editing a question. Negative points for incorrect answers is available for the following question types: Matching Multiple answer Multiple choice For example, if a Multiple Choice question awards 5 points for a correct answer and 0 points for skipping it, you can set -20% (or -1 point) for each of the incorrect answers. Valid negative point percentage values for a question are -100.0 to -0.0. Do It 1. On the Test Question Settings page, select Specify negative points options for answers. NOTE: The negative points option appears only if you enabled the Specify partial credit options for answers option and click Submit. 2. For each appropriate question, select the check boxes for Allow Partial Credit and Allow Negative Scores for Incorrect Answers. 3. For each incorrect answer, type a negative percentage in the Partial Credit % box and click Submit. Help Question Settings AWARD EXTRA CREDIT © 2014 Blackboard Inc. 12 Quick Steps The Facts When you enable extra credit and apply it to an individual question, a correct answer results in adding the points listed in the question's Points box to the points earned for the test. An incorrect answer does not result in a point deduction. Questions designated as extra credit are removed from the total points calculation for the purpose of grading. If a student answers all questions correctly on a test with an extra credit question, he or she is awarded a score greater than 100% on the test. You enable extra credit after you create questions. Do It 1. On the Test Question Settings page, select Provide option to assign questions as extra credit and click Submit. 2. For each appropriate question, click the Points box. 3. In the pop-up box, select the check box for Extra Credit and click Submit. Help © 2014 Blackboard Inc. Question Settings 13 ADD QUESTIONS You can add new questions to your test in two ways. 1: Create Question Drop-down List On the Test Canvas, point to Create Question on the action bar to access the drop-down list. The questions are listed in alphabetical order. Select the type of question you want to add to your test. The question is added to the bottom of the list of questions. © 2014 Blackboard Inc. 14 2: Add Question Feature You can add new questions exactly where you want them on the Test Canvas. Click the plus sign before or after another question and choose a question type. You can also reuse questions from other tests or pools, and upload questions from your computer. These functions are presented last in this workshop. © 2014 Blackboard Inc. 15 QUESTION TYPES The following table summarizes the question types available in Blackboard Learn. To learn more about each question and see an example in Blackboard Help, click the link in the Question Types column. Question Types Description Calculated Formula Students are required to make a calculation and respond with a numeric answer. NEW: You can select the number of decimals or significant figures for the correct answers generated by the system. Calculated Numeric Students are required to make a numeric answer. Either/Or Students are asked to respond using a selection of predefined twochoice answers. Essay Students write a detailed response to a question or statement. File Response Students are required to upload a file from their computers or from Course Files/Content Collection as the answer to the question. Fill in the Blank Students type text or numbers to complete a statement. Fill in Multiple Blanks Students complete the sentence by typing the appropriate word or phrase for each blank. Hot Spot Students are presented with an image and are asked to select a particular area. Jumbled Sentence Students complete a sentence by selecting words or phrases from a drop-down list. Matching Students are presented with two lists of items and must pair items from one list with the other list. Multiple Choice © 2014 Blackboard Inc. Students select the single correct answer from a series of answers. 16 Question Types Description Multiple Answer Students select one or more correct answers from a series of answers. Opinion Scale/Likert Students answer based on attitudes or reactions using a comparable scale. Ordering Students are presented with a list of items to place in order according to criteria specified in the question. Quiz Bowl Students are presented with an answer they must provide the question to. Short Answer Students enter an answer into one or more text boxes. True/False Students select the validity of a statement. The process for adding questions to tests is similar for each question type. At a minimum, you must add the following information: Question text Answer Point value Depending on the test’s question settings, you may add more information for each question. For example, you may also select the display order for the answers of a specific question. NOTE: You can associate rubrics with Essay, File Response, or Short Answer questions. Rubrics © 2014 Blackboard Inc. 17 Quick Steps ASSIGN POINTS The Facts After selecting the questions, you can assign or edit point values. Do It 1. On the Test Canvas, click a question’s current point value to access the Update Points pop-up box. 2. Edit the points. 3. Click Submit. Help © 2014 Blackboard Inc. Add Questions 18 BEST PRACTICE Test higher levels of learning by adding videos to your tests. Video is ideal for providing visual cues and testing observation skills, evaluation skills, and comprehension. Use the mashups function in the content editor to add YouTube™ videos to your test questions. A mashup combines elements from two or more sources. When you view a YouTube video in a Blackboard Learn course as part of the course content, you are experiencing a mashup. Mashups also include Flickr® photos and SlideShare presentations. TIP: After selecting a YouTube video, choose No for the Show YouTube Information option if you do not want to show YouTube’s suggested videos at the end of playback. © 2014 Blackboard Inc. 19 ORDER the QUESTIONS The final step in creating a test is to order the questions. By default, questions are presented to students in the same order you added them to the test. If the order of the questions is not important, skip this step. After you deploy a test—add it to a course area for student access—you can change the order of questions without affecting grades on existing submissions. In addition, when you deploy a test, you can choose to display the questions randomly each time the test is taken. On the Test Canvas, reorder questions by pointing to a question to show the double-tipped arrow and then dragging it to its new location. Alternatively, use the keyboard accessible reordering tool ( ) on the action bar. Select a question, and use the up and down arrows below the title box to adjust the order. © 2014 Blackboard Inc. 20 TRY IT For this activity, use your Practice Course. Use the tests tool Create a test. Select the test’s question settings. Add four to six questions. Add at least one of the questions using the “plus sign”—the add question feature. Reorder your questions. Optionally, edit the point value for one or more questions. © 2014 Blackboard Inc. 21 BEST PRACTICE Use Tests as a Teaching Tool In online teaching, the role of an instructor often shifts toward guide and mentor. By using online assessments, you have the opportunity to use a test as a teaching tool, rather than purely as an evaluation mechanism. Looking at Tests in a New Way Traditionally, assessments measure knowledge at a specific point in time. However, assessments can be used as tools to improve subsequent learning. Pre-tests provide a snapshot of students’ current understanding of a topic and prime them for receiving new knowledge. Students can derive personal satisfaction by comparing their pre-test scores with a post-test score. Tests that allow students more than one attempt provide opportunities for relearning and reassessment. Instructions can improve retention and comprehension by providing students with answer feedback and special assignments between test attempts. Frequent Testing and Motivation Frequent tests not only assess what students know, but also enhance later retention, a phenomenon known as the testing effect. In an experiment, students either took repeated quizzes or were permitted to study the material repeatedly. Students in the former group performed better on a delayed test of their knowledge. The study-only group had read the passage about 14 times, but recalled less than the repeated testing group, which had read the passage only 3.4 times. “Clearly, testing enhances long-term retention through some mechanism that both different from and more effective than restudy alone.” 1 Many studies show similar results, including one that shows frequent testing resulted in higher examination grades, especially for students with low GPAs and high levels of procrastination behavior.2 Frequent tests give students the motivation to study at regular intervals during the semester, which results in better long-term retention than cramming. © 2014 Blackboard Inc. 22 Frequent opportunities for students to test their knowledge can be especially effective in an online course. Inform students they will be tested often with less emphasis placed on the grades obtained and more emphasis on the learning process. Feedback Students often state they need timely and high-quality interaction with their instructors. In Seven Principles for Good Practice in Undergraduate Education, Chickering and Gamson (1987) identify prompt feedback as one of the principles. One efficient way to accomplish this interaction is by creating feedback for online test questions. While the score itself is feedback, students will find your comments valuable.3 In Blackboard Learn, you can provide automatic feedback for each answer, such as: Praise for correct answers. Reasons why an answer is incorrect—a teachable moment. References to textbook pages or online course content to help students learn the material for incorrect answers. Appropriate humor. By developing a testing strategy that uses frequent opportunities for students to test themselves, feedback for learning, and an emphasis on long-term retention, you may see improved student outcomes. © 2014 Blackboard Inc. 23 STEP TWO: DEPLOY the TEST Now that you have learned how to create tests and add questions, the next step is to deploy the test: 1. Add the test to a content area, learning module, lesson plan, or folder. 2. Make the test available to students by editing test options. In addition to setting availability, you can also edit the self-assessment, presentation, and feedback options. Carefully selecting the options most appropriate for your course can enhance student learning and encourage academic honesty. © 2014 Blackboard Inc. 24 Quick Steps ADD TESTS to CONTENT AREAS The Facts You can add each test only once to a specific content area, learning module, lesson plan, or folder. In the Grade Center, a column is created automatically for each test you deploy in your course. Do It 1. Change Edit Mode to ON and access the content area where you want to add the test. 2. On the action bar, point to Assessments and click Test. 3. On the Create Test page, select a test from the Add Test box. 4. Click Submit. The Test Options page appears. Help © 2014 Blackboard Inc. Deploy Tests 25 TEST OPTIONS PAGE Test options allow you to edit the name and description of a test. Other options control the availability, presentation, and feedback. These options are available only after a test is deployed to a content area. Reminder: When you add a test to a content area, it is "deployed." To edit options for a deployed test, access a test's contextual menu and select Edit the Test Options. If you change the name or description of the test, the changes appear in the content area and the Grade Center column is updated with the new name. Availability Test availability is set after you add a test to a content area. When you make a test unavailable, it is deployed to a content area, but a link does not appear to students. When Edit Mode is ON, instructors and course builders can see unavailable tests. All the test options are discussed in the following Blackboard Help links. Test Options Edit Tests © 2014 Blackboard Inc. 26 BEST PRACTICE Instructors are often concerned about academic honesty when considering online assessment. While you may not be able to prevent cheating completely, you can take steps to minimize it.4 Awareness Have students sign an academic integrity policy that defines cheating and outlines expectations for honesty. Use email or chat to verify the level of a student’s test content knowledge after an unexpectedly good performance. Test Construction Limit both duration of the test’s availability and time limit for taking it. Randomize the order in which the questions appear and the order of each question’s answers. Provide a different test for those taking the test late. Use a test as a learning tool, rather than as a measure of fact recall. Allow students more than one attempt with the intent they will return to the course content to learn what they missed the first time. Create a follow-up assignment requiring them to demonstrate knowledge in areas not mastered at the time of the test. Pair online objective tests with subjective measures, such as writing assignments, projects, applied problems, and group work. Technology Administer tests in a proctored location, if possible. If you suspect two students of collaborating on tests, compare the times the tests were taken, and their answers and grades. © 2014 Blackboard Inc. 27 TRY IT For this activity, use your Practice Course. From the Assessments content area, access the following tests and edit the settings as directed. Practice Quiz Make the test available to students. Allow three attempts. Do not include the score in the Grade Center. Change the test results and feedback options. Unit 4 Quiz Add an announcement to alert students to its availability. Set the timer and turn on Auto-Submit. Change the Test Presentation mode to All at Once. © 2014 Blackboard Inc. 28 REUSE QUESTIONS You can add existing questions to a test using the Reuse Question function on a test’s action bar. From the drop-down list, you can add the following items: Questions sets: Collections of test questions retrieved from selected tests and pools. From each question set, you specify how many questions to display. Random blocks: Sets of test questions retrieved at random from one or more question pools to be presented each time the test is taken. Existing questions: The Find Questions feature allows you to add any questions that you have already created in other tests and pools. Pools are discussed after this section. © 2014 Blackboard Inc. 29 The Find Questions Feature The Find Questions page consists of the active filter area that displays all the questions that meet your search criteria. This is a dynamic list that changes automatically as you select or clear criteria. Make your selections in the Browse Criteria section in the left panel. You can select questions from pools and tests, or narrow by question type or metadata. You can also use the Search current results box to type a phrase, word, or part of a word to find matching questions. In the right panel, select the check boxes for the questions you want to add to your test. They are added to the Selected Questions area at the bottom of the screen. You can expand the area to view your list of questions or delete questions you don’t want to add to this test. Find Questions © 2014 Blackboard Inc. 30 Question Sets You can specify a random selection of questions to be presented each time a test is taken. To do this, you create a question set or random block. A question set is a collection of questions retrieved from existing tests and pools. From this set, you specify how many questions to present in the test. The specific questions displayed are randomly chosen each time the test is taken. Question sets can draw questions from both tests and pools. When added to a test, a question set shows the total number of questions in the set, and the currently set number of questions to display to a student taking the test. The default is one. You can change the number to display at any time. To be sure students are presented with a question set each time, enter a number less than the total number of questions. NOTE: When you add questions to a test using the question set feature, each question is linked, not copied. Therefore, if you change the original question, the revised version of the question appears. Questions Sets © 2014 Blackboard Inc. 31 Random Blocks Rather than selecting specific questions to include in a test, you can specify that a random selection of questions be presented each time the test is taken. Unlike question sets, random blocks draw questions from pools only. You cannot draw random blocks of questions from tests. Question selection for random blocks does not have a search function or the capability to browse by metadata. You assign a point value to each question in the block. You cannot assign separate point values for individual questions in the same random block. NOTE: When you add questions to a test using the random block feature, each question is linked, not copied. Therefore, if you change the original question, the revised version of the question appears. Random Blocks Upload Questions You can write test questions offline in a specially formatted text file and upload it into your tests and question pools. Once uploaded, you can edit and use the questions exactly like the questions that you create inside Blackboard Learn. Visit the following topic in Blackboard Help to learn how to format the questions and file. Upload Questions © 2014 Blackboard Inc. 32 QUESTION POOLS A question pool is a collection of questions that is stored for repeated use. Questions in a pool are almost identical to test questions. They can contain feedback, metadata, and all the other elements of a test question, but they do not contain point values. When creating pools, you may want to only include questions of approximately the same difficulty. Instructors typically use pools to create a database of questions they can reuse in multiple tests. For example, an instructor for a Sustainable Living course might create a question pool for each of the central topics in the course. In the following illustration, the instructor created three pools: Diet, Housing, and Energy. You cannot deploy pools or make them visible to students. However, during test creation, you can find questions to include by searching pools. When you select pool questions for a test, links are created between the test and the pool questions. Changes to linked questions produce a warning message that the changes take effect everywhere the question appears. TIP: You can export a pool of questions from one course, and then import into another course. Questions Pools © 2014 Blackboard Inc. 33 TRY IT For this activity, use your Practice Course. If you experience any issues with finding questions, check that you have allowed pop-up windows for your server and try again. Reuse questions Build a pool. Access the tests tool and create a new test with a random block that uses questions from two pools. Select the check box for the Multiple Choice question type and Submit. On the Test Canvas, change the point value for the questions in the random block and display at least five questions. Reminder: All questions in a random block must have the same point value. Preview the questions in the random block. Next, use the find questions feature to add two more questions to your test. Select the check boxes for All Tests, and the Essay and True/False question types. Select all the questions that appear in the right filter area. Expand the Selected Questions area to view the list. Now, Submit. Assign different point values to these new questions. Feel free to experiment with the question set feature. FREQUENTLY ASKED © 2014 Blackboard Inc. 34 What is proctoring? Local proctoring is a common practice when an online test is being used for formal evaluation purposes, such as in the case with a midterm or final exam. Students can arrive at a computer lab at a designated time where a proctor checks identification and ensures students do not collaborate or cheat on the test. Proximity and common geographic areas are typically necessary factors to organize local proctoring of tests. If students are geographically dispersed, proctoring becomes more challenging, but not impossible. In some cases, instructors are able to set up local proctoring by making arrangements with professors or instructors at a local college, school, or library. You can also learn about online proctoring and restricting test taking by location in the following Blackboard Help topics. Kryterion Konnect Online Proctoring Restrict Tests by Location Can I reuse a test given at the beginning of the course? What is the best way to achieve this? Yes. You can copy the test. Access a test’s contextual menu and click Copy. The copied test appears in the list and has a number appended to the name, such as Unit 1 Quiz(1). You can edit this name, if needed. If you want to use a test in another course, you can export it and import it to the other course. SPOTLIGHT on YOUR COURSE Now that you have experienced an online test, it’s time to think about implementing tests in your course. Begin by taking an inventory of the resources already available to you. © 2014 Blackboard Inc. 35 Resources Notes Print exams from a face-to-face Create new test questions based on your print materials. course you have already taught Find out if assistance for this task is available. Textbook test banks Import them into Blackboard Learn. Other instructors Are there other instructors in your department who can collaborate with you? Online resources Be discriminating—fully assess the quality of any online resource you find. After you create your online assessments, test your tests. Before deploying your test, preview it, answer questions as a student, and have a volunteer do the same. Keep the following questions in mind: Have your students been notified of the time and date and location of the test? Are the instructions clear? Is the formatting easy to read? Is any of the wording unclear or ambiguous? Are the correct answers selected? Are appropriate points given for correct and incorrect answers? Are the test settings correct? Double-check settings such as dates available and number of attempts. Has feedback been added to continue the learning process? GOING FURTHER To continue learning about the tests tool, visit these Blackboard Help topics. Presently, survey information is included with the tests topics as they share many of the same workflows. If you want to learn more about surveys, visit the combined tests and surveys topics. © 2014 Blackboard Inc. 36 Blackboard Help Test Question Status Test Results Grade Tests Resolve Student Issues With Tests My Grades Student View: Tests Bibliography Roediger, H. L. & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17, issue 3. 2Tuckman, B. (2000). Using frequent testing to increase students’ motivation to achieve. Paper presented at the 7th biannual International Conference on Motivation, Leuven, Belgium. 3Chickering, A. W. & Gamson, Z. F. (1987). Seven principles for good practice in undergraduate education. American Association of Higher Education Bulletin, 3-7. 4Rowe, N. C. (2004). Cheating in Online Student Assessment: Beyond Plagiarism. Online Journal of Distance Learning Administration Volume 7, Number 2. © 2014 Blackboard Inc. 37