Policy Title Academic Programs External Assessing Policy Policy Number 2-03 Functional Field Student Learning by Coursework Related Policies Responsibility of Status Approved By Curriculum and Academic Policies Committee In-revision Active Draft # Proposed Draft # Academic Council Approval Date Effective Date Revision History Number Date Revision # 1 Revision # 2 Revision # 3 Revision # 4 Revision # 5 Contact Office e-mail address Phone Number DVC-AACS By Main Changes 1 Academic Programs External Assessing Policy Sultan Qaboos University 1. POLICY TITLE Academic Programs External Assessing Policy 2. DEFINITIONS AND ABBREVIATIONS 2.1 ADRI: A four step tool (Approach, Deployment, Review and Improvement) used in analaysing a topic. 2.2 External Assessing: is a process conducted by a peer staff to assess the validity, reliability, practicality and the effectiveness of students assessments. Recommendation are made on any needed changes that will lead to improvement in the opportunities made available to students to help them attain the program outcomes. 2.3 External Assessor: is an individual having appropriate and relevent knowledge, skills, competencies, expertise and experience in the field area. Professional experience is an added value. Appendix II provides guidlines for External Assessors recruitment. 3. POLICY STATEMENT SQU is committed to apply external assessing to its academic programs. 4. PURPOSE/ REASONS FOR POLICY The main aims of this policy are: 4.1. To ensure academic and quality standards. 4.2. To provide opportunity to externally evaluate the effectiveness of students assessments in testing students’ attainment of learning outcomes. 4.3. To provide a systematic and centralized approach to ensuring that assessment instruments are appropriately designed according to the set standards. 4.4. To ensure the fulfillment of internal and external reference points requirements including those of the professional bodies. 4.5. To clearly define the roles, responsibilities and authorities of all parties involved in External Assessing. 4.6. To assure the public on SQU’s academic and quality standards. 4.7. To contribute to evaluative reports which will inform decision-making and will be used for accreditation purposes. 4.8. To respond to stakeholders’ needs. 4.9. To disseminate good practices. 5. GENERAL PRINCIPLES The following main principles constitute the basis of this policy: 5.1. SQU responsibility for the standards and quality of the awards made in its name are discharged in part through an external assessing. 5.2. The external assessing does not result in judgment. The findings are to be listed as good practices or recommendations for improvement. 5.3. The external assessing focuses more on the students assessments than on university systems or programs. 5.4. External assessing are conducted annually. 5.5. Relevent internal and external reference points (these may iclude but not limited to National Qualifications Framework, Audit Manuals, SQU Regulations and Policies and where appropriate profesional bodies requirements) are made available to External Assessors. 5.6. External assessing should have input from students, alumni, employers, relevant stakeholders and possibly from other departments within the University. 5.7. External Assessors’ reports are submitted to the office of the DVC-AACS. 1 Academic Programs External Assessing Policy Sultan Qaboos University 5.8. External Assessors’ adher to a centrally agreed unified report structure guidelines (Appendix I). 5.9. It is the responsibility of the relevent Dean and Head of Department to implement the needed changes and remedies recommended by the External Assessors. 5.10. Good practices reported by External Assessors are documented and shared with the University community. 5.11. The University is responsible for ensuring the availability of resources needed for implementing this policy. The University will arrange for an honorarium and costs associated with the participation of the External Assessors. 6. SCOPE OF APPLICATION This policy applies to academic programs offered by the University. 7. POLICY OWNER (The policy owner is a staff member(s) or body(s) within SQU responsible for developing, overseeing, reviewing and updating the policy under consideration.) The Policy owner should be appointed by the Academic Council. Any development on the proposed policy will be communicated to the Quality Assurance Office. 8. APPROVAL BODY Academic Council. Meeting number: Meeting date: 9. PROCEDURES A) General: 9.1. A central university register of External Assessors is kept by the office of DVC-AACS. The register includes a list of prominent academic staff and other professional people who have held senior post at their institutions. 9.2. Administrative responsibility of external assessing rests with the office of the DVC-AACS. 9.3. External assessing of undergraduate and postgraduate programs is preferably scheduled at the same period. B) External Assessing Stages: 9.4. Nomination of External Assessors The Head of the relevant department would propose through the Dean to the office of DVC-AACS a list of three potential External Assessors names at least four months before the scheduled date of the external assessing exercise. External Assessor Form, see Appendix II, is to be completed for each potential External Assessor. Nominations could be based on the list of existing names given in the central register or in lieu new names can be suggested by the Head of the Department and the central register is updated accordingly. 9.5. Contacting External Assessors The office of DVC-AACS will, within no less than three months before the scheduled date of the external assessing exercise, contact the potential External Assessors as per the order proposed by the Dean. The potential candidates are requested to respond to the invitation within 10 working days from the date of the invitation. If invitation is accepted, the potential candidate is requested to forward any queries/questions to clarify information to the relevant Head of Department, within no more than 20 working days from 2 Academic Programs External Assessing Policy Sultan Qaboos University the date of acceptance of the role. The Head of Department is to respond to these queries within no more than 10 working days from the date of such enquiry. 9.6. Conduct of External Assessing. The external assessing visit activities include: 9.6.1. Review of a sample of students marked work covering selected core courses from different levels (years) in the program. The sample must be a representative one covering excellent, good, borderline, failed results of course-work and final examination works for both undergraduate and postgraduate as appropriate. Dissertations and project works and other similar submitted works shall be included in the sample. 9.6.2. Review of assessment instruments, model/key answers and marking schemes. 9.6.3. Review students’ assessment analytical reports. 9.6.4. Meet with current students of various levels. Students to be interviewed by External Assessors are identified by the Head of Department and notified to attend a meeting with the External Assessor 10 working days or more before the date of the interview. 9.6.5. Meet, as appropriate, with the Dean, Head of Department, academic and supporting staff. 9.6.6. View the Department facilities and resources related to teaching and learning. 9.7.External Assessor Briefing On the last day of the visit, the External Assessor will meet with the concerned Head of Department and brief him/her with the main findings, good practices and areas needing further developments. The Head of the Department is thus given an opportunity to clarify/comments on factual errors, if any. 9.8. External Assessor Reporting The External Assessor issues his/her report by no later than one month from the briefing date. The report is forwarded to the DVC-AACS. The report shall outline any good practices and recommendations for improvement. ADRI, or any other similar tool, may be used for reaching to any conclusions. Suggested report structure is given in Appendix I. 9.9. Department Action Plan The Department is to conduct a meeting to assess the findings in the External Assessor report. The meeting shall focus on how to address any recommendation given in the review report and the development of an action plan in response to the report. The action plan should outline needed resources and timeframe for its implementation. The Head of the concerned Department is principally responsible for drafting an action plan to address the recommendations given in the review report. The action plan is forwarded by the relevant Head of Department through the relevant Dean to the DVC-AACS office. 9.10. DVC-ACCS Commentary Report The DVC-AACS prepares an overall commentary report on the external assessing process outcomes including reference to any proposed action plans to address areas needing improvement pertaining to programs in question. This report is to be forwarded first to the Quality Assurance Office for review and feedback and then to the Academic Council for final discussion and approval. 9.11. Administrative Matters 9.11.1. The term of an External Assessor is up to 3 years and may be extended for only one additional year. However, the term can only be renewed after a break of 3 years from the end of a previous term. 9.11.2. A minimum of five working days is suggested for the completion of the external assessing visit and related activities. 9.11.3. There should be a good match between the number of External Assessors and the amount of work load assigned to them. 9.11.4. An office is made available for the External Assessor. Access to all needed resources and documents are made available to him/her. 3 Academic Programs External Assessing Policy Sultan Qaboos University 10. RELATED POLICIES 11. RESPONSIBILITY FOR IMPLEMENTATION Office of Deputy Vice Chancellor for Academic Affairs and Community Service, DVC-AACS, – Issues University schedule of external assessing activities and contacts potential External Assessors. Receives review reports and action plans and discuss measures and actions in response to any recommendations made in the External Assessors’ report. Writes an overall commentary report on the external assessing process outcomes including reference to any proposed action plans. Monitor progress in making necessary changes as a result of the report’s recommendations. Relevant Dean – Ensures that external assessing activities are conducted as per University issued schedule. Participates, as appropriate, in the external assessing activities. Proposes, in coordination with the relevant Head of Department, a list of potential External Assessors names. Monitors progress in making necessary changes as a result of the report’s recommendations. Forwards reports to DVCAACS. Relevant Head of Department – Nominates potential External Assessors, reflects on queries comments made by External Assessors, makes available for review all needed student work materials and documents, avails staff and students for meetings with External Assessors. Develops an action plan in response to any recommendations made in the External Assessor report and implements any needed changes and modifications. Documents and reports to the DVC-AACS, through the Dean, progress in implementation of corrective measures. Staff – Effectively participate in external assessing activities including any analytical commentary. Quality Assurance Office – Disseminates good practices emerging from external assessing findings. Reports on the consistent implementation of the policy across the University and on the effectiveness of the policy in achieving its intended aims. 12. ISSUING OFFICE Quality Assurance Office 13. REVIEW 13.1. Heads of Departments and academic staff shall periodically participate in evaluating the effectiveness of this policy in achieving intended goals. 13.2. The DVC-AACS shall assist in reviewing activities and shall report on the consistent implementation of the policy across the University. 13.3. The Quality Assurance Office shall review the effectiveness of the policy in achieving its intended aims. 14. KEY RISKS Identifying potential risks associated with the introduction of policies, in particular new ones, is considered a good practice. Line managers have the responsibility to embed risk management practices in their day-to-day operations. This may include taking the necessary measures to eliminate or control such risks. Main risks associated with this policy are listed below: 14.1 Possible lack of objectivity in reporting; i.e. reproducing previous years’ reports only with minor changes. 14.2 Possibility of undeclared conflict of interests. 14.3 Inconsistent application of the policy in different departments or colleges. 14.4 Possible non adherence to timelines of scheduled events. 4 Academic Programs External Assessing Policy Sultan Qaboos University 14. APPENDICES APPENDIX I – Suggested Report Structure APPENDIX II – External Assessor Nomination Criteria 5 Academic Programs External Assessing Policy Sultan Qaboos University Appendix I. SUGGESTED REPORT STRUCTURE The External Assessor report may benefit from a unified structure or standing headings. The report will provide detailed analytical commentary and discussion on all the items listed under the unified headings. External Assessors are required to point out their findings in the form of good practices and areas needing development. The following unified headings are suggested: 1. Assessments and Examinations – This section discusses various ways and tools used to analyze assessment instruments effectiveness in measuring student’s attainments of the program outcomes and educational objectives (attributes). 1.1. Verify if assessment criteria (e.g. outcome-referenced versus norms-referenced), methods (Examinations, assignments, projects, case study, presentations, posters, etc.), weightings, are set at an appropriate level and effective in leading students to attain learning outcomes and graduate attributes. 1.2. Verify if assessment instruments are valid, reliable and practical. This may cover depth and breadth of all assessment components; i.e. coursework and final examinations. Verify if assessment instruments are made more demanding and challenging with progression in the program. 1.3. Verify if students’ performance is judged against the set standards. 1.4. Verify if formative assessments are adequate and systematically applied. 1.5. Verify if students’ assessment results reflect their attainment of learning outcomes. 1.6. Verify that different moderation methods including double blind marking are used and effective. 1.7. Verify that assessment boards’ practices are systematic and adhere to current policies and regulations. 1.8. Verify if assessment analytical commentary reports impact on the enhancement/ improvement of assessment methods and policies. 1.9. Verify that assessment feedback is systematic and assisting students in attaining learning outcomes. 1.10. Verify if the integrity of assessments is systematically assured. This may include physical security of assessments, invigilation and management of student requests for grade alterations. 2. Teaching Quality – This section discusses elements and activities which would contribute to effective teaching which in turn is essential to ensure the standards and quality of academic programs. 2.1. Discuss the appropriateness of teaching methods used, academic staff teaching load and availability to students. 2.2. Describe what developmental activities have the academic staff participated including Teaching and Learning Certificate. 2.3. Discuss what extra actions have been taken to enhance students’ interaction with academic staff to enhance learning opportunities. 2.4. Are ranks and qualifications of academic staff and students/academic staff ratio appropriate and benchmark well with acceptable standards? 2.5. What are the criteria used to assign academic staff to teach on undergraduate and postgraduate programs? Have the criteria been adhered to? 6 Academic Programs External Assessing Policy 3. 4. 5. Sultan Qaboos University Facilities, Equipment and Resources – This section reviews the adequacy of facilities and resources necessary to program viability. 3.1. Discuss the quality level of teaching materials – textbooks, hand-outs, laboratory manuals, assignments, etc. 3.2. Discuss the adequacy of class rooms, laboratories and other technology and instructional facilities and their alignments with program needs. 3.3. Discuss the quality level of administrative support, facilities, equipment and resources and how students and academic staff inputs on these are ensured. 3.4. Discuss the adequacy of library services and their impact on program effectiveness. 3.5. Discuss the quality level of research resources, such as funding, equipment, computer facilities, technical support and conferences. Students Feedback – Reflect on the need to continue offering the program based on its relevance and currency. 4.1. Discuss how students’ needs are accommodated through optional modules and interdisciplinary pathways. 4.2. Discuss students’ satisfaction with the educational opportunities offered and the supporting services made available to them. 4.3. Verify if systematic approach is used for program’s response to changes in discipline issues, technology and employer, student, market and other society needs. Staff Profile and Feedback – This section deals with the external assessor reflection on the staff profile and satisfaction with the working environment. 5.1. Review and comment on the staff profile (specialization, rank, age, gender, etc.). 5.2. Discuss staff satisfaction with work environment including any developmental opportunity and support. 6. Summary of Good practices and Recommendations –This section summarizes any observed good practices and recommendations. The External Assessor is requested to specify any rectification actions as deemed necessary and suggested timelines to complete these actions. 7. Follow Up – This section focuses on the implementation of last year’s report. Establish if recommended actions given in last year’s report have been appropriately addressed. Reference is to be made to any presented evidences. 7 Academic Programs External Assessing Policy Sultan Qaboos University Appendix II. EXTERNAL ASSESSOR NOMINATION CRITERIA Appendix II gives guidelines on the selection of External Assessors. Listed below are the main points to be noted while nominating an External Assessor: 1. Possession of appropriate qualifications, knowledge and skills related to the subject area they are nominated to consider. 2. Where necessary, having professional expertise and knowledge of professional and/or vocational regulations and standards. 3. Preferably someone who has acted as an External Assessor, an external reviewer or an advisor to in curriculum design of similar programs. 4. Familiarity with and having an understanding of the type of programs they are nominated to consider. 5. Known to be able to make an objective contribution to the tasks he/she has been selected for. 6. Willingness to avail themselves at the agreed dates and times. 7. Willingness to produce the needed reports within the agreed timeframe. 8. Declaring any potential conflict of interests. 9. Having a capacity to make good judgment on all the components of the scope of work. 10. Having appropriate knowledge and understanding of the Omani higher education sector. 11. Has gone through formal training in one or more method of External Assessing or institutional review. 12. Having significant experience in senior positions within reputable organizations. 8