Professorial Faculty Rank and Status Expectations and Review Guidelines College of Family, Home and Social Sciences Advice from the Associate Dean and The College Rank and Status Committee* Updated May 8, 2008 The purpose of this document is to provide FHSS faculty expectation guidelines to assist in improving the quality of the rank and status review process (R&S). It is supplemental to the University Rank and Status Policy (R&S), and is cross referenced to the university document where deemed necessary.** Who to hire and retain are among the most important questions that the university faces. To earn a commitment of relatively secure employment, a BYU faculty member must demonstrate a record of excellence after being hired. This document spells out FHSS professorial faculty expectations for their rank and status file. Attending to these expectations can also help faculty gauge whether they are on the right trajectory in annual stewardship reviews. It also provides information for departments and schools that can be utilized in refining faculty expectation documents and assessing faculty performance. These College guidelines are intended to be broad enough to encompass diverse disciplinary perspectives in FHSS. Working within these and university R&S guidelines, department and school faculty are encouraged to create documents that provide more specific disciplinary expectations regarding acceptable publication outlets, as well as teaching and citizenship expectations (URS 1.5; 3.4.3; 3.3.2; 3.2.2). The information provided here should be helpful to candidates preparing their third year review files and assembling their portfolio with an eye towards expansion in the final CFS review. Principles and recommended practices apply to all stages of review (3rd year, associate, and full – URS 5). Recommendations for department chairs and R&S committees for helping the evaluation process go smoothly are also provided. We believe that we can work together to improve best practices in the college. In that regard, we are always open to suggestions that will facilitate the process of review. Contents Part I: Advice to the Candidate, 2 Preparing the File , 2 Scholarship Expectations, 5 Teaching Expectations, 9 Citizenship Expectations, 10 Portfolios, 11 Part II: Advice to Department R&S Committee, 12 Part II: Advice to Department Chair, 18 References, 21 *Prepared by the Associate Dean over R&S and the College R&S Committee with input from department chairs, school directors, department/school R& S committee chairs, and the FHSS Dean over multiple drafts commencing in April 2007. **Cross references to the University R&S Policy are referenced in text as URS plus section number Part I: Advice to the Candidate The most helpful files reflect candidates who anticipate the needs of conscientious reviewers. Candidates are encouraged to be planning well in advance of the submission deadline about their self-assessment, what research samples to include and tracking their teaching loads and course evaluations over time. A current and complete vita as well as an annotated publication list is critical to the review process. Annual stewardship interviews, accompanied by written feedback to the candidates, should provide constructive input on teaching, scholarship and citizenship that is useful for preparing the file. Additional helps for excelling in all three of these areas of faculty responsibility are provided by the BYU Faculty Center for Teaching and Learning and can be found at http://fc.byu.edu/topics.htm Preparing the File This is an opportunity to make your best case for CFS and/or promotion. It is worth taking the time and effort to compile your file in an organized and compelling manner. Focus on your strengths, and outline your professional contributions in teaching, scholarship, and service. However, it is not helpful to employ hyperbole or pad the file with tangential information. Be sure to explain how you have attended to any feedback and concerns that were raised in previous annual and/or third year reviews. There is a checklist provided in the file binder that can assist in assuring that you have included all materials. Attach the checklist to the inside cover of the binder before you turn it in. When assembling the file, please copy double sided wherever feasible. Sample files of several successful candidates are available to look at in the dean’s office. For final R&S reviews, many of the file components described below, including the vita, annotated publication list, self assessment, and teaching summaries should be prepared during April and May of the review year in order to meet June external reviewer solicitation deadlines in departments that send all of these materials out for review (see specific guidelines on pp. 14-15). File materials can be modified and the file can be updated at the time they are submitted to the department in mid-September. Department chairs can notify internal and external reviewers of submitted work accepted for publication at any time while the file is under review. For third year reviews, the file should be prepared in the fall prior to the winter semester that the third year review occurs (see College R&S Calendar). The annotations and self assessment guidelines described below should help reviewers better understand the candidate’s perceived contributions to scholarly products (particularly multiple authored), their quality and impact on the field, how they have moved disciplinary knowledge forward, and any special circumstances that might have helped or hindered scholarly productivity (URS 4.5). Teaching and citizenship contributions can be similarly contextualized (see below). Vita. Make sure the vita is easy to read; keep it focused on professional contributions. Clearly specify the order of authorship for multiple authored publications and professional conference presentations using a standard bibliographic style (e.g., APA, ASA, and Chicago Manual). For conference presentations, indicate whether it was a poster or symposium paper, and whether you were the actual presenter. Be sure to create 2 separate sections for abstracts, books, book chapters, peer reviewed journal articles, book reviews, proceedings, text books, outreach publications, and any non-peer reviewed publications (e.g., professional organization newsletter articles, encyclopedia entries). Annotated publication list. Place an annotated list of publications following the vita (separate tab). The annotated publication list should consist of one to two short paragraphs providing information outlined below for each of your publications. For coauthored publications, specify your contributions to the work .For each publication produced, particularly since you were hired at BYU, briefly explain how it contributes to your research agenda and to your disciplinary specialty area in a few brief sentences. This is where you should also briefly document the quality and impact of each publication outlet (see journals and books sections below), and explain why each outlet was selected for disseminating your scholarship. Annotating a reasonable number of publications prior to BYU hire can also be helpful in explaining the context for more current work. Showing how your publications are threaded together in thematic ways should be part of your self assessment, not your annotated publication list. Also, help reviewers understand disciplinary norms for authorship order listings in multiple-authored work. The order placement may be alphabetical, reflect equal authorship regardless of contribution, or listed in actual order of contributions as agreed upon by the authors. Some disciplines also have norms for whether chairs and committee members are listed as co-authors on published products stemming from the dissertation. Authorship listings that place the candidate’s name first, followed by “with” a list of coauthors in the vita and annotated publication list are confusing to reviewers, particularly when the published work clearly shows an authorship order. Candidates should avoid this practice. Regardless of disciplinary protocol for authorship order listings, statements describing actual contributions to each co-authored publication is important for reviewers to ascertain candidate competency and the quality of their work. Departments can help candidates with this by explicating disciplinary norms for authorship ordering in R & S expectation documents. Works in progress and in press. Works in preparation, in submission, in review, or under contract, do not count as accepted publications, and should not be included in the annotated publication list. However, since they can point to your future scholarly trajectory, they can be included in the supplemental file. They should be mentioned in the self-assessment and may be noted on the vita. For “revise and resubmits,” include communication from the editor in the file preceding the article or book manuscript. For all unpublished but accepted journal articles, include the final acceptance letter from the journal editor indicating that these are actually in press. These should be included in the annotated publication list. The college policy on counting books is somewhat different from articles for two reasons: first, unlike article acceptances, a book contract is not a firm commitment by a press to publish, and, second, a completed book manuscript takes longer to publish than an article. Ideally, a book should be published a year or two before the review so that there is sufficient time for book reviews to be included in the folder. Most candidates for CFS will be able to achieve this ideal, and a candidate who does not have a book in hand by the sixth year is strongly advised to delay going up until the seventh year. However, 3 given the length of time it takes to publish a book (one to two years), and the additional delay of obtaining reviews, in rare cases a candidate may go up with the book in press (and thus without reviews). In press is defined as in production (e.g. copyedited or galley proofs, not merely under contract or under revision). A letter from the press stating a firm date of publication is also required if the book is not in hand. Similarly, candidates for full professor will be considered if a book is in press (as heretofore explained). However, they should carefully consider delaying an application for rank advancement until the book is in hand since it would probably strengthen their case. This policy puts book disciplines on a par with article disciplines, since accepted but not yet published articles are counted for purposes of review. Similarly, for book chapters, if it can be demonstrated that the chapter has been accepted by the volume editor (and publisher) and a publication date is scheduled, they can be included for review purposes. These can also be included in the annotated publication list. Constructing the Self Assessment The self assessment is a critical document for helping reviewers understand how you contextualize your performance and productivity. Accomplishments in scholarship, teaching, and citizenship can usually be described in 8-10 single-spaced pages. If the selfassessment appears unfocused, there is risk that your message will be lost. Be sure not to delve into issues that aren’t pertinent to the evaluation process. Remember that you are trying to convey contextual information to reviewers, some who know little about your discipline, subspecialty, or the nature of the courses that you teach. As you construct the self-assessment, explain why you targeted certain journals and academic presses, why you chose certain approaches to classroom instruction, and why you selected service opportunities as valuable. We advise you to consult your mentor or other established professors in your department or field and obtain their feedback when constructing your self-assessment. To reiterate, for candidates preparing for their final CFS and/or promotion review, the self assessment should be prepared during April and May prior to June external reviewer solicitation deadlines, particularly for those in departments that send this out to external reviewers (see solicitation section on p. 14). Third year candidates should be preparing their self assessment during the fall prior to the Winter 3rd year review cycle. The evaluation of faculty contributions is not an exact science, so anything you can do to proactively allay concerns and demonstrate your competence in the self assessment is helpful. In the end, it all boils down to independent assessments of the difficulty, quality, originality, and impact of your teaching, scholarship, and citizenship at department, college, and university levels (supplemented by external reviews where appropriate). As with any peer review process, there may be honest differences of opinion at different levels of evaluation. For example, although college and university reviewers are encouraged to give “appropriate deference to the department’s perspective” (Section 7.8 of the University R&S document), it is possible to give appropriate deference to other’s positions and still disagree. This is less likely to happen when candidates clearly document how their contributions are in line with national disciplinary 4 standards that are hopefully reflected in departmental expectations (Clarification provided in communication with Academic VP Richard Williams 7-10-07). SCHOLARSHIP EXPECTATIONS Scholarship that is read, critiqued, cited, and influences the work of academic peers is important for consequential impact in any discipline. Critically peer reviewed work is the currency of our professions. Blind, peer review is typical in most journal disciplines. National academic press editorial board peer review, that often includes additional external peer review, typically plays into book and book chapter publication decisions. Evaluators read your publications to assess quality of organization, breadth and depth of inquiry, relevance to field, originality, methodological rigor, style and analysis. They make judgments about whether your interpretations and conclusions are compelling in the context of a clearly defined, thematic trajectory that adds to disciplinary knowledge. In light of this, explain how your scholarship fits together into a thematic, innovative, and high quality program of inquiry. For example, discuss how your publications build on one another in developing a coherent research agenda that provides evidence for the uniqueness, quality and rigor of your scholarship. Tell how it is relevant to broader disciplinary inquiries. Describe how your work is innovative in your field. Demonstrate how various themes in your scholarship are tied together in ways that point to your developing or having achieved national prominence in your specialty area. Show how your contributions meet disciplinary norms for advancement at peer institutions. If yours is a book and journal discipline, explain why both book chapters and journal articles are important. If yours is a journal article discipline and you are publishing mainly book chapters, tell the reviewers why. If yours is a book discipline, tell why the academic presses you have selected are the best (top ranked) for your specialty area in ways that are suggested below. If you are publishing in outlets not typically considered to be in your disciplinary arena, explain why. If you haven’t been publishing primarily to your national disciplinary audience, help reviewers understand why public policy, trade, practitioner or other audiences you are targeting are important for your academic work. The myth persists that review committees look only at the quantity of output. However, numbers of publications are not usually the most important factor. There should be a consistent flow of publications that is normative for your discipline, while taking into account the complexity of your work. Faculty who build a trajectory towards higher impact publications are typically viewed more positively than those with a trajectory built on greater numbers of lower quality, easier-to-publish research products. Publishing some papers in outlets of non-demonstrable or low impact targeting specific audiences is typically not problematic, as long as the work is deemed to be of acceptable quality. However, it is difficult to build a case for CFS and promotion when much of the record is based on non-critically peer reviewed scholarship, third tier publications, practitioner focused outreach that repackages the scholarship of others, or work published by presses without strong disciplinary reputations. It is important, particularly in article disciplines, to explain any lengthy gaps in your publication record. These can often be due to intensive project conceptualization, 5 data gathering, retooling, publisher delays, or personal issues. Help reviewers understand why the hiatus occurred, and how dry spells have been or are being overcome. Activities including advocacy, outreach, moonlighting, evening school teaching, and textbook writing are typically not viewed favorably when they detract from disciplinary knowledge production and quality teaching. Candidly evaluate your strengths and weaknesses and changes you are making to overcome any deficiencies. Multiple authored work.. Although publishing with prior mentors is generally encouraged, you should show how you are developing an independent research program. Document how you are providing leadership if you are working with a team, beyond contributions that move only portions of a project forward. If your only contributions to published work are consistently that of a methodologist or statistician, for example, questions will be raised about your ability to create an independent trajectory of lead and/or sole-authored scholarship. Leadership usually involves conceptualizing and identifying gaps in disciplinary knowledge, synthesizing the literature toward hypothesis formulations, executing appropriate methodologies, and performing and/or assuring that the data is properly analyzed and presented in the published work. While much of this workload can be shared with co-investigators and students, there should be evidence that you are the driving force in projects that contribute to your focused research agenda. Project leadership can also be reflected in large scale studies with colleagues, where everyone has carved out portions that they take ownership of. If publications in your discipline are primarily multiple authored, provide a context for this with documentation (e.g., author number means and standard deviations for your primary disciplinary journals). Help reviewers understand what is typical with regard to numbers of sole, lead and/or co-authored papers in your discipline. In journal disciplines, a third to half of peer reviewed multiple-authored papers at a minimum are typically lead authored by successful candidates. Many of these should be independent of prior mentors. Also, document how your publication rate, the quality, and impact of your work stacks up with same-rank faculty norms in your field regarding where scholarship is typically published at your career stage. Explain the type of methodologies that you employ, the labor intensiveness of data collection in your specialty area, and the nature of your sources (such as the use of archival data). This will help reviewers unfamiliar with your discipline better understand what a reasonable publication rate should be for the type of scholarship that you do. Publication impact.. Since many reviewers will not be practicing in your discipline, it is important to help them understand the challenges of producing consequential research in your field. For example, top tier journals in some disciplines may be easier to access than in others. The effort to publish in a top tier generalist journal in one discipline may be commensurate with the effort to publish in a second tier, but highly regarded specialty journal in another. It is understandable that early in your career you may publish in more specialized journals that are stepping stones to publishing in higher tier generalist journal publications as your career progresses. For most disciplines, it is particularly helpful to document how much your work has been cited by other scholars, excluding self-citations. Start this process prior to being evaluated for promotion to associate professor so you can document a trajectory that is leading to national or international recognition and full professorship. A faculty member’s citation 6 record is available in the dean’s office that provides an example of one way to this (contact the dean’s secretary). Some disciplines have recently published papers on these topics that can help with this type of contextualization (e.g., Byrnes, 2007; Giles & Garand, 2007; Grable, 2006; Holoski, 2006; Wuchty, Jones, & Uzzi, 2007). Journals. You should tout the impact and quality of publication outlets in the annotated publication list and in your self-assessment. This can include criteria such as: impact scores, verifiable rejection rates, citation counts (excluding self citations), documented familiarity or reputation in your field, circulation numbers, etc. Note and provide documentation for which publications were peer reviewed. For articles published in journals not listed in the Social Sciences Citation Index (or with low impact scores relative to other journals in your discipline), it is particularly important to provide reviewers with information regarding why this was the best outlet for your work. For example, non-citation indexed journals often publish special issues on a particular topic that target specialized audiences. Providing other externally verifiable information regarding the rigor and reach of the journal can be helpful (e.g., editorial board quality, national professional organization research journal, rejection rate, peer review standards). Reviewers outside your specialty area are well aware of citation index strengths and limitations (e.g., Altbach, 2006; PLoS Editorial, 2006; Essays of an Information Scientist, 1983), so they also want to see other evidence for the impact and quality of your outlets where ever possible. Some disciplines have published articles that summarize national disciplinary views of journal quality and impact that can be helpful in this process (e.g., Sellers, et al., 2006; Holosko, 2006). A number of online resources are available to help, each with its strengths and limitations: Institute for Scientific Information – ISI data base, Scopus, Google Scholar. Books and Book Chapters. For book disciplines in particular, it is important to explain why you selected a certain press to publish your work. Evaluating the scholarly contribution, originality and impact of your book is most importantly done by your peers in the department, profession, college and university reading the book. The reputation of the press that publishes your book also matters, as it does with journals in journal disciplines. You should aim for the most respected presses in your field. In your selfassessment describe the process you underwent to select the press publishing your book and how it compares to other presses in your field. You might include other scholars in your field who have also published their first or second book with your press. If your book is in a series you might explain the significance of the series. Other information that helps external reviewers of your work is the process of peer review undertaken at the press publishing your book, what journals regularly review the books published by the press, and awards that your press has won in recent years in your field. Information about production quality, copyediting, visibility, and marketing can also be helpful to reviewers who are less familiar with your discipline. For those seeking promotion to the rank of full professor, comment on whether second and subsequent books were published by a higher-ranking press, or at least a press of comparable rank that has strong national or international visibility. Book chapters are less likely to be rigorously peer reviewed, and generally don’t make pivotal contributions relative to single volumes or peer reviewed journal articles in final R&S decisions. For some disciplines, it is recognized that book chapters provide opportunity to identify gaps in the literature and build stepping stone conceptualizations 7 that provide direction for future peer reviewed studies. They can also provide a foundation for single volumes. Disciplinary handbook chapters are often weighted more. It is also important with book chapters to help reviewers ascertain whether the contribution is part of a volume with prominent editors and scholars in their field, indicating national/international visibility and connections with disciplinary peers. Text Books. Textbooks are a synthesis of the existing knowledge base rather than original disciplinary contributions focused on the creation of new knowledge. While valued in the teaching domain, they are not typically evaluated on par with scholarly output. It is thus unwise to include them as one of the three works in your portfolio of scholarship. Professional meeting contributions. Briefly describe your contributions to international, national and regional professional meetings in your discipline in ways that illustrate your consistent engagement in the discipline and your networking with professional peers. Document which presentations have led to publications and papers in progress. Pay particular attention to the ratio of presentations to publications. If presentations far outweigh the number of publications, explain why those presentations have not led to publications. Be clear in your vita as to which presentations were symposium papers or posters, and whether you were the actual presenter for co-authored papers presented at conferences. Cue reviewers as to any important paper symposia you have participated in with leading figures in your field, or provide any other information that indicates that you are tracking towards or have arrived at national or international prominence in your discipline. Expectations unique to full professor. For those being evaluated for promotion to full professor, it is important to provide evidence of national or international recognition by disciplinary peers and significant influence on scholarly discourse in the field. This recognition, stemming primarily from a body of quality scholarship, often leads to tangible evidence of prominence in one’s discipline. This evidence can include: citation counts relative to national peers in your discipline, published reviews of books by nationally recognized academic presses, service on editorial boards of respected journals, editorships, invitations to write external reviews for tenure and promotion, service on federal grant review committees, offices in national or international professional organizations, invitations to serve as commentators or discussants in disciplinary conferences and other venues. Faculty who are not publishing quality scholarship in visible and rigorously peer reviewed disciplinary outlets are typically less likely afforded significant opportunities to serve in these capacities. Grant writing expectations. It is recognized that successful external grant writing is more difficult at BYU, given the limited social science grant procurement, technical staffing, and oversight infrastructure, and that it is strategically curtailed for a variety of reasons (see Samuelson 2005, University Conference address online). Notwithstanding, high quality research in many disciplines requires significant extramural funding, more than what can be procured in the BYU intramural system. Despite this, some FHSS faculty are successful in the extramural funding enterprise. From your disciplinary perspective, help internal and external evaluators understand how 8 this plays out in your own research program (if deemed necessary), how you have been compensated, and what you are doing to be competitive in this endeavor. Future roadmap for scholarship. Finally in your self-assessment, provide a roadmap as to future research contributions to help evaluators estimate your trajectory beyond this review. Make your case as strong as you can. Given the length of time between books (sometimes 6 to 10 years), it is particularly important for faculty in book disciplines to provide evidence that the next project is well underway. Documentation can include grants, journal articles, book chapters, or conference papers that point to solid progress in project conceptualization. TEACHING EXPECTATIONS Teaching is carefully considered in the evaluation process. Excellence in teaching is expected of all BYU faculty members and a high threshold must be sustained in order to gain CFS and promotion. Performance in the scholarship and teaching areas of the faculty appointment is often highly correlated. However, the type of documentation for demonstrating high teaching performance is more straightforward and doesn’t vary as much by disciplinary standards, requiring less elaboration in this document relative to scholarship. Less space devoted to teaching expectations should not be interpreted as reflecting a heavier emphasis on scholarship in the review process. Additional recommendations for presenting your teaching accomplishments are outlined below in the Teaching Portfolio section. Tout your strengths and what you have done or are doing to overcome any weaknesses. Teaching Philosophy. For your self evaluation, describe your philosophy of teaching for lower and upper division courses as well as graduate courses where applicable. If you have not been teaching both lower and upper division courses, be sure to explain why. Explain how you promote rigor in terms of providing opportunities for synthesizing disciplinary knowledge, and enhancing critical thinking, writing and numeracy skills (where applicable). These themes can be further tailored to meet specific course objectives in your annotated list of courses taught (Tab 1b). Responsiveness to feedback. Find ways to document your improvement as a teacher since the last review. If there are extenuating circumstances or specific data (e.g., low student evaluations in a course, recurring negative themes across semesters in student comments) that need explanation, please do so. For faculty undergoing university rank and status review, careful attention should be paid to formative peer review feedback in both lower and upper division courses when writing the self assessment (see Peer Review of Teaching Section on pages 16 and 17). While students can gauge the effectiveness of your course delivery, faculty peers are in the best position to assess the quality, rigor, and currency of your instruction relative to disciplinary norms. Where necessary, evidence for improvement and specific ways this has been accomplished and/or is being accomplished should be explicated. It is recommended that formative peer reviews be included in the R&S file, in addition to the summative peer review that the department will insert after you have submitted your file. This provides evaluators with multiple data points on course content and delivery and provides an opportunity for you to demonstrate how you have been responsive to constructive feedback about teaching over time. 9 Contextualize your grade distributions, teaching loads each semester and spring/summer term, and number of new preps relative to departmental norms and expectations for upper and lower division courses. Be explicit in describing how you have responded to student concerns, including recurrent negative themes reflected in student comments. .If you are not teaching both upper and lower division courses regularly, explain why. Also, explain any release time and how that time was used productively. Pay particular attention to any recurrent negative themes in student comments. Discuss how you have responded to any concerns raised in formative peer reviews of your teaching, course materials, and learning activities. As recommended in the University R&S document, there should be written feedback from formative peer reviews that occurred over several semesters prior to your current, summative review (URS 3.3.2). Be sure to describe your involvement in other teaching activities. Supervising graduate students, mentoring undergraduate students, teaching invited lectures, and supervising internships are some of the other areas that fall in the teaching domain. Finally, provide a roadmap as to future teaching contributions to help evaluators estimate your trajectory beyond this review. Make your case as strong as you can. The BYU Faculty Center for Teaching and Learning has some excellent suggestions for how to respond to student concerns (http://fc.byu.edu/topics.htm). CITIZENSHIP EXPECTATIONS As detailed in section 3.2 of the university rank and status document, citizenship at BYU entails adhering to the highest standards of personal behavior, exemplifying honor and integrity, being loyal to the Church and mission of the University, performing administrative assignments, serving on department, college, and university committees, interacting with students and colleagues with civility and respect, promoting harmony and collegiality, providing leadership and service in national or international professional organizations, lending professional expertise to give service to the community and the Church, and attending and participating in department, college, and university meetings. Reviewers presume that the candidate supports the aims of a BYU education and is living in accordance with standards required for BYU employment. Comment briefly and do not over do these points. The self-assessment is a good place to help reviewers understand the quality and impact of your professional service, especially for those being considered for promotion to full professor. As you consider suggestions for activities to describe in your citizenship portfolio below, you should contextualize many of those activities here. For example, some assignments are more time consuming or important than the name and description in your citizenship portfolio implies. This is a good place to help reviewers understand all this. Explaining how you balance professional service opportunities with producing quality peer-reviewed scholarship and engaging in high quality teaching can be helpful to reviewers in assessing how you establish and prioritize your commitments. 10 Be sure to also explain how you view your role as a stakeholder in creating a culture of high quality expectations for scholarship and teaching in your school or department. Briefly identify specific ways you are helping build the intellectual and collegial climate in your department in ways that not only build national or international reputations for you and your colleagues, but how you contribute to a work environment that promotes both group and individual achievement. Context for Research Portfolio Provide an annotated bibliography of the three contributions chosen. These three should be separate from, but referred to, the annotated publication list so that reviewers realize that they are somewhere else in the file. As with the rest of the annotated publication list, explain the content, the quality of the outlet, and your perceptions of impact. In addition, explain why you chose these three contributions to include in your portfolio. Provide high quality double-sided copies of articles; package any books in a separate envelope. Teaching Portfolio (Evaluation of Teaching Section) List courses taught chronologically in a teaching summary page. This includes a list of courses taught each semester, number of students enrolled in each course, overall course and instructor ratings, and average course GPAs. Staff in the Dean’s Office can provide you with an electronic template for reporting this information. Comprehensive single page course rating summaries and student comments for each semester under the period of review are also included in the evaluation of teaching section. You can print the single page summaries of student ratings and the student comments for each class from the student ratings website. No other solicited or unsolicited student comments should be included. Be sure to also include an annotated list of courses taught to help reviewers understand what each course is about and how it fits into the overall department curriculum. On a separate page, include teaching contributions other than through formal courses. For example, these might include graduate students supervised, undergraduate student mentoring, invited teaching lectures, and internships supervised. The detailed forms for student evaluations using the old paper and pencil method should also be included in the file under Summary of Student Evaluations. All open-ended student comments from the old system should be typed and included. The teaching portfolio should include the most recent course syllabi for all classes that you have taught since your last review. Also include examples of out of class and/or in class learning assignments, and exams. Other than syllabi, it is not necessary to include all of these materials for all courses in the binder, but it would be nice to include those that demonstrate quality across the breadth of your teaching assignment (e.g., large G.E. course and smaller field course). Include in a separate folder any additional teaching materials you might deem appropriate for all courses. 11 Citizenship and Service Portfolio We try to limit younger faculty’s contributions to university, college, and department committees. Overloading junior faculty with too many course preps and major citizenship assignments, allowing them to become extensively involved in local and national professional service, or permitting them to perform extensive service for others (e.g., statistical analyses) diminishes their ability to focus on solid teaching and research and is strongly discouraged. Associate professors also need to be judicious in balancing these demands as they prepare for the full professor status review. In light of this general directive, list, contextualize where necessary, and briefly describe BYU administrative and committee assignments. Place your contributions to committees in context (what the committee does, amount of time commitment, accomplishments). Also list and place in context any national/international professional service. Be sure to briefly describe service on editorial boards of respected journals, editorships, ad hoc national academic press book or journal reviewing, invitations to write external reviews for tenure and promotion, service on federal grant review committees, offices in national or international professional organizations, and book review, abstract or newsletter writing that demonstrates national or international engagement and visibility in your discipline. It is fine to list outreach or professional contributions to the community (including outreach publications). For example, providing evidence that the faculty member is called upon to share their disciplinary expertise with local and national media can be an indicator that the candidate is becoming the “go-to” person in their specialty because of their scholarly reputation. However laudable church or other similar contributions are, we recommend that these not be included. As with any other major university, the BYU rank and status review process focuses on professional contributions. Faculty Development Account for leaves or release time and specifically explain how you used the time to further your goals of scholarly achievement or improved teaching. Discuss how you benefited from other on and off campus development opportunities (e.g., those provided by the Faculty Center). Part II. Advice to the Department/School Rank and Status Committee The University Rank and Status Policy explains the required procedures for the process (URS 7). These procedures insure fairness to the candidate and the institution. Deviations from them expose the candidate and the university to risks. Committee members are urged to carefully read and study the university document. Committee members are also encouraged to read the advice given to the candidates from the college in this document that elaborates FHSS expectations. They should also review department documents that specify disciplinary norms and expectations. In your report (letter) to the chair, give a summary of the evidence, the issues discussed in weighing the evidence (the analysis of the evidence), and the conclusions 12 reached. Impartially weigh the evidence, considering what recommendation is best for the institution. State clearly the vote of the committee. The conclusion is reached by majority vote but a minority report can be filed. Department committee members do not have to be full professors to carry out their committee assignment, which includes voting privileges on those being considered for promotion to full professor (correspondence with Academic VP Richard Williams, October 17, 2006). The department committee may advise the candidate when questions are asked about file creation. However, once the file is submitted, the committee serves not as an advocate, but as a judge for the institution in order to insure the quality of the faculty for students. It is hoped that a complete picture of the quality and quantity of the contributions of the candidate emerges. If more evidence is needed after the file is turned in, please seek it. Any additions to the file after the original submission should be clearly dated and noted as an addendum. The department level is the best place to uncover the needed evidence. As the committee evaluates the file, anticipate questions that might arise from the department, the college, or the university and seek data or explanations that clarify the nature of the contributions. For example, questions regarding whether a candidate’s perceptions of their contributions to co-authored work are accurate is a recurrent issue. If the department review committee deems it necessary, they may invite any co-authors to submit a short statement regarding the candidate’s contributions. The department committee should also carefully read the publications, evaluate their quality, rigor, and potential impact on the discipline. They should independently assess the quality of outlets using the means stated above; candidate claims are not enough. The R&S committee at the department/school level is extremely important to a quality R&S process. Although each person or committee in the review process does an independent evaluation, deference is typically given to local judgments provided that a convincing case is made. It is important that the evaluations be wise, credible, and impartial. An objective view of the candidate’s strengths and weaknesses should be presented, along with a projection of the likely future trajectory supported by evidence. ANALYSIS OF SCHOLARSHIP Consider the research portfolio, other published research, “revise and resubmits” papers submitted, other working papers or works in process. The later work is essential in order to assess the pipeline and future contributions. Consider weaknesses and strengths, quality of outlet, and impact of the research. For promotion to full professor especially, it might be helpful to examine citations or other indications of impact. Please read and assess the quality of each piece in light of the standards of the profession for the rank and status sought (see above advice to the candidate). Because of the committee’s disciplinary expertise, the focus should be on assessing the quality of the work by carefully reading it, rather than by relying on others’ perceptions of quality. External Letters External reviewers should be selected from “at least three faculty members at well regarded academic institutions who have achieved reputations in the relevant field.” (section 7.9.6). Sometimes three reviewers are not enough, particularly in cases where 13 reviewers provide perfunctory evaluations. We advise you to set a goal of five reviewers; more are unnecessary. Given the number of reviewers who are typically unavailable, it is best to initially create a larger list of first and second choices with an eye towards actually obtaining five reviews. Select reviewers who will analyze the strengths and weaknesses of the candidate’s work, put it into a broader context, and evaluate it relative to national peers. To obtain the most appropriate reviewers, we urge you to solicit letters in early May before scholars have made other commitments. Letters from external reviewers should be solicited from individuals who would reasonably be expected to offer an impartial evaluation of the candidates work. All letters requested from outside reviewers are included in the file. It is important that reviewers should be at “arm’s length.” This means they should not be solicited from faculty who were or are at the institution that granted the candidate’s degree. The reviewers should not be co-authors, collaborators, national young scholar award mentors (the committee can see information from co-authors but they should not be the formal evaluators), or those with whom the candidate has a close personal or professional association. Reviewers should have the rank and status at least equal to that sought by the candidate. Reviewers can be suggested by the candidate but the majority should be selected judiciously by the committee and the chair. The candidate can also name reviewers whom he or she thinks cannot provide an impartial critique of their work, along with a brief written rationale, to the committee and chair for their consideration. It is entirely appropriate to exclude these reviewers. There is also wisdom in choosing reviewers (at least one) to evaluate properly whether the candidate’s contributions are significant to the broader discipline. That is, sub-fields can be defined narrowly but may have little impact on the profession generally. Sometimes, a club of national colleagues in a specialization is quite small and insular, making it difficult to gauge how a candidate’s work is viewed by the larger discipline. Therefore, a mix of specialists and those who can speak to generalist field perceptions is highly encouraged. The department committee’s report should outline why the reviewers were chosen, their stature in the profession, and their relevant knowledge of the reviewer (URS 7.9.6). If reviewers are selected from institutions of potentially perceived lesser stature than BYU, a rationale should be provided. A waiver of the right to see external review letters (Appendix C) should be presented to the candidate (who should be encouraged to sign it). Preparing the Solicitation Disciplinary norms for what is sent out to external reviewers vary across the college. The university R&S document recommends sending out the candidate’s curriculum vita, scholarship samples, teaching assignment information, a signed waiver, and “a summary of the university and department standards for assessing scholarship.” (URS 7.9.2 and 7.9.6). It is also normative in some FHSS disciplines/sub-disciplines to include an annotated publication list and/or self-assessment. The annotations and self assessment can help external reviewers better understand the candidate’s perspectives on their contributions to scholarly products (particularly multiple authored), their quality and impact on the field, how they have moved disciplinary knowledge forward, their research 14 trajectory, and any special circumstances that might have helped or hindered scholarly productivity. External reviewers can agree or disagree with candidate perspectives and provide their own views. In some FHSS disciplines, it can also be argued that this additional information has potential to lead external reviewers to be less independent in their judgments of scholarship quality and contributions. It should be kept in mind that the external review process is conducted in a manner similar to critical peer reviews of scholarly work (e.g., journal articles). From this perspective, care should be taken not to sway reviewers with additional material beyond the scholarly output of the candidate. Department faculties are in the best position to decide and declare in department R & S expectation documents what is best to send out to external reviewers in light of their disciplinary norms. It is appropriate to briefly summarize any special circumstances (e.g., maternity leaves) in cover letters for files that go out to external reviewers. For departments that do send out annotated publication lists and self assessments, candidates should be preparing these materials in April and May of their review year in order to meet June external reviewer solicitation deadlines noted below (see college R&S Calendar). These can be revised up to the time that the file is submitted to the department. Department chairs can inform external reviewers and university evaluators of any submitted work that is accepted for publication during the period of review. As noted earlier, the university rank and status document recommends that a summary of the university and department standards for assessing scholarship be included (section 7.9.6). However, we strongly recommend that reviewers be asked in the request letter to evaluate the candidate according to their own perceptions of national standards in the discipline. A list of departmental expectations and standards often unintentionally signals that reviewers should adhere to these criteria. Rather than providing an independent critique of the candidate’s scholarship, reviewers often conform themselves to the standards set by the department, regardless of what the standards are. Comments like “Dr. ______ met the departmental checklist for promotion” are not very helpful. Reviewers can be informed by department expectations, but they should be encouraged to make judgments about professional standing based on their sense of how the candidate measures up to national disciplinary norms. A copy of a sample request letter is found in Appendices D & E of the University document. Add to it as you think useful, to help the reviewer frame the relevant questions. Be sure to request the return of the reviewer’s vita and include it with a copy of the solicitation letter in the file with their letter. It is up to departments to decide whether to and how much financial compensation should be offered to external reviewers in accordance with disciplinary norms. Timing of the Solicitation We have had several late files because external review letters have not arrived. Presumably this is due to procrastination or strategically waiting until the last moment in expectation that an “acceptance” might arrive. This isn’t fair to the candidates and jeopardizes the equity of the process. We recommend that letters be solicited by no later than May 10, and earlier is better (see college R&S calendar). We are finding that many potential reviewers are already committed to their limit for the coming fall by May. Early 15 invitations provide the proper courtesy to reviewers and allows for contingencies. Be sure to obtain more than 3-5 commitments in case one or more drops out. The candidate materials should be sent to the reviewers by no later than mid-June, with a deadline for completion mid-August to assure 3 weeks to follow up with stragglers before candidate files are due to the department in mid-September (see college R&S calendar). Individual units have discretion over whether and how much to compensate reviewers, but in our experience payment for reviews is increasingly becoming the national norm. Packaging and Providing Context for External Reviews Include the solicitation letter and any attachments and the signed waiver in the file before the actual review letters. Also, provide the context for how the reviewers were selected (URS 7.9.6). Tell which of the reviewers were candidate and which were committee recommendations. Provide a paragraph or two on each reviewer and potential reviewer telling why they were selected. Also, include information on why first-choice selections declined to review (if that is the case). This helps evaluators get a better feel for the larger pool of potential reviewers and whether the best reviewers for each case actually responded. With the external review letters include the vitas of the reviewers in the file (double side copy where possible). The vitas help internal reviewers better understand the publication record, stage of career, and other aspects of the external reviewers’ background and perspective as related to the critique that they provide for the candidate. ANALYSIS OF TEACHING The committee should carry out their own analysis of teaching for their report. Consider the student teaching evaluations, including all the student comments, the peer evaluations, the teaching record, and all course materials (e.g., syllabi, readings, out of class learning assignments, special in-class activities, exams, pedagogy). It is assumed that all voting faculty will have read all of the student comments, the peer review of teaching, and examined the course materials from the candidate provided for the review. Highlight strengths. But also pay particular attention to any recurrent negative themes in student comments and ratings that might reflect disorganization, belittling, arrogance, sarcasm, rudeness, insults, condescending attitude, lateness, lack of class control, inability to respond to student questions, lack of quality feedback on assignments. Ascertain what the candidate has done or is doing to remedy these problems. Peer Review of Teaching The review is not meant to address curricular issues in the department, but should focus on the candidate’s teaching competencies and on the currency, rigor and quality of the courses offered by the faculty member. The committee manages (in consultation with the department chair/school director) the summative peer review process for rank and status. Ideally, this should be an outgrowth of an ongoing formative peer review process conducted for many semesters well in advance of the final review where the faculty member is provided with constructive feedback as recommended in the University R&S document (URS 3.2.2; see also BYU Faculty Center for Teaching and Learning http://fc.byu.edu/topics.htm). Peer evaluations might best assess the following areas: 16 1. Whether the course reflects the current state of the discipline. 2. The faculty member's mastery of the course content. 3. The course objectives, including whether the course meets the objectives of the curriculum of which it is a part. 4. The course organization. 5. The methods used to foster and measure learning. 6. The materials in the teaching portfolio (syllabi, textbooks, handouts, multimedia materials, assignments, learning exercises, examinations, and other course materials). 7. The faculty member's general concern for and interest in teaching. 8. The overall quality of teaching. Formative evaluation describes activities designed to provide faculty with information they can use to improve their teaching regardless of their years in the classroom. It can also be helpful to administrators in the annual review process for providing commendations and constructive feedback that can assist faculty in making course corrections. As was indicated earlier to the candidate, it is recommended that formative peer reviews be included in the R&S file. This provides evaluators with multiple data points on course content and delivery and provides candidates the opportunity to demonstrate how they have responded to constructive feedback about teaching. For both formative and summative peer reviews, candidates should have had opportunity to be reviewed in both lower and upper division courses so that their teaching abilities in introductory and advanced classes can be evaluated. The final summative peer review of teaching should be conducted in the semester prior to the 3rd year or final review (see college R&S calendar and advice to department chair below). Summative peer reviews of teaching are confidential and are not shared with the candidates. The BYU Faculty Center for Teaching and Learning has some good suggestions for how to coordinate this peer teaching review committee work in the department (http://fc.byu.edu/topics.htm). In addition, consider assigning faculty members to provide input to the committee that are in the best position to assess the candidate’s courses but can also be objective. To avoid conflicts of interest, utilize peer reviewers who are somewhat arm’s length and who can provide objective feedback (that is, avoid coinstructors or collaborators). It is recommended that you follow the peer evaluation process developed by the College Curriculum Committee (available from the Dean’s office), or the more extensive evaluation process provided by the BYU Faculty Center for Teaching and Learning. Because it relies on more than class visiting, the peer review process is crucial for judging whether the class is state of the art, and if it meets department standards for rigor in curriculum and assessment procedures. The peer reviewer should pay careful attention to course syllabi, exams, project assignments, and grading standards. Courses should be evaluated as to their currency in the field. Faculty peers are better suited to assessing these aspects of teaching than are students. The peer review of teaching should assess aspects of teaching related to delivery of course content in and out of the classroom. This includes but is not limited to class 17 visits. Peer reviewers should read all student course evaluations, including all open-ended comments and evaluate student feedback for the department colleagues and others in the process. As with course content, evidence that the faculty member is attempting to appropriately respond to student feedback over time is relevant to the evaluation. ANALYSIS OF CITIZENSHIP Citizenship contributions to the department and the university that were noted earlier in the citizenship advice section to the candidate should be carefully evaluated. Meeting these citizenship expectations is important at any level of review, but national or international professional visibility becomes more significant in the decision to award the rank of full professor. Naturally, the candidate for full will have assumed more significant citizenship responsibilities at the university, such as chairing committees or serving in responsible positions such as a department associate chair. To achieve the rank of full professor, a faculty member should also be making contributions to national or international professional organizations, such as serving as an officer or on the editorial boards of professional journals. The citizenship decision would also encompass serving on national grant review panels, such as NEH or NIH, or in other ways demonstrating a broader recognition of the candidate’s expertise in the field that are noted below in the citizenship portfolio. Regarding national or international professional service, it is recognized that labor intensive federal grant review committee service, editorships, editorial board, and ad hoc journal article reviewing requires considerable time and writing effort. These are important professional service activities. Likewise, book reviews, encyclopedia entries, abstract and newsletter writing, and outreach publications are typically viewed as professional service rather than peer-reviewed scholarship endeavors. These activities are typically limited during the early years of a career, but there should be evidence of an emerging trajectory pointing to national or international recognition as invitations to serve in these capacities are extended. This trajectory should be expanded after the CFS review in order to firmly establish a national/international presence in the discipline. Part III. Advice to the Department Chair/Director Department chairs and school directors have the important responsibility of overseeing the review process to assure that everything is functioning as it should. While much of this responsibility can be delegated, chairs should be vigilant in assuring that the faculty reviews are conducted in a timely, fair, and systematic manner. Process Oversight Items Requiring Attention With the help of the department R & S committee, be sure to carefully review the checklist at the beginning of each candidate’s binder to assure that all of the materials are included before submitting it to the college. Chairs should know the University and College processes and should ensure that the proper procedures are followed. This implies that the R&S committee of the department should be chosen during the prior winter semester to facilitate the fall reviews. For example, the committee should be in place to select and solicit external reviewers and assist in summative peer reviews of teaching by mid March for fall reviews (see college R&S calendar). These reviews should be conducted in both lower and upper division courses. 18 CFS candidates are required by the college to declare their intention to undergo final review in their 6th or 7th year by March 15 of their fifth year. This will allow time for college personnel to verify candidate review timelines with in-house documents in order to meet the university April 1 declaration deadline (see section 4.1 of university R&S document). However, for those going up in their 6th year, somewhat earlier “in house” notice in January or February is encouraged so that peer teaching evaluations can be conducted well before winter semester is completed. This will help avoid rushing summative peer reviews in the first week of the fall semester in order to meet the midSeptember file submission deadline. Conducting peer teaching reviews after the file is submitted to the department often detracts from time and energy that should be dedicated to carefully evaluating each candidate’s record in a compressed time frame. While it may be necessary to conduct some summative peer reviews of teaching during spring or summer term, course dynamics are often atypical given the number of temporary students enrolled at BYU, and thus may not provide the best circumstances for a fair review. Peer teaching reviews for third year candidates should be conducted in the fall semester prior to their winter review. Candidate Evaluation Tips The chair often has unique knowledge of the contributions, strengths, and weaknesses of each candidate. It is important to contextualize the evaluation of each candidate in terms of annual stewardship review feedback that has been provided on an annual basis. Discuss how responsive the candidate has been to constructive input on their performance in teaching, research, and citizenship. Also describe how well the candidate has been mentored by assigned faculty, as well as by others in and out of the university. It is a wise system that universities have developed that has a parallel system of peers and administrators weighing in on rank and status decisions. Both of their perspectives are invaluable. Please resist the temptation to be an advocate, glossing over or hiding evidence of weaknesses. This may unduly complicate the review process by reducing the credibility of a very important assessor in the process and often will harm a fair review of the candidate. In recent reviews, chair letters have been more helpful but in the past this has been a serious problem. We commend chairs for their hard work and efforts to submit thoughtful letters. The best chair letters demonstrate that the chair is thoroughly acquainted with the scholarship of the candidate and has come to an independent judgment of its quality. It is appropriate for chairs to familiarize themselves with candidate files while the department R & S committee review process is occurring. Independently assess the strengths and weaknesses of the candidate's research and help place it in context for college and university committees. Since the chair typically finalizes his or her letter after the department rank and status committee and department faculty deliberations, the chair is especially encouraged to include pertinent information that has not been provided in the committee report. 19 Faculty Meeting Oversight Tips Fall dates should be calendared in the April and May for department faculty meetings to be conducted separately for the purpose of reviewing each candidate. This helps assure that the discussion for one candidate doesn’t intrude on the time of another. These meetings should be scheduled several days apart from each other so that faculty members will have ample time to carefully attend to each file. Notifying faculty members of fall review meeting dates in the spring will help assure non-interference with other activities (e.g., visiting scholars). During the fall, a similar process should be utilized for conducting summative peer reviews of teaching and setting up winter semester meetings well in advance for upcoming third year reviews (see college R&S calendar). To assure that candidates are fairly evaluated by their departmental colleagues, it is important that the department R&S committee’s written report is made available to the faculty well prior to the meeting in which each candidate is discussed. This allows faculty time to digest the report, make their own judgments of the file in the context of the committee report, and prepare their comments to share with the larger group. At the departmental meeting, all CFS track faculty should be invited (except the one being reviewed), including those who have not yet attained CFS. The chair should make a strong statement of confidentiality (nothing leaves the meeting). Department secretaries and other staff should not be present. After the committee presents its report, candid deliberation should ensue. Because faculty who do not have status equal to or above that sought by the candidate cannot vote, they will be dismissed after the discussion. The remaining faculty will vote by secret ballot. This should be recorded by the Chair and clearly noted in the file as the number for, the number against, and number abstaining. Be sure to follow guidelines outlined in section 7.9.10 in the university document for who is eligible to attend the meeting and vote on which candidates. There has been some confusion recently over whether the faculty votes for promotion to associate professor should be separate from votes for CFS. University policy still requires a separate vote (correspondence with Academic VP Richard Williams, Oct. 17, 2006). In this process, it is important to encourage faculty to carefully consider why we would want to give someone CFS if they are not eligible for promotion. 20 References Altbach, P. G. (2006, May 8). The tyranny of citations. Inside Higher Education Newsletter. http://www.insidehighered.com/ Bauer, K., & Bakkalbasi, N. (2006). An examination of citation counts in a new scholarly environment. D-Lib Magazine, 11 (9). http://www.dlib.org/ Byrnes, J. P. (2007). Publishing trends of Psychology faculty during their pretenure years.Psychological Science, 18(4), 283-286. Garfield, E. (1983). Current comments: How to use citation analysis for faculty evaluations, and when is it relevant? Essays of an Information Scientist (Parts 1 and 2),6 (44,45), 354-362 and 363-372. Giles, M.W., & Garand, J.C. (2007). Ranking political science journals: Reputational and citational approaches. PS: Political Science and Politics 40 (4):741-751 Grable, J.E. (2006). Personal finance, financial planning, and financial counseling publication rankings. Journal of Personal Finance, 5, 68-78. Holoski, M. (2006). Assessing the impact of fourteen core Social Work journals. Research on Social Work Practice, 16(4), 449-454. Nisonger, T. E. (2004). Citation autobiography: An investigation of ISI database coverage in determining author citedness. College and Research Libraries Journal. PLoS Editors (2006). The Impact Factor Game. PLoS Medicine. http://journals.plos.org/plosmedicine/information.php Seller, S. L., Mathiesen, S. G., Smith, T., Perry, R. (2006). Perceptions of professional social work journals: Findings from a national survey. Journal of Social Work Education, 42, 139-160. Wuchty, S., Jones, B., & Uzzi, B. (2007). The increasing dominance of teams in production of knowledge. Science, 316, 1036-1039. 21