BUS-MHR 8780 - Introduction to Micro Research Methods in Management THE OHIO STATE UNIVERSITY FISHER COLLEGE OF BUSINESS Spring 2013, Second Term Professor: Office: Office Phone: E-Mail: Office Hours: Class Hours: Howard J. Klein, Ph.D. 748 Fisher Hall 292-0719 Klein_12@fisher.osu.edu Monday 3:00-3:30, Tuesdays 4:00-5:00, and by appointment Mondays, 12:00 – 3:00 in 700 Fisher COURSE DESCRIPTION: This class is designed to familiarize doctoral students with the fundamentals of behavioral research in the organizational sciences. To be successful as a researcher (and a consumer of research) you need a working understanding of all phases of the scientific process. The scientific process employs both theory and data in an effort to describe, explain, predict and/or control a phenomenon of interest. This process consists of four key linkages. Linkage I involves moving from theory to hypotheses generation, and entails such topics as theory building, ascertaining the current state of knowledge and generating hypotheses. Linkage II concerns designing a study to provide data relative to those hypotheses, and deals with the issues associated with the reliability and validity of measurement, sampling, and research design. In Linkage III, that data is analyzed in order to draw inferences with respect to the theory and hypotheses. Finally, the process comes full circle in Linkage IV, where the results are used to support, amend or refute the theory. Critical topics associated with this linkage include the interpretation, generalizability, and presentation of research results. This class uses a seminar format requiring (a) active, through reading and evaluation of assigned materials, (b) written assignments, and (c) rigorous, engaged group discussions. This class will not deal with technical aspects of using statistics (your statistics classes serve that purpose) nor will it provide in-depth practice in conducting research (research experiences with faculty is needed for that). This course provides the foundation for you to learn more from those research experiences. COURSE OBJECTIVES: This seminar will prepare you to undertake original programs of behavioral research. You will learn about the numerous methodological options available to researchers, the strengths and weaknesses of each, and how to assess the appropriateness of each given the research questions of interest. We will consistently return to the themes of choices, constraints, and tradeoffs. You will also gain an understanding of the ethical standards used in the conduct of research with respect to both the treatment of research participants and the dissemination of research findings. After completing this course, you should be able to: (a) communicate about empirical research with the community of scholars in your field, (b) critically review the methods used in empirical studies -- for yourself in the journal articles you read, for peers seeking feedback, or for journal editors, and (c) design your own research to maximize the possible knowledge to be gained from it, while at the same time recognizing its limitations. MHR 8780 2 Spring 2013 INSTRUCTIONAL PHILOSOPHY: This course is designed to lay the foundation for your research endeavors and I'm here to help you set a solid foundation. I am committed to working with you to create a positive learning environment but each of us is responsible for the success of this class. What you get out of the course is, ultimately, up to you. I expect each you to take responsibility for your own learning and to actively contribute to the class. My goal is to strikes a balance between my presenting material and your involvement in discussions and assignments. If things get boring, tell me; if you're having trouble, let me know. I'm looking forward to an enjoyable, stimulating course. I’ll do everything I can to make it so. WEEKLY ASSIGNMENTS: Required Text: Kerlinger, F. N., & Lee, H. B. (2000). Foundations of behavioral research, (4th Edition). Cengage Learning - Wadsworth. ISBN# 9780155078970 Readings. Additional readings for the course are taken from a variety of academic journals and books to provide a comprehensive treatment of the topics covered. Assigned readings will be made available on Carmen along with discussion questions for each class sessions. Web Casts. For several classes, the assignment will include watching a web cast and downloading the slides corresponding to those web casts. The web casts are approximately 1 to 1 and one-half hours and should be viewed prior to the class session for which they are assigned. Below are the log in instructions for accessing the assigned web casts and PowerPoint slides as well as any other topics that may interest you. Go to http://carma.wayne.edu/Login.asp the website for the Center for the Advancement of Research Methods and Analysis Enter my e-mail address: klein.12@osu.edu and the password: MHR911 Click on Video Library (the last link on the page) GRADING: The graded requirements for this course reflect the things you will be expected to know and do as academics. Students will receive a letter grade for all but one of these requirements (the CITI training), each of which is described in more detail below. The following table indicates the relative weighting of the graded requirements in determining final grades. Research Proposal 50% Contribution to Class Discussions 20% Faculty Research Report 15% Peer Critique 15% Complete CITI Training -Total: 100% CITI Training. Students will be required to complete the training requirement for OSU researchers involved in researching involving human subjects if they have not already done so. This assignment is not graded but must be completed as evidenced by submitting a copy of the completion report. Information about this web-based course can be found at http://orrp.osu.edu/irb/training/citi.cfm. MHR 8780 3 Spring 2013 GRADING (continued): Research Proposal. Each student will formulate a research proposal with a focus on the methods section of that proposal. The proposal should describe a feasible, original research project relating to your area(s) of interest. I encourage you to choose a topic you are interesting in pursuing (i.e., the goal should be to subsequently conduct the research you propose). You may but do not have to further refine an idea used for a previous seminar. You may consult with faculty to help develop, evaluate, and refine your thinking, but the proposal should be for your own original new research, not something you have started doing with a faculty member. See the page 9 of this syllabus for additional details on expected content of your proposal. Faculty Research Report. This assignment requires that you make contact with one of the faculty members in your department, ask him or her to share a relatively recent article reporting the results of some type of empirical research, and conduct a brief interview with that professor. You are to read that article carefully and to prepare a brief (5-7 minute) presentation to the class on that professor’s views of research, areas of expertise, and the study you read. See the page 10 of this syllabus for additional details on expected content of your presentation and suggested interview questions. Students are expected to coordinate their efforts so as to avoid duplication of faculty (i.e., each student should report on a different faculty member’s work). If you have already begun working with a faculty member and are somewhat familiar with that person’s research, you should select a different faculty member so that you use this opportunity to make additional connections. Since you will be hearing about my research and views all quarter, I am not an eligible subject for this assignment. Class Contribution. Substantive contributions to class discussions are expected and critical for learning to occur in a seminar format. Note that contributing is not the same as attending or participating (though the others are prerequisites for contributing). Contributing is defined as providing comments that demonstrate knowledge, application, or integration of course material, respectful responses to and elaboration on the comments of others, refraining from overly long monologues or war stories, and actively engaging in in-class activities. Ask questions and contribute your thoughts and personal experiences whenever relevant. You are expected to have read all assigned materials and prepared answers to the discussion questions. Review those materials prior to class and to come to class with ideas, concerns, and questions on the material. You do not have to formally write out your answers to the discussion questions, but you should bring thorough notes and be prepared to speak from them, should you be called upon in class (and you will be!). There is not always a "correct" answer to the questions but you are expected to have an opinion and to back up your opinion. My teaching style is informal. Feel free to interrupt or ask questions at any time. The questions you ask will likely be of help to other students in class. Feedback regarding your contribution level is available upon request. Peer Draft Critique and Response. You will be asked to critically review a classmate’s draft research proposal. In doing so, you will provide constructive, written feedback to your classmate (and to me). You likewise will receive a peer critique of your own draft proposal. Use that critique to improve your proposal. When you submit your proposal, you will need to also provide a letter to the “editor” (me!) explaining what was changed in the final paper (or the rationale for not making changes) to address the “reviewer” comments. The 10% of your final grade for this assignment breaks down to 6% for the peer critique you provide to your classmate and 4% for your letter responding to the critique you receive. MHR 8780 4 Spring 2013 COURSE POLICIES: Anyone needing special accommodation because of a disability or other unique circumstances should notify me as early as possible. Students with special needs are responsible for making me aware of their situation. I expect you to honor the academic honesty standards outlined in OSU’s Code of Student Conduct (Section 3335-23-04). Any suspicion of academic misconduct will be acted upon in accordance with university policy. The ideas and work on the assignments for this class must be your own -- they are not group projects. I also reserve the right to use Turnitin or similar products to detect plagiarism or excessive undocumented material. CLASS SCHEDULE: DAY (DATE) TOPIC MONDAY (3/4) Introduction to Course & Connections to Prior Coursework MONDAY (3/11) No Class – Spring Break MONDAY (3/18) Choosing Variables & Measures I MONDAY (3/25) Choosing Measures II & Samples MONDAY (4/1) Research Ethics & Cultural Considerations CITI & Faculty Research Reports Due MONDAY (4/8) Choosing Data Collection Sources and Methods; MONDAY (4/15) Choosing Designs Draft Research Proposal Due MONDAY (4/22) Choosing Conclusions Peer Critique Due MONDAY (4/29) Research Proposal due RESEARCH METHODS RESOURCES: Below is a list of valuable research methods resources of which you should be aware: Research Methods Division of the Academy of Management Website http://division.aomonline.org/rm/joomla Cohen, J., Cohen, P., West, S.G., and Aiken, L.S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences. Lawrence Erlbaum and Association: Hillsdale, NJ. Corbin, J., & Strauss, A. (2008). Basics of qualitative Research (3rd ed). Newbury Park: Sage. Dillman, D. A., Smyth J. D., & Christian, L. M. (2008). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). New York: Wiley. Drasgow, F. & Schmitt, N. (2002). Measuring and analyzing behavior in organizations. San Francisco: Jossey Bass. Goldacre, B. (2010). Bad Science: Quacks, Hacks, and Big Pharma Flacks. London, Faber & Faber. Hunter, J. E. & Schmidt, F. L. (2004). Methods of meta-analysis (2nd ed.). Thousand Oaks, CA: Sage. MHR 8780 5 Spring 2013 RESOURCES (continued): Klein, K.J. & Kozlowski, S.W.J. (2000). Multilevel theory, research, and methods in organizations. San Francisco: Jossey-Bass. McGrath, J.E., Martin, J., & Kulka, R.A. 1982. Judgment calls in research. Beverly Hills, CA: Sage. Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory. (3rd Edition). NY: McGraw-Hill. Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum. (P&S) Rogelberg, S. G. (2002). Handbook of research methods in Industrial and Organizational Psychology. Oxford, UK: Blackwell. Rubin, H. J. & Rubin, I. S. 2005. Qualitative interviewing: The art of hearing data. 2nd Ed. Thousand Oaks: Sage Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental Designs for generalized causal inference. Boston, MA: Houghton Mifflin. Van de ven, A. H. (2007). Engaged scholarship: A guide for organizational and social research. Oxford University Press. READING ASSIGNMENTS: I. THEORY HYPOTHESES WEEK 1 Introduction to Course Kerlinger & Lee (2000) Chapters 1-2, & pages 455-463 (Research Design as Variance Control). Campion, M. A. (1993). Article review checklist: A criterion checklist for reviewing research articles in applied psychology. Personnel Psychology, 46(3), 705-718. Robison, S. J. 1990. Top 10 things you should know about doing research in an organization. The Industrial-Organizational Psychologist, 27(3), 79-81. Hodgkinson, G. P., & Rousseau, D. M. (2009). Bridging the rigor relevance gap in management research: It is already happening! Journal of Management Studies, 46, 534-546. Recommended but not required: Sackett, P. R., & Larson, J. R., Jr. (1990). Research strategies and tactics in industrial and organizational psychology. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 419-489). Palo Alto, CA: Consulting Psychologists Press. [a great overview of everything we’ll be covering in this class] McGuire, W. J. (1997). Creative hypothesis generating in psychology: Some useful heuristics. Annual Review of Psychology, 48, 1-30. II. HYPOTHESIS DATA WEEK 2 Choosing Variables & Measures I Webcast - #22 Issues with Group Measurement; Dr. Katherine Klein Kerlinger & Lee (2000) Chapters 3 & 27. Becker, T. (2005). Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8, 274-289. Hackman, J.R. (2003). Learning more by crossing levels: Evidence from airplanes, hospitals, and orchestras. Journal of Organizational Behavior, 24, 905-922. MHR 8780 6 Spring 2013 Harnish, K.A., Hulin, C. L., & Roznowski, M. (1998). The importance of individuals' repertoires of behaviors: the scientific appropriateness of studying multiple behaviors and general attitudes. Journal of Organizational Behavior, 19, 463-480 Bagozzi, R. P., & Edwards, J. R. (1998). A general approach for representing constructs in organizational research. Organizational Research Methods, 1, 45-87. Johnson, R. E., Rosen, C. C., Chang C. H., Djurdjevic, E., & Taing M. U. (2012). Recommendations for improving the construct clarity of higher-order multidimensional constructs. Human Resource Management Review, 22(2), 62-72. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78 (1), 98-104. Dalton, D. R. (1986). The practical applicability of test-retest reliability on a decision making algorithm: Exhaustive Alternatives. Journal of Irreproducible Results, 3, 278. Recommended but not required: Kerlinger & Lee (2000) chapter 26. LeBrenton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815-852. WEEK 3 Choosing Measures II & Samples Webcasts - Watch at least one or both of the following: #20 Non-responses to Organizational Surveys; Dr. Steven Rogelberg #15 Contributing to Applied Psychology with Laboratory Research; Dr. John Hollenbeck Kerlinger & Lee (2000) Chapters 8 & 28. Schwab, D. P. (1980). Construct validity in organizational behavior. Research in Organizational Behavior, 2, 3-43. MacKenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P. (2011). Construct measurement and validation procedures In MIS and behavioral research: Integrating New and existing techniques. MIS Quarterly, 35(2), 293-334. Molloy, J. C., & Ployhart, R. E. (2012). Construct Clarity: Multidisciplinary Considerations and an Illustration Using Human Capital. Human Resource Management Review, 22, 152-156. Highhouse, S., & Gillespie, J.Z. (2009). Do samples really matter that much? In C.E. Lance & R.J. Vandenberg (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences (pp. 249-268). New York: Routledge. Murphy, K. (2002). Using power analysis to evaluate and improve research (pp. 119-138). In S.G. Rogelberg (ed.), Handbook of research methods in industrial and organizational psychology. Malden, MA: Blackwell. Mook, D.G. (1983). In defense of external invalidity. American Psychologist, 38, 1379-1387. Highhouse, S. (2009). Designing experiments that generalize. Organizational Research Methods, 12 (3), 554-566. WEEK 4 Research Ethics & Cultural Considerations Webcasts - Watch at least one or both of the following: #21 Methodological issues in Cross-Cultural Research; Dr. Michele Gelfand #65 Cross Cultural Research Methods; Dr. Mark Peterson Kerlinger & Lee (2000) chapter 17. MHR 8780 7 Spring 2013 Schaffer, B. S., & Riordan, C. M. (2003). A Review of Cross-Cultural Methodologies for Organizational Research: A Best- Practices Approach. Organizational Research Methods, 6(2), 169-215. Rousseau, D. M., & Fried, Y. (2001). Location, location, location: Contextualizing organizational research. Journal of Organizational Behavior, 22, 1-13. Academy of Management. (2005). The Academy of Management code of ethics. Rosenthal, R. (1994). Science and ethics in conducting, analyzing, and reporting psychological research. Psychological Science, 5 (3). 127-133. Stroebe, W., Postmes, T., & Spears, R. (2012). Scientific Misconduct and the Myth of SelfCorrection in Science. Perspectives on Psychological Science, 7(6) 670-688. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359-1366. Kacmar, K. M. (2009). An ethical quiz. Academy of Management Journal, 52 (3), 432–434. WEEK5 Choosing Data Collection Sources and Methods Webcasts - Watch one or more of the following: # 40 Measurement of Affect and Episodic Events; Dr. Howard Weiss # 41 Question and Context Effects in Organizational Survey Data; Dr. Adam Meade # 70 Practical Issues in Developing a Measure; Dr. Fred Oswald Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104-121. Schriesheim, Powers, Scandura, Gardiner, & Lankau. (1993). Improving construct measurement in management research: comments and a quantitative approach for assessing the theoretical content adequacy of pencil–and-paper survey-type instruments. Journal of Management, 19, 385-417. Whitley, B. E. (2002). Chapter 11: Survey research. In Principles of research in behavioral science (2nd ed.). Burr Ridge, IL: McGraw-Hill. pp. 343-371. Schwartz, N. (1999) Self-reports: How the questions shape the answers. American Psychologist, 54(2), 93-105. Podsakoff, P.M., MacKenzie, S.M., Lee, J., & Podsakoff, N.P. (2003). Common method variance in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879-903. Spector, P. E. (2006). Method variance in organizational research: Truth or urban legend? Organizational Research Methods, 9, 221-232. Recommended but not required: Kerlinger & Lee (2000) chapters 25, 29-31. WEEK 6 Choosing Designs Webcasts - Watch at least one or both of the following: #24 Longitudinal Data Analysis; Dr. Robert Ployhart #64 Quasi-Experimental Research, Dr. Adam Grant Kerlinger & Lee (2000) chapters 18-24. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental Designs for generalized causal inference. Boston, MA: Houghton Mifflin. Pages 53-61; 156-160; 247-257. MHR 8780 8 Spring 2013 Kelly, J. R., & McGrath, J. E. (1988). Time and the logic of method. In On time and method, (pp. 9-28). Newbury Park, CA: Sage. Edmonson, A. C., & McManus, S. E. (2007). Methodological fit in management field research. Academy of Management Review, 32(4) 1155-1179. Mitchell, T. R. (1985). An evaluation of the validity of correlational research conducted in organizations. Academy of Management Review, 10, 192-205. Scandura, T. A., & Williams, E. A. (2000). Research methodology in management: Current practices, trends, and implications for future research. Academy of Management Journal, 43, 1248-1264. Recommended but not required: Stone-Romero, E. (2002). The relative validity and usefulness of various empirical research designs (pp. 77-98). In S.G. Rogelberg (ed.), Handbook of research methods in industrial and organizational psychology. Malden, MA: Blackwell. III & IV. DATA INFERENCE THEORY WEEK 7 Choosing Conclusions Webcasts - Watch one or more of the following: # 56 Missing Data: Problems and Prospects; Dr. Daniel Newman # 48 Computer-Aided Text Analysis: Tips and Techniques; Dr. Jeremy Short # 82 Crafting Qualitative Organizational Research; Dr. Ann Cunliffe Jick, T.D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24, 602-611. Corbin, J., & Strauss, A. (2008). Chapters 1 and 2 from Basics of qualitative research (3rd ed). Newbury Park: Sage. Cunliffe, A. L. (2011). Crafting Qualitative Research: Morgan and Smircich 30 years on. Organizational Research Methods, 14(4), 647-673. Smith, P. C., Budzeika, K. A., Edwards, N. A., Johnson, S. M., Bearse, L. N. (1986). Guidelines for clean data: Detection of common mistakes. Journal of Applied Psychology, 71, 457460. Asimov, I. (1988). Chapter 17: The relativity of wrong. The relativity of wrong. NY: Doubleday. pp. 213–225. Anderson, C. (2008). The End of Theory: The Data Deluge Makes the Scientific Method Obsolete. Wired Magazine: 16.07. http://www.wired.com/science/discoveries/magazine/16-07/pb_theory Recommended but not required: Switzer, F. S., & Roth, P. L. (2002). Coping with missing data. In S. G. Rogelberg (Ed.), Handbook of research methods in Industrial and Organizational Psychology. (pp. 310-323). Oxford, UK: Blackwell. MHR 8780 9 Spring 2013 Research Proposal Guidelines [with approximate page budgets] Proposals should conform with the predominate format style in your field (e.g., Academy of Management Journal Guidelines, APA Guidelines) and are not to exceed 25 pages (doublespaced in a 12-point font with 1” margins on all sides). Introduction - Because the focus of this assignment is on the on the methods section, the introduction section should be abbreviated from what you would typically see in a journal article submission. However, it must contain sufficient detail to (a) demonstrate the “value added” to the literature and (b) evaluate the appropriateness of the proposed methodology. 1. Identify the specific problem, purpose of the study, and corresponding research question(s). Explain how this study will make an important contribution from both theoretical and applied perspectives. [2-3 pages] 2. Briefly review the relevant literature on this topic and use that literature to develop specific, testable hypotheses. In doing so, be sure to provide conceptual definitions of all constructs that will be measured in the study. [4-6 pages] Method - This section should reflect what would be found in a journal submission but should contain even more detail and be worded in future rather than past tense. The methods should be spelled out so clearly that another graduate student could pick up your proposal and know exactly how to conduct the research. 3. Describe the sample you would use, explain why that sample is appropriate, and identify how participants would be recruited and selected. Note how participants will be assigned to conditions (if applicable), and any incentives offered for participation. [1 page] 4. Describe the setting and procedures to be followed. Include the instructions to be presented to participants and what the participants will do and when and how they will do it. Also describe any experimental tasks, manipulations, and/or apparatus or materials that will be used. [1-3 pages, depending on complexity] 5. Describe the specific type research study and research design you will use and justify those choices. [1-2 pages] 6. Describe all constructs that will be measured and/or manipulated. Provide operational definitions of all variables in the methods section. Indicate what types of data collection methods will be used (i.e., how will data be assessed and from where/whom?). Provide evidence regarding the reliability and validity of each established measure and describe how you would go about establishing the reliability and validity of any new measures. [24 pages] 7. Discuss the analytical approach that you would use to test your hypotheses and the specific statistical information used to determine support for each hypothesis. [1 page] Discussion - This section should also be abbreviated from what you would typically see in a journal article submission but should include the following: 8. Outline the implications of the study (assuming that the study was conducted as described and the hypotheses were supported) for both theory/research and practice. [1-2 pages] 9. Provide a thorough treatment of the study’s strengths and limitations. [2-4 pages] 10. Identify suggestions for future research that would follow from the expected findings in this study. [1-2 pages] Appendices - Include a complete list of references, any figures or tables, and your point by point response to the peer review you received with your proposal submission. These materials do not count against the page limit. MHR 8780 10 Spring 2013 Faculty Research Report Contact the professor you would like to use for your report, explain the assignment, request an article and make an appointment to meet with and interview that faculty member. Obtain and read the article prior to meeting with the faculty member. Below are some possible interview questions (you should ask some of these and some of your own) and an outline for preparing your presentation to the class. In addition to giving an oral report, have a handout to distribute to the class (2 pages maximum) that summarizes what you learned about conducting research and research methods from this assignment. Faculty Interview guide General questions What kinds of things do you study and how do you go about studying them? (topics, methods, samples, target audience/outlets) Is there a particular methodology you use most frequently? Why? What aspects of the research process do you find most enjoyable and rewarding? Most difficult and frustrating? How many research projects do you typically have going at any one time? How often do you work with co-authors? Why? If you do, how is the work usually distributed? How do you make time for research? In your opinion, what makes for a great research article? What do you see as the biggest obstacles to being a successful researcher? Questions relating to the article What can you tell me about this study that is not in the article? What was the most challenging aspect of conducting this study? If you had the chance to do this study over again, what would you have changed? Identify what you perceive to be a trade-off made in the design of the study and inquire as to why the decision was made to go in the direction that was taken. How does this article fit within your broader research interests? Have you or are you planning to conduct any research that follows-up on the results of this study? Report and presentation outline What did you learn about the kind of research this faculty member conducts? What did you learn about this faculty member’s approach to research? What did you learn about this faculty member’s thoughts about the research process? Summarize the example of this person’s research Purpose, findings, importance Methods - setting, sample, type of design; strengths and weaknesses