The Wednesday Workshop Dissertation Skills for MA and MSc Students: Preparing the Research Workshop workbook Louise Livesey This workshop will: - Cover some of the pre-writing (preparatory and analysis) activities that form part of a standard dissertation process - Give a basic introduction to designing research, both for primary and secondarybased projects and dissertations - Explore basic meanings of methodology and methods, data collection and analysis, epistemology and ontology and their part in the research design process Introduce basic principles for students when considering ‘ethical research’ Teaching points: The research process Choosing your research question Ethics and ethical research Critical analysis of text Database and other sources key word searches Literature Review including - Purpose - Searching, choosing and analysing sources - Synthesis and mapping your ideas - Structure Research design process including - Steps - Qualitative and quantitative approaches - Research methods - Pilot studies - Data analysis Conclusion There is a great deal to do before you start writing your dissertation; the research process has various stages you must work through. You may move backwards and forwards, and start writing during a ‘research’ stage, but as long as you work through the phases, you will be successful. 1 www.brad.ac.uk/academic-skills Activity Jot down what activities you would perform or questions you would ask next to each stage, and draw a line above the first stage of writing. organisation & planning •1 •2 plan proposal research question literature review •3 •4 •5 •6 research •7 first draft •8 revise & edit proof & submit •9 You will notice that there are more preparation stages than writing stages. This gives you an indication of the percentage of time you need to take for the pre-writing and writing elements of the process. It always takes longer to prepare than you think it will, so start as soon as you can. 2 www.brad.ac.uk/academic-skills Choosing a research question isn’t straightforward. There are a lot of ideas and elements to consider before you make your final choice on what you are going to investigate and how you are going to do it. In effect, you have to analyse the field and your own research interest. This means asking questions. Activity Suggestions are at the back What questions should you ask or what issues should you consider when deciding on your research question? 1 2 3 4 5 You should be considering and incorporating ‘ethics’ and what ‘ethical research’ is into your choice of research question as the ethical implications will determine what you may be allowed to investigate (depending on the educational establishment you attend). What you must consider when planning for and undertaking ‘ethical research’: Protect those you provide you with your information Protect the innocent and vulnerable Protect yourself and the university from harm and litigation It is vital you ensure your research is conducted in an ethical manner, causes no harm and retains the confidentiality of those who participate; for students working with people, animals or tissues, this is especially pertinent. Applying critical analysis throughout all stages of your research is key to a successful dissertation, especially in the early stages when deciding your research field and question, and during your literature review. But how do you do it? What does it involve? What does it mean? Critical analytical thinking requires you to… Stand back from whatever information you are given Examine it in detail from many angles including those based on other theorists’ or writers’ points of view MORE THAN ONCE Check closely if it is completely accurate Check to see if each statement follows logically from what has been said before Look for any flaws in the reasoning, evidence or how conclusions are drawn Evaluate any supporting evidence Understand and explain why different people may have arrived at different conclusions Have the ability to lay out why one argument is preferred to another Not to agree with questionable statements without further investigation 3 www.brad.ac.uk/academic-skills Check for hidden assumptions Check for words, phrases and other elements trying to persuade you into agreement Consider where a viewpoint leads, i.e. what conclusions would follow and are they suitable and rational, and if the viewpoint should therefore be reconsidered. Activity When reading for any stage of your research process, there are basic building blocks and elements you should be looking for. With a neighbour or in a small group, jot down on the small whiteboards what you think you these aspects are in a text. Activity Now you are aware of what to look for to analyse a piece of writing, use the frame below to assess the text found on the handout marked ‘Rochborough Health’. Critical questions What is the main line of reasoning (the main argument?) Is the line of reasoning clear both in the introduction and in the conclusion? What is the key evidence used to support the main argument? Is the evidence presented in a way that develops the argument and leads clearly to the conclusion? Analysis of text When was the evidence produced? Is it up to date? Is it still relevant? Is there sufficient evidence to prove the case? Is the evidence relevant? What might be missing? What (if any) would have been a better order in which to present the evidence so as to strengthen the line of reasoning? Are there any examples of flawed reasoning? Attempts to persuade the reader through an appeal to the emotions? Is evidence 4 www.brad.ac.uk/academic-skills interpreted and used correctly? Has the writer given sufficient consideration to alternative points of view? Give examples General key questions you should be asking: ? ? ? ? ? ? ? ? ? ? ? Why How far How much How often To what extent How do we know this is true How reliable is this source What could be going on below the surface What do we not know about this Which is preferable For what reasons Before you start to read, you need to know what to look for. You need to break the brief or question down, even if it’s your own question. Have a look at this question: Are glasses or contact lenses better for correcting short sight? What do you think you should be looking for? What key search words are in this question or suggested by this question? Key words: Glasses, contact lenses; short sight, correction Other words: Myopia, near sight; treatment, alleviation, reduction; spectacles Activity You have a go with the following question…. What is the most engaging way to deliver educational mobile content? Key words: Other words: Subject librarians will be able offer advice on conducting key word searches. A literature review is NOT just a summary of everyone else’s thoughts and discoveries, but a dialogue between you and what has been written to contextualise and underpin your work rather than substitute for it. Your research and dissertation will not simply repeat what has gone before. You will synthesise the ideas, debates, arguments, theories and points of view in the review to develop your own work and see what you will contribute to the field and topic. 5 www.brad.ac.uk/academic-skills The purposes of the search and review are to: Develop a breadth of understanding about your field Understand how research is conducted in your subject Gain ideas for a topic (question) of your own Identify what has been covered in depth already, so you are clear about how your project can contribute in an original way and where it fits Put your own research into context for yourself and your reader A literature review is a process with a number of stages. Locating the literature Selecting & assembling sources Reading sources critically Analysing the sources Synthesising to develop your own ideas Mapping ideas Formulating potential structures We will now look at these in more detail not forgetting the purpose of the search and review. Where is potentially useful information found? *Reading lists *Google Scholar *Specialist peer-reviewed academic journals *External experts *Reference lists in reading list sources *Specialist databases *University Subject Librarian What are the different types of sources available? Examples of primary sources Examples of secondary sources Articles in peer-reviewed academic journals (printed or online) Books written by the originator of a particular theory, idea, model, or practice Critics of ideas expressed in journals, books or media providing they have built a strong reputation for intelligent comment in the subject area Originators of original creative work, e.g. authors, artists, musicians, film and stage directors Conference papers presenting original research or ideas by their originators Reports written by originator of proposals, particularly if supported by an influential body, agency or institution Original Court and parliamentary proceedings Autobiographies Minutes of important meetings Professional or trade journals, unless they were presenting original research findings Newspaper reports Tutor handouts (unless presenting original work by the tutor)) Text books summarising the work of others Reference books Wiki Internet sites Other Internet sites summarising the work of others Biographies Student dissertations Unpublished conference papers 6 www.brad.ac.uk/academic-skills Don’t forget there is a difference between primary and secondary research processes, and primary and secondary sources. Examples: Performing an experiment on measurement and monitoring of air pollution is called primary research – you are doing the experiment and obtaining results yourself. However, you will use primary and secondary sources to investigate which experiments have been done in this area before, who are the leading lights in this arena, etc – these are secondary sources. Analysing statistics produced by various television companies to discover any demographic trends in viewers is called secondary research as you are using sources that have been created or developed by someone else. The sources you use, the statistics, are primary sources. However, as above, you will use secondary sources to find out if there are any previously produced trend information, any theories or hypotheses that could help with explaining any trends you find, etc. What criteria is used to select sources? Expertise of Author General questions And in addition for electronic information Credibility of Source General questions And in addition for electronic information What qualifications and experience does the author/originator have? Is the author well-known in his or her field? Do other writers in the field of study refer to this author in their work? Do your tutors mention this author? Is it clear who is taking ‘ownership’ of the online information presented? Is there a link to any named author’s Email address? Is the publisher/originator of the evidence a reputable one? Are sources of evidence presented done so in a credible way, e.g. properly referenced? Does the bibliography presented by the author seem comprehensive and wide in its coverage? Does the author present all relevant background and context information of the ideas presented, e.g. explain in a convincing way the rationale to his or her ideas? Is the information presented still valid? For example, if it was written over five years ago are the ideas still applicable today? If it is a research finding, is the research methodology carefully presented to the reader? Do your tutors recommend this Internet site? Why has this site been established – is it 7 www.brad.ac.uk/academic-skills Relevance of Evidence clear from the introduction? Who sponsors and/or pays for the site? Who is the intended readership for the site? Are there any open or possible biases in the site? Were you linked to this site from a reliable source? Does it look professional? Is the site easy to navigate and use? Does the resource follow good principles of design, proper grammar, spelling and style? Was the site updated recently? The first two elements connect with this third key criterion for selecting evidence. Whether it is from a printed or electronic source, you will select evidence that best supports the point you are making in the assignment. The formula for choosing relevant sources would be: elements ‘author expertise’ + ‘credibility of source’ + the most relevant evidence to the point you are making How are sources read and analysed critically to assess their use specifically for your? What are you looking for? Broad overview of your field: Origins – broad survey of material on your topic, tracing research on it or specific issues back to their origins Breadth – read widely around the subject, selecting texts that represent well the various perspectives taken by experts in this area Recurrent issues – note recurrent themes and issues that might have a bearing on your project Schools of thought – how are these issues treated by the key schools of thought in your discipline Journals – browse abstracts and identify relevant articles to read in full People – find out who the leading figures in primary research related to your selected topic are Topicality and currency – what issues are most recently reported in your field and how are these relevant to your topic Pendulum swings – different perspectives or interpretations come into favour or are rejected, so find out where the pendulum sits currently and what are the alternative positions Methods – what are the standard methods for research topics similar to yours, and what are the current methodological issues or problems researching in this area. What are the challenges facing researchers in this field, and why do they arise. Ideology – what ideological differences are there regarding your topic and does this affect how the research is undertaken. Will this affect you? Application – How is research being applied in the field? Gaps – What aspects are not yet fully researched and do researchers point to any future research that will help advance understanding of the topic. Trajectory and links – How has published research in this area changed in direction over time, and be clear about how understanding of this topic has developed through successive pieces of research. Consider how each piece of research or set of ideas 8 www.brad.ac.uk/academic-skills has influenced others. Look at the lit review section of the books, articles and theses to see what research they refer to. Making specific choices about the rest of your search: Become increasingly focused regarding – specialised content; relevance to your title; reflecting on what your are finding and how it may be useful; and analytical and critical of the material you are considering for inclusion Include specialist material – what is at the cutting edge of research in your subject Demonstrate your understanding – of the field as a whole; in the depth of your specialist topic; of what is most significant or what is less so Be selective – start to make decisions about how you will make use of the material in your own work (see the checklist on the handout) Decide how much you will use (see the checklist on the handout) Clarify your purpose – make decisions about what to read, not or include in your own work (see the checklist on the handout) How do I synthesise and map/plan ideas? Most recent contributors Different direction Origin Origins: who started the line of enquiry Direction of travel: who built on it by adding to or going in a different direction to the original piece of work Contributory players: note details of who added to the direction of travel you are most interested in Most recent contributors: what has been published in last few years and does it affect your approach Themes: note themes and issues to take into consideration as you chart the information Development of original idea/ theory Most recent contributors Synthesis is about developing your own work and opinions based on the ideas, debates, arguments, theories and points of view you have read about during the review. It is choosing what is relevant, linking it to other elements, and interpreting it into something new. Example: You are one of four friends that go out for a meal and watch a film, but one other friend is unable to go. You telephone this absent friend and tell them all about the night before. You don’t merely report or summarise – at seven o’clock we entered the restaurant, at half past seven John ate his started and said ‘that sauce is really tasty’, at seven thirty five Michelle received a call, etc. OR we all went to an Italian restaurant, ate our starters, ate our main meals, etc. You harmonise or interpret what you ate, what you discussed, what you saw, etc. and created something new. This is synthesis. 9 www.brad.ac.uk/academic-skills There are many ways to plan or map your ideas – you can use a mind or concept map, use diagrams, use ‘family trees’, use pieces of paper stuck to a board, use specialist software. One every University computer, there is a software programme called MindGenius 3 (click on the start icon, click on all programmes, and scroll down to Productivity). This will help you to build a map of your ideas and can also be used to provide a structure for your literature review. What are the potential structures you can use? One common way to approach a literature review is to start out broad and then become more specific. Think of it as an inverted triangle: First briefly explain the broad issues related to your topic; you don't need to write much about this, just demonstrate that you are aware of the breadth of your subject. Then narrow your focus to deal with the studies that overlap with your research. Finally, hone in on any research which is directly related to your specific investigation. Proportionally you spend most time discussing those studies which have most direct relevance to your research Other structures you can use for your literature review include: Chronological – You simply start with the earliest sources and references to your topic and research, and work forwards to the most recent. This structure is safe and easy. Earliest Most recent Relationship to your own research topic –. You start with the sources and references that are farthest away from your own topic and research, and work inwards to those that are closest to your work. This structure is fairly easy too. Farthest Closest A theme B theme C theme Thematic – If you have spotted multiple discrete themes during your reading, you could discuss them individually. You MUST keep the themes discrete, whilst linking them together by what they 10 www.brad.ac.uk/academic-skills have in common. You will also employ sub-structures too. For example, if you have grouped your themes together, how do you order them? What structure do you use? Similarly, there will be many references within a theme, how do you structure these? Simply use the chronology or relationship methods as discussed earlier, depending on which one seems to fit the best. This is probably the most difficult structure but it can be the most effective. Most literature reviews have sub-headings to help group the relevant research into themes or topics. This gives a focus to your analysis, as you can group similar studies together and compare and contrast their approaches, any weaknesses or strengths in their methods, and their findings. So now you’ve decided on your question but how do you design research? Your research design is the way that you set up and conduct your research to find the answer to your research question or test your hypothesis. They include: Your methodology – the principles that inform the way you make choices and decisions in designing, undertaking and writing up your research. Your choice of raw material – documents, data or characteristics for participant selection Your chosen methods – including any conditions you create for undertaking your research Materials you develop – e.g. information provided to participants Your methods of recording information – at each stage of your research Analytical methods or employ – e.g. statistical approaches, formulae or analytical techniques you apply to your raw data or source materials Both objectivity and subjectivity can be appropriate to good research, depending on the discipline and the nature of the research. Objective research: the aim is to remove your personal feelings and response from the equation so that the findings could be the same irrespective of who carried out the research. Easier to attain for ‘hard’ sciences. Subjective research: acknowledges that the methods and results will vary as a result of the researchers themselves – the results would not be exactly the same without their personal input. Used across all disciplines but valued differently by each. Let’s have a look at the relationship between and the basic meanings of important theoretical and operational terms relating to your research. The research design process – WARNING: DIFFICULT STUFF COMING UP! Remember, research design is a process with a number of stages all playing their part to the outcome of information you will analyse to put forward a conclusion. 11 www.brad.ac.uk/academic-skills Activity Examples at the end The diagram below shows the flow from ontology to data collection. With a neighbour, note down your definition of these terms. Ontology Epistemology Definition Definition Methodology Definition Data Collection (Methods) Analysis Activity (use glossary for definition of paradigm) Using the blank frame and sheet with phrases on, work out which elements make up which paradigm, cut them out and place them on the frame. Quantitative and qualitative approaches The former focuses on what can be measured in an objective way whilst the latter endeavours to extend our understanding of human behaviour, individually and collectively, by investigating underlying causes for people acting and thinking as they do. 12 www.brad.ac.uk/academic-skills What is a research method? Activity Read the following statements and decide which are advantages or disadvantages for quantitative or qualitative approaches. Each is allocated a number; please jot down the number of the statement in the box under the appropriate heading. Advantages Quantitative Disadvantages Advantages Qualitative Disadvantages 1. They are useful for in-depth analysis of individual people, businesses, events and occurrences. 2. Projects that use quantitative approaches are generally easier to plan and to contain in size, making them ideal for student projects. 3. They enable study on a broader scale, such as through online surveys, generating large amounts of data which can be analysed relatively easily using relevant software. 4. The results can lack ecological validity. 5. They can be unpredictable, making them harder to manage and contain. 6. They help establish patterns such as trends in behaviour, or in science, ‘universal laws’. 7. The research questions or hypotheses tend to be very precisely articulated, allowing the possibility of precise answers. 8. They tend to more open-ended, allowing a greater set of responses to emerge. 9. Findings may be useful to the particular case but not more generally applicable. 10. They have greater ecological validity. 11. There is a risk of gaining rather banal results. 12. The scale of the research makes it easier to draw reasonably valid generalisations in a relatively short timeframe. 13. Not everything is easily measured. There is a risk of gaining skewed understanding of a phenomenon as a result of omitting those aspects that can’t be measured. 13 www.brad.ac.uk/academic-skills Activity What would be your suggested methods of data collection for each paradigm based on the table above? Use of primary or/and secondary sources Positivistic Paradigm Methods Post-positivistic Paradigm Methods Pilot studies These are a great way to see what works and what doesn’t, and can assist in producing a strong written proposal. When writing it up you need to include: What you set out to do What you did The results A succinct, critical outline of what works, any flaws that were apparent and what would improve or fine-tune the research How you plan to adapt your approach and methods for when you start to collect data Keep it brief and to the point: Avoid praising the parts that went well Don’t apologise or make excuses for anything that needs to be improved Do include copies of materials used, or data collected during the pilot. You’ve collected and collated your data, what next? Analysis. This process involves discovering what the research tells you – your findings. Be selective and focused: ask yourself which data is relevant to your research question – you won’t need the whole mountain of data you may have accumulated. You must be meticulous and careful at all times or your findings will be questionable. Ask yourself if what you expected was confirmed or was there anything contradictory and surprising. Tip – analyse your data in batches as it becomes available. You can then integrate it and see whether you need to change your work directions. It also indicates what kinds of findings are emerging. 14 www.brad.ac.uk/academic-skills Consider the following for your conclusion: ? ? ? ? The significance of your findings, i.e. what is the likelihood that your results could have been discovered by chance or how statistically significant is it? Generalisability, i.e. can others use your model, and either replicate it or develop it? Reliability, i.e. could another researcher carry out the same research with the same kind of group and replicate your findings (although need not be identical findings)? Validity, i.e. have your methods provided appropriate findings for the research question/s you have asked? References Burnett, J. (2009) Doing Your Social Science Dissertation. London:Sage Publications Ltd. Cottrell, S. (2003) The Study Skill Handbook. 2nd Ed. Basingstoke: Palgrave Macmillan. Cottrell, S. (2014) Dissertations and Project Reports. Basingstoke: Palgrave Macmillan Dunleavy, P. (2003 ) Authoring a PhD. Basingstoke: Palgrave Macmillan. Farlex Incorporated. Huntingdon Valley: Farlex Incorporated Search term: paradigm http://www.thefreedictionary.com Accessed 25.3.14 Dr. Raddon, A. (2010) Early Stage Research Training: Epistemology and Ontology in Social Science Research. Leicester: University of Leicester http://www2.le.ac.uk/colleges/socsci/documents/research-training-presentations/EpistFeb10.pdf Accessed 25.3.14 http://www.literaturereviewhq.com/3-great-methods-to-structure-your-literature-review/ Accessed 28.3.14 Wallace, M. and Wray, A. (2011) Critical Reading and Writing for Postgraduates. 2nd ed. London: Sage Publications Ltd. Wikipedia Foundation Incorporated (2014) Search terms: ontology, epistemology, methodology http://en.wikipedia.org/ Accessed 28.3.14 Wisker, G. (2008) The Postgraduate Research Handbook. 2nd ed. Basingstoke: Palgrave Macmillan. Lydia Lunning, L. Writing The Literature Review. Minneapolis: Waldon University https://www.youtube.com/watch?v=GpErYDb6PsY Accessed 28.3.14 With thanks to the LDU and Jennifer Rowland of the Library. Answers Research question activities or questions Consider level of interest, availability of resources, usefulness for ‘field’ via literature search Check key words which define the topic. Ask questions to direct research. Use questions to make a rough plan. Consider the methodology and meta-research needed? What experiments need to be designed? Is data collection involved? What you should look for when critically thinking when reading 15 www.brad.ac.uk/academic-skills Identifying the line of reasoning Critically evaluating the line of reasoning Questioning surface appearances and checking for hidden assumptions or agendas Identifying evidence Evaluating the evidence according to valid criteria Identifying the conclusion Deciding whether the evidence given supports these conclusions Key and other word searches Key words: engagement, mobile content, education Other words: interest, learning (?games) Other words: app, phone, tablet Other words: learning (? age range) Examples using different philosophical and methodological approaches Positivistic Post-positivistic Ontology World is knowable and measurable World is perceived and interpreted Epistemology Knowledge Understanding Methodology Survey Survey Data collection Highly structured questionnaire Loosely structured interviews methods Quantitative National Student Survey on student satisfaction Qualitative Student learning styles and approach to study at individual university Quantitative and qualitative Quantitative Advantages Disadvantages 3,7,12,6,2 13,11,4 Advantages 8,1,10 Elements of paradigms Positivistic Paradigm Concerned with testing hypotheses Uses large samples Data is highly specific and precise High reliability Low validity Generalises from sample to population Produces quantitative data Post-positivistic Paradigm Concerned with generating theories Uses small samples Data is rich and subjective Low reliability High validity Generalises from one setting to another Produces qualitative data Methods of paradigms Use of primary or/and secondary sources Primary Primary Primary Primary Secondary Primary Secondary Qualitative Positivistic Paradigm Methods Structured detached-observer non-participant involved observation Questionnaire: structured closed questions Experiment Structured interview closed questions Use of statistical data Disadvantages 9,5 Post-positivistic Paradigm Methods Involved observer and participant observation Questionnaire: less-structured open questions Less-structured interview openquestions Focus group Use of reports 16 www.brad.ac.uk/academic-skills Primary Close measurement 17 www.brad.ac.uk/academic-skills