CVOH 221 – Topics in Occupational Health and Safety Fall 2018 Session #7 Session #7 Agenda • • • • • • • • Environmental scan | Warm up Planning data analysis Managing data collection Preparing for data analysis Quantitative data analysis techniques Assignment #1 | Grading explained Assignment #2 | Expectations Group exercise Warm-up | Environmental scanning Emerging and interesting news that have value? Warm-up | Environmental scanning • Instructor bullets. CVOH 221 | Status • 2nd assignment due > 2 weeks • 1st presentation > 4 weeks from today • November 16th > drop date w/o grade • 6 weeks till course completion • The next 4 lectures, including today: – data analysis; – evidence based conclusions; – proposing recommendation; Research genesis recap 1. 2. 3. 4. 5. 6. 7. Indentified a research need for a company. Completed a literature review examining existing evidence. Established a “research question”. Established project variables. Settled on a research project design. Initiated participant recruitment through informed consent. Selected a tool to capture data. 8. 9. 10. 11. 12. Establish a data analysis plan. Collect and aggregate project data. Analyze the gathered data. Interpret results and formulate evidence-based conclusions. Establish clear recommendations and present findings. Planning data analysis • Understand what data you are / will be collecting! – – – – – WSIB injury statistics Survey responses Behaviour measurements Interview / focus group data Observations • What demographics data do you intend to gather? – Age, gender, job tenure, nationality Planning data analysis • Why are you collecting “that” specific data? • Why are you collecting the indentified demographics? • Why are looking for numbers vs. words vs. both? Planning data analysis • Ultimately, your data set needs to present evidence that address your research question and can offer a strong base for relevant conclusions and recommendations. • Although, until you see and asses your data you may not yet know the full scale of your analysis, you still need a plan for moving forward. Planning data analysis If you know you’re collecting these components, what data analysis can you feasibly plan? • Independent variable > OHS training (job specific) • Dependent variable > OHS knowledge (job specific) • Research design > pre-post survey (or just post) + focus group (3 months post training) with 30 staff • Demographics > age, job tenure, educations level. Quantitative analysis Managing data collection • Create and use a project folder to store documents. • Prepare participant instrument / consent packages. • Organize responses from participants and ensure that all documents are appropriately identified. • Ensure to have a mechanism to track data collection activities and respond quickly to missing questionnaires. • Allocate time to conduct progressive data entry. • If someone is assisting your data collection, provide accurate instructions and minimize their influence. • Establish a clear and open communication channel. Preparing for data analysis • Develop a coding system for your data files, and code all the questionnaire (survey) data points. • Create an excel spreadsheet and enter your data! • Verify data entry accuracy! Is there missing data? • Backup your data prior to commencing the analysis. Preparing for data analysis • Coding system: • • • • • Strongly disagree, (1) | or (5) if reversed Disagree, (2) | or (4) if reversed Neither agree or disagree, (3) Agree, (4) | or (2) if reversed Strongly agree , (5) | or (1) if reversed • “Company management cares about the health and safety of its employees” (Positive = 1 - 5) • “Company management does not care about the health and safety of its employees” (Negative = 5 - 1) Preparing for data analysis • Missing data points, what do you do? – Complicated method = use a statistical simulation. – Simple method = use a sample average for the item. • Do NOT code the data point as zero. • Do NOT ask the participant to retrospectively provide a response on the item or re-complete the survey. Q3. The average is 2.25 for the PRE sample, & 3.25 for the POST sample Data analysis prep | demographics • Demographics data is also coded and entered! • Will allow you to understand the project sample. • Investigate and identify commonalities, variations and other interesting descriptors within sample. • Enable you to conduct cross-tabulations! Data analysis prep | Instruments You need to understand the final output (unit of analysis) of your data collection instrument! • Sum of all items vs. individual analysis? – Safety climate score vs. OHS practices assessment • Total number correct responses? – Knowledge based scoring (percent or total) • Average rating or frequency of response? – Average rating of an item – Percentage selected (frequency scale) Descriptive statistical analysis • Most likely the limit of your statistical analysis! • The most commonly used practical outputs in OHS. • • • • • Total numbers, frequencies, percentages, ratios, ranks. Mean, median, mode (central tendency measures) Percentiles Standard deviation, variance Cross-tabulations Descriptive stats | Frequency & ratios • Frequency tables – an organized approach to looking at numbers and visualizing multiple variables • Ratios – relationship between two numbers Descriptive stats | CTM • Mean – the average for the data sample or the reviewed collection of numbers. The most commonly used measure in practical OHS. • Median – the value that splits the data sample down the middle. Only used in a skewed data set*. • Mode – most frequently appearing number within the data sample. Rarely used in statistical analysis. Descriptive stats | CTM - skewness • Skewness (x-axis) - an unnatural data distribution generally caused by extreme data points also referred to as outliers. Descriptive stats | CTM - kurtosis • Kurtosis (y-axis) – describes the sample distribution along the vertical axis or the “peakedness” of the probability distribution. Descriptive stats | St-dev • Standard deviation (+/-) – a measure of data distribution and the amount of variation in the sample. • Represented by a range: mean is 15 ± 2.5 How to approach a skewed distribution? Descriptive stats | Variance • Variance – average deviation from the mean (SD2) • Experimental variance – differences associate to the manipulation of independent variables • Error variance – observe difference due to presence of confounding variable or unexplained sources. • Most of the powerful statistical analysis tests use variance within their formula. (e.g. ANOVA) Descriptive stats | Percentiles • Percentiles – similar to ranking and percentage but with an element or self-assessment or benchmarking. • Performance is as good or better than x-percent. 90TH PERCENTAILE IF ABOVE AVERAGE. DOING BETTER THAN THE REST Descriptive stats | Cross-tabulations Cutting and stratifying the available data across established variables for future interpretation. • Analyzing data across demographics: – The mean of responses for female participants. – The frequency of responses for participant that self-indentified as newcomers to Canada. – The average number of correct responses for male participants, in the 20 – 30 age group. – Workplace stress assessment scores for workers in the 35 – 50 age group, with 5+ years of services, selecting English as primary language. Assignment #1 | Grading rubric • Instructor document Take 15 minutes! Your thoughts? • The employee, a sales manager for a food company, was taking his son and colleagues daughter to lunch in his company vehicle when the 2nd driver ran a red light and collided with the vehicle. The other driver was employed with a company that was registered with the WSIB as a “Schedule 1” employer. • The injured party wanted to sue, however the other driver claimed that it should be consider a workplace related accident bound by the WSIA. – The employer of the driver causing the accident registered with the WSIB after the event. – On the same trip he planned to see a client and pick up a cheque for his employer, which was a regular task for him Assignment #2 | Expectations • Review Assignment #2 guide. Statistical analysis • There’s no need for fear! You’ve likely done it already! • As an OHS professional you most likely don’t need to know how to conduct statistical analysis, but you do need the capacity to understand the analysis. • WSIB has a very robust statistics department! what an health and safet rep need to know overall 1. 2. 3. 4. 5. T-test P-value SEM’s and CI’s Correlation Linear regression Statistical analysis • How do you compare two groups or samples and demonstrate whether there is an actual difference (or none) between their results and or performance? – Plant A had 65 lost time injuries in 2014, whereas Plant B had 74 lost time injuries. Are they truly different? • How do you demonstrate that a group of participants improved their knowledge and or performance following training (pre-post testing)? – 77 participants completed an OHS knowledge gaps survey and scored an average of 63%. Following 3-days of OHS training, 77 participants again completed the same survey and scored an average of 72%. Did their knowledge actually improve? Statistical analysis • Science works by a method of refutation? • What is meant by significant difference? • When do you know or can claim that there is a significant difference between x and y? Statistical analysis | T-test • T-test (students t-test) is the most common analysis technique used to determine whether there is a significant difference between two groups or post intervention. • The analysis incorporates the inherent error variance and determines whether observed variance is due to the independent variable. • Two types: – Independent samples – Paired / repeated measures • The output is the p-value what is the probabilit that u are seeing a value due to luck Statistical analysis | P-value • P-value – the probability that the observed difference could have occurred simply by chance, and there is really no difference between groups or no intervention effect. – – – – – – Alpha (α) the cut-off point established at 0.05 (5%) (p<0.05)* Alpha (α) of 0.01 (1%) are frequently seen (p <0.01)** When below α cut-off significant difference is declared P-value can be shown as value e.g. p = 0.0345* Does p-value indicate effect size, e.g. p = 0.047 vs. p = 0.001? Hard to believe that p = 0.07 is not as good as 0.05, TRENDS! TRENDS means that u ma need more participants in order to get to under 0.05 use a two tailed t-test than a 1 tailed t test Statistical analysis | SEM • Standard error of the mean (SEM) – representation of mean variation for the sample distribution. • Similar to SD, but offers a measure of variance relevant to the statistic or population mean if the statistical analysis was consistently repeated (68% range) if the lines are large u can assume that there is a significant variation through observing the table visuall Statistical analysis | Correlation • Correlation – a statistical test to determine a presence of a relationship between two variables. • An analysis to understand whether a defined variable has an associated effect on secondary variable. Statistical analysis | Correlation • How does one interpret a correlation analysis? – Pearson r: 0 – 1 • Two elements to analysis: – 1. Direction (Negative or Positive); – 2. Strength 0 - .3 = no correlation .3 - .6 = low or weak correlation .6 - .8 = moderate correlation .8 – 1 = high or strong correlation • Has a slope or a line of best fit • Not a causal relationship! Statistical analysis | Linear regression • Linear regression – function related to correlation that is used for predictive and modeling purposes between the independent and dependent variable. • A statistical function that can be used to forecast a result and also retrospectively estimate the result. – Output: r2 (function of r in correlation) – Equation: Y = bX + a (a = intercept; b = slope) – Predictor = x-axis; predicted = y-axis Statistical analysis | Linear regression • I don’t expect you to execute a regression analysis! • Let’s look at some examples of a linear regression. Note the difference in visual representations. • Consider how you can predict the result! OHS research project • How will you analyze your data sample? • What is the end goal of your project? • What do you feel is feasible? Group exercise 1. Please pair with someone who has a computer! 2. Open the Excel file posted on the course site! 3. Review the questions / prompts! Parting shots • Assignment #2 due in 2 weeks! – Hard copy + turnitin! • At this point you should have: – – – – – Identified a project sample Developed an informed consent Settled on the research and data collection strategy Develop / selected your data collection instruments Started considering quantitative data analysis. • Next class: measurement & evaluation continues: – qualitative analysis; reporting your results.
