Week 12 Slides
Interview Assignment Feedback
• Display Competence & Conscientiousness
–
–
–
–
What changes did you decide to keep?
How did you decide to score?
What KSAO gap did the interview fill?
Some of you included assessments that were not in the
scenario.
• What was the purpose of the interview?
– Fill the KSAO gap or measure all KSAO’s?
• Be careful with the selection plan
– Use SME judgments if you can
• Pilot your instructions and forms
• You can make your points back by revising and resubmitting
once.
Assigned Readings
• Allen et al (2013) Socialization
• Orvis et al (2008) Contract Breach
Section 6: Raw data Interpretation (10
points)
• Create a correlation matrix for our scale, TI,
and PW.
– Summarize the results presented in the
correlation matrix. Does it make sense
theoretically?
• Summarize the sample by analyzing the
demographic variables and reporting the
sample size.
– Generalizable?Explain.
Preliminary Cleaning
• Deleted those that did not finish.
• Removed all identifiable data
• Converted old SAT scores
– There may still be some cases I missed
• Removed the GRE question
What you need to do!
• Determine what data should be included
– Incomplete data?
– Response sets?
– Working and tenure?
– Sample size
• Code the Occupational (free-response) data
into meaningful categories.
• Dummy code accordingly.
Statistical Procedures
• Screen out bad data
• Conduct demographic and frequency analyses for the
demographic variables you feel are relevant.
• Generate reliability estimates.
– Should any items be removed to improve reliabilities?
• If so, are the items bad or do they need to be reverse scored?
• EFA for conscientiousness
– Method: Principle Axis Factoring; Rotation: Promax
• Make adjustments to scale based on the reliability, factor
analysis, and definition.
• Compute scales for TI, Conscientiousness, and PW
• Run correlational analysis of TI, Conscientiousness, and, PW
Turn in to me
• All syntax (organized and labeled)
• Part of section 6 & 2
– A list of what cases you removed and why
• Also; explain if you dummy coded any variables and then
compared screening vs. no screening and decided not to
screen.
– Discuss and summarize the data regarding the freeresponse occupation variable.
– Output: Demographics (in APA table form),All EFA
(highlight pattern matrix with your loadings) and
correlation (in APA table form).
Full Week 11 Slides
Final Project Information
• Due Thursday November 21st
– Turn in on Dropbox
• Around 8-10 Pages APA style.
– Also include your output and syntax
• Worry more about showing competence than
providing a one-sided convincing argument
• I will help except for the Monday, Tuesday and
Wednesday in which it is due.
– Bring specific questions.
Section 1: Content Analysis (10
points)
• Define Conscientiousness and content
domain. Do you think there is enough
evidence of content validity.
• How congruent is the scale with the content
domain? Contamination and deficiencies.
• Would you make any changes to the current
scale (i.e., directions, response options, or
question content)?
Section 2: Factorial Validity (10
points)
• What factor structure do you expect to see with
our scale?
• Calculate reliability. High enough? Should items
be removed?
• Summarize how many observed factors from the
EFA and name the factors. Is the scale
homogeneous? How do you know?
• Is there “enough” evidence of factorial validity?
What are the implications for the usefulness of
the scale?
Section 3: Construct Validity (10
points)
• Come up with an expected pattern of
relationships
– Convergent & Discrimination (relative)
• Compare expectations to observed
relationships
– What findings were unexpected?
– How does this influence your interpretation of the
evidence of validity?
Section 4: Criterion-Related Validity
(10 points)
• -Summarize the results presented in the two
regression tables.
– R2 , Beta values?
• -Does conscientiousness explain enough
unique variance in outcomes when compared
to other assessments. Is there enough
evidence to convince you of it’s utility?
• -Is multicollinearity an issue? Elaborate.
Section 5: Recommendation (10
points)
• -Make an official recommendation given the current situation
– Talk about the different types of evidence and potential
tradeoffs/preferences.
– Should we add the scale?
• Discuss the selection and baserates for before and after adding the
measure. Is there enough of a change to impress you?
• Does the organization currently have an issue with adverse impact?
– Does adding our measure help or harm in any significant way?
• Is adding the scale financially and practically feasible? Explain. Are
you concerned with faking? If so, what can we do to address the
issue?
Section 6: Raw data Interpretation (10
points)
• Create a correlation matrix for our scale, TI,
and PW.
– Summarize the results presented in the
correlation matrix. Does it make sense
theoretically?
• Summarize the sample by analyzing the
demographic variables and reporting the
sample size.
– Generalizable?Explain.
Preliminary Cleaning
• Deleted those that did not finish.
• Removed all identifiable data
• Converted old SAT scores
– There may still be some cases I missed
• Removed the GRE question
What you need to do!
• Determine what data should be included
– Incomplete data?
– Response sets?
– Working and tenure?
– Sample size
• Code the Occupational (free-response) data
into meaningful categories.
• Dummy code accordingly.
Statistical Procedures
• Screen out bad data
• Conduct demographic and frequency analyses for the
demographic variables you feel are relevant.
• Generate reliability estimates.
– Should any items be removed to improve reliabilities?
• If so, are the items bad or do they need to be reverse scored?
• EFA for conscientiousness
– Method: Principle Axis Factoring; Rotation: Promax
• Make adjustments to scale based on the reliability, factor
analysis, and definition.
• Compute scales for TI, Conscientiousness, and PW
• Run correlational analysis of TI, Conscientiousness, and, PW
Turn in to me
• All syntax (organized and labeled)
• Part of section 6 & 2
– A list of what cases you removed and why
• Also; explain if you dummy coded any variables and then
compared screening vs. no screening and decided not to
screen.
– Discuss and summarize the data regarding the freeresponse occupation variable.
– Output: Demographics (in APA table form),All EFA
(highlight pattern matrix with your loadings) and
correlation (in APA table form).