Evaluation Of Technical Quality
Dahlia Salmon
Capella University
Introduction
The test used in this assessment are used in businesses and organization to assess human
behavior and personalities. This study is to evaluate both the validity and the reliability of testretesting of all three of the assessment tests. The Fundamental Interpersonal Relations OrientationBehavior (FIRO) -B Scales, The Hogan (HDS), and the Wonderlic Personnel test-revised (WPTR).
These tests focus on employees and potential employees, their learning capabilities stress levels,
their personalities, IQs, and cognitive abilities. The purpose of the research is to address whether
these tests have proven to be effective, and if the measures accurately predict the effectiveness of
these tests.
Technical Review Article Summaries
The FIRO-B is a measure of verbal and social abilities. FIRO-B was designed to test
cognitive abilities in highly intelligent sample (Gorkan et al., 2010). The FIRO-B is used
commonly in businesses and organizations globally for different purposes such as training upper
management functions, and the enhancement of leadership and their effect on others,
The FIRO-B measures the verbal and spatial abilities, and was designed to evaluate
cognitive abilities (Gorkan, 2010). The Test is a questionnaire made up of 54 items with 24 of
the questions completed on a 6-point scale (from 1= nobody to 6=most people). The tests
compare their behavior and pattern to their peers and the remaining questions were formatted on
a point scale (from 1=never to 6= usually). These questions describe the usual patterns of
behavior. The data for these FIRO-B shows reliability of the measure and acceptability range
of .68 to .91 (Furnham & Moyle, 2000). The test data for the FIRO-B test shows reliability of the
measures (Furnham & Moyle, 2000), and leadership capabilities.
Reliability of the FIRO-B
The reliability of the FIRO-B data was given to 296 University Freshmen. The results
found that the means and the reliability coefficient were not as high as those given to another
group in the FIRO-B manual. On the overall scores analyses the total of the six scales along with
the total in the need areas gives the best reliability coefficients. Researchers have found in the
data that they get the best result from the FIRO-B by taking advantage of the higher correlations
found in the overall scores (.81) and the sums of the wanted and expressed scales (.75) (Wiley &
Sons, 1973).
The FIRO-B instrument is used in numerous ways to measure leadership development,
team building, individual interpersonal effectiveness, and retention. The FIRO-B instrument can
be used along with the MBT1 Instrument for leadership development in workshops for team
building. The FIRO-B measures three needs; the need for inclusion, need for control, and the
need for affection, and shows the extent to which a person will go to seek relationships, or to
lead and control others. A numerical score and categorical score are both given to respondents of
the FIRO-B test that measure the needs such as the Total inclusion, and the Total Expressed. A
sample of US National individuals took the FIRO-B test in 1997 to examine if the reliability of
the test was consistent in each area and found that in all areas the reliability of the test was
consistent in each area and found that in all areas the reliability ranges from .7 to .85 on the
scales.
Validity of the FIRO-B
The validity of the FIRO-B instrument has shown by a number of researchers, that the
studies of the measure of the FIRO-B instrument have proven to measure leadership abilities,
personal values (.03 to .27), and help individuals develop relationships and friendships (.43
to .46) in correlation with other instrument such as the MBTI form instrument (.56 to .29) and
the CPI instrument (.48 to .51) (Hamer & Schnell, 2000).
A study was conducted to investigate the interpersonal relationship and the differences
between employee of service and manufacturing organizations; there were 437 employees
chosen from 56 employers from the service and manufacturing organizations in an area call
Kolkata and the FIRO-B instrument was used to collect data from the respondents. There were
significant differences found between the manufacturing and the service organizations regarding
the Expressed Inclusion, Expressed Control, Total Expressed, Total Inclusion, and Total Control
(Mitra & Ishita, 2019). These individuals were mostly males, and the participants had a
minimum graduation level and higher on the scale of the measurement, the test shows a
consistent reliability with the Expressed Inclusion =.87, the Wanted Affection =.96, The
Expressed Control =.93 Wanted Control =.86 Expressed Affection=. 86, Wanted Inclusion=.85.
The results of the test between these groups shows the employees of the service
organization have a higher need for Expression and a greater need than their manufacturing
counterparts (Mitra & Ishita, 2019). The result shows how based on the very nature of their job
in customer service the employees of the service industry will be more in control and forward
thinking. The result shows that the reliability and validity of this test is effective.
Reliability of the HDS
The development of reliable instrument to measure the dark-side traits uses variables that
have been introduced to measure the predictability of worker derailment, and failures that occur
in senior level jobs. The HDS measures the different personalities of people and their job
performance. The HDS correlation coefficient measure validity between -1 and 1 in the scoring.
The closest the measure is to 1 the stronger the test. The predictive value of the Hogan test is .29
but combined with the HPI and the MVPI the performance climbs to .54. Example of the
reliability is; predicting the validity of ibuprofen to reduce pain is .14, while another example is
the relationship between a structured job interview and a job performance is .18 (Hogan). These
are just some of the quantitative ways to measure the validity of the HDS. The HDS is a nonclinical measure made up of 168 items and a participant need a score of 1 scale. Then test is
grouped with 11 true or false questions and is testing for disfunctions in personalities that can
impact individuals and relationships in the workplace. The higher the scores are on these tests the
higher the risk of work-related issues (Hogan & Hogan,2001).
A study of the HDS measures in use was conducted using two different psychological
businesses in Britain, the study was conducted over ten years. The service consultant businesses
have different specialties, but both companies were assessed for middle management
development programs (Furnham, 2017), and the assessments were used for promotional
measures. The participants were instructed to conduct the assessment online, and as part of the
program they were each given feedback on how they score through an independent
psychological consultant. In addition, they assessed for intelligence and creativity. The
intelligence and creativity test results were not results that were given to individuals, instead the
test was given to researchers who were given ethical permission to evaluate them.
Results were similar in the stress tolerance across both groups, and in one of the tests
labeled cluster A, the test results were negative in association with stress symptoms such as
excitement and caution, while cluster B shows a correlation to positivity, and cluster C shows
positivity. Excitable and cautions were more important finding, and the dark-side factors were
shown in poor stress tolerance (Furnham, 2017). The findings show that cluster A and B
negatively affected job reliability, cluster C had a positive impact. In this study a number of
dark-side traits were highlighted and shows that they are powerful negative predictors.
Reliability of test-retest can be evaluated in two ways; internal consistency and test-retest
reliability responses. The test-retest uses correlations of scores from more than one assessments,
the HDS shows that the short-term retest reliabilities is .81 for the HPI is .70 for the HDS and .79
for the MV1I (Hogan). To ensure accuracy of these tests, the same questions are asked in
separate ways to measure reliability, and validity. The HDS recommends to organization and
industries continued evaluations. The HDS test have been founded to be above industry standard
(HOGAN).
Validity of the HDS
One test used to prove validity is the dark-side factor, which shows what qualities are
displayed during stressful times that can cause problems in relationships and can prevent
successes. Hogan assessment tool has .54 predictive validity of .18. and several other tests are
performed to learn more about the dark-side trait; however, the HDS test measures the stress
tolerance and reliability of this trait. With far better reliability. The organization, utilizing these
measures are testing for resilience, composure, calm, coping with stress and being tolerant.
Furnham et al., 2016 described these concepts as coping or bouncing back after experiencing
setbacks. Both the bright-side and the dark-side characteristics are important to organizations
because they are related to personality trait, namely neuroticism and conscientiousness. These
personality traits have shown to be the strongest predictors of job success in all different
occupations (Furnham et al., 2012).
Some researchers, from their studies, found that the dark-side score is usually around 1
being optimal, standard deviations above 1 is considered a success, and a score of 2 deviations
above the normative outcome shows a person with poor management style.
In the Quadratic analyses used in the findings of the dark-side, factor show that the darkside can be a success and failure. The HDS measures has been used in many different studies and
have proven to be a strong and reliable test (Hogan & Hogan 1997).
A poll taken by The Society for Human Resource Management (SHRM) shows 18% of
495 professionals find the test helpful in predicting job-related behavior in their organization.
The Reliability of the WPT
Another important test is the Wonderlic Test, about 2.5 million people all over the world
take this test as part of their employment possibilities. The correlation of the Wonderlic (WPTR)
measures is .085 to 0.93. and has a strong validity with the Wechsler adult Intelligence scale. The
psychometric of the scores in retest are consistent. The WPT test is popular for use in preemployment testing. The WPT is easy to administer, and the test contains 50 written questions
with 12 minutes of test time (Bell et al., 2002). The scores measure intelligence and IQ scales
and can also be applied to check how the test behaves, and the role of the participants. The
results of this information can predict the reliability of the test. The WPT gives empirical data
and helps psychologists in their decisions in certain situations.
This study is to assess the validity of the WPT and to determine the measures that are
more closely aligned with the fluid or crystalized intelligence, not just theoretical purposes
(Kaufman & Kaufman, 1998). Testing was performed using 22 men who participated from a
small Southeastern college- average age of 27 years (SD=8.6, range 19 to 54) Also included by
race were 73.1% white 14.9% African American, 4.5 % Hispanics, 7.5% Asian and others. The
students were given both the WPT and the Kaufman Adolescent and Adult Intelligence Test
(KAIT) for control of the effects of the tests. The test was separated in two installments, one
given at the beginning of the semester, and the other at the end. The Kait was given separately.
The KAIT measure the general intelligence of ages 11 to 85 years. The crystalized and
fluid composite of the test measures the IQ scores (M=100; SD=15) and was developed by
Cattell and Horn (1963, 1978). The results from the study of the students shows the WPT
correlated positively and significantly with the KAIT composite IQ scores (r=66, p<.0001),
Crystalized IQ scores (r=62, p<.001), and fluid IQ scores (r=.54, p<.001) (Bell et al., 2002).
Internally the WPT shows a consistent reliability with scores ranging from .88 to .94 and the
alternate-form reliability estimated to range from .73-95 (Wonderlic, 1992). Further results show
the WPT has a strong association with fluid crystalized abilities, it is a solid choice of instrument
for individuals looking for a quick assessment tool to measure intelligence.
To check stress tolerance of the WPT one of the most important organizations to evaluate
is the National Football League (NFL). The research done to collect data to determine correlation
of the quarterback in the NFL between 2002 to 2012 to see if a quarterback is more productive
during the early stages of the draft, or the latter part. The WPT test score found to be relevant to
the NFL performance despite all previous materials on the test. The authors, however, found no
clear evidence that Wonderlic scores are correlated with draft position (Pitts & Evans, 2018).
The authors collected data from many sources which shows all the quarterbacks drafted between
2002 and 2012, who completed the WPT test. and four years of data were collected to measure
the productivity in the NFL (Pitts & Evans, 2018).
The WPT is applied to NFL draft event to test on players both on the field and tested on
paper for them to get drafted. The reason is to test the players thinking capabilities and their
cognitive abilities. The test helps the managers, the coaches and the scouts in their selection
process. Soring high on the test shows how a player thinks and will perform on the field.
Validity of the WPT
The test is calculated by the number of questions that are answered within the 12 mins the
testers are given the score is 0 to 50, with 0 been lowest 50 is highest. The questions equal the
same amount, so guessing is acceptable. The test scores of the players can be compared to each
other and is used to match players based on their scores. The scores are different for each
position; a quarterback with a score of 20 or more is more sought after while the average player
scores 20 out of 50 that computes to an IQ of 100. The comparison used is shown below:
Offensive Tackle: 26
Tight End: 22
Wide Receiver: 17
Center: 25
Safety:19
Fullback: 17
Quarterback: 24
Linebacker: 19
Halfback: 16
Guard: 23
Cornerback: 18
A quarterback is more likely to be recruited, if his school had a prior quarterback that was
recruited. The authors did not find enough evidence to show that the WPT score has any impact
on the draft position (Pitts & Evans, 2018), however they did find evidence that the WPT scores
correlates with the NFL quarterback’s performance.
The evidence by the authors found that the WPT results can be used as an indicator of a
quarterback’s valve, but found the NFL is either unaware or is not willing to use the WPT
information when they are making their selection (Pitts & Evans 2018). The authors admit they
may need more research done but feels that the NFL should be considering their findings, since
the results could be valuable to the coaches, managers and executives (Pitts & Evans, 2018).
They found the results of the WPT scores to be valid and could improve the selections of
quarterbacks in the NFL draft.
Organizations and companies find the WPT a useful tool in helping them to hire the best
people in sales, accounting, marketing, supervisory positions, or football players. The WPT helps
the companies to hire the right people for the positions.
Conclusion
The findings of the technical qualifications of these three studies done by many
researchers, found the FIRO in all areas, shows reliability of the measures and show a high
correlation in the test overall scores, is used in many ways to measure the different aspects of
human intelligence in the workforce. The HDS also used in many studies and has shown to be
strong, dependable, and valid. The HDS is used in organizations for their customer service and
other manufacturing industries to assess the performance and stress levels of their employees
and, these industries have found the test helpful in predicting performance. The third test
included is the research is the WPT, also commonly used, it is found to be a an easy test to apply
and assessed, and is seen as a solid choice of instrument for use in organizations. These three
tests have been found to be valid and reliable.
References
Humandevelopmentsolutions.com. (2021). Retrieved 9 October 2021, from
https://www.humandevelopmentsolutions.com/views/img/firo/aside/History-ValidityFIROB.pdf.
Batey, M., Chamorro-Premuzic, T., & Furnham, A. (2010). Individual Differences in Ideational
Behavior: Can the Big Five and Psychometric Intelligence Predict Creativity
Scores? Creativity
Research Journal, 22(1), 90-97. https://doi.org/10.1080/10400410903579627
Flanagan, D., Genshaft, J., & Boyce, D. (1999). Test Reviews : Kaufman, A. S., & Kaufman, N.
L. (1993). Kaufman Adolescent and Adult Intelligence Test (KAIT). Circle Pines, MN:
American Guidance Service. Journal Of Psychoeducational Assessment, 17(1), 62-89.
https://doi.org/10.1177/073428299901700108
Furnham, A. (2008). Psychometric Correlates of FIRO-B Scores: Locating the FIRO-B scores in
personality factor space. International Journal Of Selection And Assessment, 16(1), 3045. https://doi.org/10.1111/j.1468-2389.2008.00407.x
Gilligan, J. (1973). FIRO-B: Norms and reliability revisited. Journal Of Clinical
Psychology, 29(3), 374-376. https://doi.org/10.1002/1097-4679(197307)29:3<374::aidjclp2270290327>3.0.co;2-3
Hicks, K., Harrison, T., & Engle, R. (2015). Wonderlic, working memory capacity, and fluid
intelligence. Intelligence, 50, 186-195. https://doi.org/10.1016/j.intell.2015.03.005
Hogan, R., & Hogan, J. (2001). Assessing Leadership: A View from the Dark Side. International
Journal Of Selection And Assessment, 9(1&2), 40-51. https://doi.org/10.1111/14682389.00162
Leading Source for Verified Practice Tests - JobTestPrep. JobTestPrep. (2021). Retrieved 9
October 2021, from https://www.jobtestprep.com/.
Mitra, D., & Chatterjee, I. (2016). Interpersonal Relationship Needs of Employees of Private
and Public Sector Organizations: A FIRO Perspective. Prabandhan: Indian Journal Of
Management, 9(7), 5. https://doi.org/10.17010/pijom/2016/v9i7/97783
Pitts, J., & Evans, B. (2017). Evidence on the importance of cognitive ability tests for NFL
quarterbacks: what are the relationships among Wonderlic scores, draft positions and
NFL
performance outcomes? Applied Economics, 50(27), 2957-2966.
https://doi.org/10.1080/00036846.2017.1412081