peer
reviewed
W. Lee Grubb III (grubbw@mail.ecu.edu) is an
Assistant Professor of Management at East Carolina University.
Abstract
As
organizations continue to strive to hire the most productive employees, the use
of both cognitive and non-cognitive selection and assessment tests has
continued to increase. Situational judgment tests continue to grow in
popularity as selection and assessment tools because they are valid predictors
of one’s future performance and because they assess a wide variety performance
related constructs. Despite the growth in popularity, very little is known
about the ability of situational judgment tests to assess one’s emotional
intelligence. Emotional intelligence research has increased over the last ten
years and some researchers have expressed the belief that job applicants’
emotional intelligence should be considered as well as their cognitive
abilities. This study investigated the relationship between a situational
judgment test and an emotional intelligence test and is a first step in
understanding how the two measures are related.
Over
the last decade, the use of situational judgment tests as selection and
assessment measures has increased as research has indicated that situational
judgment tests are valid predictors of future job performance (McDaniel,
Hartman, & Grubb, 2003; McDaniel, Morgeson, Finnegan, Campion, &
Braverman, 2001; McDaniel & Nguyen, 2001; O'Connell, McDaniel, Grubb,
Hartman, & Lawrence, 2002; Weekly & Jones, 1999). Situational judgment
tests are personnel screening devices designed to measure a respondent’s
judgment regarding work place dilemmas. Situational judgment tests are
typically comprised of several different situations, each followed by a series
of choices. After reading the situation, the respondent is asked to choose a
specific response that represents a course of action for responding to the
aforementioned situation. The following is an example of a situation and
response choices from the Work Problems Survey (Smith & McDaniel,
1998).
You are in the middle of a difficult job and you ask your boss for help.
Your boss won’t help.
A. Get help from someone else.
B. Tell the boss you don’t like the boss’ attitude.
C. Go to the boss’ supervisor and complain.
D. Refuse to do the work.
E. Ask for a meeting with your boss’ supervisor.
A burgeoning body of literature reveals that
situational judgment tests assess a variety of constructs including general
mental ability, conscientiousness, emotional stability, agreeableness and job
experience (McDaniel et al., 2003; McDaniel & Nguyen, 2001; Nguyen, 2001;
O'Connell et al., 2002; Pereira & Schmidt Harvey, 1999; Weekly & Jones,
1999). Hence, situational judgment tests are correlated with different
constructs such as cognitive ability and conscientiousness that have repeatedly
been shown to be valid predictors of job performance.
McDaniel and Nguyen (2001) and Ployhart and Ryan
(2000) remarked on the importance of increasing our understanding of
situational judgment tests and what other constructs they measure. In the past,
situational judgment tests were often designed specifically to predict
supervisory behavior (Smith & McDaniel, 1998) but there is interest in
developing situational judgment tests that can be used to provide predictive validity
for multiple jobs to decrease the time and cost of developing specific tests
for different jobs (Ployhart & Ryan, 2000). As situational judgment tests
as selection and assessment tools increase in popularity, additional
constructs, relevant to job success, must be investigated to learn more about
how they are assessed by the situational judgment tests. The purpose of this
study is to assess the degree that different emotional intelligence dimensions
are measured by a situational judgment test that was
developed to offer predictive validity for most jobs.
While there is still much to be learned about the
design of situational judgment tests, there are several characteristics that
make them appealing personnel screening tests. Situational judgment tests have
been shown to produce smaller race based differences than general mental
ability tests (Chan & Schmitt, 1997; Clevenger, Jockin, Morris, &
Anselmi, 1999; Motowidlo et al., 1990; Motowidlo & Tippins, 1993; Nguyen,
2001). Further, situational judgment tests are valid predictors of future
performance and they may offer incremental validity beyond tests of general
mental ability (McDaniel et al., 2001; McDaniel et al., 2002).
Although the literature regarding situational
judgment tests continues to grow, McDaniel and Nguyen (2001) reported that
there is a paucity of research that has investigated the non-ability correlates
of situational judgment tests. As the use of situational judgment tests and
other selection and assessment tests grows in popularity in industry, so should
our understanding of the different constructs measured by these instruments.
In recent years, the use of emotional intelligence tests as predictors of
performance and life success has increased (Bar-On, 2000; Goleman, 1995;
Gowing, 2001; Mayer, Salovey, & Caruso, 2000). Bar-On (1997)
described his instrument to assess emotional intelligence and the main
components of his model of emotional intelligence. Bar-On’s (1997) emotional
intelligence model includes an overall emotional intelligence score referred to
as the Total EQ for emotional quotient and five main dimensions of emotional
skills: 1) intrapersonal skills, 2) interpersonal skills, 3) adaptability, 4)
stress management and 5) general mood. In addition, an inconsistency
scale and a positive impression scale are used as validity indicators to
examine the responses for cases of careless or random responses and cases of
inflated self-perceptions or attempts to inflate one’s score by faking good.
Bar-On (2000) offered a brief review of the five main components. First,
intrapersonal skills refer to self-understanding and self-awareness and the
ability to express one’s feelings and ideas. The second component,
interpersonal skills, is described as the ability to be aware of, appreciate,
and understand others’ feelings and the ability to establish and maintain
mutually satisfying relationships with other individuals. The third component,
adaptability, refers to one’s ability to accurately assess one’s feelings with objective
external cues and accurately assess the immediate situation. In addition,
adaptability refers to one’s ability to remain emotionally flexible to change
one’s thoughts as situations change and to aid in problem solving. The fourth
component, stress management, is the ability to cope with stressful situations
and to control one’s emotions. Finally, general mood is described as the
ability to be optimistic, express positive feelings, and enjoy ones’ self and
others.
Although research regarding emotional intelligence has been popular, several
studies have remarked that emotional intelligence is very similar to
personality dimensions (Davies, Stankov, & Roberts, 1998; Mayer, Salovey et
al., 2000; Newsome, Day, & Catano, 2000). Some studies show moderate
correlations between personality dimensions and facets of emotional
intelligence (Bar-On, 2002; Parker, 2001). These correlations are what one
might expect when correlating items that measure similar but different
constructs. Because of the relationship between personality measures and
emotional intelligence measures and the relationship between personality
measures and situational judgment tests, there should be a positive
relationship between situational judgment tests and emotional intelligence.
Hypothesis 1:
Total emotional quotient scores will account for
significant variance in the situational judgment test scores.
Although significant variance is expected based on the relationship between
personality measures and emotional intelligence measures, scale comparison
between personality measures and emotional intelligence measures can be complex. McCrae (2000) provided a conceptual correspondence
between the Costa and McCrae (1992) five-factor personality model and the
Bar-On (1997) proposed aspects of emotional intelligence. Emotional stability,
the personality dimension most strongly correlated with situational judgment
test scores (McDaniel et al., 2003) contains items that closely map to three
different scales from Bar-On’s mixed model of emotional intelligence, general
mood, intrapersonal skills and stress management. Similarly, conscientiousness,
the second most strongly correlated personality dimension with situational
judgment test scores (McDaniel et al., 2003) contains items that closely map to
two different scales from Bar-On’s mixed model of emotional intelligence,
adaptability and interpersonal skills.
With a relationship established between some of the Big Five factors used to predict
performance and Bar-On’s constructs of emotional intelligence, the nature of
the emotional intelligence constructs used to predict performance can be better
understood. Bar-On (2002) described two studies that have investigated the
relationship between emotional intelligence and job performance. The first
study was an investigation of 100 banking employees in the Philippines that
compared cognitive ability and emotional intelligence as predictors of
performance (Jae, 1997). Jae (1997) reported positive correlations between all
emotional intelligence scales and performance. The order of the individual
scale correlations with performance was: stress management (r = .52),
adaptability (r = .49), intrapersonal (r = .48), general mood (r
= .39) and interpersonal skills (r = .38).
The second study involved over two thousand males from the Israeli Defense
Forces (Fund & Bar-On, 2002). The results of this study indicated that
general mood and stress management were the greatest predictors of performance
from Bar-On’s (1997 & 2002) emotional intelligence scales. The results of
these two studies, as well as the description of what each scale of the Bar-On
emotional intelligence instruments measure, helped determine the posited order
of the relationships between the emotional intelligence scales and the
situational judgment test.
First, the emotional intelligence scale related to interpersonal relationships
and social awareness, interpersonal skills, taps into conscientiousness,
agreeableness, and openness to experience. Because the interpersonal skills
scale taps into three personality correlates of situational judgment test
scores and because of its reported predictive ability, I posit:
Hypothesis 2:
The interpersonal scale will account for significant
variance in the situational judgment test scores.
Emotional stability, the personality dimension most strongly correlated
with situational judgment test scores (McDaniel et al., 2003; McDaniel &
Nguyen, 2001) contains items that closely map to three different scales from
Bar-On’s mixed model of emotional intelligence. Stress management is the most
represented dimension of emotional intelligence contained within the
personality dimension of emotional stability. In addition, Jae (1997) reported
that stress management offered the strongest single scale correlation with
performance. Therefore, I posit:
Hypothesis 3:
Stress management will account for significant
variance in the situational judgment test scores.
Next, conscientiousness, the second most strongly correlated personality
dimension with situational judgment test scores (McDaniel et al., 2003;
McDaniel & Nguyen, 2001) contains items that closely map to two different
scales from Bar-On’s mixed model of emotional intelligence, adaptability and
interpersonal skills (McCrae 2000). The adaptability scale relates to reality
testing, flexibility and problem solving. These skills are believed to directly
relate to judgment in the workplace and therefore I posit:
Hypothesis 4:
Adaptability will account for significant
variance in the situational judgment test scores.
Of the remaining personality dimensions, openness to experience and
extraversion are not strongly correlated with situational judgment test scores
(McDaniel & Nguyen, 2001). The remaining emotional intelligence scales are
intrapersonal skills and general mood. Intrapersonal skills may match up with
some aspects of emotional stability, extraversion and openness to experience
(McCrae 2000) but only emotional stability is strongly correlated with
situational judgment test scores. Similarly, general mood may share some
characteristics with emotional stability and extraversion (McCrae 2000).
Although I do not believe that the intrapersonal scale and general mood will be
strongly correlated with situational judgment test scores, Fund and Bar-On
(2002) reported that general mood was the strongest predictor of performance
and Jae (1997) reported a strong correlation between the intrapersonal scale
and performance. Therefore I posit:
Hypothesis 5:
General mood will account for significant variance in
the situational judgment test scores.
Hypothesis 6:
Intrapersonal skills will account for significant
variance in the situational judgment test scores.
Methods
Participants
284 undergraduate business students from a large southeastern public university
participated in the study in exchange for partial course credit. The sample
size was reduced to 215 after accounting for attrition, missing data, random
responding and excluding subjects that were determined to be faking good by the
Bar-On EQ-i:S. The average age of the participant was 24 years old (SD
= 7.2). 49% of the respondents were male and 2% of the respondents did not
indicate their gender. Roughly 57% of the respondents were White, 21% of the
respondents were Black, 11% of the respondents were Asian, 2% of the
respondents were Hispanic, 6% of the respondents indicated they were not
characterized by any of the aforementioned races and 4% of the respondents did
not indicate their race.
Procedure
A battery of tests was administered to groups of business students at a large
southeastern university. The students were instructed to respond to a situational
judgment test, the Work Problems Survey and Bar-On’s short form
emotional intelligence test, the EQ-i:S. The respondents were instructed
to answer the questions as honestly as possible.
Measures
The Work Problems Survey is a situational judgment test designed by
Smith and McDaniel (1998). The test was chosen over other different situational
judgment tests because the Work Problems Survey was designed by industry
experts specifically as a test that would tap into several different relevant,
performance predicting constructs (general mental ability, personality and job
experience) and could be used to offer validity for a variety of different jobs
(Smith & McDaniel, 1998). Often situational judgment tests are developed
and designed in industry for one specific type of job. However, the Work
Problems Survey was designed to be used in different settings and has been
successfully used in the past by both practitioners and academics and has been
shown to be a versatile, valid and reliable test with alphas consistently in
the mid .70s to the low 80s. (Grubb, 2003; Nguyen, 2001; Smith & McDaniel,
1998).
The test contains 31 different situations, each
followed by five different possible responses to the situation. Respondents are
asked to rate the responses by indicating the best and worst action for each
different situation. The test taps into several constructs including stable
personality traits, cognitive ability, job experience and age (Smith &
McDaniel, 1998).
For
use in this study, Bar-On’s model and measure of emotional intelligence was
chosen from a number of different mixed model, self-report scales for several
reasons. Mayer et al., (2000) discussed several emotional intelligence measures
including their own emotional intelligence measure, the Multifactor
Emotional Intelligence Scale (MEIS), the EQ-i, a scale from
Bar-On (1997), a test developed by Goleman (1995), as well as a measure
developed by (Schutte et al., 1998). Additionally, the Emotional Competence
Inventory (Boyatzis, Goleman, & Hay/McBer, 1999) and the EQ-Map
(Cooper, 1996), were considered. Of the measures mentioned, the measure
developed by Schutte et al., (1997), the EQ-i and the EQ-Map were
the only self-report measures. Further review of the measures indicated that
the EQ-Map was designed for personal assessment and was not intended for
selection. Of the two remaining, the EQ-i and the measure developed by
Schutte et al., (1997), the EQ-i is better known and has received more
research attention. In addition, the EQ-i was developed to help
determine what makes some individuals more successful and productive than
others (Bar-On, 1997) which makes it more applicable to a study of selection
and assessment measures.
Bar-On (2002) developed a short version of the original 133-item measure. The
short version contains 51 questions and measures the same five main components
of emotional intelligence and because Bar-On’s model of emotional intelligence
is a mixed model of emotional intelligence, it should tap into several
different constructs that can be measured with situational judgment tests.
Mayer et al., (2000) described some of the different constructs related to
mixed models of emotional intelligence and mentioned practical intelligence,
general intelligence, social desirability, and different aspects of the Big
Five model of personality, as well as many others.
The Bar-On EQ-i:S, a short version of the EQ-i was chosen
over the standard version of the EQ-i for practical administration
purposes. The EQ-i:S is a short form emotional intelligence test based
on the Bar-On EQ-i (1997). The instrument contains 51 items that measure
information across 8 different scales. The instrument includes 5 main emotional
intelligence scales: intrapersonal skills, interpersonal skills, stress
management, adaptability and general mood. In addition, an overall emotional
intelligence score referred to as one’s Total EQ is created. One’s Total EQ is
a score based on one’s performance on the 5 main emotional intelligence scales
and is used as an indicator of one’s overall emotional intelligence. Two
additional scales referred to as an inconsistency index and a positive
impression scale are used to screen for random or careless responses and
exaggerated positive responses.
The
instrument’s main emotional intelligence scales and the positive impression
scale describe traits or characteristics of a person and the respondent selects
a response based on a five-point Likert scale to report the degree that they
believe the statement is representative of his or her self. The responses are
anchored at 1 = very seldom or not true of me and 5 = very often true of me or
true of me. All of the 5 main emotional intelligence scales and the global
measure of one’s emotional intelligence are reported to have alpha reliability
coefficients ranging from .71 to .93 (Bar-On, 1997).
Results
To ensure that the results from the EQ-i:S were valid, a confirmatory
factor analysis using varimax rotation was conducted using SPSS 12.0. Although
the solution presented more factors with eigenvalues greater than 1, the main
emotional intelligence scales were well supported as the first five factors
containing stress management, general mood, adaptability, interpersonal skills
and intrapersonal skills. Bartlett’s test of sphericity for multivariate
normality was significant with a value of 4796.18 (p < .01) and the
KMO test score was .798 indicating that the data set had sufficient structure
and was adequate for factor analysis. In order to further investigate the
individual factors, the same analysis was conducted suppressing coefficients
less than .30.
The
factor analysis revealed a fourteen-factor model with eigenvalues greater than
1 but several of the factors appeared to echo the main dimensions of emotional
intelligence. The first factor, stress management, revealed an eigenvalue of
8.96 and explained 17.57% of the variance. Next, general mood, with an
eigenvalue of 4.33 explained an additional 8.48% of the variance. The third
factor, adaptability accounted for 6.07% of the variance with an eigenvalue of
3.09. The fourth factor, interpersonal skills, had an eigenvalue of 3.02 and
accounted for 5.92% of the variance. The fifth factor, intrapersonal skills,
revealed an eigenvalue of 2.26 and explained 4.37% of the variance. The sixth
factor appeared to echo the stress management scale and loaded strongly on
three of the stress management items with an eigenvalue of 1.98 that accounted
for 3.88% of the variance. In addition, a seventh factor appeared to strongly
echo the intrapersonal scale with four items that loaded strongly from the
intrapersonal scale with an eigenvalue of 1.68 that accounted for 3.30% of the
variance. An eighth factor loaded heavily on the positive impression scale, an
instrument validity scale, with an eigenvalue of 1.50 that accounted for 2.95%
of the variance. The eight-factor model accounted for 52.54% of the total
variance explained.
Although
there were six other factors that posted eigenvalues between 1.01 and 1.42 that
explained between 1.99% and 2.79% of the variance (equaling 13.69% of the total
variance), they were in large comprised of a smaller number items that belonged
to different scales. Although it is possible that the additional factors may
represent other facets of emotional intelligence with subtle differences,
investigating the potential for different facets of emotional intelligence is
beyond the purview of this study. In addition, the subsequent factor loadings
are not surprising given that many of the individual scale items address
similar but different facets of emotional intelligence. The individual scales
of emotional intelligence have moderate to high intercorrelations and thus they
may have a tendency to load with different factors (See Table 1 below.).
Table 1
Situational Judgment Test Scores and Emotional Intelligence
Scales: Alpha Reliability and Correlation Matrix
|
|
Alpha |
1 |
2 |
3 |
4 |
5 |
6 |
7 |
|
1. SJT Score |
.80 |
1.0 |
|
|
|
|
|
|
|
2. Intrapersonal |
.81 |
.15 |
1.0 |
|
|
|
|
|
|
3. Interpersonal |
.81 |
.19 |
.27 |
1.0 |
|
|
|
|
|
4. Stress Mgmt |
.85 |
.14 |
.30 |
.06 |
1.0 |
|
|
|
|
5. Adaptability |
.76 |
.10 |
.25 |
.28 |
.18 |
1.0 |
|
|
|
6. General Mood |
.85 |
.20 |
.59 |
.36 |
.46 |
.34 |
1.0 |
|
|
7. Total EQI Score |
.74 |
.24 |
.74 |
.56 |
.63 |
.51 |
.85 |
1.0 |
Correlations
at or above .14 are significant at p < 0.05
Correlations
at or above .19 are significant at p < 0.01
N = 215
Factor
analysis was not conducted on the Work Problems Survey because it is a
generally accepted principle that situational judgment tests do not typically
produce interpretable or significant factor structures (McDaniel & Whetzel,
2005). Due to the factorially complex nature at the item level, situational
judgment tests are more of a measurement method and do not contain items
specifically used to capture unidimensional constructs. The factorially complex
items in a situational judgment test therefore do not load cleanly into
distinct factors.
Next,
the hypotheses were tested using simple and multiple regression to determine
the amount of variance the emotional intelligence scales accounted for in the
situational judgment test scores. Hypothesis 1 predicted that the total
emotional quotient scores would account for significant variance in the
situational judgment test scores. Hypothesis 1 was supported using simple
linear regression R = .24 (p < .01). The total emotional
intelligence quotient accounted for 5.6% of the variance in the situational
judgment test score. Because the total emotional quotient score is based on the
scores from the five main facets of emotional intelligence in Bar-On’s mixed
model, the other main scales were subjected to multiple regression to determine
the level of variance that each individual scale contributed to explaining the
situational judgment test scores.
Multiple
regression, using the enter method, was conducted with the emotional
intelligence scales interpersonal skills, stress management, adaptability,
general mood and intrapersonal skills to examine the amount of variance
explained by each scale. The model summary was significant, R = .24 (p
< .05) and accounted for 6% of the variance however none of the individual
scales revealed a significant β.
An
additional multiple regression equation using the stepwise method was conducted
with the emotional intelligence scales to examine the individual amount of
variance explained by each scale. This method allows for variables to be
included or excluded from the regression equation depending on the amount of
variance the individual variable explains. The amount of variance explained by
each individual scale may increase or decrease depending on the presence or
absence of the other variables and the weaker variables may be removed from the
equation depending on the amount of variance they contribute. The model summary
was significant, R = .20 (p < .01) and accounted for 4% of the
variance. Interestingly, only general mood met the entry requirement and was
retained for the equation (β = .20, p < .01). Based on this
regression output, the majority of the subsequent hypotheses were not
supported.
Hypothesis 2 predicted that the interpersonal scale would account for
significant variance in the situational judgment test scores. Hypothesis
2 was not supported.
Hypothesis 3 predicted that stress management would account for significant
variance in the situational judgment test scores. This hypothesis was not
supported.
Hypothesis 4 predicted that the adaptability scale would account for
significant variance in the situational judgment test scores. This hypothesis
was not supported.
Hypothesis 5 predicted that general mood would account for significant variance
in the situational judgment test scores. Hypothesis 5 was supported. General
mood accounted for significant variance in the situational judgment test scores
(β = .20, p < .01).
Finally, Hypothesis 6 predicted that intrapersonal skills would account for
significant variance in the situational judgment test scores. Hypothesis 6 was
not supported.
Discussion
Although several of the hypotheses were not supported, it is understandable
when one considers the high degree of intercorrelation between the emotional
intelligence scales. When measures are highly correlated it is possible to have
a significant R with the regression equation and not have any of the
individual variables reveal a significant β. The total emotional
quotient score boasted the highest magnitude relationship with the situational
judgment test scores because the total emotional quotient score is comprised of
the five main emotional intelligence scales and thus it should explain the most
variance because of its broad band width of constructs covered.
General mood, the only single scale to account for variance in the situational
judgment test scores is highly correlated with the total emotional quotient
scores. When one considers the magnitude of the correlation between the total
emotional quotient and the general mood scale (.85) and the magnitude of the
correlations between general mood and the other emotional intelligence scales,
it seems to indicate that the general mood scale is saturated as a global
emotional intelligence measure. The high degree of correlation between the
total emotional quotient and general mood helps explain how the inclusion of
the general mood scale in a regression equation would draw variance explained
in the situational judgment test scores from the other emotional intelligence
scales. In addition, this may help explain the earlier research by Fund and
Bar-On (2002) that stated general mood was the strongest individual scale
predictor of performance.
The purpose of this study was to investigate the relationship between emotional
intelligence and a situational judgment test. As predicted, the relationship
between global emotional intelligence, the total emotional quotient score, and
the situational judgment test scores was significant. The single scale that
accounted for the most variance in the situational judgment test scores was the
general mood scale. The general mood scale involves such things as self-motivation,
optimism and happiness. It is however important to note that the situational
judgment test was not designed to measure emotional intelligence. Further
research involving the use of situational judgment tests to measure emotional
intelligence is warranted.
This study is the first to examine the relationship between emotional
intelligence and a situational judgment test. Much more is known and understood
about situational judgment tests and their ability to predict performance. Although
it seems clear that the relationship between emotional intelligence and
situational judgment tests will be positive, additional research is warranted
to determine the magnitude of the variance explained and which emotional
attributes are most strongly related to situational judgment tests.
The main implications of this research are twofold. First, the current study
helps to advance the literature in that it reveals another “noncognitive”
construct that can be assessed with situational judgment tests. As noted
earlier, (McDaniel & Nguyen, 2001) the literature regarding situational
judgment tests and their ability to measure general mental ability is clearer
than the literature regarding non-ability constructs such as personality and
emotional intelligence. This is the first study to investigate the relationship
between emotional intelligence and situational judgment tests and the results
help advance the theory regarding situational judgment tests. Simply knowing
that a positive relationship exists between the situational judgment test and
emotional intelligence helps us understand more about the versatility of
situational judgment tests and their ability to measure different constructs.
Second, from a practitioner standpoint the current results may lead to the
development of situational judgment tests that are able to more accurately
assess a wide range of desired employee characteristics and abilities in one
test. Although it is widely recognized that general mental ability tests are
the strongest predictors of future performance (Grubb, Whetzel & McDaniel,
2004), the literature has also revealed that situational judgment tests can
offer predictive validity beyond that of general mental ability tests
(O’Connell et al., 2002; McDaniel et al., 2002). The development of situational
judgment tests that are able to more accurately assess emotional intelligence
dimensions may enable practitioners to hire better employees, more efficiently,
through the use of fewer, more comprehensive selection and assessment tools.
Limitations
Although this study offers an
interesting look at an additional construct captured by situational judgment
tests there are several limitations that need to be addressed. First, the
sample used in the current study consisted of junior and senior level students
in a school of business. There is some question as to whether or not the
results would have been different if the sample had been comprised of older
respondents with more work experience. Because situational judgment tests have
shown moderate correlations with age and work experience the results of the
current research are likely an underestimate of the population correlation.
Interestingly however this is not to say that the sample used in the current
study is inappropriate. Situational judgment tests were originally designed to
measure the judgment and potential of supervisors and managers but in more
recent times, situational judgment tests are considered to be measurement
methods and can be designed to offer predictive validity for many different
types of jobs. Future research may replicate the current study with an older
more experienced sample to compare results.
Second, the results of this study are limited in that they compare two specific
measurement instruments. The literature regarding emotional intelligence
continues to grow and the results presented from this research should not be
considered an investigation into the specific emotional intelligence dimensions
provided by Bar-On. As the literature continues to grow, additional dimensions
of emotional intelligence may be discovered. As well, the Bar-On model differs
from other mixed models of emotional intelligence and those models may account
for more or less variance in situational judgment test scores. Although Bar-On
and his emotional intelligence instruments are well known in both academic and
practitioner circles and although the Work Problems Survey has been used
both in academic and practitioner settings, the results of the current study
are not generalizable to other emotional intelligence or situational judgment
tests.
Finally, although the Work Problems Survey was designed to be a valid
predictor of success with several different types of jobs, it was not
specifically designed to measure emotional intelligence. Because situational
judgment tests are considered to be measurement methods, a test could be
designed to specifically measure one’s emotional intelligence but a tradeoff is
presumed to exist. Part of the appeal of situational judgment tests is that
they tap into several different success-related constructs and what adds to the
capturing of one construct may detract from the capturing of another. Again,
the research presented in this paper is simply an extension of research that
seeks to further our understanding of what situational judgment tests can be
used to measure.
References
Bar-On, R. (1997). Bar-On Emotional Quotient
Inventory (EQ-i):
Technical Manual. Toronto:
Multi-Health Systems.
Bar-On, R. (2000). Emotional and social intelligence:
Insights from the
Emotional Quotent Inventory. In J. D. A. Parker (Ed.), The
handbook of emotional
intelligence (pp. 363-390). San Francisco: Josse-Bass.
Bar-On, R. (2002). Bar-On Emotional Quotient
Inventory: Short Technical Manual. Toronto, Canada: Multi-Health Systems.
Barrick, M. R., & Mount, M. K. (1991). The big
five personality dimensions and
job performance: A meta-analysis. Personnel
Psychology, 44, 1-26.
Boyatzis, R.E., Goleman, D., & Hay/McBer. (1999).
Emotional Competence Inventory. Boston: HayGroup.
Chan, D., & Schmitt, N. (1997). Video-based
versus paper-and pencil method of assessment in situational judgment tests:
Subgroup differences in test performance and face validity perceptions. Journal
of Applied Psychology, 82, 143-159.
Clevenger, J., Jockin, T., Morris, S., & Anselmi,
T. (1999). A situational judgment test for engineers: Construct and
criterion related validity of a less adverse alternative. Paper presented
at the 14th Annual Conference of the Society for Industrial and Organizational
Psychology, Atlanta, GA.
Clevenger, J. P., & Haaland, D. E. (2000). The
relationship between job knowledge and situational judgment test performance.
Paper presented at the 15th Annual Conference of the Society for Industrial and
Organizational Psychology, New Orleans, LA.
Costa, P. T., & McCrae, R.R. (1992). Revised
NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI)
professional manual. Odessa, FL: Psychological Assessment Resources.
Cooper, R.K. (1997). EQ Map. San Francisco:
AIT and Essi Systems.
Davies, M., Stankov, L., & Roberts, R. D. (1998).
Emotional intelligence: In search of the elusive construct. Journal of
Personality and Social Psychology, 75, 989-1015.
Fund, S., & Bar-On, R. (2002). Examining the
contribution of emotional intelligence to occupational performance. Unpublished
manuscript.
Goleman, D. (1995). What's Your EQ. Retrieved,
from the World Wide Web: http://www.utnereader.com/
Gowing, M. K. (2001). Measurement of individual
emotional competence. In D. Goleman (Ed.), The emotionally intelligent
workplace (pp. 83-131). San Francisco: Josse-Bass.
Grubb III, W. L. (2003). Situational judgment and
emotional intelligence tests: Constructs and faking. Unpublished
doctoral dissertation. Virginia Commonwealth University, Richmond, VA.
Grubb III, W. L., Whetzel, D. L., & McDaniel, M.
A. (2004). General mental ability
tests in industry. In J. Thomas (Ed.), Comprehensive
handbook of psychological assessment.
Hurtz, G. M., & Donovan, J. J. (2000).
Personality and job performance: The Big
Five revisited. Journal of Applied Psychology, 85,
869-879.
Jae, J. H. (1997). Emotional intelligence and
cognitive ability as predictors of job performance in the banking sector.
Unpublished master's thesis. Ateneo de Manila University, Philippines, Manila.
Mayer, J. D., & Salovey, P. (1997). What is
emotional intelligence? In D. J. Sluyter (Ed.), Emotional development and
emotional intelligence: Implications for educators (pp. 3-31). New York:
Basic Books.
Mayer, J. D., Salovey, P., & Caruso, D. (2000).
Models of emotional intelligence. In R. J. Sternberg (Ed.), Handbook of
intelligence (pp. 396-420). New York: Cambridge University Press.
McCrae, Robert R., (2000). Emotional intelligence
from the perspective of the five factor model of personality. In R. Bar-On,
& J.D.A. Parker (Eds.), The Handbook of Emotional Intelligence (pp.
263-276). San Francisco: Josse-Bass Inc.
McDaniel, Michael A., Hartman, Nathan S., & Grubb
III, W. L., (2003). Situational Judgment Tests, Knowledge, Behavioral
Tendency, and Validity: A Meta-Analysis. Paper presented at the 18th
Annual Conference of the Society for Industrial and Organizational Psychology,
Orlando.
McDaniel, M. A., Morgeson, Frederick P., Finnegan,
Elizabeth B., Campion, Michael A., and Braverman, Eric P. (2001). Use of
situational judgment tests to predict job performance: A clarification of the
literature. Journal of Applied Psychology, 86, 730-740.
McDaniel, M. A., &
Nguyen, N. T. (2001). Situational
judgment tests: A review of practice and constructs assessed. International
Journal of Selection and Assessment, 9, 103-113.
McDaniel, M. A., & Whetzel, D. L. (2005)
Situational judgment research: Informing the debate on Practical intelligence
theory. Manuscript submitted for publication.
Motowidlo, S., Dunnette, M. D., & Carter, G. W.
(1990). An alternative selection procedure: The low-fidelity simulation. Journal
of Applied Psychology, 75(6), 640-647.
Motowidlo, S., & Tippins, N. (1993). Further
studies of the low-fidelity simulation in the form of a situational inventory. Journal
of Occupational and Organizational Psychology, 66, 337-344.
Mullins, M. E., & Schmitt, N. (1998). Situational
judgment testing: Will the real constructs pleas present themselves.
Paper presented at the 13th Annual Conference of the Society for Industrial and
Organizational Psychology, Dallas, TX.
Newsome, S., Day, A. L., & Catano, V. M. (2000).
Assessing the predictive validity of emotional intelligence. Personality and
Individual Differences, 29, 1005-1016.
Nguyen, N. (2001). Faking in situational judgment
tests: An empirical investigation of the work problems survey. Unpublished
doctoral dissertation. Virginia Commonwealth University, Richmond, VA.
O'Connell, M. S., McDaniel, M. A., Grubb, W. L., III,
Hartman, N. S., & Lawrence, A. (2002). Incremental validity of
situational judgment tests for task and contextual performance. Paper
presented at the 17th Annual Conference of the Society of Industrial
Organizational Psychology, Toronto, Ontario.
Parker, J. D. A. (2001). Conceptual and predictive
validity of the emotional intelligence construct: Empirical support using young
adults. Paper presented at the 10th Biennial Meeting of the International
Society for the Study of Individual Differences, University of Edinburgh
Pereira, G. M., & Schmidt Harvey, V. (1999). Situational
judgment tests: Do they measure ability, personality or both. Paper
presented at the 14th Annual Conference of the Society for Industrial and
Organizational Psychology, Atlanta, GA.
Ployhart, R. E., & Ehrhart, M. G. (2001). Effects
of response instructions on criterion-related validity, construct validity, and
reliability of situational judgment tests. Paper presented at the 16th
Annual Conference of the Society for Industrial and Organizational Psychology,
San Diego, CA.
Ployhart, R. E., & Ryan, A. M. (2000). A
construct-oriented approach for developing situational judgment tests in a
service context. Manuscript submitted for publication.
Schutte, N. S., Malouff, J. M., Hall, L. E.,
Haggerty, D. J., Cooper, J. T., Golden, C. J., & Dornheim, L. (1998).
Development and validation of a measure of emotional intelligence. Personality
and Individual Differences, 25, 167-177.
Smith, K. C., & McDaniel, M. A. (1998). Criterion
and construct validity evidence for a situational judgment measure. Paper
presented at the 13th Annual Convention of the Society for Industrial and Organizational
Psychology, Dallas, TX.
Weekly, J. A., & Jones, C. (1999). Further
studies of situational tests. Personnel Psychology, 52, 679-700.
