See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/312212692
Measuring Subjective Movie Evaluation Criteria: Conceptual Foundation,
Construction, and Validation of the SMEC Scales
Article in Communication Methods and Measures · January 2017
DOI: 10.1080/19312458.2016.1271115
CITATIONS
READS
15
588
1 author:
Frank M. Schneider
Universität Mannheim
95 PUBLICATIONS 1,243 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Social media & ostracism: Consequences and coping View project
Effects of election promises on trust in politicians and vote choice View project
All content following this page was uploaded by Frank M. Schneider on 08 March 2019.
The user has requested enhancement of the downloaded file.
Running head: MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
Please cite as:
Schneider, F. M. (2017). Measuring subjective movie evaluation criteria: Conceptual foundation,
construction, and validation of the SMEC scales. Communication Methods and Measures, 11, 49–75.
https://doi.org/10.1080/19312458.2016.1271115
Measuring Subjective Movie Evaluation Criteria: Conceptual Foundation, Construction, and
Validation of the SMEC Scales
Frank M. Schneider
University of Mannheim
Author Note
Frank M. Schneider (PhD, University of Koblenz-Landau, Germany), Institute for Media
and Communication Studies, University of Mannheim, Germany
Correspondence concerning this paper should be addressed to Frank M. Schneider, Institute
for Media and Communication Studies, University of Mannheim, Haus Oberrhein,
Rheinvorlandstr. 5, 68159 Mannheim, Germany. E-mail: frank.schneider@uni-mannheim.de,
Phone: +49 621 181-3938, Fax: +49 621 181-393.
Acknowledgements
I thank Frieder Schmid for assisting in collecting the data, Carina Weinmann for her
feedback on an earlier version of this paper as well as the anonymous reviewers and the editor,
Jörg Matthes, for their valuable recommendations and thoughtful comments that helped to
improve the paper during the editorial process.
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
2
Abstract
Audiences’ movie evaluations have often been explored as effects of experiencing
movies. However, little attention has been paid to the criteria viewers use when they evaluate a
movie or its specific features. Adding to this, the present research introduces the idea of
subjective movie evaluation criteria (SMEC), conceptualizes SMEC as the mental representation
of important attitudes toward specific film features, and describes the scale construction for their
measurement and its validation process. Findings from pilot work and 2 studies including over
1,500 participants provide first evidence that 8 dimensions—Story Verisimilitude, Story
Innovation, Cinematography, Special Effects, Recommendation, Innocuousness, Lightheartedness, and Cognitive Stimulation—are largely determined by stable individual differences,
substantially but differentially related to film-specific constructs and personality traits, and that
the SMEC scales are reliable and valid instruments for measuring subjective movie evaluation
criteria.
Keywords: subjective movie evaluation criteria; important attitudes toward specific film features;
scale construction; scale validation
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
3
Measuring Subjective Movie Evaluation Criteria: Conceptual Foundation, Construction, and
Validation of the SMEC Scales
“I have a foolproof device for judging whether a picture is good or bad. If my fanny squirms, it's
bad. If my fanny doesn't squirm, it's good. It's as simple as that.”
—Harry Cohn (*1891, †1958), founder and president of Columbia Pictures
Is evaluating a movie really as simple as suggested by these famous words from Harry
Cohn? Or, do we have a set of more elaborated criteria we can apply when we evaluate a movie?
If we talk with others about films, want to buy a DVD at an online store, or visit movie websites
like www.rottentomatoes.com or www.IMDb.com, we come across a plethora of movie
evaluations. We also find that individuals judge the same movies quite differently and that these
judgments are based on different criteria. For example, the movie Babel was reviewed on Rotten
Tomatoes as “elegantly written and shot, harrowing to watch and realistic” or as “unusual
aesthetic force, no coherent idea or narrative logic” or as “one of the most challenging and
saddest movies of the year”, or as “higher kitsch” (see also Chamorro-Premuzic, Kallias, & Hsu,
2014). Such observations give reason to think about the concept of movie evaluation criteria
from the perspective of lay audiences because evaluating movies is not limited to film theorists,
movie critics, or festival juries—“[it] is something that we all do all of the time” (Carroll, 2003,
p. 148). Thus, the present article deals with the notion of criteria lay audiences use for evaluating
narrative films, or briefly subjective movie evaluation criteria (SMEC), and addresses the
question what is important when we watch and evaluate a movie.
Conceptually speaking, it is assumed that SMEC are mental representations of important
attitudes toward specific film features that have been formed during our cinematic socialization
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
4
(cf. Valkenburg & Cantor, 2000; Valkenburg & Janssen, 1999). These specific film features can
be broadly categorized as film-inherent features (e.g., story, cinematography), film-external
features (e.g., awards, film critiques), or anticipated effects of use (e.g., features that might be
entertaining or require cognitive effort to cope with).1 Like attitudes, SMEC can vary in their
importance. Especially important attitudes have been found to be stable over time, resistant to
persuasion, influential in information processing, and predictive of behavior (e.g., Boninger,
Krosnick, Berent, & Fabrigar, 1995; Eaton & Visser, 2008). Thus, in mimicking important
attitudes, SMEC might be consequential for encountering movies and corresponding evaluative
responses.
After briefly summarizing why movie evaluations and SMEC are relevant and reviewing
previous operationalizations of film-related evaluation criteria, the empirical part of this article
focuses on the number of SMEC dimensions, their scope, and on the question how we can
measure them.
Movie Evaluations and the Role of SMEC
From motivational frameworks for movie theater attendance (e.g., Austin, 1986;
Palmgreen, Cook, Harvill, & Helm, 1988; Tesser, Millar, & Wu, 1988) and mood management
(Zillmann, 1988) to more recent and general attempts to delineate the underlying motivations and
gratifications of entertainment consumption (e.g., Oliver & Raney, 2011; Tamborini et al., 2011;
Wirth, Hofer, & Schramm, 2012), communication scholars have been interested in why people
expose themselves to movies. Even more studies have been conducted to investigate the effects
of movies on their audience (Young, 2012). However, surprisingly little research has been done
with regard to movie evaluations, although they seem to play an important part not only after, but
also during, and before movie exposure (for a detailed overview, see Schneider, Vogel, Gleich,
& Bartsch, 2014).
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
5
Evaluation After Exposure
First, movie evaluations have often been operationalized as post-hoc judgments after
exposure (e.g., enjoyment or appreciation as evaluative responses to movies; e.g., Oliver &
Bartsch, 2010). These evaluations are not only determined by the (perceived) features of the
movie itself; they are also determined by characteristics of the situation (e.g., going to the cinema
theater alone or with others; discussing a movie with others at a bar or in an online forum;
multitasking while watching a movie at home; etc.), and by individual characteristics (e.g.,
preferences, expectations, attitudes, motivations, traits, and states; e.g., Krcmar, 2009).
According to the disposition-content congruency hypothesis (Valkenburg & Peter, 2013, p. 232),
the content of a movie is more likely to have an impact if it is congruent to the characteristics of
the viewer. For instance, an individual’s greater preference for suspenseful films led to a better
evaluation of a suspenseful movie (cf. Vorderer, Knobloch, & Schramm, 2001), viewers who
watched clips from film festival submissions and had more movie expertise evaluated these clips
as more interesting (Silvia & Berg, 2011), sensation seeking and neuroticism predicted the
evaluation of violent and arousing content (Chamorro-Premuzic et al., 2014), and the need for
cognition affected mystery enjoyment (e.g., Knobloch-Westerwick & Keplinger, 2008). Thus, if
viewers hold stable, important attitudes toward specific film features (i.e., SMEC) and encounter
congruent film features, it seems likely that these attitudes affect their evaluation of a movie as
well. From this perspective, we can look at movie evaluations as dependent variables that are
partly determined by SMEC. Most importantly, if SMEC are causally related to movie
evaluations, they may also exert their influence when movies are evaluated while watching.
Evaluation During Exposure
During movie exposure, the interplay of cognitive and affective processes hints to the role
of movie-related evaluations as a mediating variable. For instance, from the perspective of
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
6
appraisal theories of emotions (e.g., Bartsch, Vorderer, Mangold, & Viehoff, 2008; Scherer,
2013; Schramm & Wirth, 2010), cognitive appraisals are the key to understand how emotions are
elicited. Movie-specific evaluative processes might be related to these more general cognitive
appraisals. In fact, these appraisals do underlie basic evaluation principles (i.e., appraisal criteria
or stimulus evaluation checks, cf. Scherer, 2013). These stimulus evaluation checks include,
among others, the novelty, the intrinsic pleasantness, and the norm compatibility of an event, and
the coping potential with regard to an event. Thus, it seems possible that corresponding SMEC
(e.g., novel or pleasant aspects of story-telling or cinematography) influence the set of appraisal
criteria or even become some of them. Consequently, the mediational process between the film
stimulus and the emotional response via cognitive appraisals might be partly determined by
SMEC. Furthermore, it also seems likely that evaluating cinematic information while watching a
movie could alter involvement. For example, if someone thinks it is important that a movie does
not contain brutal scenes, but perceives cues for brutality, and evaluates them as offensive, she or
he might stop or get less involved in watching the movie. Alternatively, these evaluated cues can
trigger different modes of reception—for instance, switching from an emotionally-involved to a
more production-oriented mode of reception (Suckfüll & Scharkow, 2009).
Evaluation Before Exposure
Finally, before movie exposure, prior aggregated evaluative information about movies
from external sources can be seen as determinants of movie choice and influential factors for
responses to films. This is widely acknowledged in the movie marketing literature, especially
with regard to awards and nominations, critical acclaim and professional critique, and word-ofmouth recommendations (for overviews, see e.g., Hadida, 2009; Simonton, 2009). In addition,
experimental studies stress the importance of critical acclaim and word-of-mouth for evaluative
responses to movies (e.g., d'Astous & Touil, 1999; Jacobs, Heuvelman, ben Allouch, & Peters,
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
7
2015; Wyatt & Badger, 1990). In other words, recommendations or prior aggregated movie
evaluations from others might influence the way we select, process, and respond to movies. In
this sense, movie evaluations can be seen as independent variables. If we deem them to be
important for our own choices of and judgments about movies, these recommendations might
qualify as SMEC. This might be especially true when we intentionally seek out such filmrelated, external information.
With regard to SMEC as potentially dispositional determinants of movie evaluations (see
above), individuals may also choose movies congruent to their traits, needs, motivations,
preferences, SMEC, or actual states more or less consciously. This has been addressed, for
instance, by research on uses and gratifications (U&G), gratifications sought and obtained
(GSGO), selective exposure, and mood management (e.g., Knobloch-Westerwick, 2015; Krcmar,
2009). However, these approaches focus rather on the users’ driving forces and anticipated
effects of use than on the specific film-inherent content or features like cinematography,
aesthetics, or story (cf. Swanson, 1987, p. 245; Wolling, 2009, p. 85).
In contrast to these approaches and to explicitly include media-inherent features, Wolling
(2004, 2009) proposed the theory of subjective quality assessment (TSQA) that is based on
expectancy x value assumptions and GSGO models (e.g., Palmgreen & Rayburn, 1985), but
extends them in two major ways: (a) Whereas expectations can be seen as the probability that a
media object will possess a specific feature, the TSQA postulates that users desire that a media
product should possess specific features. If two or more desired features are in conflict (e.g.,
perhaps a happy and a realistic ending cannot be desired at the same time), the importance of
each desired feature might play a crucial role in predicting the global judgment about a movie
across features. Although SMEC are defined as important attitudes, which are conceptually
different from desires, the idea of varying importance is common two both concepts. (b) Distinct
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
8
from these desired features is the second component in the TSQA: the subjective evaluation of
the encountered features. In GSGO, these evaluations are derived from experiences with or
behavioral responses to the specific features and are weighted against expectations (Wolling,
2009, p. 89), whereas in the TSQA, evaluations of specific features are generated through the
interplay between the desired or undesired features and the perceived features (for details, see
Wolling, 2009, pp. 89–92). Likewise, the interplay between SMEC and congruently perceived
features might influence specific evaluative responses. Most importantly with regard to SMEC,
Wolling presents some early thoughts on evaluation criteria of media products in general, that is
on which dimensions can the features be systematized (p. 96). For instance, he proposes three
preliminary and broad criteria dimensions (i.e., reality, originality, and the effects of the features)
that comprise different features.
In sum, the TSQA seems to be a useful framework that explicitly incorporates parameters
that can be related to evaluative processes and criteria. It extends U&G, mood management, and
GSGO by relating evaluations not only to the effects of use but also to specific product features.
This distinction is important, especially for delineating SMEC because they are not only related
to anticipated effects of these features but also to film-inherent and film-external features.
Unfortunately, we cannot derive a set of movie evaluation criteria from the TSQA with regard to
movies. Instead, Wolling suggests drawing from existing “catalogues of criteria” (p. 95) as a
starting point. Especially previous research on subjective evaluation criteria with regard to TV
shows (e.g., Greenberg & Busselle, 1996; Gunter, 1997; Himmelweit, Swift, & Jaeger, 1980;
Wolling, 2004) or within specific target groups (e.g., children, Nikken & van der Voort, 1997;
Valkenburg & Janssen, 1999) provide such catalogues that are briefly reviewed because they
might be useful for the purpose of the present study.
Previous Research on Evaluation Criteria
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
9
Previous research on evaluation criteria is scarce and has focused mainly on criteria in the
context of film journalism or TV quality research. Table 1 gives a summary of the relevant
studies that will be briefly introduced in the following paragraphs.
To my best knowledge, Linton and Petrovich (1988) conducted the only study that
focused specifically on movie evaluation criteria. They asked students to rate the importance of
movie attributes when evaluating a movie and found two underlying dimensions: foreground
(e.g., storyline, acting, scenery, music) and background (e.g., editing, photography, director).
A more elaborated and multidimensional approach evolved from a survey on newspaper
film critics (Wyatt & Badger, 1988). The surveyed film journalists rated the importance of
review elements, movie characteristics for writing film critiques, and the functions of a movie.
In sum, the approach of Wyatt and Badger seems to offer first interesting insights into the
professional perspective on film criticism and evaluation criteria. However, it is questionable
whether these results can be applied to the ordinary movie audience. A study from a project that
dealt with films in daily newspapers examined this question more closely (Rössler, 1997).
Among other studies in this project, Rössler surveyed German newspaper film critics and found
that their ratings on the importance of movie evaluation criteria differed from the ratings of
moviegoers. In line with these results, a US-American study showed only a weak
correspondence between the ratings of film critics and consumers (Holbrook, 1999).
Only a few studies that have dealt with TV evaluation criteria focused on fictional
formats that might be related to movies in general (e.g., Gehrau, 2008; Greenberg & Busselle,
1996; Himmelweit et al., 1980; Wolling, 2004). The major drawback of these studies lies in the
simultaneous measurement of predictor (evaluation criteria) and criterion variables (evaluative
responses)—both are explicitly related to the program watched. Hence, it is impossible to draw
any conclusions about the causal relationship between general evaluation criteria and the specific
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
10
evaluative responses to a particular film. Unfortunately, what is even worse, we cannot infer
anything about independent criteria or their consistency from these results. Furthermore, Gunter
(1997) questioned the content validity of the often used adjective lists. Instead, he suggested to
“go directly to viewers themselves even before the attribute scales are produced as a source of
insights concerning what those scales might be” (p. 14). In his own research on soap operas,
Gunter (1997) developed genre-specific scales in two steps, consisting of a qualitative focus
group and a quantitative survey approach.
One branch of TV quality research has focused on the criteria children use to evaluate TV
programs. Nikken and colleagues (e.g., Nikken & van der Voort, 1997; Nikken, van der Voort,
& van Bochove, 1996) found seven types of quality criteria when asking mothers how a good
children’s TV program should be. These types plus two additional criteria were then used to
assess children’s quality criteria for TV programs (Nikken & van der Voort, 1997). The results
resembled those of the mothers-study, especially when taking only fictional TV program genres
into account. However, as Valkenburg and Janssen (1999) argued, the study is limited in that
Nikken and van der Voort (1997) largely used items from the list of the mothers-study.
Therefore, Valkenburg and Janssen (1999) used a focus group approach to collect children’s
criteria and then conducted a paper-and-pencil questionnaire study. Their analyses resulted in an
eight-dimensional solution and were quite similar for Dutch and US children (see Table 1).
Summary and Purpose of the Present Research
Because there is no approach to the SMEC of lay audiences, it is important to first shed
some light on criteria found in the context of film critics and reported on selected work from TV
quality research. Although TV research has accumulated a large body of literature, it is limited
to this medium and thus cannot be merged with film audience research without hesitation:
Watching TV comprises many more formats and genres irrelevant for movie evaluation (e.g.,
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
11
news, magazines, and talk shows). Furthermore, it is supposed that TV is associated with
different evaluations and viewing motives (Finn, 1997). Despite these constraints, we could gain
insights into research methods applied to quality assessments that might be useful for the purpose
of the present research—especially the combination of qualitative and quantitative approaches
during a multi-step development of measures and the call for an inductive and data-driven
development of the criteria by asking the lay audience (e.g., Gunter, 1997; Himmelweit et al.,
1980; Valkenburg & Janssen, 1999). Moreover, the TV quality studies included to some amount
narrative and fictional stimuli. Thus, if carefully interpreted with regard to movies, these
findings give us a first impression of the research done so far. Although previous research has
accumulated evidence for the existence of subjective evaluation criteria, taken together, findings
from film criticism and TV quality research show some shortcomings with respect to the goals of
the present research: (a) Film critics might have different criteria than laypeople (e.g., Holbrook,
1999; Rössler, 1997). (b) TV programs contain specific and varying features. Their one-to-one
application to movies hardly seems justifiable (e.g., viewers might apply different criteria to
evaluate news, talk shows, game shows). (c) Even children seem to already have developed
sophisticated, multidimensional criteria for assessing TV programs (Nikken & van der Voort,
1997; Valkenburg & Janssen, 1999). Thus, it seems reasonable to assume that more experienced
viewers apply even more or more elaborated criteria. Furthermore, it is also possible that the
criteria children use differ from those adolescents and adults apply. (d) Adjective lists seem to
be inappropriate as a measurement instrument because they might be too abstract to capture
specific film features (e.g., Gunter, 1997). (e) The simultaneous assessment of criteria
importance and evaluation of stimuli already watched might lead to erroneous conclusions due to
confounding (e.g., Gunter, 1997; Wolling, 2009). (f) Most of the instruments applied were ad
hoc scales and thus, maybe only appropriate to the respective samples. (g) The use of
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
12
exploratory data analysis to examine the construct space was not followed by further
confirmation or validation procedures.
The present research differs from previous research in some important aspects: Of course
and first of all, the topic is movies across media, not TV shows. Second, the individual steps in
developing a measurement instrument are carried out in separate studies (e.g., item development,
scale construction). In addition, confirmatory and construct validation studies are conducted.
Moreover, detailed reporting on the psychometric properties of the scales is warranted. Finally,
the broad conceptualization of SMEC as important attitudes toward specific film features offers
the opportunity to connect them to a plethora of attitude theories (e.g., Gawronski, 2007;
Schneider, 2012) and also to use them for further examination of the criteria dimensions within
the TSQA (Wolling, 2009).
In line with the recommendations of previous approaches (e.g., Gunter, 1997;
Himmelweit et al., 1980; Valkenburg & Janssen, 1999) and because we have only little
theoretical and empirical knowledge about SMEC, as a first step, an inductive, data-driven
approach to scale development seems to be the most appropriate strategy to gain a deeper
understanding of the content domain.2 Although previous research has tried to answer the
question what is important when we watch and evaluate a movie and accumulated evidence for
the existence of SMEC, we still do not know their number, scope, or stability—and we do not
have any proven measure. Thus, it seems fruitful to start exploring them in more detail and
develop a methodologically sound instrument for their measurement.
Pilot Studies for Item Pool Development
To collect everyday language terms of movie evaluation criteria, 258 participants (age:
M = 26.44, SD = 9.68, range 18–73; 73% female) filled out questionnaires with an open-ended
item (“In general, what do you think what kind of criteria can be applied to evaluate a movie?”).
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
13
Responses to this open-ended question informed the construction of 117 items that broadly
covered all aspects that were investigated or found in previous research (see above). Every item
consisted of three components: (a) “When evaluating movies, how important for you personally
is/are...”, (b) derived from the answers to the open-ended question, a phrase including the
criterion (e.g., an item concerning the verisimilitude of a film’s story was worded “...that the
story the film tells is realistic?”), and (c) a rating scale. All items were positively phrased to
avoid artifacts only due to the negative wording. The rating scales were constructed as 5-point
unipolar with construct-specific labels for the response options: 0 (not at all important), 1
(slightly important), 2 (moderately important), 3 (very important), 4 (extremely important).
The items were pretested applying the item panel method with 14 participants (mostly
graduates). In an item panel, panelists review and discuss in several rounds if the content and
wording of items are comprehensible and appropriate for the measured construct (cf. Wilson,
2005, pp. 59–61). Results led to minor wording revisions and the exclusion of some items. The
final version—including 93 items for measuring SMEC—was created as an online-questionnaire
and subjected to another pilot test. This pilot test was conducted as a mix of thinking-aloudtechnique (n = 2) and exit interviews (n = 6) by two interviewers (cf. Wilson, 2005). The
interviewers’ discussion of the results led to minor changes in item wording and the inclusion of
an example how to fill out the online questionnaire.
Study 1
Study 1 aimed to explore the latent structure of SMEC and to reduce the number of items.
Method
Sample and Procedures. Cover letters including the link to an online questionnaire
were distributed to students via mailing lists. Participants were offered a chance to win shopping
vouchers for Amazon as an incentive to take part in the study. To further increase the number of
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
14
participants, all recipients were asked to forward the link to other persons they knew. The
“Start”-button on the welcome page was clicked by 659 people; 506 of them completed all
SMEC items (N), 500 participants completed the entire questionnaire (age: M = 30.4, SD = 11.9,
range = 16–77; 65% female; 93% had at least finished secondary school).
Measures. Besides socio-demographic items, the major part of the online questionnaire
consisted of 93 items to measure SMEC and two items to check for methodological issues (i.e.,
perceived difficulty of responding to questions, specific reference movies for answering the
items). The response scale was provided as described above.
Data analytic procedure. Prior to analysis and using SPSS, all 93 variables were
examined for multivariate outliers, univariate normality by assessing skewness and kurtosis
(skew < 2, kurtosis < 7), and item difficulties (cut-off criteria for exclusion were < .20 or > .80).
No outliers were excluded. The distributions of 13 items were severely skewed, had a highly
positive kurtosis, or showed unsatisfactory item difficulties; therefore, these items were
excluded. The remaining 80 criteria items were analyzed with CEFA software (Browne, Cudeck,
Tateneni, & Mels, 2010). Following the recommendations of Fabrigar, Wegener, MacCallum,
and Strahan (1999), exploratory factor analysis (EFA; maximum likelihood [ML] extraction,
oblique Geomin rotation, no row standardization; Browne, 2001) was run to assess the
dimensionality of the movie criteria and identify reasonable corresponding indicators.3 RMSEA,
RMSEA “close” fit (CFit; RMSEA ≤ .050), and parallel analysis were investigated to determine
the number of factors. Items without salient loadings (less than |.40| or less than |.50| for the
upper bound of the 90% CI) or with multiple salient cross-loadings were removed considering
90% CI for the loadings in the pattern matrix. The primary goal was to achieve a simple
structure.
Results
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
15
For the 80 items included in EFA, a good model fit was obtained for a 10-factor solution
(RMSEA = .047, 90% CI of RMSEA [.045, .048], CFit = .999; parallel analysis also suggested a
10-factor solution). However, 25 items had no salient loading on any of these factors. Thus,
these items were excluded, and EFA was run again with the remaining 55 items. This procedure
was repeated until adequate fit, salient loadings, and non-salient cross-loadings were achieved.
After a total of eight processing cycles, an adequate fit was achieved for an eight-factor solution
with 32 items (RMSEA = .052, 90% CI of RMSEA [.046, .057], CFit = .291; parallel analysis
also suggested an eight-factor solution). The final solution is presented in Table 2, which also
provides latent factor correlations and Raykov's ρ.4
Discussion
The first factor is labeled Story Verisimilitude (SV) and comprises three items reflecting
correspondence to (contemporary) reality. Similar factors representing realism have been found
in previous studies (e.g., Greenberg & Busselle, 1996; Gunter, 1997; Nikken & van der Voort,
1997; Valkenburg & Janssen, 1999). The second factor is labeled Story Innovation (SI) and
consists of two items reflecting the originality of the story. This is also consistent with
dimensions found in previous research like originality (e.g., Greenberg & Busselle, 1996). The
third factor is labeled Cinematography (CI) and comprises four items reflecting core cinematic
techniques. It is related to film theoretical approaches but has been found in audience research
too (e.g., Gunter, 1997). The fourth factor is labeled Special Effects (FX) and consists of four
items also reflecting cinematic techniques, but focusing more on technical effects. It has been
investigated as a film-inherent factor or an element of film critiques before (e.g., Neelamegham
& Jain, 1999; Rössler, 1997). The fifth factor is labeled Recommendations (RE) and comprises
five items reflecting external resources for film evaluation. Indicators of RE were found to be
important factors in influencing movie attendances and evaluations (e.g., Möller & Karppinen,
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
16
1983; Neelamegham & Jain, 1999). The sixth factor labeled Innocuousness (IN) consists of four
items reflecting lack of potentially unpleasant characteristics. This dimension represents
anticipated audience responses and is consistent with previous research on children’s TV quality
criteria (e.g., Nikken & van der Voort, 1997; Valkenburg & Janssen, 1999) and also related to
factors that represent the opposite pole of this dimension like sex and violence (e.g., Wyatt
& Badger, 1988). The seventh factor is labeled Light-heartedness (LH), comprises five items,
and reflects amusement and escapism and has also been found in previous studies (e.g.,
Greenberg & Busselle, 1996; Nikken & van der Voort, 1997; Valkenburg & Janssen, 1999).
Finally, the eighth factor is labeled Cognitive Stimulation (CS), consists of five items, and
reflects the viewer’s cognitive processes such as cogitation or learning and is consistent with
previous factors (e.g., Himmelweit et al., 1980; Nikken & van der Voort, 1997).
Taken together, most of these factors are not independent of each other. However, they
share only little variance (Mdn of all latent correlations is .14). Therefore, it seems to be justified
to interpret them as distinct dimensions. Because none of the factors is highly correlated with
another one, it is also unlikely that there are underlying higher factors.
Study 2
Given the initial findings in Study1 concerning the factorial structure and scope of the
SMEC dimensions, Study 2 had three goals: (1) to cross-validate the latent structure discovered
in Study 1 with another sample and to improve structural validity, (2) to provide evidence for the
substantive validity of the different SMEC (i.e., to show that these criteria can be better described
as rather enduring and cross-situationally consistent constructs than transient constructs that are
susceptible to situational influences), and (3) to locate the SMEC in the nomological network of
related constructs by examining their discriminant and convergent validity. Although nothing is
yet known about possible relationships to other constructs relevant to SMEC, it stands to reason
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
17
that—based on the disposition-content congruency hypothesis (Valkenburg & Peter, 2013)—the
different SMEC dimensions have their own particular relationships with (a) more general
personality traits (e.g., the Big Five), (b) thematically related constructs from specific domains
(e.g., need for cognition and sensation seeking), and (c) film-specific constructs like film
expertise, film genre preferences, and film-specific evaluations.
The Nomological Network of SMEC and Related Constructs
Big Five personality factors. Although there is surprisingly little research on how the
Big Five contribute to movie-related motives, preferences, and evaluations, the scarce empirical
evidence clearly demonstrates that individuals seek movies that reflect and reinforce facets of
their personalities (e.g., Chamorro-Premuzic et al., 2014; Rentfrow, Goldberg, & Zilca, 2011).
The Big Five dimensions—Neuroticism, Extraversion, Openness to Experience, Agreeableness,
and Conscientiousness—describe personality traits on an abstract and broad level (e.g.,
Goldberg, 1990). As research on Conscientiousness and movie-related motives, preferences, and
evaluations is scarce and connections are not theoretically clear, it is not further considered here.
Neurotic (or emotionally instable) individuals are more likely to experience negative
affective states as anxiety, anger, or depressed mood. They are self-conscious, impulsive, and
vulnerable to stress. Neuroticism was found to be negatively correlated with preferences for
action-adventures, intellectual and mainstream films, and violent media content (e.g., Kallias,
2012; Krcmar & Kean, 2005; Weaver, 1991).
Extraverts tend to be friendly, gregarious, assertive, active, excitement-seeking, and
cheerful. In a few studies, Extraversion was positively associated with the liking of surrealist
(Swami, Stieger, Pietschnig, & Voracek, 2010) and violent films (Krcmar & Kean, 2005).
People who are open to experiences appreciate aesthetics, reflect on their emotional
states, are interested in adventurous activities, are intellectually curious and liberal, and have a
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
18
vivid imagination. They also seem to like surrealist films (Swami et al., 2010) and violent
content if it was of aesthetical value (Krcmar & Kean, 2005). Furthermore, Openness was also
positively related to artistic and information seeking viewing motivations and to preferences for
art, drama, and science-fiction genres as well as negatively related to hedonistic viewing
motivations (Kallias, 2012). This is also in line with a strong positive relationship to eudaimonic
entertainment motivations (Oliver & Raney, 2011). Moreover, Silvia and Berg (2011)
demonstrated that Openness to Experience is also related to expert knowledge about movies.
Agreeable people think that most other people are trustworthy. They are friendly,
empathetic, generous, and helpful. People who scored high on Agreeableness preferred lighthearted and entertaining programs, such as romances and comedies, whereas low agreeableness
was related to viewing or liking action-oriented, suspenseful, violent, dramatic, or horror movies
(for overviews, Chamorro-Premuzic et al., 2014; Kallias, 2012). Furthermore, Agreeableness
was positively related to affiliation and hedonistic viewing motivations, preferences for
mainstream genres and negatively related to aggressive viewing motivations (Kallias, 2012).
Sensation seeking. Sensation seeking, “the need for varied, novel, and complex
sensations and experiences and the willingness to take physical and social risks for the sake of
such experience” (Zuckerman, 1979, p. 10), manifests itself in individually different levels of
optimal arousal. To establish these levels, high sensation seekers search for stimulating
sensations to avoid boredom, whereas low sensation seekers rather enjoy calm environments
instead of excitement. For instance, whereas high sensation seekers liked horror, suspenseful,
violent, action, or sexual content more, and preferred high-arousal films, low sensation seekers
preferred light films, and avoided violent and sexually explicit content (for overviews, see
Chamorro-Premuzic et al., 2014; Kallias, 2012).
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
19
Need for cognition. The need for cognition, the “tendency to engage in and enjoy
thinking” (Cacioppo & Petty, 1982, p. 130), has been found to be positively correlated with the
enjoyment of complex narratives (Knobloch-Westerwick & Keplinger, 2008), a productionoriented mode of reception (Suckfüll & Scharkow, 2009), preference for an unspoiled story
(Rosenbaum & Johnson, 2016), and with eudaimonic entertainment motivations (Oliver
& Raney, 2011).
Film expertise. Art expertise is widely known to influence the perception and evaluation
of visual arts (e.g., Augustin & Leder, 2006). With regard to movies, Silvia and Berg (2011)
argued that the expert’s domain knowledge should facilitate the processing of aesthetic,
challenging, and complex stimuli and should allow them to more easily build representations and
connect them to other experiences.
Film genre preferences. Media genre preferences result from positive experiences with
specific genres and evolve over time (Knobloch-Westerwick, 2015, p. 15). To subsume a movie
under a specific genre category might help the potential viewers, for instance, to choose among a
vast selection of movies according to their preferences, thereby reducing uncertainty, and to
facilitate the comprehension of the plot during or after watching a new movie (e.g., interpreting a
misfortune in a comedy as funny, in a drama as tragic, or interpreting open endings). Some
findings from research on movie genre preferences and broader personality traits were presented
above and showed that individuals differ in their movie preferences (e.g., Chamorro-Premuzic et
al., 2014; Kallias, 2012; Rentfrow et al., 2011).
In addition, preferences for dramas, science fiction, and non-fiction (e.g., documentaries)
were positively related to eudaimonic gratifications, whereas preferences for comedies and action
adventures were negatively correlated with eudaimonic gratifications, but positively associated
with hedonic motivations (Igartua & Barrios, 2013; Oliver & Raney, 2011). However, prior
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
20
research on movie preferences did not have the opportunities to go beyond the broad and fuzzy
categories of film genres, which are often open to subjective speculation and interpretation.
Thus, leaving aside violent film content (e.g., Chamorro-Premuzic et al., 2014; Krcmar & Kean,
2005), specific film features across genres have not been examined in detail yet, with one
exception: Möller and Karppinen (1983) showed that the viewers’ tendencies to stress
characteristics like film critiques and viewing motives (e.g., extending views and opinions and
aesthetic experiences) were positively correlated with the preference for drama.
Film-specific evaluations. In entertainment research, recent developments distinguish
between two distinct but most often associated types of evaluations concerning effects of use,
hedonic and eudaimonic entertainment experiences (e.g., Bartsch & Schneider, 2014; Lewis,
Tamborini, & Weber, 2014; Oliver & Bartsch, 2010; Wirth et al., 2012). Whereas hedonic
entertainment experiences refer to longstanding concepts like enjoyment and pleasure (cf.
Vorderer, Klimmt, & Ritterfeld, 2004), the concept of appreciation is prevalent in research on
eudaimonic entertainment experiences. It is defined as “an experiential state that is characterized
by the perception of deeper meaning, the feeling of being moved, and the motivation to elaborate
on thoughts and feelings inspired by the experience” (Oliver & Bartsch, 2010, p. 76). As
discussed above with regard to movie evaluations after exposure, film-specific evaluations might
not only be related to features of the movie but also to the viewer’s traits (e.g., Krcmar, 2009).
Characterizing the associations between SMEC and related constructs. SV reflects
the importance of realistic and contemporary elements in a story. Therefore, a positive
relationship with preference for documentaries and negative correlations with preferences for
genres that deal with unrealistic or fantastic stories (e.g., science fiction and animation) and with
film expertise, which focuses on aesthetic and artistic aspects, are expected. As Bartsch and
Schneider (2014) pointed out, contrary to the mood management factor lack of semantic affinity
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
21
to real world concerns (Zillmann, 1988) that contributes to hedonic entertainment experiences,
eudaimonic entertainment is related to realistic and socially relevant topics and truth-seeking
motivations (Oliver & Raney, 2011). Thus, SV should be positively associated with
Appreciation.
SI represents original and novel aspects of a film’s story. This should lead to positive
associations with traits like Sensation Seeking, Extraversion, and Openness to Experiences. SI is
also assumed to be positively correlated with film expertise because film experts are
knowledgeable about many stories and thus value new and innovative aspects of movies.
Innovative stories are often based on ideas about the future or include some kind of
“experimental” adaptation of archetypical content (e.g., a hero who saves the world). Thus, SI
should be positively related to preferences for science fiction and avant-garde genres.
CI focuses on visual and aesthetic aspects of movies such as camera or cutting. Thus,
associations with film expertise, Openness to Experiences, Extraversion and Sensation Seeking
should be positive. Suckfüll and Scharkow (2009) found a positive relationship between need for
cognition and a production-oriented mode of reception. Thus, it seems likely that CI is also
positively associated with Need for Cognition. Furthermore, CI should be positively related to
preferences for avant-garde and other genres in which visualization is important (e.g., science
fiction, animation).
FX is also characterized by visual aspects but focuses more on additional technical
effects. Special effects are often assumed to be innovative parts of a movie (e.g., trailers contain
the most eye-catching, arousing, and exciting special effects). Thus, FX should be positively
associated with Sensation Seeking and Extraversion. It is also likely that FX is positively related
preferences for light, non-serious genres (e.g., action, animation, and action comedies) as movies
of these genres, which are most often produced to be entertaining mainstream blockbusters,
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
22
contain special effects and action at the same time. In contrast, negative relationships with more
serious genres like drama, documentary or avant-garde are expected and FX might also be
negatively correlated with constructs that reflect more aesthetic and cognitive effort (i.e., film
expertise and Need for Cognition, respectively). In addition, it is assumed that the nuances of CI
and FX become obvious in different magnitudes of correlations, especially with the genre
preferences described above.
RE takes the importance of external information about films into account. Therefore, it
should be positively related to film expertise and to preferences for genres that receive critical
acclaim in the media and awards (e.g., dramas; cf. Oliver, Ash, Woolley, Shade, & Kim, 2014).
Furthermore, as neurotic people are more anxious and submissive, and cannot easily make up
their minds, they might rely more on film-external features like word of mouth or film critiques
when evaluating movies. Finally, people who have a high need for cognition might be more
likely to scrutinize cinematic information than to enjoy motion pictures that have already been
interpreted or spoiled by others (cf. Rosenbaum & Johnson, 2016).
IN focuses on the absence of aversive scenes (i.e., film elements that are disgusting,
frightening, violent, or enraging). Thus, IN should show negative associations with Emotional
Stability and Sensation Seeking as well as with preferences for genres that usually include scenes
with explicit or negatively arousing content (e.g., thriller, horror, tragedy). In addition, IN should
be positively connected to preferences for not offensive, harmless genres (e.g., romance and most
often comedy), and consequently, to Agreeableness.
Because LH mainly represents positive mood and fun, it should also be positively
associated with Fun and preferences for light entertainment genres. However, these positive
relationships are not expected for serious or negative genres (e.g., avant-garde, drama, horror).
With regard to personality traits, a positive correlation with Agreeableness but negative
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
23
correlations with Openness for Experience and Need for Cognition are assumed. Moreover, LH
should also negatively correlate with film expertise.
Finally, CS is characterized by thought-provocation, broadening knowledge,
communicating values, and taking action. Thus, contrary to LH, CS should be positively
correlated with Appreciation and preferences for serious genres (e.g., drama, avant-garde), and
negatively related to mainstream genres like comedy and action. Moreover, CS should also be
positively related to cognitive effort in information processing (e.g., Need for Cognition) as well
as to traits that reflect engaging in stimulating activities (e.g., Sensation Seeking, Extraversion,
and Openness to Experiences).
Method
Participants and samples. Two samples from different studies were combined to
validate the latent structure. In Study 2a, 152 students filled out questionnaires including the
SMEC items; 147 completed all SMEC items (age: M = 22.65, SD = 4.26, range = 18–47; 64%
female; more than 85% were undergraduates enrolled in social sciences courses). In an
additional sample, an online-study (Study 2b) was conducted that used recruiting strategies
different from Study 1 (e.g., undergraduates received course credit for distributing a link to the
online questionnaire via their social network profiles or mailing lists; a press release including a
link to the study was distributed; a variety of special interest online platforms about cinema and
movies posted the link on their news website, in their weekly newsletter, on Facebook, Twitter,
or online forum). Participants were offered a chance to win shopping vouchers for Amazon as an
incentive. The “Start”-button on the welcome page was clicked by 849 people; 659 of them
completed all SMEC items, 587 of them completed the entire questionnaire (age: M = 29.74, SD
= 10.64, range = 14–72; 55% female; 86% had at least finished secondary school; Mdn of cinema
attendances per month was 1).
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
24
Together, both samples yielded a total of 806 valid cases (147 from the first, 659 from the
second sample). The total sample was randomly split into two halves R1 and R2—each
comprising 403 cases (see below). The 587 participants who completed the entire questionnaire
in Study 2b were used for investigating the nomological network of SMEC.
Exactly four weeks after completing the online-questionnaire in Study 2b (t1),
participants, who wanted to take part in the lottery (n = 371), were invited for the second part of
the study (t2). On average, the time between t1 and t2 was 31.08 days (SD = 6.81). The “Start”button on the welcome page was clicked by 282 people (participation rate: 76%); 275 of them
completed all SMEC items (age: M = 29.88, SD = 10.97, range = 14–70; 60% female; 88% had
at least finished secondary school); 273 participants completed the questionnaire at both
measurement occasions.
Measures. All final 32 SMEC items from Study 1 were included in Study 2a and 2b (t1
and t2). Moreover, one item (Item si3: “. . . that a movie shows something that has never been
shown in a film before?”) that was supposed to load on SI was added as a third indicator for this
factor.
Besides socio-demographic items, the online-questionnaire in Study 2b also included the
following measures to investigate external validity: If not indicated otherwise, all measures were
applied using 5-point Likert scales ranging from 0 (completely disagree) to 4 (completely agree).
The Big Five dimensions Extraversion (ρ = .72), Emotional Stability (ρ = .65), Openness
to Experience (ρ = .50), and Agreeableness (ρ = .53) were assessed using the German version of
the Ten-Item Personality Inventory (Muck, Hell, & Gosling, 2007).
Zuckerman’s (1979) sensation seeking construct was measured by the eight-item Brief
Sensation Seeking Scale (Hoyle, Stephenson, Palmgreen, Lorch, & Donohew, 2002), which was
transferred into German via translation–back-translation method (ρ = .80).
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
25
To assess need for cognition, the German version of the 16-item Need for Cognition scale
(Bless, Wänke, Bohner, Fellhauer, & Schwarz, 1994) was applied (ρ = .87).
Film expertise operationalized as a knowledge base for facilitating aesthetic experiences
was measured using an adaptation of the Aesthetic Fluency in Film scale (Silvia & Berg, 2011).
Participants rated their subjective film expertise on 10 terms reflecting important names related
to film production, history, and theory (e.g., “Sergej Eisenstein”, “Mise-en-scéne”, or “The
Cabinet of Dr. Caligari”) on 5-point scales ranging from 0 (I have never heard of this artist or
term) to 4 (I can talk intelligently about this artist or idea in art). Raykov’s ρ was .90.
Furthermore, Pearson correlations with film viewing frequency (r = .30, N = 563) and
preferences for avant-garde films (r = .42, N = 587) provided preliminary support for criterion
validity.5
Eleven items for measuring film genre preferences—action, animation, avant-garde,
comedy, documentary, drama, horror, romance, science-fiction/fantasy, thriller, and tragedy—on
a response scale ranging from 0 (completely dislike) to 3 (completely like) were included. To get
a fine-grained picture of how SMEC are related to different types of genres, it was necessary to
take into account subtle nuances that differentiate between similar genres (e.g., action, sciencefiction, and thriller). Thus, only the genres drama and tragedy, which were highly correlated
(r = .67), were combined; all other genre preferences were represented as single item measures.
For measuring film viewers’ evaluative responses to specific movies at t2, the German
version of the Oliver and Bartsch (2010) scales—Appreciation (Raykov’s ρ = .90) and Fun
(ρ = .94), each consisting of three items—was included. Before filling out the scales,
participants were asked to recall the last movie they had seen and to respond to the gratification
items with that movie in mind.
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
26
Data analytic procedures. To confirm the structural validity of the SMEC scales, EFA
within the confirmatory factor analysis framework (E/CFA) was conducted for analyzing Sample
R1. One of the reasons for not directly moving into a strict CFA framework was the question if
indeed all cross-loadings and error covariances could be fixed to zero—as it is done in CFA.
Because the latent structure is far from being established at this early stage of scale development,
E/CFA is seen as an appropriate intermediate step between EFA and CFA (Brown, 2006, pp.
193–202). Compared to EFA, E/CFA provides more tools for inspecting misspecifications (e.g.,
a variety of fit indices, standardized residuals, or modification indices). To avoid capitalization
on chance, Sample R2 was used to replicate the findings from E/CFA in Sample R1. All analyses
were conducted with EQS 6.1 (Bentler & Wu, 2005) and robust ML estimation because of
violation of multivariate normality (Satorra & Bentler, 1994). Robust test statistics and cutoff
values for assessing model fit were used (Kline, 2011).
To accomplish the second goal and to examine whether the SMEC scales assessed rather
trait- or state-like constructs, latent state–trait (LST) analyses were conducted (e.g., Schneider,
Otto, Alings, & Schmitt, 2014; Steyer, Geiser, & Fiege, 2012). Testing an LST model requires
(at least) two parallel test halves at two points of measurement for each of the SMEC. Test
halves for each SMEC were generated by randomly assigning the indicators of each SMEC
construct to either one or the other half. Several restrictions had to be imposed to obtain good
fitting models (for details, see Figure 1).
Finally, convergent and discriminant validity were examined by inspecting partial
correlations between the SMEC and relevant constructs in their nomological network, controlling
for age and gender.
Results
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
27
Validity of the latent structure. Although Model 1b clearly replicated the pattern found
in Study 1 (see Table 3), modification indices suggested that freely estimating some error
covariances would improve the model. Additionally, several cross-loadings were salient (see
Table 4). Thus, five items were removed.6 The E/CFA of Model 2 (R1) includes the remaining
28 items and shows improved model fit (e.g., ∆rCFI = .016; rRMSEA includes zero as lower
bound CI). Further support for structural validity comes from the replication of Model 2 via
CFA (R2; see Table 3). All indices indicate good or acceptable model fit.
Stability and reliability. Based on the estimated variance components, the LST
coefficients in Figure 1 were calculated.7 First, all SMEC scales demonstrate high reliabilities
across both measurement occasions. On average, 88% is due to systematic variance indicating
that the scales are reliable. Second, reliabilities are largely determined by stable interindividual
differences (i.e., common consistencies were 65% on average). Third, indicators contain small,
but statistically significant proportions of test-half-specific variance (i.e., method specificities
were 6% on average). Last, on average, 16% variance is due to systematic, but unstable effects
of the situation or interaction. For instance, occasion specificities for IN and LH vary about 10
percentage points between t1 and t2. Additionally, SV, CI, and RE show high proportions of
occasion specificity compared to common consistency.
Convergent and discriminant validity. Previous research on the uses and effects of
movies found differences in age and gender (e.g., Chamorro-Premuzic et al., 2014). Thus, Table
5 presents the partial correlations between the SMEC and the constructs relevant to the
nomological network, after controlling for gender and age.
Discussion
The latent structure found in Study 1 was replicated in Study 2. Furthermore, the results
of the LST analyses clearly demonstrate that the scales reliably measure stable individual
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
28
differences in SMEC that depend to a small extent on systematic situational or interactional
effects present at the measurement occasion.
Concerning construct validity, most of the examined partial correlations between related
constructs and SMEC, controlling for gender and age, were as expected.
More specifically, SV was negatively correlated with preferences for genres that deal
with rather unrealistic content (e.g., science fiction, animation, and action) and with film
expertise. Furthermore, as hypothesized, it was positively associated with Appreciation.
Unexpectedly, it was not statistically significantly related to preference for documentaries. One
reason for this might be that the story in a film is considered being mainly fictional and narrative.
Although it seems to be important that such stories are based on historical facts, are realistic, and
address contemporary issues, this does not mean that these stories need to be real.
SI was expectedly associated with film expertise, Openness to Experiences, Extraversion,
Sensation Seeking; and preferences for avant-garde and science fiction. The additional positive
relationship with Fun and preferences for drama/tragedy might hint to the importance of
innovative stories for entertainment experiences as well as for serious genres.
As expected, CI was closely positively connected to film expertise. Furthermore, as
hypothesized, positive relationships with preferences for genres that typically focus on visual and
aesthetic styles (e.g., for avant-garde, science fiction, and animation) as well as with the Need for
Cognition, Openness to Experiences, Extraversion and Sensation Seeking clearly draw a
distinction between CI and FX as expected. CI did not significantly correlate with Fun or
Appreciation. This might be due to the fact that aesthetic aspects build a distinct factor of
audience gratifications in the Oliver and Bartsch (2010) studies that could be included in future
studies.
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
29
FX was, as assumed, positively related to preference for action, comedy, and animation
and negatively associated with preference for avant-garde and dramas. Remarkably, FX was not
significantly correlated with Sensation Seeking and Extraversion, which might hint to the fact
that not all special effects need to be highly arousing. Instead, FX was positively associated with
Agreeableness. This is in contrast to previous findings that showed a positive correlation
between preferences for and enjoyment of action-oriented content and Psychoticism, Sensation
Seeking, and Extraversion. However, more recent findings also suggested a positive relationship
between Agreeableness and a preference for mainstream genres (Kallias, 2012). Furthermore, as
expected FX was negatively correlated with Need for Cognition and film expertise, emphasizing
the distinction between FX and CI.
As expected, RE was positively associated with preference for drama/tragedy and
negatively related to Need for Cognition and Emotional Stability. Unexpectedly, it was
positively but not significantly correlated with film expertise. Maybe due to the fact that film
experts are already knowledgeable and competent they do not need to rely on the
recommendations of others. However, they must surely have the external sources to keep their
expertise up-to-date. Perhaps items concerning these resources are not included in the RE scale.
Thus, adding more items to investigate the uniqueness of the facets of the scale might be also an
effective way to cope with the substantial method specificity (see Figure 1).
Participants who tend to rate IN as important had less film expertise. As expected, IN is
positively associated with harmless genres (e.g., romance) and Agreeableness, and negatively
with genres that are assumed to include frightening and violent content (e.g., horror) and
Emotional Stability and Sensation Seeking. Contrary to expectations, IN correlated only low
with preference for comedy but positively with Appreciation. Despite drawing a distinction
between IN and LH, this result is difficult to explain. Perhaps this is so because comedies can
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
30
also include scenes in which disgusting things happen, schadenfreude is evoked, or aggressive
behavior is celebrated (e.g., Dumb and Dumber), thereby leading to rather mixed emotions.
With regard to LH, nearly all correlations are in line with previous assumptions and
reflect the semantic closeness to mood management factors or U&G dimensions like
entertainment or escapism as well as to the Fun scale, and Agreeableness. At the same time, the
magnitudes are rather low or moderate. Thus, the results point in the right direction but clearly
draw a distinction between the constructs. In addition, LH is negatively connected to constructs
or genres that reflect some kind of aesthetical interest or cognitive effort (e.g., film expertise,
preference for avant-garde, Need for Cognition, Openness to Experiences).
As expected, CS was negatively related to preferences for action and comedy (although
the latter correlation was not statistically significant) and positively related to serious genres
(e.g., drama, avant-garde, documentary). Furthermore, CS was as assumed positively related to
Appreciation as well as to Need for Cognition and broader aspects of stimulation (e.g., Openness
to Experience, Extraversion, and Sensation Seeking). The scope of CS and the results indicate a
close connection to the meaningful and thought-provoking aspect of eudaimonic entertainment.
Taken together all partial correlations in the nomological network showed only small to
moderate magnitudes. Hence, the constructs share rather little proportions of variance with
SMEC. These low partial correlations indicate discriminant validity and also help to draw
distinctions between the SMEC themselves (e.g., SV and CS, CI and FX, IN and LH). However,
it is quite evident that measuring the evaluative importance of, for instance, aesthetic movies has
a very different level of specificity than such broad personality traits like Openness to
Experiences. Therefore, in future studies, a higher level of correspondence, for example,
between CI and the openness-facet aesthetics could be achieved by measuring personality traits
with the many-faceted scales of the NEO-PI-R (Costa & McCrae, 1992).
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
31
A final note concerns the theoretical relationship between SMEC and film-specific
evaluations. If SMEC are conceptualized as stable and important attitudes toward specific film
features, individual differences in SMEC might predict film-specific evaluation like enjoyment
or appreciation. Similar ideas have been proposed by GSGO models (e.g., Palmgreen &
Rayburn, 1985). However, as argued in the TSQA (Wolling, 2009), such an influence might be
only exerted when viewers also perceived the features they consider as important. As
participants in the present study only rated their experiences with the last movie they have seen,
it is unknown whether they perceived the corresponding features. Although the SMEC were
developed to be generally applicable across a variety of movies, there might be movies, for
example, without any special effects. Thus, future research should not only include featurespecific evaluations but also ask if these specific film-features have been perceived.
General Discussion
The major goal of the present research was to delineate what is important when we watch
and evaluate a movie. Taken together, the studies provide evidence that subjective movie
evaluation criteria are much more complex than the introductory quote from Harry Cohn
suggested. Individuals differ in the criteria they use when they evaluate movies. Eight distinct
dimensions reflect film-inherent features (Story Verisimilitude, Story Innovation,
Cinematography, Special Effects), film-external features (Recommendation), or (anticipated)
effects of use (Innocuousness, Light-heartedness, Cognitive Stimulation). These findings are in
line with previous studies on film critiques and TV quality but extend them by exploring SMEC
in the field of movies and applying current standards in scale construction and validation.
Furthermore, LST analyses supported the assumption that individual differences in SMEC are
rather trait-like.
Future Research Directions
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
32
Besides the preliminary support for the conceptualization of SMEC as important attitudes
toward specific film features, it remains an open question how these SMEC emerge and develop
during the life-span. Research provides some evidence that media preferences crystallize during
different developmental levels (Valkenburg & Cantor, 2000). During socialization, we do not
only develop likes and dislikes for film genres. Rather, it seems plausible that we even develop
fairly stable likes and dislikes for specific film features. Moreover, in the course of time we gain
knowledge about movies, film production, and cinematic experiences. Furthermore, film
features are not independent of each other; specific features frequently appear together with
certain other features because the movies are produced in correspondence with a specific film
genre category (e.g., a romantic comedy combines a happy ending, funny scenes, maybe a less
innovative story, no special effects, etc.). Consequently, the development of SMEC is
accompanied by the development of mental representations of genre categories as well as genre
preferences and distinguishing between SMEC and genre preferences is a creative, longitudinal
endeavor. Thus, applying theories that deal with cognitive as well as socio-emotional
development might help to disentangle both constructs and clarify their relationships
(Valkenburg & Cantor, 2000). In addition, answering these questions about how SMEC develop
and what the antecedents of SMEC are, would contribute considerably to the substantive validity
of the construct.
As outlined in the introductory paragraphs, SMEC are only one part of the puzzle
concerning movie evaluations but they could help to examine evaluative processes before,
during, and after movie exposure more deeply. Thus, we must carefully differentiate between
SMEC and perceived features that might result from the interaction of a viewer with a specific
movie, leading not only to evaluations of multiple experiences (cf. Oliver & Bartsch, 2010) but
also to evaluations of film-inherent and film-external features. Whereas the SMEC can be
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
33
conceptualized as mental representations of important attitudes towards specific film features, the
evaluative responses can be seen as resulting from those activated representations. The scales
developed here provide an appropriate measurement tool only for SMEC. Thus, future steps
should include the development of appropriate tools for measuring perceptions of and evaluative
responses to specific film features. This approach is similar to the TSQA (Wolling, 2009) which
proposes that the interplay of desired and perceived media product features results in the
evaluation of these features. In search for a catalogue of criteria that could be tested within the
TSQA, the SMEC provide a first step and a sound instrument. Hence, SMEC can be used to
address research questions such as: How do different SMEC contribute to the selection or global
judgment of a movie, dependent or independent from perceived features?
Furthermore, SMEC might work as a fine-grained alternative to preferences and sought
gratifications because they do not only refer to anticipated effects of use (i.e., IN, LH, and CS)
but also to film-external (RE) and film-inherent features (SV, SI, CI, FX), thereby taking aspects
of content into account (Swanson, 1987; Wolling, 2009). Future studies should investigate
whether SMEC might be better predictors of movie evaluations compared to genre preferences
within and across specific genres.
In addition, as SMEC are conceptualized as mental representations of important attitudes,
the concept can easily be connected to and explored from the perspective of recently proposed
theories in the field of social cognition (e.g., Gawronski, 2007; see also Schneider, 2012).
With these contributions and research questions in mind, it is also important to mention
limitations and propose solutions. First, the convenience samples were predominantly conducted
online and largely consisted of younger and higher-educated participants, thereby leading to a
sample bias and presumably to systematic bias. Although moviegoer statistics show that the
highest frequency of movie attendances in Germany is among adolescents and students (FFA
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
34
Filmförderungsanstalt, 2011), this certainly calls for further studies, which might be conducted
under other circumstances (e.g., in field studies at cinema theaters) and include samples with
different socio-demographic or ethnic backgrounds.
Secondly and as it is true with every scale, the conceptualization and measurement of
SMEC need to prevail over time. More studies should be conducted to broaden the nomological
network by taking recent theoretical advances into account. For example, eudaimonic
entertainment motivations and experiences (e.g., Oliver & Raney, 2011; Wirth et al., 2012) might
help to disentangle the antecedents and consequences of specific SMEC (e.g., CS and LH).
Moreover, they might enrich the conceptualization of SMEC. Due to the fact that the modelling
of the construct space and the SMEC dimensions were data-driven and based on the subjective
ideas of lay audiences, it is important to prove that more parsimonious models with fewer
dimensions do not compete with the present structure. This could be accomplished by testing the
predictive and incremental power of the SMEC and complementary constructs, for example, with
regard to specific evaluations of movies belonging to different genres.
Finally, as attitude importance is only one strength-related attribute among many others
(e.g., attitude accessibility) and is based on self-reports, it might be interesting to examine
whether multi-methodological approaches can enrich the measurement (e.g., by developing and
applying indirect measures; cf. Hefner, Rothmund, Klimmt, & Gollwitzer, 2011).
The SMEC scales are developed to address the question what is important when we
evaluate a movie. Hopefully, a number of researchers will elaborate on the SMEC construct and
scales, improve them, and find them useful to delineate the role of these criteria in the underlying
processes and effects of evaluating movies.
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
35
References
Augustin, M. D., & Leder, H. (2006). Art expertise: a study of concepts and conceptual spaces.
Psychology Science, 48, 135–156.
Austin, B. A. (1986). Motivations for movie attendance. Communication Quarterly, 34, 115–
126. https://doi.org/10.1080/01463378609369627
Bartsch, A., & Schneider, F. M. (2014). Entertainment and politics revisited: How non-escapist
forms of entertainment can stimulate political interest and information seeking. Journal of
Communication, 64, 369–396. https://doi.org/10.1111/jcom.12095
Bartsch, A., Vorderer, P., Mangold, R., & Viehoff, R. (2008). Appraisal of emotions in media
use: Toward a process model of meta-emotion and emotion regulation. Media
Psychology, 11, 7–27. https://doi.org/10.1080/15213260701813447
Bentler, P. M., & Wu, E. J. C. (2005). EQS 6 structural equations program manual. Encino, CA:
Multivariate Software. Retrieved from www.mvsoft.com
Bless, H., Wänke, M., Bohner, G., Fellhauer, R. F., & Schwarz, N. (1994). Need for Cognition:
Eine Skala zur Erfassung von Engagement und Freude bei Denkaufgaben [Need for
cognition: A scale measuring engagement and happiness in cognitive tasks]. Zeitschrift
für Sozialpsychologie, 25, 147–154.
Boninger, D. S., Krosnick, J. A., Berent, M. K., & Fabrigar, L. R. (1995). The causes and
consequences of attitude importance. In R. E. Petty & J. A. Krosnick (Eds.), Attitude
strength: Antecedents and consequences (pp. 159–189). Hillsdale, NJ: Erlbaum.
Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY:
Guilford Press.
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
36
Browne, M. W. (2001). An overview of analytic rotation in exploratory factor analysis.
Multivariate Behavioral Research, 36, 111–150.
https://doi.org/10.1207/S15327906MBR3601_05
Browne, M. W., Cudeck, R., Tateneni, K., & Mels, G. (2010). CEFA: Comprehensive
Exploratory Factor Analysis (Version 3.04). Retrieved from http://faculty.psy.ohiostate.edu/browne/
Burisch, M. (1984). Approaches to personality inventory construction: A comparison of merits.
American Psychologist, 39, 214–227. https://doi.org/10.1037/0003-066X.39.3.214
Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social
Psychology, 42, 116–131. https://doi.org/10.1037/0022-3514.42.1.116
Carroll, N. E. (2003). Engaging the moving image. New Haven, CT: Yale University Press.
Cattell, R. B. (1988). Psychological theory and scientific method. In J. R. Nesselroade & R. B.
Cattell (Eds.), Handbook of multivariate experimental psychology (2nd ed.) (pp. 3–20).
New York, NY: Plenum Press.
Chamorro-Premuzic, T., Kallias, A., & Hsu, A. (2014). What type of movie person are you?
Understanding individual differences in film preferences and uses: A psychographic
approach. In J. C. Kaufman & D. K. Simonton (Eds.), The social science of cinema
(pp. 87–122). New York, NY: Oxford University Press.
Costa, P. T., & McCrae, R. R. (1992). The Revised NEO Personality Inventory (NEO-PI-R) and
NEO Five-Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological
Assessment Ressources.
d'Astous, A., & Touil, N. (1999). Consumer evaluations of movies on the basis of critics'
judgment. Psychology & Marketing, 16, 677–694. https://doi.org/10.1002/(SICI)15206793(199912)16:8<677::AID-MAR4>3.0.CO;2-T
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
37
Eaton, A. A., & Visser, P. S. (2008). Attitude importance: Understanding the causes and
consequences of passionately held views. Social and Personality Psychology Compass, 2,
1719–1736. https://doi.org/10.1111/j.1751-9004.2008.00125.x
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use
of exploratory factor analysis in psychological research. Psychological Methods, 4, 272–
299. https://doi.org/10.1037/1082-989X.4.3.272
FFA Filmförderungsanstalt. (2011). Der Kinobesucher 2010: Strukturen und Entwicklungen
[German Federal Film Board: The Moviegoers 2010]. Berlin. Retrieved from
http://www.ffa.de/download.php?f=b35f7381f893e02729cfac5c62bfa735&target=0
Finn, S. (1997). Origins of media exposure: Linking personality traits to TV, radio, print, and
film use. Communication Research, 24, 507–529.
https://doi.org/10.1177/009365097024005003
Gawronski, B. (Ed.). (2007). What is an attitude? [Special Issue]. Social Cognition, 25(5).
Gehrau, V. (2008). Fernsehbewertung und Fernsehhandlung: Ansätze und Daten zu Erhebung,
Modellierung und Folgen von Qualitätsurteilen des Publikums über Fernsehangebote
[Television evaluation and television action: Approaches and data relating to the
recording, modeling and consequences of quality assessments of television programs by
viewers]. München: Reinhard Fischer.
Goldberg, L. R. (1990). An alternative 'description of personality': The Big-Five factor structure.
Journal of Personality and Social Psychology, 59, 1216–1229.
https://doi.org/10.1037/0022-3514.59.6.1216
Greenberg, B. S., & Busselle, R. W. (1996). Audience dimensions of quality in situation
comedies and action programmes. In S. Ishikawa (Ed.), Quality assessment of television
(pp. 169–196). Luton, UK: University of Luton Press; John Libbey Media.
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
38
Gunter, B. (1997). An audience-based approach to assessing programme quality. In P.
Winterhoff-Spurk & T. H. A. van der Voort (Eds.), New horizons in media psychology:
Research cooperation and projects in Europe (pp. 11–34). Wiesbaden: Westdeutscher
Verlag.
Hadida, A. L. (2009). Motion picture performance: A review and research agenda. International
Journal of Management Reviews, 11, 297–335. https://doi.org/10.1111/j.14682370.2008.00240.x
Hefner, D., Rothmund, T., Klimmt, C., & Gollwitzer, M. (2011). Implicit measures and media
effects research: Challenges and opportunities. Communication Methods and Measures,
5, 181–202. https://doi.org/10.1080/19312458.2011.597006
Himmelweit, H. T., Swift, B., & Jaeger, M. E. (1980). The audience as a critic: A conceptual
analysis of television entertainment. In P. H. Tannenbaum (Ed.), The entertainment
functions of television (pp. 67–106). Hillsdale, NJ: Erlbaum.
Holbrook, M. B. (1999). Popular appeal versus expert judgments of motion pictures. Journal of
Consumer Research, 26, 144–155. https://doi.org/10.1086/209556
Hoyle, R. H., Stephenson, M. T., Palmgreen, P., Lorch, E. P., & Donohew, R. L. (2002).
Reliability and validity of a brief measure of sensation seeking. Personality and
Individual Differences, 32, 401–414. https://doi.org/10.1016/S0191-8869(01)00032-0
Igartua, J.-J., & Barrios, I. (2013). Hedonic and eudaimonic motives for watching feature films.
Validation of the Spanish version of Oliver & Raney’s scale. Communications, 38, 411–
431. https://doi.org/10.1515/commun-2013-0024
Jacobs, R. S., Heuvelman, A., ben Allouch, S., & Peters, O. (2015). Everyone's a critic: The
power of expert and consumer reviews to shape readers’ post-viewing motion picture
evaluations. Poetics, 52, 91–103. https://doi.org/10.1016/j.poetic.2015.07.002
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
39
Kallias, A. (2012). Individual differences and the psychology of film preferences (Doctoral
thesis). University of London. London. Retrieved from http://research.gold.ac.uk/7813/
Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New
York, NY: Guilford Press.
Knobloch-Westerwick, S. (2015). Choice and preference in media use: Advances in selective
exposure theory and research. New York, NY: Taylor & Francis.
Knobloch-Westerwick, S., & Keplinger, C. (2008). Murder for pleasure: Impacts of plot
complexity and need for cognition on mystery enjoyment. Journal of Media Psychology,
20, 117–128. https://doi.org/10.1027/1864-1105.20.3.117
Krcmar, M. (2009). Individual differences in media effects. In R. L. Nabi & M. B. Oliver (Eds.),
The Sage handbook of media processes and effects (pp. 235–250). Thousand Oaks, CA:
Sage.
Krcmar, M., & Kean, L. G. (2005). Uses and gratifications of media violence: Personality
correlates of viewing and liking violent genres. Media Psychology, 7, 399–420.
https://doi.org/10.1207/S1532785XMEP0704_5
Lewis, R. J., Tamborini, R., & Weber, R. (2014). Testing a dual-process model of media
enjoyment and appreciation. Journal of Communication, 64, 397–416.
https://doi.org/10.1111/jcom.12101
Linton, J. M., & Petrovich, J. A. (1988). The application of the consumer information acquisition
approach to movie selection: An exploratory study. In B. A. Austin (Ed.), Current
research in film. Audiences, economics and law (Vol. 4, pp. 24–45). Norwood, NJ:
Ablex.
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
40
Möller, K. E. K., & Karppinen, P. (1983). Role of motives and attributes in consumer motion
picture choice. Journal of Economic Psychology, 4, 239–262.
https://doi.org/10.1016/0167-4870(83)90029-6
Muck, P. M., Hell, B., & Gosling, S. D. (2007). Construct validation of a short five-factor model
instrument: A self-peer study on the German adaptation of the Ten-Item Personality
Inventory (TIPI-G). European Journal of Psychological Assessment, 23, 166–175.
https://doi.org/10.1027/1015-5759.23.3.166
Neelamegham, R., & Jain, D. (1999). Consumer choice process for experience goods: An
econometric model and analysis. Journal of Marketing Research, 36, 373–386.
https://doi.org/10.2307/3152083
Nikken, P., & van der Voort, T. H. A. (1997). Children's views on quality standards for children's
television programs. Journal of Educational Media, 23, 169–188.
https://doi.org/10.1080/1358165970230206
Nikken, P., van der Voort, T. H. A., & van Bochove, E. (1996). Maternal quality standards for
children's television programs. Journal of Educational Media, 22, 41–54.
https://doi.org/10.1080/1358165960220104
Oliver, M. B., Ash, E., Woolley, J. K., Shade, D. D., & Kim, K. (2014). Entertainment we watch
and entertainment we appreciate: Patterns of motion picture consumption and acclaim
over three decades. Mass Communication and Society, 17, 853–873.
https://doi.org/10.1080/15205436.2013.872277
Oliver, M. B., & Bartsch, A. (2010). Appreciation as audience response: Exploring entertainment
gratifications beyond hedonism. Human Communication Research, 36, 53–81.
https://doi.org/10.1111/j.1468-2958.2009.01368.x
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
41
Oliver, M. B., & Raney, A. A. (2011). Entertainment as pleasurable and meaningful: Identifying
hedonic and eudaimonic motivations for entertainment consumption. Journal of
Communication, 61, 984–1004. https://doi.org/10.1111/j.1460-2466.2011.01585.x
Palmgreen, P., Cook, P. L., Harvill, J. G., & Helm, D. M. (1988). The motivational framework
for moviegoing: Uses and avoidances of theatrical films. In B. A. Austin (Ed.), Current
research in film. Audiences, economics and law (Vol. 4, pp. 1–23). Norwood, NJ: Ablex.
Palmgreen, P., & Rayburn, J. D., II. (1985). An expectancy-value approach to media
gratifications. In K. E. Rosengren, L. A. Wenner, & P. Palmgreen (Eds.), Media
gratifications research. Current perspectives (pp. 61–72). Beverly Hills, CA: Sage.
Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied
Psychological Measurement, 21, 173–184. https://doi.org/10.1177/01466216970212006
Rentfrow, P. J., Goldberg, L. R., & Zilca, R. (2011). Listening, watching, and reading: The
structure and correlates of entertainment preferences. Journal of Personality, 79, 223–
258. https://doi.org/10.1111/j.1467-6494.2010.00662.x
Rosenbaum, J. E., & Johnson, B. K. (2016). Who’s afraid of spoilers? Need for cognition, need
for affect, and narrative selection and enjoyment. Psychology of Popular Media Culture,
5, 273–289. https://doi.org/10.1037/ppm0000076
Rössler, P. (1997). Filmkritiker und Publikum: Diskrepanzen und Übereinstimmungen.
Ergebnisse einer Befragung von Filmrezensenten und Kinogängern [Film critics and the
public: discrepancies and agreements: Results of a survey of film critics and moviegoers].
Media Perspektiven, 28, 133–140.
Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in
covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variables
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
42
analysis: Applications for developmental research (pp. 399–419). Thousand Oaks, CA:
Sage.
Scherer, K. R. (2013). The nature and dynamics of relevance and valence appraisals: Theoretical
advances and recent evidence. Emotion Review, 5, 150–162.
https://doi.org/10.1177/1754073912468166
Schneider, F. M. (2012). Measuring subjective movie evaluation criteria. Conceptual foundation,
construction, and validation of the SMEC scales (Dissertation). Universität KoblenzLandau. Landau. Retrieved from http://nbn-resolving.de/urn:nbn:de:hbz:lan1-7813
Schneider, F. M., Otto, L., Alings, D., & Schmitt, M. (2014). Measuring traits and states in
public opinion research: A latent state–trait analysis of political efficacy. International
Journal of Public Opinion Research, 26, 202–223. https://doi.org/10.1093/ijpor/edu002
Schneider, F. M., Vogel, I. C., Gleich, U., & Bartsch, A. (2014, May). Evaluating movies: A
review of movie evaluations and a framework for investigating underlying psychological
processes. Paper presentation at the 64th Annual Conference of the International
Communication Association (ICA). Seattle, WA.
Schramm, H., & Wirth, W. (2010). Exploring the paradox of sad-film enjoyment: The role of
multiple appraisals and meta-appraisals. Poetics, 38, 319–335.
https://doi.org/10.1016/j.poetic.2010.03.002
Silvia, P. J., & Berg, C. (2011). Finding movies interesting: How appraisals and expertise
influence the aesthetic experience of film. Empirical Studies of the Arts, 29, 73–88.
https://doi.org/10.2190/EM.29.1.e
Simonton, D. K. (2009). Cinematic success criteria and their predictors: The art and business of
the film industry. Psychology & Marketing, 26, 400–420.
https://doi.org/10.1002/mar.20280
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
43
Steyer, R., Geiser, C., & Fiege, C. (2012). Latent state-trait models. In H. Cooper, P. M. Camic,
D. L. Long, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research
methods in psychology, Vol 3: Data analysis and research publication (pp. 291–308).
Washington, DC US: American Psychological Association.
Suckfüll, M., & Scharkow, M. (2009). Modes of reception for fictional films. Communications,
34, 361–384. https://doi.org/10.1515/COMM.2009.023
Swami, V., Stieger, S., Pietschnig, J., & Voracek, M. (2010). The disinterested play of thought:
Individual differences and preference for surrealist motion pictures. Personality and
Individual Differences, 48, 855–859. https://doi.org/10.1016/j.paid.2010.02.013
Swanson, D. L. (1987). Gratification seeking, media exposure, and audience interpretations:
Some directions for research. Journal of Broadcasting & Electronic Media, 31, 237–254.
https://doi.org/10.1080/08838158709386662
Tamborini, R., Grizzard, M., Bowman, N. D., Reinecke, L., Lewis, R. J., & Eden, A. (2011).
Media enjoyment as need satisfaction: The contribution of hedonic and nonhedonic
needs. Journal of Communication, 61, 1025–1042. https://doi.org/10.1111/j.14602466.2011.01593.x
Tellegen, A., & Waller, N. G. (2008). Exploring personality through test construction:
Development of the Multidimensional Personality Questionnaire. In G. J. Boyle, G.
Matthews, & D. H. Saklofske (Eds.), The Sage handbook of personality theory and
assessment. Personality measurement and testing (Vol. 2, pp. 261–292). Los Angeles,
CA: Sage.
Tesser, A., Millar, K. U., & Wu, C.-H. (1988). On the perceived functions of movies. The
Journal of Psychology: Interdisciplinary and Applied, 122, 441–449.
https://doi.org/10.1080/00223980.1988.10542949
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
44
Valkenburg, P. M., & Cantor, J. (2000). Children's likes and dislikes of entertainment programs.
In D. Zillmann & P. Vorderer (Eds.), Media entertainment: The psychology of its appeal
(pp. 135–152). Mahwah, NJ: Erlbaum.
Valkenburg, P. M., & Janssen, S. C. (1999). What do children value in entertainment programs?
A cross-cultural investigation. Journal of Communication, 49, 3–21.
https://doi.org/10.1111/j.1460-2466.1999.tb02790.x
Valkenburg, P. M., & Peter, J. (2013). The differential susceptibility to media effects model.
Journal of Communication, 63, 221–243. https://doi.org/10.1111/jcom.12024
Vorderer, P., Klimmt, C., & Ritterfeld, U. (2004). Enjoyment: At the heart of media
entertainment. Communication Theory, 14, 388–408. https://doi.org/10.1111/j.14682885.2004.tb00321.x
Vorderer, P., Knobloch, S., & Schramm, H. (2001). Does entertainment suffer from interactivity?
The impact of watching an interactive TV movie on viewers' experience of entertainment.
Media Psychology, 3, 343–363. https://doi.org/10.1207/S1532785XMEP0304_03
Weaver, J. B. (1991). Exploring the links between personality and media preferences.
Personality and Individual Differences, 12, 1293–1299. https://doi.org/10.1016/01918869(91)90203-N
Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ:
Erlbaum.
Wirth, W., Hofer, M., & Schramm, H. (2012). Beyond pleasure: Exploring the eudaimonic
entertainment experience. Human Communication Research, 38, 406–428.
https://doi.org/10.1111/j.1468-2958.2012.01434.x
Wolling, J. (2004). Qualitätserwartungen, Qualitätswahrnehmungen und die Nutzung von
Fernsehserien: Ein Beitrag zur Theorie und Empirie der subjektiven Qualitätsauswahl von
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
45
Medienangeboten [Quality expectations, quality perceptions and viewing of television
serials: A contribution to the theoretical and empirical study of the subjective selection of
media content]. Publizistik, 49, 171–193. https://doi.org/10.1007/s11616-004-0035-y
Wolling, J. (2009). The effect of subjective quality assessments on media selection. In T.
Hartmann (Ed.), Media choice. A theoretical and empirical overview (pp. 85–101). New
York, NY: Routledge.
Wyatt, R. O., & Badger, D. P. (1988). What newspaper film critics value in film and film
criticism: A national survey. In B. A. Austin (Ed.), Current research in film. Audiences,
economics and law (Vol. 4, pp. 54–71). Norwood, NJ: Ablex.
Wyatt, R. O., & Badger, D. P. (1990). Effects of information and evaluation in film criticism.
Journalism Quarterly, 67, 359–368. https://doi.org/10.1177/107769909006700213
Young, S. D. (2012). Psychology at the movies. Oxford, UK: Wiley-Blackwell.
Zillmann, D. (1988). Mood management through communication choices. American Behavioral
Scientist, 31, 327–340. https://doi.org/10.1177/000276488031003005
Zuckerman, M. (1979). Sensation seeking: Beyond the optimal level of arousal. Hillsdale, NJ:
Erlbaum.
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
46
Footnotes
1
See Neelamegham and Jain (1999), Linton and Petrovich (1988), or Wolling (2009), for
instance, for similar categorizations.
2
Burisch (1984) distinguishes three primary approaches to the development of self-report
inventories (external, inductive, and deductive). In an inductive approach, no preconceived set
of dimensions are derived from theory. Instead, empirical data analysis should reveal important
dimensions and their interrelationships. Furthermore, whereas deductive scale constructionists
aim at accommodating the initial constructs, inductive scale developers examine them (Tellegen
& Waller, 2008, p. 261). Thus, the inductive approach can be characterized as theory-building
rather than theory-testing. In personality psychology, there is a long tradition of inductive scale
development, for example, several Big Five inventories based on the lexical approach or the
Multidimensional Personality Questionnaire. For more details, the interested reader is referred to
Tellegen and Waller (2008, p. 261–263), who discuss some advantages of an inductive approach,
especially as a first step in the “inductive–hypothetico–deductive method” (Cattell, 1988).
3
The Kaiser-Meyer-Olkin coefficient was good (.85), measure of sample adequacy
coefficients were all above .70, and the Bartlett test of sphericity was significant, thus indicating
applicability of EFA. Mardia’s coefficient for multivariate skewness was not statistically
significant, thus indicating no violation of multivariate normal distribution. The coefficient for
multivariate kurtosis was statistically significant. However, the pattern matrix of the ML
extraction correlated over .98 with the pattern matrix of a principal axis factor analysis, which
requires no distributive assumptions, thus indicating no bias.
4
Throughout the text the composite reliability estimator Raykov’s ρ is reported because
Cronbach’s α underestimates reliability in case of congeneric measures models (Raykov, 1997).
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
47
5
More details on the psychometric properties of the scale are available from the author.
6
Freely estimating error covariance parameters or excluding items needs theoretical
justification. On the one hand, the reasons for correlated error variances seem to lie in redundant
item content. For instance, Item cs1* (“that the movie is thought-provoking”) and Item cs2
(“that the movie is intellectually challenging”) are obviously synonymous. In a similar vein,
Items ci3 and ci4* as well as lh4* and lh5 seemed to be semantically related (for item wording
see Table 2). Based on their psychometric properties, the items with asterisk* were preferred,
while the competing ones were dropped. On the other hand, two items (fx4 and re3) were
excluded because their loading patterns were ambiguous. For instance, re3 had only low
loadings on RE (λ = .29), but also loaded similarly on LH (λ = .19). One reason might be that
friends provide information concerning the movie’s entertainment value, whereas the remaining
items of the RE dimension cover aspects of the movie itself. The Item fx4 seems to be a
summary item for the FX dimension; it also statistically significantly cross-loaded on CI
(λ = .36) and RE (λ = .12) and showed the lowest loading on FX (λ = .57) compared to the other
FX-items. Therefore, it was excluded, too.
7
As in classical test theory, the reliability (Rel) represents the ratio of the true score
variance to the observed variance. However, in LST theory, the true score variance can be
further decomposed into a latent trait component representing stable individual differences (its
proportion is reflected in the common consistency coefficient cCon), a latent state residual
representing systematic situational influences (occasion specificity coefficient Spe), and—in case
of method effects—a method factor component representing systematic variance of the
measurement instrument or, as here, the test halves (method specificity coefficient mSpe). For
further details, see Schneider et al. (2014) or Steyer and Schmitt (1990).
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
48
Table 1
Overview of selected studies dealing with movie evaluation criteria or related constructs
Source
Object
Criteria dimensions (Examples for operationalization)
Gehrau
Evaluative
Adjectives:
(2008)
responses to TV  Emotional Entertainment (e.g., entertaining, pleasant, good)
shows
 Cognitive Information (e.g., realistic, serious, credible)
 inconsistent findings for further dimensions across the different stimuli
Content and production quality:
 Technical production (e.g., photography, editing, sound)
 Content (e.g., story, relevance, characters)
Greenberg Evaluative

& Busselle responses to TV 
(1996)
shows



Gunter
(1997)
Attributes of
soap operas








Himmelweit a) Evaluative

et al. (1980) responses to TV
shows

Appreciation (e.g., enjoyable, entertaining) [for comedies & action–adventures]
Real vs. funny (e.g., realistic, serious–light) [only for comedies one combined factor]
Fairness (e.g., fair, gentle) [only for comedies]
Modern (e.g., new, modern) [only for comedies]
Originality (e.g., unusual, original) [for comedies & action–adventures]
Study details
Survey after viewing
excerpts from TV shows and
films (N = 563; German TV
research panel); principal
component analyses with 20
adjectives regarding the
general quality and 14 terms
regarding the quality of the
content and production
Survey after viewing
situation comedy/action
program (N = 1,345 US
students); factor analysis
with 44 adjective pairs
regarding general attributes
of the program
Real-time responses and
Verisimilitude (e.g., “The characters were true to life.”)
focus group discussions for
Established Characters (e.g., “All the characters clearly belonged to the local area.”)
item development and survey
Tension/drama (e.g., “There were a number of problems going on that surprised and
(N = 3,000; British TV
intrigued me.”)
research panel); no factor
Entertainment/Involvement (e.g., “I couldn’t wait to find out what happened next.”)
Coherence/Cohesion (e.g., “Each of the storylines in this episode had a clear beginning, analyses, only internal
consistencies of the eight
middle, and end.”)
Technical Professionalism (e.g., “The camera work balanced long and close-up shots.”) dimensions comprising 26
items developed in the focus
Contrast and Balance (e.g., “It included both light and serious storylines.”)
group study were assessed in
Plot/Setting (e.g., “The setting was recognizable and reasonably familiar.”)
the survey
Impact Reactions (informative, moving, exciting, disturbing, trivial, annoying,
Survey (N > 1,000; BBC
convincing, and absorbing)
viewing panel); ratings on 17
Stylistic Attributes (e.g., realistic, lighthearted, funny, informative, complicated, brutal, attributes
and violent)
(table continues)
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
49
Table 1 (continued)
Source
Object
Criteria dimensions (Examples for operationalization)
Study details
b) Stylistic
 Approach to/Avoidance of Potentially Upsetting Stimuli (e.g., “A sad ending just
preferences with
makes me miserable.”)
regard to TV
 Aloofness or Involvement in Programs (e.g., “I prefer programs that appeal to my
series
heart.”)
 Preferences for Low/High Cognitive Effort (e.g., “I prefer programs that you have got Factor analysis with 20 items
to make a real effort to understand.”)
 Preference for Real vs. Fantasy Content/Presentation (e.g., “I prefer programs set in
everyday surroundings.”)
 Liking for Thriller/Action Content (e.g., “What makes a plot interesting is the action.”)
Linton &
Petrovich
(1988)
Important
 Foreground (e.g., storyline, characters, acting, scenery, music, etc.)
attributes for
 Background (e.g., editing, photography, casting, director, etc.)
movie evaluation
Survey (N = 192; students);
factor analysis with 15 items
developed in a pilot study
Nikken & Children’s
Van der
quality standards
Voort
in TV programs
(1997)









Credibility (e.g., “A children’s program should give a balanced image of reality.”)
Comprehensibility (e.g., “. . . should be understandable for all children.”)
Entertainment (e.g., “. . . should be funny.”)
Aesthetic Quality (e.g., “. . . should contain beautiful images.”)
Involvement (e.g., “. . . should capture a child’s attention.”)
Presence of Role Models (e.g., “ . . . should feature persons a child wants to be like.”)
Innocuousness (e.g., “. . . should not make a child sad.”)
Restfulness (e.g., “. . . should set a child at ease.”)
Thought Provocation (e.g., “. . . should make a child curious.”)
Survey (N = 427 Dutch
children); principal
components analysis of 52
items mainly derived from
maternal quality criteria of
children’s TV programs
(Nikken et al., 1996)
Rössler
(1997)
a) Important
evaluation
criteria in film
critiques




Technical Qualities (e.g., sound, camera, music, and editing)
Plot (e.g., director, actors, dramaturgy, and plausibility)
Educational Engagement (e.g., political ambition, social relevance)
Entertainment (e.g., entertainment value, stars, and special effects)
Survey on German
newspaper film critics (N =
150); factor analysis with 15
items
b) Important
evaluation
criteria from
viewers’
perspective
 Entertainment (suspense and fun), educational engagement (political and social
relevance), aesthetic value
Survey on German
moviegoers (N = 225); five
items selected from a)
(table continues)
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
50
Table 1 (continued)
Source
Object
Valkenburg Children’s
& Janssen criteria for a
(1999)
good TV
program
Criteria dimensions (Examples for operationalization)
 Interestingness (e.g., “show somebody I would really like to be”)
 Romance (e.g., “be about love”)
 Realism (e.g., “show things that are real”)
 Violence (e.g., “be about brave and strong heroes”)
 Humor (e.g., “be full of jokes”)
 Innocuousness (e.g., “not show any violence that children can imitate”)
 Comprehensibility (e.g., “be easy to understand”)
 Action (e.g., “contain a lot of action”)
Wolling
(2004)
Quality
dimensions of
TV shows




Wyatt &
Badger
(1988)
a) Film critics’  Sex/Violence (e.g., vulgarity, sex, violence, nudity, MPAA rating)
important movie  Production Elements (e.g., editing, design, cinematography)
characteristics
 Performance Elements (e.g., lead actor, director, plot, screenplay)
 Production Difficulty (e.g., production difficulty, cost, genre)
Survey (N = 166; US
newspaper film critics);
factor analysis with 22 items
b) Content
 Personal Impression/Judgment (e.g., subjective responses, evaluative adjectives)
elements of film  Objective Reporting (e.g., facts of the film)
critiques
 Audience Reaction (e.g., discussing audience reaction, viewers who might enjoy)
Factor analysis with 13 items
c) Film critics’
important
functions of a
movie





Conflict Management (e.g., harmony, aggression)
Relation to Reality (e.g., authenticity)
Story (e.g., temporal closeness, suspense, erotic)
Production (e.g., details, humor)
Study details
Focus group discussions and
paper-and-pencil
questionnaires (N = 200
Dutch and US children);
principal components
analysis with 44 items
Survey (N = 100); principal
component analysis with 23
items (analyses within each
not across dimensions)
Aesthetic Experience (e.g., artistic experience, aesthetic experience, and self-education)
Entertainment (e.g., diversion, escape, and relaxation)
Factor analysis with 25 items
Arousal (e.g., danger, emotional arousal, and sexual stimulation)
Subject of Conversation (e.g., basis of conversation, what’s talked about)
Ethical Value (e.g., model of behavior, reinforce values)
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
51
Table 2
Latent structure of subjective movie evaluation criteria: Descriptive statistics (M, SD), results of ML-EFA (loadings in the pattern matrix, CI,
communalities h², and latent factor correlations), and Raykov’s ρ (N = 506)
No.a
Wording
M SD
SV
SI
sv1 that the story the movie tells is based on
.73
true facts (e.g., based on a historic event 0.86 0.98
[.66, .81]
or tells the life story of a real person)?
sv2
.65
that the story the film tells is realistic? 1.72 1.21
[.58, .72]
sv3 that the movie addresses contemporary
.56
0.88 0.91
issues?
[.49, .64]
si1 that the movie tells a story in a novel
.71
1.98 1.07
way?
[.61, .82]
si2
.69
that the story the film tells is unusual? 2.12 1.06
[.59, .80]
ci1
the film’s camera work and the shots? 2.54 1.04
ci2 the way how the film is cut or how
individual shots are cut together?
ci3 the illumination and lighting in the
movie?
ci4 the color scheme in the movie (e.g., the
use of black-and-white, red as a symbol
or signal color, etc.)?
fx1 that the production of the film was
extravagant?
fx2
the digital (post-)editing of the film?
2.43 1.08
2.11 1.10
2.27 1.09
1.12 1.03
1.42 1.13
fx3 the movie illusions (e.g., special effects
such as fire, explosions, stunts, combat 1.73 1.17
scenes)?
fx4 the technical design of the film
1.91 1.08
altogether?
CI
FX
RE
IN
LH
CS
h²
.55
.43
.40
.56
.54
.90
[.87, .93]
.83
[.80, .86]
.79
[.75, .83]
.79
.71
.64
.58
[.52, .64]
.49
.68
[.62, .74]
.20
.67
[.13, .27] [.60, .73]
.50
.57
.66
[.60, .73]
.54
.23
.64
[.16, .31] [.57, .71]
.53
(table continues)
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
52
Table 2 (continued)
No.a
Wording
re1 the award(s) for the film (e.g., Oscar,
Golden Globe, Golden Palm, Golden
Bear, German Film Award, Grimme
Award)?
re2 the reviews of the film in the “media”
(e.g., press, radio & TV, movie sites in
the Internet)?
re3 the opinions of friends, acquaintances,
etc. about the film?
re4 how the film is advertised (e.g., on TV
or movie trailers, posters, newspaper,
and magazine ads)?
re5 the fact that the film is considered a
classic movie?
that the movie is free of scenes that you
in1
find disgusting?
that the movie is free of scenes that
in2
make you angry?
that the movie is free of scenes that you
in3
find frightening?
that the movie is free of scenes
in4
containing violence?
that the film puts you in a cheerful
lh1
mood?
M
SD
CI
FX
RE
IN
LH
CS
h²
.79
[.73, .84]
.61
1.19 1.04
.71
[.65, .77]
.47
1.72 1.04
.58
[.51, .64]
.39
1.02 0.96
.25
.51
[.16, .33] [.44, .58]
.38
1.14 1.08
.45
[.38, .52]
.30
1.81 1.53
1.41 1.35
1.43 1.36
1.10 1.21
2.24 1.18
2.37 1.00
lh3 that you find the movie entertaining?
3.03 0.84
lh4 that you find the movie relaxing?
2.21 1.17
that the movie takes your mind off
everyday things?
SI
1.03 0.98
lh2 that you find the movie humorous?
lh5
SV
2.60 1.13
.24
[.15, .32]
.92
[.89, .95]
.83
[.79, .87]
.73
[.69, .78]
.52
[.45, .58]
.83
.73
.61
.39
.83
[.78, .89]
.70
[.63, .77]
.59
[.52, .67]
.57
[.50, .65]
.43
[.34, .51]
(table continues)
.70
.49
.43
.38
.26
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
53
Table 2 (continued)
No.a
Wording
cs1 that the movie is thought-provoking?
cs2
that the movie is intellectually
challenging?
cs3 that the movie communicates values?
that watching the film broadens your
knowledge?
that the movie motivates you to do
cs5 something (e.g., politically, socially,
search for information)?
cs4
M
SD
SV
SI
CI
FX
RE
2.73 1.00
2.41 1.03
1.98 1.16
2.15 1.08
.20
[.11, .29]
1.65 1.10
IN
LH
CS
.82
[.78, .87]
.82
[.76, .87]
.57
[.50, .64]
.58
[.50, .65]
The items were introduced with the phrase “When evaluating movies, how important for you personally is/are...”
*p < .05
.69
.67
.43
.47
.56
.41
[.49, .63]
Latent factor correlations
SV
SV
.71
SI
SI
–.04
.76
CI
CI
–.01
.14*
.82
FX
FX
–.02
.16*
.25*
.75
RE
RE
.19*
.17*
.06
.13*
.72
IN
IN
.15*
-.10*
–.02
.06
.08
.86
LH
LH
.11*
.17*
.04
.35*
.28*
.34*
.78
CS
CS
.36*
.20*
.17*
–.02
.15*
–.03
.08
.79
Note. ML-EFA = exploratory factor analysis with maximum likelihood extraction and Geomin-rotation, no row standardization; factor
loadings < .20 are omitted; factor loadings > .40 are in bold typeface. 90% CI are reported in brackets.
Raykov's ρ for each factor is presented in the diagonal in italics in the lower part of the table; latent factor correlations below the diagonal.
SV = Story Verisimilitude, SI = Story Innovation, CI = Cinematography, FX = Special Effects, RE = Recommendation, IN = Innocuousness,
LH = Light-heartedness, CS = Cognitive Stimulation.
a
h²
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
Table 3
Fit Indices for Model Comparison of EFA, E/CFA, and CFA Models of Eight SMEC Dimensions
Model Analysis Extraction N (Sample)
No. Items SB-χ²/df rRMSEA rRMSEA 90%CI SRMR rCFI
a
1a
EFA
ML
506 (Study 1) 32
2.35
.052
.046, .057
–
–
b
1b
E/CFA Robust ML 403 (R1)
33
1.39
.031
.024, .038
.020
.977
b
2
E/CFA Robust ML 403 (R1)
28
1.16
.020
.000, .031
.017
.993
2
CFA
Robust ML 403 (R2)
28
1.92
.048
.042, .053
.061
.926
Note. SB-χ²/df = Satorra–Bentler χ²/df; rRMSEA = robust root mean square error of approximation; SRMR = standardized root
mean square residual; rCFI = robust comparative fit index.
a
For Model 1a non-corrected fit indices are presented (i.e., χ²/df, RMSEA, and RMSEA 90% CI).
Differences compared to Model 1a: Model 1b (Item si3 added), Model 2 (Item si3 added; Items ci3, fx4, re3, lh5, and cs2 removed).
b
54
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
Table 4
Latent Structure of the SMEC Scales: Exploratory Factor Analysis within the Confirmatory Factor Analysis Framework (E/CFA; nR1 = 403)
and Confirmatory Factor Analysis (CFA; nR2 = 403)
SV
SI
CI
FX
RE
IN
LH
CS
Item E/CFA CFA E/CFA CFA E/CFA CFA E/CFA CFA E/CFA CFA E/CFA CFA E/CFA CFA E/CFA CFA
sv1
.72 .71
sv2
–.21
.57 .61
sv3
.65 .55
si1
.71 .70
si2
–.21
.20
.82 .66
si3
.15
.74 .70
ci1
.90 .88
ci2
.80 .89
ci4
.58 .62
fx1
.13
.62 .72
fx2
.23
.64 .77
fx3
.65 .76
re1
.71 .70
re2
.17
.23
–.17
.69 .75
re4
.20
.51 .59
re5
.54 .52
in1
.84 .82
in2
.59 .71
in3
.84 .83
in4
.20
.68 .81
lh1
.82 .82
lh2
.74 .81
lh3
–.23
.70 .52
lh4
.70 .70
cs1
–.25
.71 .65
cs3
.22
.80 .75
cs4
.78 .72
cs5
-.22
.88 .79
Note. Factor loadings with p > .05 are omitted; factor loadings > .40 are in bold. For abbreviations of the SMEC dimensions see text; for
item wording see Table 2.
55
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
56
Table 5
Partial Correlations Between the SMEC Scales and External Criterion Constructs,
Controlling for Gender and Age
Validation Construct
SV
SI
CI
FX
RE
IN
Emotional Stability
–.02
.02
–.03
–.02
–.10
–.11
Extraversion
.05
.01
.06
–.03
.09
.11
Openness to Experiences
–.01
–.03
–.07
–.07
.17
.16
Agreeableness
–.03
.05
.00
.06
.11
.12
Sensation seeking
.02
.00
.02
.13
.13
–.12
Need for Cognition
–.07
.04
.14
–.14
–.14
–.17
Film expertise
.07
–.12
.21
.33
–.12
–.28
Film genre preferences
Romance
.02
–.01
–.04
.10
.14
.20
Comedy
.02
.03
.02
.04
.19
.14
Action
.01
.04
.05
–.12
.09
.37
Animation
.08
.03
.01
–.18
.13
.22
Thriller
–.02
.07
.06
.11
.11
–.20
Science-Fiction/Fantasy
–.03
–.05
–.25
.12
.14
.17
Horror
–.08
.07
.05
–.04
.08
–.35
Drama/Tragedy
.04
.11
.13
–.20
.13
–.23
Avant-garde
–.04
.00
.19
.23
–.29
–.19
Documentary
.06
–.01
.03
–.10
–.09
–.17
Film specific evaluationsa
Fun
.00
.01
.11
.01
.04
.17
Appreciation
.01
.04
.04
.07
.20
.12
Note. N = 587. Significant correlations (p < .05, two-sided) are in bold.
a
Film specific evaluations were measured at t2 (N = 273).
LH
.01
.02
–.10
.16
–.08
–.16
–.41
CS
.00
.13
.18
.03
.13
.15
.06
.19
.41
.28
.11
–.02
.05
–.09
–.37
–.43
–.17
–.05
–.07
–.15
–.03
–.01
–.10
–.10
.20
.16
.15
.10
-.02
–.02
.23
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
57
MEASURING SUBJECTIVE MOVIE EVALUATION CRITERIA
58
Figure 1. LST coefficients reliability (Rel), common consistency (cCon), occasion specificity (Spe), and method specificity (mSpe) for SMEC
dimensions on two points of measurement (N = 275; aggregation across test-halves within occasions; for formulas see Steyer & Schmitt, 1990).
The LST models were specified as follows: Error variances were freely estimated for SV, SI, and FX, constrained on each occasion for CI and LH
and constrained on all occasions for RE, IN, and CS. Both method factors were freely estimated for SV and SI and constrained for RE and IN. One
method factor was fixed to zero due to lack of significance for CS (test half 1) and for CI, FX, and LH (test half 2). All model fit indices indicated
good fit (i.e., all ps > .05, all RMSEAs < .06 and all 90% CIs included 0; all SRMRs < .06, and all CFIs > .99).
View publication stats