Primož Južnič, Polona Vilar & Tomaž Bartol,
University of Ljubljana, Slovenia
What do researchers think about
altmetrics and are they familiar with their
abilities?
LIBRARIES IN THE DIGITAL AGE (LIDA) 2014
Zadar, Croatia
1.
The debate on how to measure scientific quality and
quality of scientific research has been going on for
decades.
 Using citations as an absolute proof of quality has
been frequently labelled “controversial”, either if
used as an indicator of assessing the quality of
research work, both directly (citations of individual
papers and other publications) and indirectly
(journal impact factors).

2.


The rise of the Web 2.0 (Social web) has given
the main incentive to the creation of
altmetrics, which are social web metrics for
academic purposes.
It can be used in an evaluative role and as an
information seeking aid, both tasks reserved
until recently for traditional bibliometrics.
3.


If altmetrics are to be trusted then the claims
about both of these tasks must be acceptable
and verifiable.
Most of the research presenting arguments
about the extent of the use and importance
of social web simply quotes user statistics
obtained from administrators of different
social networking websites.
We have to find out more about the possibility of
altmetrics, and to explore the applicability, use and
acceptance of altmetrics sources and indicators in
the scientific community.
 Since it is still unclear how and to what extent the
social networking platforms are used, by whom and
for what purpose.

The objective of this study is to assess the
representativeness
and
validity
of
altmetrics’ indicators with the help of
scientific community.

Results are a part of extensive longitudinal
survey of Slovenian scientists which investigated
their information behaviour (preferences, opinions,
use) and their views on research results evaluation.
 The principal objective was to investigate whether
Slovenian researchers essentially use social
networking sites and perceive them as an important
part of their professional work as social networking
is regarded as a part of possible new metrics.
We had following research questions:
1. Do younger researchers use social
networking tools more than older
researchers?
2. Does the period of three years present an
important difference in the acceptance of
social networking tools among researchers?
3. Do the researchers regard altmetrics as
alternative methods for evaluation of their
research work?

Online survey (open April – May 2014)
 10 content questions (eg. about information resources
they use/prefer and science metrics )
 6 demographic questions (age, gender, employment,
experience, research areas)

Random sample of active researchers (data from
ARRS – Slovenian Research Agency.)
 E-mail invitation for every eighth (592 individuals)



114 acceptable
answers
41 % female
Age structure:

Research area
(ARRS classif.):
Research area
%
Natural Sciences
33
Social Sciences
19
13
Age
%
Technical Sciences
20-30
9
Humanistic Sciences
31-40
39
Interdisciplinary Research
16
41-50
26
Medicine
17
51-60
14
Biotechnical sciences
11
above 60
12
9

Heavy use of
 web search engines – general and scholar (84%
often/always),
 e-journal sites and specialised bibliographic databases
(WoS, Scopus…) (63% often/aways)
 Library catalogue (COBISS) (43% often/aways)

They like electronic materials:
 (54% prefer e-) (43% cite over 80% resources in
electronic form)
which are not so surprising anymore, as the results
are almost identical to ones three years ago.


They hardly use popular social networking tools:




Facebook (87 % never/ almost never),
Twitter (89% never/almost never)
blogs (77% never/almost never),
forums (66% never/almost never),
Even specialized social networking tools are not so
popular:




Mendeley (87 % never/ almost never),
CiteULike (93% never/almost never)
LinkedIn (74% never/almost never),
ResearchGate (60% never/almost never),
ResearchGate seems to be a little bit more popular than
other tools
14% said they often use it, bot none (0%) always.

Age and gender were not an important factor in
social network use.

Research discipline has some influence, but not
really important one.
We asked the respondents to provide their opinion
on the uses of altmetrics in the evaluation of
scientific research.
Three possible answers were offered:
1.
2.
3.
I am familiar with
I am not familiar with but I'm interested
I am not familiar with and I'm not interested
19% of respondents (mostly male) reported on
their familiarity with altmetrics.
Surprisingly many (71%) said that they are not
familiar with it but are interested!
Age was not an influential factor.
Research discipline also had some influence. On
average, fewer Natural scientists and more
Medicine researchers are familiar with it; more
scientists from Engineering fields are not
familiar with it and are not interested.
There were also some correlations between the
awareness of altmetrics tools and the use of
Web 2.0 tools.
Many of those who claim to be familiar with
altmetrics often use ResearchGate and
LinkedIn, and occasionally use Facebook and
Twitter.
We also wished to investigate the possible acceptance of
different altmetrics' indicators on the part of the
researchers as a measure for evaluation.

Number of downloads of articles from scientific
journals/publications was the indicator agreed or
partially agreed upon by the majority of respondents.
Only 12% disagreed or had no opinion.

Similar answers were obtained related to the
possibility of using the number of downloads of
publications from repositories, although these
received little more answers related to “No opinion”.
References to research results in mass media also
received the same very positive acceptance
(77%).
Two other indicators, references to research
results in social networks and statistics from
the programs such as ResearchGate and
Mendeley were little less popular, as expected,
but still receiving positive acceptance (51% and
66%).
Some respondents mentioned successful transfer
of knowledge to industry as important
reference.
Even if the researchers themselves are not very
regular users of social netowork tools they do
not completely reject such a possibility to
evaluate research results. In fact, most
researchers partially agreed with this option.
Researchers from the Humanities are more in
favour of using the data on downloads, as
expected as they usually oppose the use of
citation data.
Slovenia has a very sophisticated system of tracking
the publication patterns of scientists, and the
respective citation impact, which is also very
transparent as it is publicly available through two
interconnected systems - COBISS and SICRIS.
 The motivation for the updates on publishing
activities is very strong among the researchers.
 They regularly access COBISS and SICRIS, also in
order to follow the publishing activity of their
colleagues and associates.

A weak use of social networking tools does not
seem to prevent the researchers from being open to
the possibilities of employing new methods of
research evaluation. Such non-use is more related
to the lack of time in a highly competitive world of
science, and also to pragmatism.
 If the scientists do not perceive some concrete
benefits, either in a better quality of information
resources or improved prestige, they will not use
such tools.
