The Future of Scholarly Communication | UC Berkeley
“Data citation refers to the practice of providing a reference to data in the same way as
researchers routinely provide a bibliographic reference to printed resources. The need to
cite data is starting to be recognised as one of the key practices underpinning the
recognition of data as a primary research output rather than as a by-product of research.
While data has often been shared in the past, it is seldom cited in the same way as a journal
article or other publication might be. This culture is, however, gradually changing.
If datasets were cited, they would achieve a validity and
significance within the cycle of activities associated with
scholarly communications and recognition of scholarly
effort.”
Australian National Data Service
http://www.ands.org.au/guides/data-citation-awareness.pdf
The Future of Scholarly Communication | UC Berkeley
Two assumptions in this statement?
One, that by virtue of being citable, data achieve
an equal footing with traditional publications in
merit review.
Two, that data standing alone, without both an
interpretive layer (such as an article or book) and
having been peer reviewed, will be weighted the
same as articles and books in Tenure and
Promotion (T&P) decisions.
The Future of Scholarly Communication | UC Berkeley
Drivers of Faculty Behavior
Today, put these two assumptions within the
context of our research findings regarding drivers
of faculty behavior, including possible motivations
and incentives to share and cite data.
(Harley et al., 2007, 2010, 2011)
i.e., what are the incentives for faculty to make
their data citable and to cite the data of others?
(This suggests that the data will actually be shared—
we have data/insights on this probability…)
The Future of Scholarly Communication | UC Berkeley
Giving it Away? Sharing Varies by Discipline
arXiv model not a universal answer
• First w/ trusted circle of colleagues. Modicum of privacy needed.
• Conferences/seminars. Impt.
• Public Posting of Working papers in some disciplines (different
than post-prints).
• Not posted until it has been vetted by inner circle and deemed good
enough. “Penultimate” drafts.
Preprint/Working Paper cultures
High paradigm/quant. fields, e.g., Physics, Astrophys, etc. —arXiv
Econ, Quant Poli Sci –SSRN, IRs, personal websites
Preprints culture non-existent
Bio? Non existent in MCB before publication—commercial potential,
uber competitive fields.
History/Arch/Musicology? Early sharing rare.
The Future of Scholarly Communication | UC Berkeley
Drivers (cont’d)
Based our research in the Future of Scholarly
Communication Project, the primary drivers of
faculty behavior are:
•Career advancement
•Moving the field forward
•Credit and attribution
The Future of Scholarly Communication | UC Berkeley
Academic Values and Scholarly Communication Practices:
An Analysis of Faculty Choice and
Opinion in Twelve Disciplines
Diane Harley, Ph.D., Principal Investigator
Center for Studies in Higher Education (CSHE)
University of California, Berkeley
The Future of SC Project Website and Associated Links:
Project site: http://cshe.berkeley.edu/research/scholarlycommunication
Report: Assessing the Future of SC: http://escholarship.org/uc/cshe_fsc
Peer Review in Academic Promotion and Publication
Report: Peer Review: http://cshe.berkeley.edu/publications/publications.php?id=379
BRDI. August 22, 2011. Copyright 2011, Center for Studies in Higher Education
The Future of Scholarly Communication | UC Berkeley
Funded by the A.W. Mellon Foundation
•
Assess the criteria by which faculty decide when and in what
venues to communicate the results of scholarly research—at all
stages. Informal and formal SC.
• Analyze what roles universities and faculty play in resolution of the
perceived “crisis(es) in scholarly communication.”
• Premise: Disciplinary traditions and culture matter. Plus
personality and institution type.
• Team effort (co-authors: Acord, Earl Novell, King, Lawrence).
• No agenda beyond rigorous, non-advocacy Soc. Sci. research.
The Future of Scholarly Communication | UC Berkeley
Methods
Research Universities
• 45 mostly elite research institutions
• 160+ scholars, + many informal conversations and meetings
• 12 disciplinary case studies
• Literature reviews
• Daily environmental scans (news, listservs, blogs, twitter,
announcements, etc.)
Planning project. 2005-2006
N=5 Anthropology, Biostatistics, Chemical Engineering, Law and
Economics, English-language Literature.
Main project. 2007-Present
N=7 Astrophysics, Archaeology, Biology, Economics, History,
Music, Political Science
The Future of Scholarly Communication | UC Berkeley
Interview Protocols/Topics
(1) Tenure and promotion, making a name
(2) Criteria for disseminating research at various stages (publication
practices, new publication outlets, new genres)
(3) Sharing (what, w/ whom, when, why or why not?)
(4) Collaboration (w/ whom, when, why or why not?)
(5) Resources created and consumed; Needs, discoverability,
priorities, data creation and preservation.
(6) Public engagement
(7) The Future
The Future of Scholarly Communication | UC Berkeley
Is T&P a Flexible System?
• Stellar Publication record impt for T&P. Service, Teaching,
Public Engagement secondary (age and institution dependent).
• “Groundbreaking, moves field forward, judged of high quality by
internal and external reviewers, original”
• Quality over quantity. Metrics such as impact factor often viewed
w/ suspicion.
• Exceptions to the “rules” made—qualitative review is stated ideal.
• Databases, cell lines, critical editions, software: credited in T&P
decisions; rarely sole criteria in most fields and are not given
equal weight as publications.
• New journals (OA), New Genres, etc. are acceptable as long as
peer reviewed.
The Future of Scholarly Communication | UC Berkeley
Peer Review and the Advancement Process
• Heavy reliance on peer reviewed publications to aid Institution/T&P
committees and external reviewers in evaluation of scholarly work.
• Impt. of external reviews in T&P. Damning: “No one has heard of
you.” “High impact” publications way of making a name.
• Many worry that lack of peer review is associated with newer,
untested forms of scholarly publication. Incl. OA.
• Advancement process can and should be supportive (and
unprejudiced) of non-traditional publishing models, provided that
peer review is strongly embedded.
• Written policies: New electronic genres should NOT be undervalued
in consideration of advancement. Does actual practice vary?
Committees not seeing many examples that deviate from the norm.
The Future of Scholarly Communication | UC Berkeley
Data curation and preservation alone are simply not
considered to be a high level of scholarship-Examples abound in every discipline.
Emphasis in Tenure and Promotion (T&P) is on
interpretive work vs. cataloguing or curating.
The Future of Scholarly Communication | UC Berkeley
Biology
New and emerging forms of scholarship are not
valued much in their own right, but supplemental
articles describing a novel tool or resource may be
considered in review.
The work of developing software or data resources
can be perceived as less valuable “tool
development” rather than scholarship.
The Future of Scholarly Communication | UC Berkeley
Archaeology
Developing and maintaining databases or
resource websites is considered a “research
technique” or “services to scholarship.”
The Future of Scholarly Communication | UC Berkeley
Astrophysics
Developing astronomical instrumentation and
software, posting announcements in the form of
listserv-based circulars and telegrams, and
database creation are considered “support” roles
and are usually ascribed a lower value than
publication in advancement decisions.
The Future of Scholarly Communication | UC Berkeley
History
A historian who published only footnoted sources, but
no interpretation of the sources in the form of a book,
would not be promoted.
New and emerging forms of scholarship (e.g., curating
data sets, creating Web-based resources, blogs) are
valued only insofar as they are ancillary to the book.
The Future of Scholarly Communication | UC Berkeley
Political Science
Some scholars in political science create and publish
data sets. Similar to the biological sciences, these
efforts can earn a scholar increased visibility when
other researchers use their data.
Credit is only received for this work if a strong
publication record based on the data accompanies
it.
The Future of Scholarly Communication | UC Berkeley
Generally, chairs, deans, and provosts report seeing
few alternatives to traditional scholarship in tenure
and promotion dossiers at highly competitive research
universities.
The young leading the way is a mythology.
Exceptionally conservative and hew to the norms set
by their mentors.
Data curation and other data-sharing activities may be
"best left until after tenure,“ as with other activities
not directly in service of high impact publication.
The Future of Scholarly Communication | UC Berkeley
Peer Review is the Coin of the Realm
Despite a litany of complaints, peer review is:
– THE value system supporting assessment of and perceived
quality of research.
– The primary mechanism through which research quality is
nurtured.
– The primary mechanism through which research is made both
effective and efficient.
– Excellent quality filter for the proliferating mass of scholarly
information available on the web.
More difficult for time-pressed scholars to sift through it all.
The impt of peer review's role in providing filters cannot be overstated.
The Future of Scholarly Communication | UC Berkeley
TIME and the Need for Filters
There are only 24 hours in a day. The time-pie is finite.
Research, teaching, publishing, grant writing, service, peer
reviewing, etc. –all take up most, if not all, of the pie.
Fatigue set in long ago.
Prediction: If the goal is to make data sharable and citable,
faculty are not the ones in a position to add data curation
and preservation to the suite of demands on their time.
Someone else is going to have to do it, and pay for it.
The Future of Scholarly Communication | UC Berkeley
Peer Review of Data Requisite?
Creating citable data sets vs. peer reviewed publications. Not
recognized equally in T&P. So what’s the incentive to create and how
will quality be established?
Is “audited” enough? Machines/Usage Patterns? Or will data sets
have to be formally peer reviewed to count?
How will this happen? Open peer review a viable model?
The failed Nature Precedings experiment suggests not
…and other evidence—see lit. review in our Peer Review report
http://cshe.berkeley.edu/publications/publications.php?id=379
Open peer review assumes people have unlimited time to troll
through unvetted work/raw data and provide comment.
The Future of Scholarly Communication | UC Berkeley
Factors Working Against Open Peer Review
Peer review already takes an immense amount of faculty
time -- publication, graduate student mentoring, grants,
writing external letters, providing feedback to colleagues,
etc.
Will readers trust findings that are openly peer reviewed by
a readership, which, in some cases, is completely in lieu of
peer review conducted by experts?
Popularity contests? Motivating the “right” peers to
provide commentary can be problematic.
The Future of Scholarly Communication | UC Berkeley
Overload? Changing Journal Policies (1)
The Journal of Neuroscience recently announced its
decision to cease the publication of supplementary
data, citing two main reasons (Maunsell 2010).
First, reviewers cannot realistically spend the time
necessary to review that material closely.
Second, critical information on data or methods can
be lost in a giant supplementary data package.
The Future of Scholarly Communication | UC Berkeley
Overload: Changing Journal Policies (2)
"Enough is Enough" Christine Borowski, July 4th, 2011
editorial published in the Journal of Experimental Medicine
(JEM).
“Complaints about the overabundance of supplementary
information in primary research articles have increased in
decibel and frequency in the past several years and are now
at cacophonous levels. Reviewers and editors warn that they
do not have time to scrutinize it. Authors contend that the
effort and money needed to produce it exceeds that
reasonably spent on a single publication. How often readers
actually look at supplemental information is unclear, and
most journal websites offer the supplement as an optional
download…”
The Future of Scholarly Communication | UC Berkeley
Academic Values and
Scholarly Communication Practices:
An Analysis of Faculty Choice and Opinion
in Twelve Disciplines
Diane Harley, Principal Investigator
Center for Studies in Higher Education (CSHE)
University of California, Berkeley
Project Website and Associated Document Links:
Project site: http://cshe.berkeley.edu/research/scholarlycommunication
Report: Assessing the Future of SC: http://escholarship.org/uc/cshe_fsc
Report: Peer Review: http://cshe.berkeley.edu/publications/publications.php?id=379
Diane h (at) berkeley (dot) edu
Copyright 2011, Center for Studies in Higher Education