Developing Best Practices
for Supplemental Materials
Linda Beebe
June 2, 2011
SSP June 2, 2011
COLLISION OF 2 WORLDS
SSP June 2, 2011
Explosion of─
Research
Data
Accrued Knowledge
Increased
Requirements
Funding Bodies
Reporting Standards
SSP June 2, 2011
Supplemental Materials—it
sounded like such a good idea.
 The author could expand on
their research.
 Science would be better with
data needed to verify or
replicate study at little
additional cost.
 We could enhance reporting
of science with multi-media.
SSP June 2, 2011
 We looked to technology to
solve problems—but kept our
print-centric views.
 And we did it on our own—
no standards or best
practices.
Outcomes for the author?
 May get to showcase
new work that would
not otherwise be seen.
 May also risk
displaying weak work
that otherwise might
not be seen.
SSP June 2, 2011
Outcomes for the user?
 Lack of descriptive
metadata
 Discoverability issues
 Lack of context
 Concern about persistence
 No clarity on citations
 Some mystery in the main
article about what is
supplemental
─ a maze, maybe not valueadd taken as a whole.
SSP June 2, 2011
Outcomes for the publisher?
 Direct costs
 Diverted energies—
already crisis in peer
review
 Tough decisions─
 What is value-add?
 Peer review dilemma,
quality vs workload?
 Plan for migration?
SSP June 2, 2011
NISO-NFAIS Working Group
SSP June 2, 2011
Business Working Group
Co-Chairs: Linda Beebe & Marie McVeigh
 Define Supplemental Materials, structurally and functionally.
 Define related terms, such as data, citation, and article.
 Recommend methods of referencing and linking to and from
supplemental material and for providing context.
 Recommendations around metadata, persistent identifiers, and
citations .
 Recommend processes for peer review, production, and curation.
 Consider permissions and accessibility issues.
 Recommend responsibilities for authors, editors, peer reviewers,
publishers.
SSP June 2, 2011
Technical Working Group
Co-Chairs: Dave Martinsen & Sasha
Schwarzman
 Recommend metadata, persistent identifiers, and granularity of
markup needed to support practices recommended by the
Business Working Group (BWG).
 Recommend supports for referencing and linking to and from
Supplemental Materials and for handling cited references within
Supplemental Materials.
 Recommend processes for archiving, preservation, and forward
migration of various types of Supplemental Materials.
 Recommend processes for packaging, exchange, and delivery of
Supplemental Materials, taking into account variations in the
location and hosting of those materials.
 Recommend technical support for accessibility practices
recommended by the BWG.
SSP June 2, 2011
On Different Wave Lengths
 Disciplines vary in use of supplemental material.
 Differ in style systems and culture.
 Readers vary in need for information—some current
awareness, some deep digging.
 Different approaches to underlying data.
 Very different approaches to delivery systems.
 Technology enabled, but still using print.
SSP June 2, 2011
Data One Type of Supplemental
 Example of evolving ecosystem. Print world—no
datasets part of article.
 For some, almost synonymous with supplemental.
 Journal articles—indeed whole journals—devoted to
data emerging. For these data are integral content.
 Management of data in general not within scope of
recommended practices.
 Address inclusion of data when published as
supplemental (with a little aside on sharing).
SSP June 2, 2011
Illustrates Discipline Variances
 Some publishers—such as AAAS and ACS—require
posting of data in a publicly accessible repository for
replication.
 Some publishers—such as AGU—identify acceptable
repositories.
 Some publishers—such as APA—currently say only
that authors should provide data to researchers for
verification.
More calls for
transparency
SSP June 2, 2011
Data Sharing—the ideal & reality
Most ethics codes call for some level of sharing.
What they say:
 PARSE study—84% of
scientists think it useful to
link data to articles.
 In Psychology, 80% say they
share their data.
What they do:
 Only 25% said their data are
openly available.
 Only 20% actually do.
 2008—Harvard faculty voted
to require faculty to deposit
data in Harvard repository.
SSP June 2, 2011
 By 2011, only a fraction have
done so.
Reasons for not sharing─
 It takes time, may require
extensive explanations of
coding or just plain clean-up.
 They have several fears—

Loss of confidentialty

Potential harm to
subjects
 Who will curate/protect it?

Potential faulty re-analyis
 Will I be credited?

May be proven wrong

Loss of control
 I’m not finished—I can get
more articles.
SSP June 2, 2011
What we are saying. . .
Collaborative
sharing best
practice.
Professional
ethics around
secondary
analysis.
SSP June 2, 2011
Requires clear
metadata and
explanations.
Particular concern
for studies with
human
participants.
Multi-Media Another Example
 Five years ago, audio or video not possible in the
article.
 Today still generally supplemental.
 BUT some now incorporating in PDFs.
 Executables as part of the article?
 Expect much more interactive content.
SSP June 2, 2011
Supplemental Today, Not Tomorrow
“. . . over time the concept of supplemental material will
gradually give way to a more modern concept of a
hierarchical or layered presentation in which a reader
can define what level of detail best fits their interests.”
−Emilie Markus, Editor-in-Chief, Cell
Article of tomorrow may be linked chunks, not a
narrative.
SSP June 2, 2011
Challenges for Publishers
Expectations
Limiting Factors
 Quality uber alles—
 People resources
 Peer review all.
 Financial resources
 Edit to same level as article.
 Technical resources
 Maintain all links.
 Assure migration.
SSP June 2, 2011
We need some order now.
SSP June 2, 2011
Defined 3 Types of Content
Integral. Critical to understanding the work
reported, but technical issues prevent
inclusion in the framework.
Additional Content. Expansion of core
article, added detail and context; provides
layered approach for readers.
Other Related Content. Content may add
to the understanding or enable
replication; generally hosted by others.
SSP June 2, 2011
Recommended Practices
Citing
Assuring
Findability
Editing
Selecting
SSP June 2, 2011
What we are saying. . .
Selecting
• Review same level
• Useful, relevant, not
file drawer
Editing
• Publisher/Editor
determine.
• Provide notice if not.
Assuring Findability
• Consistency
• Online TOC Reference
• Indexing Coverage
• Don’t hide!
Citing
• Within article, cite &
link as for a table.
• Not in reference list
for integral.
SSP June 2, 2011
More Recommendations
Rights Management
Assuring
Accessibility
Preserving
Providing
Links &
Context
SSP June 2, 2011
What we are saying. . .
Links & Context
• Bi-directional if possible
• Links must work!
• Context is essential.
What is this? Why here?
Preservation
• Integral same level as
article
• Clarity on what can do.
• Encourage authors to
deposit elsewhere also.
Accessibility
• Should be same level as
article.
• Strive for ideal, recognize
difficulty.
Rights Management
• Treat rights same way
do for the article.
• No authority for Other
Related Content.
SSP June 2, 2011
2 Working Groups, 2 Roles
BWG—What?
SSP June 2, 2011
TWG—How?
TWG Working Group Task Forces
 Metadata—have strawman DTD
 Linking and persistent identifiers
 Packaging and exchange
 Preservation and archiving
 Accessibility
SSP June 2, 2011
Next Steps
BWG
SSP June 2, 2011
Early
summer
share
TWG
Subgroups
over the
summer
Both
Fall
meld
and
refine
Final Set of Practices
SSP June 2, 2011
Living
document for
rapidly
changing
environment.
Approved by
NISO and
NFAIS.
Serve as a
temporary
roadmap.
Shared with
the
community
and refined.
We welcome ideas!
 NISO—www.niso.org
 To see working groups:
www.niso.org/workrooms/supplemental
 Also join the Business Stakeholders’ Group at that
page.
 NFAIS—www.nfais.org
SSP June 2, 2011
THANK YOU!
Linda Beebe
Senior Director, PsycINFO
American Psychological Association
lbeebe@apa.org
SSP June 2, 2011