Quality Infrastructure Committee (QIC) Update
July 27, 2015
Agenda
• US Standards System Refresher
• Registry Approval Process
• Technical Merit and Standards Development Process
Worksheets
• What’s Next from the QIC
2
US STANDARDS SYSTEM
OVERVIEW
3
Definitions of a Standard
Common and repeated use of rules, conditions, guidelines or characteristics
for products or related processes and production methods, and related
management systems practices. (NTTAA of 1995 and OMB Circular A-119
of 1998)
Market-driven technical specification for a product, service, person, process
or system with which compliance is voluntary. (Anonymous)
4
Types of Standards
physical (measurement)
standards
Certified reference material to aid with
calibration of measurements
documentary (technical)
standards
Specific requirements for the operation of a laboratory
related to management system and competence
5
The U.S. Standards System
The U.S. standards system is voluntary, decentralized, sector and
market driven and is, sometimes, competitive and duplicative.
The system relies on cooperation and communication among:
•
•
•
•
Industry
Private sector standards organizations
Stakeholders
Government
6
Other Terms for Standards
 Best practice
 Guide
 Guideline
 Guidance
 Specification
 De facto standard
 Code
7
Codes are Special
Standards
•Building Code Example:
•
Used in the design, build and compliance process
to construct safe, sustainable, affordable and
resilient structures.
• Prevents disaster occurrence and manages
disaster impact - minimizes the risk and effects.
8
Federal Standards Policy
The National Technology Transfer and Advancement Act
(NTTAA), and the Office of Management and Budget Circular
(OMB) A119:
• Establishes a preference for federal agency use of voluntary
consensus standards over government standards
• Encourages federal participation in standards development
• Authorizes the National Institute of Standards and Technology to
coordinate conformity assessment activities of the agencies
working with state and local government and the private sector
http://www.whitehouse.gov/omb/circulars_a119/
9
Key Concepts in Voluntary
Consensus Standards Development
• Openness
All stakeholders may participate; no
single interest may dominate
• Transparency
Records/ processes open and publicly
available
Openness
• Due Process
Appeals mechanism
Consensus
• Consensus
Decisions more than majority but not
unanimity
Transparency
10
Due
Process
American National Standards
Institute (ANSI)
• ANSI Essential Requirements
• Documented procedures
• Audits to ensure processes are followed
• Available guidance documents
• Balance
• Lack of dominance
• OSAC will build off of ANSI members’
procedures for development and coordination
of standards
11
Standards Developing Organizations
(SDOs) – Who are they?
 Professional Societies whose members seek to
advance their professions, and also develop standards
 Trade Associations promote their industry's products,
and also develop standards
 International Standards Organizations – Country
member based organization like the International
Organization for Standardization (ISO) and the
International Electrotechnical Commission (IEC)
 Other Types of Organizations – Testing laboratories
and those who only develop standards
12
Relevant Forensic Science SDOs
13
REGISTRY APPROVAL
PROCESSS
14
This Process’ Relevance to OSAC Objectives
and Aims
The purpose of the Organization of Scientific Area
Committees (OSAC) for Forensic Science is to strengthen the
nation’s use of forensic science by:
• providing
technical leadership necessary to facilitate the
development and promulgation of consensus-based
documentary standards and guidelines for forensic science
• promoting standards and guidelines that are fit-forpurpose and based on sound scientific principles
• promoting the use of OSAC standards and guidelines by
accreditation and certification bodies
• establishing and maintaining working relationships with other
similar organizations
15
This Process’ Relevance to OSAC Objectives
and Aims
The aims of the OSAC are to:
• populate the OSAC Registry of Approved Standards and the OSAC
Registry of Approved Guidelines
• promote and improve the communication, dissemination and use of
forensic science standards, accreditation, and personnel competencies
• encourage forensic science service providers in the United States to
implement guidelines and standards (e.g., ISO/IEC 17025, etc.) for
quality and competency
• enlist stakeholder involvement from a broad community to provide
public comment on OSAC outputs
16
What Goes on the OSAC Registries?
Documentary standards and guidelines that have
demonstrated:
• Technical merit
• Detailed Scope
• Fitness for purpose
• Uncertainty measurement and potential bias
• Method validation, as appropriate
• Reasonable standards development process
• Due Process
• Consensus
• Openness
• Transparency
• Freedom from undue influence
• Balance of interests
17
OSAC: Standards vs. Guidelines
transparency
technical
merit
balance
due
process
openness
consensus
OSAC Registry of Approved Standards
Standard – mandatory
• Specifies uniform methods, actions,
practices, or processes, protocols
• Compliance recommended to be
mandatory and modified only under
unusual circumstances: (shall)
• Approved by FSSB
OSAC Registry of Approved Guidelines
Guideline – strongly recommended
• Suggested methods, actions, practices, or
processes to consider in absence of
applicable standards
• Best practices that are strongly
recommended but not required (should)
18
• Approved by SAC
Standards/Guidelines Registry Approval Process Overview
https://workspace.forensicosac.org/kws/groups/library/documents
?folder_id=700
Document Number/Identifier & Title
Purpose
OSAC Registry Approval Process of Published Standards
or Guidelines Process Map V1
Overarching process map and flow chart that demonstrates this
process’s steps and approvers
OSAC Registry Approval Process of Published Standards
or Guidelines Process Description V1
Provides additional details on specifics and nuances of each
step
QIC Form 1: Technical Merit Worksheet
Tool and checklist to analyze technical merit to include a
detailed scope, fitness for purpose, uncertainty measurement,
validation, and other details
QIC Form 2: Standards Development Process Worksheet
Tool and checklist to analyze a standard/guideline’s due
process, consensus, openness, transparency, freedom from
undue influence, and balance of interests during the standards
and publishing process
QIC Form 3: Registry Request
Cover sheet to your Registry Request packet, and is a signal to
the SAC and FSSB which materials are relevant
QIC Form 4: Impact Worksheet
Tool and checklist to analyze the potential impact on the
laboratory or crime scene if the standard / guideline is
implemented
QIC Template A: Notice of Intent to Add to Registry &
QIC Template B: Justification for Non-Approval
Templates to use at various stages of the registry approval
process
19
20
RA-100: Complete and Compile Registry Request Packet
The subcommittee
will then vote on
whether to move the
full packet forward to
the SAC. A 2/3
quorum is required to
vote, and a majority
vote is required to
move a packet to the
next stage.
21
RA-200-RA-600:
QIC Template A: Notice of Intent to Add to Registry OR
QIC Template B: Justification for Non-Approval
22
RA-700 – 1000:
Subcommittee
QIC
SAC
23
RA-1100-1500:
24
RA-1700-2000:
25
QIC Form -01
Technical Merit Worksheet
26
Technical Merit Worksheet - Purpose
• Use to guide discussions on the technical strength and scientific
underpinnings of a standard or guideline.
• Serve as a starting point for discussions on the appropriateness
of a standard or guideline for inclusion into the OSAC registry.
How to Use This Form
• Responses to the questions outlined in the “Technical Merit
Worksheet” are intended to be drafted and finalized during task
group and/or Subcommittee deliberations.
• A technical point of contact should be assigned to each standard or
guideline being considered for inclusion into the OSAC Registry.
• Two potential outcomes:
• Has technical merit: If the subcommittee has determined the
standard/guideline has technical merit, the subcommittee includes this
form along with the other documents required for the Registry Approval
Packet.
• Does not have technical merit: If the subcommittee has determined the
standard/guidelines does not have technical merit, this form is retained in
the Kavi workspace as a record of discussion, or serve as a basis to have
the document further edited by its authors.
Technical Merit Worksheet
29
Subcommittee Reaches Consensus
• The subcommittee at large should now review the standard
AND the initial draft of the technical merit form.
• After a reasonable amount of review and deliberation, the
subcommittee should finalize the language and technical rating
on the form.
Gradients of Agreement – Tool to
reach consensus
1- Wholehearted Endorsement
“I really like it”
2- Support with Reservations
“ I can live with it”
3- Abstain
“This issue does not affect me”
4- More Discussion Needed
“I don’t understand the issues well enough”
5 – Serious Disagreement
“I am not on board with this – don’t count one me”
Unanimity vs Consensus
Unanimity
Consensus
Unanimity refers to agreement by
every member in a particular group.
In contrast, consensus is the process
by which a group arrives at an
agreement.
• All standards making groups do not require unanimity to make
decisions. In the OSAC, 2/3 quorum is required to vote and a
majority vote is required to make decisions which will move a
document forward to the next stage. With that said,
CONSENSUS should be the primary aim.
QIC Form -02
Standards Development Process
Worksheet
33
Standards Development Process
Worksheet
34
Registry Request Form
Indicate which forms you
have provided
Indicate which registry your
packet is intended for
35
For more information or answers to
questions
• Contact your QIC liaison
• Biology/DNA: Alice Isenberg,
alice.isenberg@ic.fbi.gov
• Chemistry/Instrumental Analysis: Deborah Friedman,
deborah_friedman@sheriff.org
• Physics/Pattern: Karin Athanas, kathanas@a2la.org
• Crime Scene/Death Investigation: Sally Aiken,
saiken@spokanecounty.org
• Digital/Multimedia: Kris Cano,
kcano@scottsdaleaz.gov
What’s Next from the QIC
37
QIC Next Steps – Training Materials
• Complete and make available QIC training modules
• What is Technical Merit?
• Technical Merit Worksheet
• US Standards Development System
• Standards Development Process Worksheet
• Registry Request Form
• Impact Worksheet
• Harmonization Worksheet
TIMELINE: August 2015
QIC Next Steps – Appeals Process
• Completed first draft of Appeals process and received FSSB
comments and feedback
• Currently revising; will finalize and distribute to OSAC
Membership
TIMELINE: 1st week of August 2015
QIC Next Steps – Comment
Adjudication Process
• Complete first draft of Comment Adjudication Process
• Comment resolution categories
• Kavi process
• Share draft with FSSB for comment and feedback
• Revise as needed, and distribute to OSAC membership
TIMELINE: August 2015
QIC Next Steps – Working with and
SDO Process and Procedure
• Confirm current process map is still accurate
• Document corresponding procedure
• Share draft process map and procedure with FSSB for
comment and feedback
• Revise as needed, and distribute to OSAC membership
TIMELINE: September 2015
Karen Reczek
Chair, Quality Infrastructure Committee (QIC)
Standard Coordination Office, NIST
Karen.Reczek@nist.gov
301.975.4038
43