project
goals
reality
time
$$
technology
From challenge…
…to happy ending?
The Devil is in the Details:
Core Adventures in Implementing a
New Analytical Method
W E S T E R N A S S O C I AT I O N O F C O R E D I R E C T O R S M E E T I N G
SEPTEMBER 12, 2014
Allis Chien, PhD
Director, Stanford University Mass Spectrometry
allis@stanford.edu
SUMS – Stanford University Mass Spectrometry
•
•
•
•
Established 2000
University-wide core resource
8 staff,12 mass specs, 15K samples/year
Application areas:
Consultation
 Proteomics
 Small Molecules
 Quantitation
 Drug Discovery Support
 Open Access Lab
 Custom Projects

Seely G. Mudd Building
Stanford Dean of Research Unit
Stanford Cancer Institute Proteomics Shared Resource
Vincent Coates Foundation Mass Spectrometry Laboratory
Bio-X Mass Spectrometry Core Shared Facility
2
Required
Administrative
Elements
need
GETTING A NEW METHOD
OFF THE GROUND
new
method
funding
technology
4
need
Research project: Acute Respiratory Distress Syndrome
Project goal: broad, quantitative
profiling of common metabolites
Challenges:
• Large number of metabolites
• Diverse metabolite types
More than 190,000 patients develop
Acute Respiratory Distress Syndrome
(ARDS) every year in the United
States, which, even in the modern
era, carries an appalling mortality rate
of 22-40%.... the etiology of ARDS
remains incompletely understood and
its clinical course hard to predict.
Identifying patients at the highest risk
for developing ARDS would enable
both early treatment and facilitate
enrollment in emerging clinical trials.
Positives:
• Sharp PI, strong in statistics
• Access to patients & samples –
quality and quantity (200)
• Understands variability, need for
method validation
• Willing to invest in method
development/implementation
Technology: turnkey method kit for targeted,
quantitative analysis of 180 metabolites
“The most extensive metabolite assay in the market.
Identification and quantification of more than 180 metabolites from 5
different compound classes” [company brochure]
tech
6
Funding: Who bears startup costs for new methods?
Faculty perspective:
“If there is a large upfront cost
associated with working with them
(i.e. you have to buy some sort of
equipment that you don't already
own), it would be hard for me to help
support it at this point; on the other
hand, if it's just a matter of having
someone fly over from [Europe], but
it's relatively straightforward for your
lab to experiment with, I'd be glad to
talk about how much that would
cost. Certainly doing something with
you rather than [Company x] would be
great from my perspective for the
long-term if the costs can be similar.”
funding
Are these valid solutions?
 Core users as a group – build into
service rates
+ Seems logical, for ongoing
method development
- Usually not equitable
 First PI to use – paying for the
privilege of being first
+ No lead time required
- Heavy burden for a single PI,
often not feasible
 First few PIs – share the cost
+ Spreads out cost
- Difficult to predict & coordinate
- Slows down process
7
funding
Seed Grant Program
Request verbiage:
“A pilot grant program would address
several needs simultaneously. Such a
program could consist of several small
competitive awards per year (e.g. $5-10K
each), with the goal of putting together
solid mass spec preliminary data for
research grant proposals.
The funds would certainly support the
service center, but unlike a straight
subsidy, such a program would raise
awareness, increase involvement, and
encourage exploration. It would also
support staff development by showcasing
the collaborative nature of modern mass
spectrometry-based research. Similar pilot
grant programs have been successful at
other universities, and would represent a
well-placed investment in future research.”
December
2012
March
2013
July 2013
September
1, 2013
• Budget request to DoR
for Seed Grant Program
• Seed Program approved
for FY14
• Funding Opportunity
Announcement issued
• Earliest start date
Find a link to the FY15 FOA at http://mass-spec.stanford.edu
8
need,
funding
ARDS Seed funding proposal
Seed grant goals:
• Supporting the application of
existing capabilities to
emerging research areas
• Developing new workflows
with existing instrumentation
base to advance research
• Helping PIs generate
preliminary data to use in
proposals for larger research
grants
Project aims:
1. Set up metabolite profiling
method
2. Verify technical reproducibility
via replicates & accuracy via
orthogonal measurements
3. Apply method to 200 patient
samples
Estimated costs: >>$23K total
1. Method setup –
› time: $2,500
› materials: $3,500
2. Verification –
› time: $4,000
› materials: $2,000
3. Analysis –
› time: TBD
› materials: $11,000
need
new
method
funding
technology
Administrative stars align – then scientific work begins
December
2013
January 2014
February 2014
• PI project
• good fit for kit
• potential
funding
“Background
check”
discussions
with kit
provider
PI applies for
seed funding
March 2014
• $20K seed
grant
awarded
• leap of faith
• preinstall
activity
April
2014
3-day
installation
site visit
Acquired
Practical
Lessons
PUTTING A NEW METHOD
INTO PRACTICE
follow up
Murphy’s
Law
evaluate
new
method
install
prepare
11
Evaluation: Commercial kit for metabolite
quantitation via LC-MS/MS
Appealing:
• Covers nearly 200 metabolites –
would be time- and cost-prohibitive
to develop from scratch
• Detailed protocols from sample
preparation and MS/MS method
setup through data acquisition and
analysis
• Authentic standards & calibration
mixes provided
• Company validated the method,
has been used in a number of
publications
evaluate
Not so much:
• Regimented, loss of flexibility
• batch size – 96 well plate
• shelf life of months
• sample type – only for plasma
• limited dynamic range
• data analysis workflow
• Proprietary method, sparse
explanations
• Required >$10K investment before
testing
prepare
Make a list, check it twice
Don’t assume anything – no detail is too small to ask about and confirm
•
Applications scientist traveling from Europe for the 2-3 day install
•
Kits, software shipped ahead of time
•
Detailed 2-page preinstallation checklist of requirements, including part
numbers where relevant:
› LC and MS hardware, HPLC column, guard, holder
› lab equipment, e.g. evaporator, vortexer, shaker, fume hood, balance
› glassware, pipettes
› solvents, chemicals
› software
•
Discussed item purposes, possible substitutions
•
Didn’t think to ask about software versions…
13
install
More information enables more & better questions
Site visit revelations about proprietary methodology
• Quantitation vs. semi-quantitation
• Full calibration curve vs. 1 point external calibration
• LC-MS/MS vs. flow injection analysis (FIA)-MS/MS
April
2014
installation
site visit
May
2014
Data
collection
on
replicate
set
Corollaries to Murphy’s Law when implementing a
new method
1. It will be more complicated than expected
2. It will take more time than expected
3. It will cost more than expected
Murphy’s
Law
The Software
Compatibility Saga
• Problem:
Versioning conflicts
among Windows,
Xcalibur, Java, Oracle
database, Excel, and
proprietary data
analysis software
• Solution: Custom
scripts implemented
via remote computer
access from Europe
• Additional Time:
60 hrs of direct staff
time over 2 months
• Additional Cost:
$6,000
United Kingdom
United States
Software Requirement: UK English
Murphy’s
Law
Follow through with the plan
December
2013
Stars align:
• PI project
• good fit for kit
• potential funding
May
2014
Data
collection
on
replicate
set
January 2014
“Background
check”
discussions
with kit
provider
…June
2014…
February 2014
March 2014
April
2014
PI applies for
seed funding
$20K seed
grant awarded,
leap of faith,
preinstall
activity
3-day
installation site
visit
Software
issues
Sept.
2014
August
2014
Orthogonal
analyses
Replicate
data
released,
evaluation
in process
October
2014
200 patient
samples
17
Lessons Learned
•
•
•
need
•
new
method
funding
•
•
•
•
•
•
•
technology
•
Ask more questions
Trust and also verify
Take the physical location of
company personnel into account
Build good relationships with
support personnel
Combine action and patience
Keep up with new developments
Request funding in terms that align with
institutional goals
Account for all costs, including materials,
staff & instrument time
Allow for Murphy’s law when generating
cost estimates
Evaluate feasibility, possible vs. practical
Ask questions
Pay attention to details
follow
through
Murphy’s
Law
evaluate
new
method
install
prepare
18
Acknowledgements
Angela Rogers
Manuel Kratzke
SUMS:







Chris Adams
Ludmila Alexandrova
Karolina Krasinska
Ryan Leib
Theresa McLaughlin
Anna Okumu
Rachel Wu
Dean of Research:




Ann Arvin
Sara Bible
John Brauman
Ken Merritt
http://mass-spec.stanford.edu
Variability Survey & Discussion - A host of factors impact the development of an
analytical project. Your survey responses will help stimulate a discussion of variability
assessment practices in our community, and how they pertain to the types of experiments
that we collectively face every day. Click here to access the survey