Brent Fultz; Co-PIs are Michael Aivazis, Ian Anderson; PM is Mike McKerns
California Institute of Technology
Spallation Neutron Source (SNS)
Accelerator + Target + Instruments = B$ 1.411
Reasons for DANSE
New Science
Better Science
Ease of Use
Software Stability and Reuse
Support Early Operations of the SNS
Excitations in Solids:
An Optical Illusion of
Space-Time Correlations
Neutron Scattering
Word Pairs
{coherent, elastic}
{coherent, inelastic}
{incoherent, elastic}
{incoherent, inelastic}
Examples
diffraction
dispersions
diffuse scattering
densities of states
elastic
inelastic
Coherence: Preserves the Phase
Coherence
–k0
Q
k0
k
k
Incoherence: Arbitrary Phase
k0
Incoherence
Structure and Dynamics
QuickTime™ and a
GIF decompressor
are needed to see this picture.
QuickTime™ and a
GIF decompressor
are needed to see this picture.
QuickTime™ and a
GIF decompressor
are needed to see this picture.
http://physics.ucsc.edu/research/images/
Structure in Space; Dynamics in Time
Diffraction
Inelastic: (incoherent, inelastic) vs. (coherent, inelastic)
Correlation Functions
Diffraction, SANS gives spatial correlation functions
(Patterson function — structure)
Inelastic scattering gives space-time correlation function
(Van Hove correlation function — dynamics)
“reduction and visualization”
The DANSE Project
SNS
NCNR
Lujan
IPNS
HFIR
User
Community
(International)
DANSE Project: Big Picture
• Enable new neutron scattering science by scientific
computing.
• Allow experimentalists to rearrange modularized components,
and create new components.
• Build enough important and quality pieces to make DANSE
the choice of future developers.
• Well-defined tasks organized by subfields.
• System in place for Earned Value Management.
• M$ 12 over 5 years
Operational Goals
•
•
•
•
•
•
Provide today’s capabilities of reduction and visualization
Enable new types of science in major subfields
Standardize software design, and start to standardize usage
Provide a runtime framework onto which scientists can add
components
Support for early SNS instruments
Maintainable by SNS before the end of the project
Component Framework
Encapsulate scientist’s code within component.
A component inherits methods from the framework,
user communicates properties through framework.
Users Differ in Needs and Expertise
1. Beginning Student
2. Senior Student or Postdoc
3. Young Scientist
4. Established Researcher
5. Instrument Scientist
6. Software Developer
7. System Maintainer
One software
system must serve
everyone,
but different people
need different
interfaces.
Users: 1. Beginning Student
•
•
•
•
Complexity reduced to essential modules
Reasonable defaults for most choices
Failures must be diagnosed and explained
Help with science principles
“Dashboard-type” GUI
and
Help services
Users: 3. Young Scientist
•
•
•
•
Flexible, interactive explorations of data
Modeling and simulation packages
Compare outputs of different analyses
Software must facilitate science
Seeks empowerment from GUI
and command-line control.
Users: 6. Software Developer
•
•
•
•
•
Well-documented access to the integration layer
Portable building tools
Robust framework for debugging
Regression test suites
Development process and tools for quality and efficiency
Direct command line control,
but support for all other user interfaces.
When analysis components
are building blocks,
what will scientists build?
Facilitate discovery in
neutron scattering science by raising
computation to a higher level of abstraction.
Can scientific careers be developed by
adding value at this higher level?
Deployment Diagrams
theorist@mit
colleague@caltech
data@ornl
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Experiment teams are already
delocalized geographically.
Natural for collaborative interactions
across distributed computing resources
system@sns
DANSE and the NSF
• Leverage the DOE BES investment in SNS hardware to
do innovative science (often done under NSF grants)
• Software has strong connections to NSF emphasis on
- scientific principles
- education
• Initiate a partnership between a university-based
developer community and a national facility
• Communication and science community outreach is
central to DANSE – should promote NSF-DOE
interactions too
• DANSE, both project and product, is extensible to other
facilities and disciplines
DANSE and the SNS
• DANSE project augments SNS expertise and resources
• Early DANSE deliverables track instrument commissioning
at the SNS
• DANSE project will facilitate new science from the SNS,
especially in the early years of operations
• Good communication channels between SNS and DANSE
• Decoupling of software development efforts
- DANSE emphasis on advanced data analysis (modeling and simulation)
- SNS emphasis on data services (acquisition and curation)
• Engineering consistency: SQRL consults for the SNS,
subcontract with DANSE
Number of Instruments
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
2006
DANSE Project
HYSPEC (BL-14B)
Vision (BL-16B)
Construction
NOMAD (BL-1B)
Commissioning
TOPAZ (BL-12)
Initial Users
Spin Echo (BL-15)
General Users
Fundamental Physics (BL-13)
Sequoia (BL-17)
SNAP (BL-3)
Vulcan (BL-7)
CNCS (BL-5)
ARCS (BL-18)
Powder Diffractometer (BL-11A)
SANS (BL-6)
Liquids Reflectometer (BL-4B)
Magnetism Reflectometer (BL-4A)
Backscattering Spectrometer (BL-2)
2007
2008
2009
2010
Calendar Year
2011
2012
2013
2014
DANSE Effort Today (Personnel)
•
All senior personnel together since early 2003
•
Professional staff
-
scientific and technical leadership
assessment of what is practical to deliver
infrastructure, support tools, institutional memory
•
Postdoc Software Developers
•
Graduate Students
•
Undergraduate Students
•
External collaborators
-
mostly physical scientists interested in software
varying software skill okay, but cannot start at zero
must sustain their scientific research
transitions out of DANSE/ARCS projects have been successful
can leverage DANSE in thesis research
testing and some development
mostly in computer science
DANSE Effort Today (Technical)
•
Scientific capabilities defined, packages selected
•
ARCS inelastic software in production, pyre framework
•
Reflectometry software in production, moving to framework
•
Software development process is established
•
Common components for I/O, interface, and numerics
gives economies of scale, reusability, uniformity,
and fewer bugs
(over 4 years, 12 workshops, 6 polls, and former CED project)
DANSE Effort Today (Management )
•
Earned value management established with a Project Baseline
•
Change, risk, configuration control plans are ready
•
Infrastructure tools identified (svn, trac, build/release, gui)
•
Development processes are specified
•
Project management in place
Work Breakdown Structure
•
Think of Microsoft Project
•
Layout of all tasks. Each task has:
- description in a dictionary file
- estimate of time
- estimate of cost
- markers of task progress
• WBS maintained by Mike McKerns, Project Manager
•
Changes are expected, but must be documented.
Earned Value Management System
•
Beyond Microsoft Project, becoming a federal requirement
•
Monthly reports of progress (Earned Value)
- use markers of task progress
•
Monthly reports of expenditures (Actual Costs)
- estimates of time and cost
•
Actual Costs and Earned Value are in units of dollars
•
EVMS maintained by B. Thibadeau at SNS
Integration of:
Design Process, QA, Releases
•
•
•
State purpose of component, specifications, functional tests
Object-oriented programming allows early selection of design
patterns
Review, commit to svn
•
•
Prototype. Review the design and costs
Build to specifications, revise specs, develop certification tests
•
Deploy to all target systems
Release Milestones.
For applications, occurs
about annually.
Good time for a Review.
Features of the Development Process
•
Misperceptions of specifications are the biggest source of
risk in software projects.
•
DANSE developers write specs, so communication
problems are minimized.
•
Code inspections and QA practices further minimize risk.
•
Agile commercial practices are appropriate for a scientific
development team.
•
Good specifications and requirements can facilitate
collaborations.
Education
• Pre-service teacher education at ISU
- develop K-12 lesson plans from subprojects
- adoption by young teachers
• Nanoscience program at MSU
• Student involvement
- minority student funds
- testers of software and documentation
- some in computer science
• New role for documentation (texts by Fultz, Billinge, Ustundag)
Broader Impact
• Algorithm Development Moves to Higher Levels
- flexible integration of DANSE components and applications
- subproject science packages
• DANSE should be Useful for Other Groups
- other neutron facilities
- synchrotron radiation research
- education programs in materials/chemistry/condensed matter
• Demonstrate a New Approach to Scientific Software
Development
- professional standards with O-O design
- distributed team, now cohesive
- some interest from computational scientists
Summary of DANSE
• The road from neutron data to neutron science is paved by
computing.
• DANSE system moves data analysis at the a higher level of
abstraction where new science is found.
• Integration of Design, Development, QA, Release
Management.
• Plan, key personnel, technical and management systems are
in place for construction of DANSE. Now funded by NSF
IMR-MIP program 2006-2011.
For this Kickoff Meeting
• DANSE Subproject Teams
- Status reports
- Technical discussion
• Collaborators
- Look over DANSE status and plans
- We want to hear updates from you too
• Visitors to this Kickoff Meeting
- Do we have common interests?
- Keep in touch!
End Presentation