Software Engineering Research Center
An NSF Industry - University Cooperative Research Center
Aditya Mathur, Purdue University
Director
Robert Cowan, West Virginia University
Assistant Director
Fire invented
Outline for Presentation




The SERC Organization
- History of I/UCRCs and NSF mission
- Our Affiliates and the Administration
- Industry and University Partnerships
Current Research Projects
Technology Transfer Examples
Virtual SERC (VSERC)
National Science Foundation
Promoting the Progress of Science

Support research and education in science
and engineering
- Foster the interchange of scientific information among
scientists and engineers in the United States and foreign
countries
- Technology Transfer via industry/university partnerships
Industry/University Cooperative Research Centers
(IUCRCs)



Developed by NSF in 1970’s
50+ centers nation-wide
- SERC: I/UCRC for Software Engineering
True partnership between industry and
university researchers
Secrets of Success
The SERC IUCRC
Established: 1986
Purdue University
 Ball State University
University of Florida
 University of West Florida
Added in 1994
Oregon Associated Universities
 University of Oregon
 Portland State University
 Oregon State University
 Oregon Graduate Institute
Added in 1998 West Virginia University
Tenured Faculty
Current Affiliates












Army Research Lab: Atlanta,GA
AveStar Inc.: Fairmont, WV
Telcordia: Morristown, NJ
British Telecom Labs: Martlesham Heath, England
Dynamix Inc.: Eugene, OR
ManTech: Fairmont, WV
Motorola, Inc.: Schaumburg, IL
NASA IV&V: Fairmont, WV
Northern Telecom: Richardson, TX
Northrop Grumman: Melbourne, FL
Northrop Grumman: Rolling Meadows, IL
Tivoli Systems, Inc: Indianapolis, IN
Examples of Research Projects









Metrics,Tools for Measurement of Software Design
Quality
Methods for Assessing and Assuring Data Quality
Testing Distributed Systems
Testing Video Games
Evaluation of ROI on V&V Activities
Software Process Improvement Modeling and Tools
Computer-Based Instructional Technologies
Test Coverage Analysis and Test Optimization
Software Reuse
Industry and University Network
Companies
Universities
Army Research Lab
Bellcore
British Telecom Labs
Dynamix
ManTech
Motorola, Inc.
Nortel
Northrop Grumman, IL
Northrop Grumman, FL
Tivoli Systems, Inc., IN
Oregon Associated Schools
Partnership
Purdue University
Ball State University
University of Florida
University of West Florida
West Virginia University
provide
is composed of
Industrial Advisory Board
Collaboration
Researchers
produce
is available to
Technology and Software
Permission (2)
.
SERC Administration
SERC Policy Board
National Science Foundation
Industrial Advisory Board
Director: Aditya Mathur
Assistant Director: Robert Cowan
Site Director: Stuart Faulk
Site Director: Aditya Mathur
Site Director: Steve Thebaut
University of Oregon
Purdue University
University of Florida
Portland State University
Ball State University
University of West Florida
Site Director: Ali Mili
West Virginia University
Oregon State University
Oregon Graduate Institute
Sharing Facilities
Benefits to Universities

Source of fresh research problems

access to industrially generated data and technology

fast turn-around on proposals

publishable research

satisfaction of seeing research used in industry

potential for additional funding from other NSF programs
The Lab After Hours
Benefits to Industry

Influence selection of research projects

exploit innovations that may lead to new products

improve software quality and productivity of software engineers

new software engineering technology….access to expertise

use of unique laboratories, facilities

satisfy corporate commitment to support university research

interaction with peers

technical reports

on-site short courses and seminars

summer interns, potential future employees
SERC Funding




National Science Foundation
Participating Universities
Industrial Affiliates
- support faculty and graduate student projects
Membership Fees
- $30,000 per year full membership
- $5,000 per year limited membership (for small businesses only)
Learning the Rules
Semi-Annual SERC Showcase






Poster sessions
Researcher presentations
IAB member presentations
Potential company presentations
Software demonstrations
IAB and SERC Admin. meetings
.
Industrial Request For Proposals: IRFPs
IAB
SERC Staff
interact with
to generate
IRFPs
Broadcast IRFPs to SERC Universities
Purdue+
Florida+
Oregon+
West Virginia+
Broadcast Proposals from Researchers to Industrial Affiliates
Army Research Lab
Bellcore
British Telecom Labs
Dynamix
Motorola
Nortel
Northrop Grumman, IL
Northrop Grumman, FL
Tivoli Systems, Inc.
A Lab Mentor and
Software Evolution
Jens Palsberg
Purdue University
Problem: When requirements change, can a new software version be evolved
automatically and efficiently?
Approach: Genetic programming.
Rationale: With genetic programming we can evolve programs by means of
natural selection. To go from one software version to the next, it may be
much faster to start with the previous version instead of starting from scratch.
Status: In our current genetic programming system, medium-sized imageprocessing software can be evolved from scratch.
Goals: Quantitative assessment of how well genetic programming can evolve
new versions of software. Guidelines for conducting the evolution process.
Study applications of the technology to SERC affiliate projects, beyond image
processing.
Data Quality
Ahmed Elmagarmid
Purdue University
Problem: Approaching data quality in a more comprehensive way than what is
currently being done. Existing scrubbing tools fail to realize that data is a
complex entity that is replicated, dynamic, and tightly coupled to the business
processes.
Approach: (a) Define a framework for data quality: Including dimensions,
requirements, policies, and a taxonomy for dealing with and improving data
quality, and, (b) Develop a methodology that deals properly with the issues of
initial cleanup, continuous cleanup, and process improvements. The
methodology must be comprehensive, implementable, and measurable.
Goals: Data quality has been a major cause of customer dissatisfaction, lost
business, and inability to meet business objectives. While quality has a
clearer meaning when it comes to software, it is much more poorly defined
when it comes to data. A generally acceptable and operational definition of
data quality refers to Accuracy, Consistency, Currency, and Completeness of
data. The specific goal of this project is to develop a methodology that deals
with all aspects of data quality.
Software Maintenance:Spotlighting the Code
Norman Wilde
University of West Florida
Description: “Where in this program is functionality X implemented?”
Method: Use test cases as probes to locate functionalities
Example: Where is call forwarding implemented?
 run small set of test cases that involve forwarding a call
 run set that does not involve the call
 analyze traces to look for program components that were executed in first
set and not second set
Goal:
 Spotlight code fragments
 Tech Transfer: to make Software Recon an affiliate productivity aid on
affiliate projects
Tech Transfer Examples: Norman Wilde
Bellcore
Maintenance of OO Programs
1991 - 1993






Bellcore concerned about the maintainability of OOPs, requested SERC
research on this topic
Over two summers, UWF conducted interviews w/Bellcore programmers and
performed study of OO tools in use at Bellcore
Study identified maintenance issues and tool needs for OOPs - earliest
research on this topic
Results used in defining Bellcore’s programming environment
Bellcore/UWF jointly authored papers in “IEEE Transactions on SE” and
“IEEE Software”
UWF student hired by Bellcore to continue work
Norman Wilde: Technology Transfer (cont.)
Northrop Grumman (RM)
Software Recon tool
1995 - current

trial of Software Recon conducted at RM on a NG in-house C language code
management system

several tool improvements made based on feedback from NG programmers

C Software Recon tool now in use

NG request Ada Recon tool, now under develop.
Norman Wilde: Technology Transfer (cont.)
Northrop Grumman (FL)
Recovery of Design Threads
1995 - 1997

NG software engineers suggest Recon could be used to recover “design
threads” from JSTARS, large battle management system

collaborate on a trial, which successfully recovered threads

joint authored paper to appear in “Journal of Systems and Software”
Design Metrics
Dolores and Wayne Zage
Ball State University
Description: During the software design process, developers should be able to
infer more about the software they are developing. Computing metrics
allows one to choose the best design, as well as identify stress points that
may lead to difficulty during coding and maintenance. (Currently developing
OO design metrics.)
Method:
 Use metrics De and Di and D(G) to identify error-prone modules in a
software design.
 Use DB (design balance) and DC (design connectivity) to gain a better
understanding of a developing system.
 Various strategies have been employed by current affiliates to correct or
monitor troublesome modules.
Goal:
 Improve software development processes and products.
Technology Transfer Example: Zages
Bellcore, Northrop Grumman, Army Research Lab



NG started by using the metrics’ ability to predict error-prone modules in a
missile defense system.
Since then, incorporated DM into software development methodology as NG
strives to go from Level 3 to Level 4. Calculating the Zages’ metrics is
included in the set of requirements for new projects.
DMAA and DMAC now in use at affiliate sites.
Ball State Research Team
DM
DM
DM
SEI Level 4
DM
Northrop Grumman
Technology Transfer Example: Zages
Bellcore, Northrop Grumman, Army Research Lab


At Bellcore: DM have been excellent predictors of stress points in
telecommunications software
DM technology moving from Bellcore research to Bellcore development
groups

Bellcore and BSU joint papers

BSU DM team student interns at NGC and Bellcore in Summer 1997
Metrics Directed Verification of SDL Designs
Dolores and Wayne Zage
Ball State University
Motorola
Goal: Apply DM technology to identify stress points in SDL designs and assess
the utility and effectiveness of design metrics on such designs.
Method: Map DM primitives to SDL design artifacts. Calculate metrics and and
determine if stress points are error-prone.
Predicting the Performance of Software Process Improvements
David Raffo, Portland State University
Description: Previous research: Process Tradeoff Method (PTA) predicts the impact
of process improvements on:
 development cost
 product quality
 task schedule
Current Project:
 Further develop these methods by applying the techniques to address a
specific question: What is the impact of implementing the Personal Software
Process (PSP) in a real world development setting?
Goal:
 Accommodate new processes, process changes, and project environments
 Develop graphical and quantitative process models which can generally be
used to predict the impact of implementing CMM KPAs and other process
changes
Analysis of Software Process Models and Metrics:
Moving to Levels 4 and 5 of the CMM
David Raffo & Warren Harrison
Portland State University
Description: Develop innovative application of TQM principles to software
project management using software process models and metrics.
Approach: Develop a suite of software process and product metrics along with
software process models which together can be used in an integrated
framework to support project management and process planning decisions.
Goal: Develop a systematic approach to help companies achieve the
Quantitative Process Management, Software Quality Management, and the
Continuous Process Improvement KPAs of the CMM.
Estimating Software Reliability During Conceptual Design
Bruce D’Ambrosio
Oregon State University
Description: Traditional reliability estimation methods depend on data from
product testing. However, many software technologies are evolving so
rapidly that failure rates of existing systems may not provide a good
indication of reliability of systems under design and development, even in
the early, conceptual design phase.
Approach:
 Develop models of software reliability based on expert judgement
combined with statistical information. Exploit advances in Bayesian
probability theory to develop software reliability models which can
combine expert estimates of factors influencing reliability with statistical
information.
Goal:

Develop methods and tools for predicting the reliability of complex
software systems during early design.
Computer-aided Testing for Reusable Software Components
Mark Yang
University of Florida
Description: Reuse has been widely discussed and promoted in the software
industry, but many aspects of current practice has to be modified in order for
reuse to be practical. For example, the change of reliability when a
component is used in a new environment has not been well investigated.
Without some guidelines, we may not have the confidence to reuse a
component. How should the component be re-tested, and if necessary be
modified?
(cont.)
Computer-aided Testing for Reusable Software Components
(cont.)
Mark Yang
University of Florida
Approach:
 To study reliability estimation methods under a changing environment
 To figure out the number of new test cases needed in order to meet the
reliability requirement in a new environment
 To build a tool that takes the old testing results, usage records, new
environments, and new reliability requirement as inputs and outputs the
number of new test cases
 To assess software reliability estimation methods from the reuse point of
view. Which one is the best choice for reuse?
Goal: To build a tool that can certify reusable components based on past
testing history and present requirements
Testing Distributed Systems
Aditya Mathur and Sudipto Ghosh
Purdue University
Description:
 Testing distributed systems composed of software components for tolerance
to component failures, security and correctness of implementation.
 Developing interface based fault injection methods between components and
metrics in order to assess the adequacy of testing.
Method:
 Identify the source of errors and failures in a distributed system
 Develop and validate metrics to assess test adequacy
 Study the behavior of system under component failure
(cont.)
Testing Distributed Systems
(cont.)
Aditya Mathur and Sudipto Ghosh
Purdue University
Goal:




Develop a methodology to assess the tolerance of a distributed
system to failures occurring in one or more of its components
Develop and assess software fault injection testing methodology
for distributed systems
Develop metrics for the assessment of adequacy of tests of
distributed systems
Develop a prototype assessment tool for experimentation
Technology Transfer Examples:
Aditya Mathur and Sudipto Ghosh





Bellcore
Fault Injection Testing
1994-1997
Bellcore explored the possibility of using fault injection testing
technology for software systems
Study identified research issues involved in fault injection testing
and tool design for fault injection testing
Over 3 years Purdue and Bellcore conducted studies on the
application of the fault injection testing technique on Bellcore’s
systems
Bellcore/Purdue jointly authored paper in Quality Week 97,
Europe
Purdue student hired as a summer intern by Bellcore
Architecture Based Estimation of Software Reliability
Aditya Mathur, Pietro Michielan, Manuela Schiona
Purdue University
Description:




Traditional methods for software reliability assessment treat a
software system as a “whole” and are applicable very late in the
development cycle
We have proposed CBRE (Component-Based Reliability Estimation)
to overcome the problems associated with traditional methods.
CBRE can also be applied to obtain early estimates of system
reliability, i.e., prior to code availability.
The current focus is on



(cont.)
comparing CBRE with Laprie-Kanoun method of component-based
reliability estimation
evolving models of inter- and intra-component dependencies,
determining ways of applying CBRE in early stages of development.
Architecture Based Estimation of Software Reliability
(cont.)
Aditya Mathur, Pietro Michielan, Manuela Schiona
Purdue University
Collaboration:

This project is supported in part by a tie-project grant from the
NSF. The tie is with Professor Kishor Trivedi from the Center for
Advanced Communications at Duke University.

Other collaborators include:



Dr. Jose Maldonado, University of Sao Paulo, Brazil.
Professor Nozer Singpurwalla, George Washington University
Dr. Alberto Pasquini of ENEA, Rome, Italy
“Endless Frontier, Limited Resources”*

Industry must increase its contribution to the U.S. R&D enterprise.

Industry must overcome private sector barriers to partnerships.

Industrial R&D must grow.

Industry must focus its research priorities.

Industry must take timely advantage of the leading-edge results
coming out of U.S. universities and government labs.
*http://nii.nist.gov/pubs/coc_rd/rd_cover.html
Educational Activities

Re-training Courses

10-week, 400-hour hand-on course in Software Engineering offered to 30
employees of Raytheon during summer of 1998.

Specialized Course Modules

1-week modules in Design, Testing, WEB-site Construction and
Management, Java, and other languages.

Master in Software Engineering

Oregon Associated Universities (http://www.omse.org/)