Assuming Software Maintenance of a Large,
Embedded Legacy System from the Original
Developer
by
William L. Miller
Lawerence B. Compton
Bruce L. Woodmansee
1
The Mission
What: Transfer maitenance of an embedded
application consisting of ~200,000 executable lines of
Ada code, running on three processors from the OEM
to an organization that was in the process of being
created to support mission critical systems’ software
maintenance
Why: An econimic study indicated that organic
maintenance, even including start-up costs, would save
quite a bit of money.
How: The United States Government enlisted the
support of Georgia Tech Research Institute to guide the
transfer from the OEM.
2
Project Phases
Optimistic
anticipation
Fear and
regret
Relief, feeling of
acomplishment
3
The Problems
There was no organic team in place
There were no organic software processes in place (i.e.,
development, configuration management or quality assurance)
All of the OEM’s development systems and tools were past endof-life and no longer supported by vendors (some vendors had
ceased to exist)
Custom test equipment could not be repaired due to
obsolesence of many components
Documentation was incomplete and in some cases, inaccurate.
There was a challenge in the relationship between the
Government organization assuming the maintenance and the
OEM
4
Overall Transition Strategy
Develop processes that enabled the software engineering team
to use the current, OEM supplied source code, to build a binary
image that compared bit-for-bit to the binary image that was
generated by the OEM for the supplied source code
Validate the processes at Georgia Tech Research Institute
Build a United States Government team
Transfer the processes from Georgia Tech Research Institute to
the United States Government team and train the team
Validate that the Government team can successfully execute all
the the processes
5
Specific Issues to Overcome
in Building the System
A compiler that was no longer
supported – who was the vendor?
A host software development
system that was no longer in
production
Incomplete software build
instructions by the OEM
Obsolete test equipment
6
Overcoming Issues
in Building the System
Located the vendor for the compiler,
obtained a licence
Purchased the host software development
system on eBay
Reverse engineered the build process by
incrementally matching compiler outputs
for files to received binary load modules
Reverse engineer the existing test
equipment, creating a new design with
available components
Define a formal software development
process
7
The Software Architecture
Perfect Storm
Static Analysis of the code:
10% of the packages had afferent
and efferent coupling indices > 50,
with some as high as 200
15% of the packages had cyclomatic
complexities > 100
Many procedures / functions
consisted of more than 1000
executable lines of code
8
Specific Issues to Overcome
in Maintaining the System
The system was poorly architected
Incomplete software documentation
Delivered documentation files could not be modified
(PDF vs. Word)
9
Overcoming Issues
in Maintaining the System
To compensate for the poor architecture, the Georgia Tech
Research Institute (GTRI) technical team is larger and more
senior than an applicaiton this size would justify.
The GTRI team spent ~ 1 year reviewing code and documentation, reverse
engineering missing or inaccurate documentation
The GTRI team spent ~ 1 year implementing and testing notional changes
to the software
During year 2, the United States government team integrated with the
GTRI team and jointly implemented changes
Manually convert 18,000 pages of software documentation from
PDF to Word – documents were heavily table-oriented
A full release cycle is currently in process under GTRI technical
guidance
10
Validating the Software
Development Process
Follow the formal process to build a
system based upon unmodified source
code
Confirm that a bit-for-bit compare between the
OEM delivered load module matches the newly
generated load module
Audit all deliverable documents Word vs. PDF
Follow formal test procedures (~7,000
pages) using legacy test equipment to
establish a performance baseline
Follow formal test procedures using newly
developed test equipment
Confirm that test results are identical
11
Is this Transition Scenario
Unique
Not really, many issues identified could exist for any long-lived
software product
Internal teams and key team members can depart
A company and it’s products may be purchased by another
company
12
Lessons Learned
The most important lesson learned for a mission critical system is that
transition of software maintenance begins during intial development and
deployment
Plan for system updates and component obsolesence
Do not assume that the business relationships will always be cooperative
Documentation
Insure that every document and artifact at every release is correct and modifiable
Component obsolesence – this is not necessarily an organizational transition
issue for software maintenance
Frequently assess the availability / supportability of key components
Software build auditing
Frequently the software should be completely compiled and built without the
development engineers, using release documentation
13