Issues on Software Testing for
Safety-Critical Real-Time
Automation Systems
Shahdat Hossain
Troy Mockenhaupt
Introduction
• Increasing complexity
• Reliability depends on many factors
• Systematic and effect testing are highly
necessary
• Full avoidance of human mistakes is not
realistic
• Thorough testing can ensure software
quality
Introduction(cont.)
• Testing as an inverse to design
• Present expected results
• Separate development and testing teams
• Regression testing at every development
modification
• Concentrate on error-prone program
segments
Functional Software Testing
Strategies
• Big bang testing
• Incremental testing
• Black Box testing
• White Box testing
Black Box Testing
• Functional testing of the program
• Anticipated functionality and desired
performance
• Not complete software testing
• Representative test cases
• Boundary value conditions
• Error guessing
White Box Testing
• Inner structure
• Cases are specified for exercising
specific internal structure
• Examine as many logic paths as possible
• Logic conditions should be tested at
least once
• Should be combined with black box
testing
Software Testing Processes and
Steps
• Unit Testing
• Integration Testing
• Validation Testing
• System Testing
Unit Testing
• White Box test cases
• Inspecting outputs
• Examine basic software unit
• Test each module after it is programmed
• Module Interface
• Crucial execution paths
• Exception handling
• Local data structure
• Boundary value testing
Integration Testing
• Testing of the subsystem
• Black box testing
• One module at a time
• Top-Down and Bottom-Up approaches
Top-Down Integration Testing
• Starts with Root module
• Subordinate modules are substituted by
stubs
• Subsystems are creating by replacing
stubs
• Regression testing
• Verify all modules have been included in
the subsystem
Bottom-Up Integration Testing
• Lower level modules first tested
• Parallel testing
• Driver module uses stub
• Driver module is replaced to form
subsystem
• Finished when root node is reached
Validation Testing
• Occurs after integration testing
• Validate that the software fulfills user
requirements
• Uses black box testing
• Software defects report
• Standards defined in the SRS
System Testing
• Occurs after validation testing
• Works as a component of the overall
system
• Objective is to find discrepancies
•
•
between actual performance and
expected performance
Based on requirements analysis
description
"On-site" testing
Non-Functional Software
Performance Testing
• Complex software
systems
• The process of
•
validation and
verification
requires a larger
amount of testing
cases.
Embedded Real
Time Applications
Availability Testing
• Meeting the time constraints.
• Missing the deadline for responding to an
operation is deemed as a failure as it
may incur system malfunction.
Reliability Testing
• Low reliability system may cause
damage.
• It is highly concerned with hardware and
other support software
Survivability Testing
• Fault tolerant capability
• Quick recovery capability
• Specific fault tolerant control algorithms
required
Flexibility Testing
• Multiple requirements and OS
• 3rd party drivers and components
• Self reconfiguration
Stress Testing
• Stress testing is to test the software by
pushing the system to its limits.
Security Testing
• System security during all the phases of
software life cycle.
• Virus and safety-critical automation
software systems
Usability Testing
• Comfortability with the user's experience
• GUI design
• Increase efficiency
• Reduce possiblity of invalid operations
Maintainability Testing
• The released software needs to be
revised and upgraded during its life
cycle, therefore, it is highly desirable
that the software can be easily
maintained.
Software Maintenance
• software maintenance in the entire
software life cycle
Corrective Maintenance
• After the software is installed on the
user sites, it is possible that during
system operations, the latent software
defects are exposed, and therefore
corresponding software revision is needed
to ensure the proper running of the
software
Adaptive Maintenance
• Its intent is to ascertain that the
released software can adapt to new
requirements, which are however not in
the previous design specification
Perfective Maintenance
• New technologies need to be
incorporated into the existing software
in order to improve its performance
Preventive Maintenance
• The final type of software maintenance
•
•
is preventive maintenance. It involves
making changes to the software that
makes future maintenance activities
easier to carry out.
Check for Software Update frequently
Ability to automatically insall latest
patches
Conclusion
• Ensuring the software’s compliance with
the user requirements
• Software testing is indispensable in
safety critical systems
• All functional and non functional testings
needs to be implemented thoroughly