•ISO 26262 (automotive) up to ASIL level D
•IEC 61508 (general industrial) up to SIL 4
•EN 50128 (railways) up to SW-SIL 4
•IEC 62304 (medical devices) up to level C
•IEC 60880 (nuclear power)
Brief Overview:
Company, Software Products & Methods
Dr. Evgueni Kolossov, R&D Director
Second HiPEAC Industry Partner Program
Tallinn, 8 October 2013
www.programmingresearch.com
© Programming Research
PRQA Overview (www.programmingresearch.com)
Over 25 years track record with focus on static analysis C / C++, defect prevention,
coding standards compliance and effective code reviews
Products:
Locations
Partners
Distributors:
Germany
Japan
China
Korea
Services:
Code audits
Coding standards development
Training - languages, standards,
products
Custom integrations – compiler,
IDE, VCS
UK: Hersham (HQ)
US: Boston + San Jose
India: Bangalore
Ireland: Dublin
Netherlands: Zeist
Ukraine: Lviv
Romania: Bucharest
© Programming Research
Technical:
Standing in the Software Community
ISO C
• Committee (BSI Sector) Member
ISO C++
• Committee voting Members
MISRA C (Motor Industry Software Reliability
Association)
• Committee Founding Member
MISRA C++
• Committee Founding Member
© Programming Research
Multiple Stakeholders
Stakeholders extend through the organisation:
o Developers & Project Leads, Test operations, QA managers,
Senior VP management, Customers, Suppliers
Project Devs/Leads
Is my project
meeting compliance
on each release?
Corporate VPs
Which projects
present the
greatest risk?
QA Manager
Is our overall software
quality improving
over time?
Customers
How does this
release compare
to the last one?
Many organisations handle this in an ad-hoc manner today...
© Programming Research
PRQA Server
Central Analysis
PRQA Server
Central Analysis
PRQA
Server
Central
Analysis
Distributed
development
/outsourcing
Local Analysis
Build
Server
OEM/Customer
Summary Information
and Reports
PRQA Server
Central Analysis
PRQA Server
Central Analysis
In-house Development
© Programming Research
Management Policy
Input and control
Supervisory and
management control
Local Analysis
Build
Server
Summary Analysis
results
PRQA Global Solution
PRQA Enterprise Solution
Build
Server
Local Analysis
Local Analysis
Build
Server
Local Analysis
Build
Server
© Programming Research
Local Analysis
PRQA
Server
Central
Analysis
PRQA Professional Solution
Build Server
Build Server
PRQA
Server
Central
Analysis
Build Server
© Programming Research
Components
© Programming Research
Static Analysis Types
Bug Catchers
Lint-Like
• Inexpensive
• Limited Analysis
Capability
• High False Positive
and False Negative
rates
• Strong on Simulation
• Strong on whole program
test verification
• Multi-Language support
• Often part of Swiss Army
knife solution: bundled with
testing tools
• High false negative rates.
• Poor Language Usage,
Portability and Preventative
analysis
© Programming Research
Automatic Code
Inspection
• Strong on 4 technology types:
pattern-based, simulation,
metrics, and comprehension.
• Facilitates code review and
pretest checking with code
collaboration, sophisticated
suppression management and
measurement analysis
• Low False Positive and False
Negative rates.
• Weak on multi-language
support
• Weak on whole program static
test
What & How we are Analyzing?
- Pattern-Based Analysis – This is the scanning of source code
PatternBased
Simulation
Metrics
Structure
and checking for patterns that indicate issues correlating (within
various degrees of severity) to defects in software. This involves
checks for quality characteristics (or lack thereof) in Portability,
Style, Language Usage, and Preventative Practices
- Simulation (Deep Data Flow) is a technique in identifying likely
Run-time Defects (otherwise known as “Bugs” during execution).
It is a form of testing without actually executing the code
- Metrics – It has often been stated that if you can’t measure it, you
can’t improve it. For example, since complex code has been highly
correlated to buggy code and impacts the testability and
maintainability of software, metrics that measure these aspects can
provide great insight into the quality of the code and act as an
indicator to where concentrated improvement needs to be made.
- Comprehension – Static Analysis of code can also provide many
different views upon the actual structure of the code in forms of
graphs and diagrams that help in comprehension, helping with
architectural decisions using tools that provide meaningful
abstractions
© Programming Research
Language Misuse

PRQA is industry leader in analysis of defensive coding
practices





Not all bugs are dataflow or resource usage based


JSF++ (Joint Strike Fighter - Bjarne Stroustrup)
MISRA C++
MISRA C
HICPP (our company standard from 2003, new version 3 October
2013)
Incorrect language usage can result in hard (expensive) to detect
bugs
Our software extends the defensive language analysis to
provide for language based bug checking:


Calling an implicitly defined member function, where sister function
has been explicitly declared.
Heap object of derived type undergoing derived to base conversion
without a virtual destructor.
© Programming Research
Resource Misuse




Acquired resources are tracked to ensure that they are
released.
Checking is not limited to memory
 create, open, close, fopen, fclose, strdup, dup
Analysis uses Syntax Usage Engine; tracking is
performed by inter-function analysis within the translation
unit.
Special handling of constructors and destructors allows
for checking that resources allocated in a constructor are
freed in a destructor.
© Programming Research
Deep-flow Dataflow

Dataflow analysis provides a mechanism to detect
serious runtime behaviour problems:






Buffer overflows (security)
NULL pointer dereference
Undefined mathematical operations
Use of unset variables
much more...
Results are accurate and precise due to in-depth modelling of
the language combined with a state of the art Satisfiability
Modulo Theories (SMT Yices 2) solver.
(Dutertre, B., de Moura, L.: A fast Linear-Arithmetic solver for
DPLL(T). In: Ball, T., Jones, R. B. (eds.) Computer Aided
Verification. LNCS, vol. 4144, pp. 81--94. Springer, Heidelberg
(2006))
© Programming Research
Deep-flow Dataflow (Continue)

Analysis is performed across function boundaries within
a translation unit.



Software highlights obvious defects where cause and effect are
localized, or where project wide knowledge is required to
determine there are no issues.
Inter variable dependencies tracked ensuring low false
positives/negatives
Tracking of values referred to by pointers increases both
depth of analysis as well as improving modelling through
function boundaries.
© Programming Research
Screenshots
© Programming Research
Screenshots
© Programming Research
Screenshots
© Programming Research
General Description




First and foremost we must parse the code correctly: Requires a fully
functional C and C++ pre-processor and parser.
Initial analysis takes place during parsing where code is checked for
conformance to the respective language standard. Both parsers are
written to conform to the standards, and deviations from "legal code" is
highlighted with a message and in some cases it is then controlled under
a configuration option. The default behaviour is to comply to the
standard.
The parser builds an internal Abstract Syntax Tree for the source code
and then the rest of the analysis takes place.
Most of the analysis in QA C++ (and some in QA C) is performed in the
reverse order of the call tree, ie. 'leaf functions' are analysed before their
callers. This allows QA C++ to use information about a called function
during the analysis of the caller. This is especially important for 'dataflow'
and is a core requirement for inter function analysis within the translation
unit.
© Programming Research
General Description (Continue)




A benefit of using our own parser technology is that we are not limited in
the information that can be used for analysis. Macro history, instantiation
history etc. that can normally be dropped by the compiler when
generating code can be kept around. This aspect will again be used for
some new C++ '11 checking, for example of the 'auto' keyword.
A common dataflow engine is used by QA C and QA C++. QA C++
translates C++ constructs into an equivalent C representation and this is
then passed into the dataflow engine.
The resulting flow graph is further simplified and SMT solver is then used
to search for defects.
As part of analysis, a semantic representation is also produced and this
is then checked during Cross Module Analysis (CMA). A significant
amount of undefined behaviour goes undetected by most of the available
linkers, for example different function declarations etc. CMA performs
this checking.
© Programming Research
General Description (Continue)
Areas with limited support:
• The output format used for the semantic representation is verbose,
and in the case of C++ can result in huge amounts of information
being written (and therefore being read). As projects have
increased in size, and with libraries such as boost, this is becoming
more of an issue. We're working to change the output format to
improve this situation.
• Once this change is made we should be able to widen the scope of
our existing analysis and provide new richer analysis for the entire
program.
© Programming Research
What we are looking for?
• Collaboration in areas:
– Architectural analysis of our dataflow with the target for
implementation:
•
•
•
•
Interprocedural Dataflow Analysis
Security Issues Analysis
Multi-threading Issues
Parallel Processing Issues
– Information about compilers new features & switches
– Timing Analysis (executable, run-time)
– New methods in code parsing, etc.
• Types of Collaboration: different types are
available – subject for discussions
© Programming Research
Questions?
Evgueni_Kolossov@programmingresearch.com
© Programming Research
Customer Case study
“QA·C is above other tools
when it comes to coding
standard compliance”
Site Software Director
© Programming Research
Customer case study
“Since we began using the MISRA C
Compliance Module, the quality and
consistency of our first generation code
has skyrocketed, and our final products
have been virtually error-free”
Stuart Jobbins, Head of Software
Development
© Programming Research
Customer Case study
“QA·C++ provides an efficient,
robust, fully automated
environment to introduce and
enforce coding standards”
Benjamin Pitzer, Senior Research
Engineer
© Programming Research
Customer Case study
“With QA·C our prototypes retain much of
their integrity, because most defects are
caught early... even as our goals evolve and
the code changes, our product quality
remains consistently high.”
Dana Sawyer, Senior Software Designer
© Programming Research
Customer Case study
“For us, it’s critical to focus on
detecting issues early and tools
like QA·C will help there”
Samir Kulkarni, Head of
Productivity
© Programming Research
Independent research on Static Analysis
Tools / MISRA Compliance by TERA-Labs
Reports
every
violation
Fails to
report
true
violation
All False
Positives
No False
Positives
Dr Marijn Temmerman from TERA-Labs observed, “On paper all the
selected tools claimed to provide comprehensive MISRA C
compliance checking - but the reality was different!”
© Programming Research