CTIS 494
Quality Assurance and Reliability
by
Dr. Ozlem Albayrak
2007 Fall
by Ozlem Albayrak
Introduction to CTIS494
Course Elements:
- Students
- Instructor
- Course Material
by Ozlem Albayrak
Instructor
Ozlem Albayrak
–
–
–
–
Computer Eng. Bilkent, 1992
M.B.A., Bilkent, 1994
M.S., UMCP, 1998
Phd., Ankara, 2002
Room# 212
Phone: 290 5039
E-mail: ozlemal@bilkent.edu.tr
by Ozlem Albayrak
The Students





? Attendance
Get to Know
Why are they here?
Expectations, Interests
This is an ELECTIVE course, may require
more...
by Ozlem Albayrak
Office Hour
Tuesday: 14:40-15:30
Thursday: 8:40-10:30
e-mail or
appointment
by Ozlem Albayrak
CTIS 494 Outline (1)
Week
Subject
17.09 Introduction to Software Quality Assurance
24.09 Software Inspections and Testing
01.10 Capability Maturity Model
08.10 ISO9000 and SPICE (1 hour missing)
15.10 Software Quality Metrics
22.10 Seven Basic (B7) Tools of Quality
29.10 The Seven MP Tools
05.11 MIDTERM (30%)
by Ozlem Albayrak
CTIS 494 Outline (2)
12.11 5S for Intelligent Housekeeping in Software Development
19.11 Software QFD and Voice of the Customer
26.11 Risk Assessment and Failure Modes and Effects Analysis
03.12 Quality Measures and Statistical Methods for Trustworthy Software
10.12 Verification and Validation
(PROJECT REPORTs Submission 15%)
17.12 Testing and Evaluation (1 hour missing)
24.12 Integration, Extension and Maintenance for Trustworthiness
(PROJECT PRESENTATIONs 20%)
31.12 Review (2 hours missing)
7.12 FINAL (35 %)
by Ozlem Albayrak
Today ...
Week
Subject
17.09 Introduction
Introduction to Software
QualityQuality
AssuranceAssurance
to Software
24.09 Software Inspections and Testing
01.10 Capability Maturity Model
08.10 ISO9000 and SPICE (1 hour missing)
15.10 Software Quality Metrics
22.10 Seven Basic (B7) Tools of Quality
29.10 The Seven MP Tools
05.11 MIDTERM (30%)
by Ozlem Albayrak
Course Material
Not limited to:
 The main text book
 References
 All quality related high quality sources
by Ozlem Albayrak
References (1)



“Quality Assurance for Information Systems” by
W. E. Perry, QED Technical Publishing Group
1991.
“Statistical Quality Design and Control” by R.
E. DeVor, T. Chang, J.W. Sutherland
Macmilllan, 1992.
“Software Quality Assurance Reliability and
Testing” Chris Summers 1987.
by Ozlem Albayrak
References (2)


“Handbook of Software Quality Assurance”, G.
G. Shulmeyer, CDP J. I. McManus 1987.
“Total Quality” J.W. Dean, Jr. J. R. Evans West
1994.
by Ozlem Albayrak
Grading



1 Midterm
1 project
1 Final
30%
35%
35%
by Ozlem Albayrak
(II. Parts: Closed and Open book)
(Group of 4)
(Open book)
Course Goal:

Software Quality Assurance and
Reliability
This course provides students with the
analytical and management tools necessary
to solve software quality problems and
implement effective quality systems.
by Ozlem Albayrak
Topics include:









voice of the customer analysis,
the Six Sigma problem solving methodology,
process capability analysis,
measurement system analysis,
design of experiments,
statistical process control,
failure made and effects analysis,
quality function development, and
reliability.
by Ozlem Albayrak
What is Quality?
by Ozlem Albayrak
Concentration on:
SOFTWARE
QUALITY ASSURANCE
by Ozlem Albayrak
Requirements



Come to class prepared (search related
subjects)
Involve in in-class-case studies or exercises
Team work is important part of the course
by Ozlem Albayrak
Project

Details are written in the handouts

You will evaluate your clasmates studies

by Ozlem Albayrak
CTIS494 PROJECT DETAILS



35% of the total grade is from the projects
(same weight as final)
The course project is a group project each of
which will be developed by 4 students.
Project subjects will be announced by the
instructor. Project requires groups to submit a
report and deliver a in-class presentation.
Reports will be submitted 2 weeks before the
presentations.
by Ozlem Albayrak
Project Report specifications




Project reports will be prepared as web pages that can
be accessed by all the students and the istructor.
The groups are free to design their web pages format,
but the generated pages should be readable and
understandable.
Navigation among the pages should be carefully
organized.
The quality of the web pages will be determined by is
content, format as well as its timely distribution. Late
submissions will be heavily penalized.
by Ozlem Albayrak
Project Presentations





Project study will be presented in classroom.
Following the presentations there will be a question-answer
session during which the instructor and the other students are
allowed to ask questions.
Those students who prepare good questions will be awarded by
the instructor.
Fluency of the presentation is as important as its content, thus the
groups are urged to prepare their presentations in advance to
deliver a high quality presentation.
Duration of the presentations will be announced by the instructor.
Using time effectively is an important element.
by Ozlem Albayrak
Project Evaluation


20% of the project grade is allocated for project
presentation and 15% is for the project report.
15% includes evaluation of the following items:
–
–
–

Content
Format and ease and fluency of navigation
Timely delivery
20% includes evaluation of the following items:
–
–
–
–
Content
Fluency of the presentation
Obedience to time limits
Answers to questions
by Ozlem Albayrak
Quality exercise






Define high quality toilette paper?
Single (5 min)
In groups of two (5 min)
Reviews.......
Hw # : form a group of four
One address for the class and registration to
the group.... Do not wait for invitation register
by Ozlem Albayrak
What is quality?
by Ozlem Albayrak
Different Perspectives on Quality



Beauty is in the eye of the beholder
For the same course one may say it is highly
theoretical, while others state that it is highly
practical.
There are many different definitions and
dimensions of Quality
by Ozlem Albayrak
Quality Dimensions





Transcendent
Product based
User based
Manufacturing based
Value based
by Ozlem Albayrak
Transcendent

Quality is something that is intiutively
understood but nearly impossible to
communicate, such as beauty and love.
I like it, because I just like it....!!!
by Ozlem Albayrak
Product based

Quality is found in the components and
attributes of a product.

I like it, because the product has ....
by Ozlem Albayrak
User based

If the customer is satisfied, the product has
good quality.

I am satisfied with this product.
by Ozlem Albayrak
Manufacturing based

If the product conforms to design
specifications, it has good quality.

The product is of xxx standards.
by Ozlem Albayrak
Value based

If the product is perceived as providing good
value for the price, it has good quality.

This is the best one I can buy at this price.
by Ozlem Albayrak
Garvin’s 8 quality dimensions
Performance
Durability
Serviceability
Features
Reliability
Aesthetics
Conformance
Percieved
quality
by Ozlem Albayrak
Performance

Efficiency with which a product achieves its
intended purpose
by Ozlem Albayrak
Features

Attributes of a product that supplement the
product basic performance
by Ozlem Albayrak
Reliability

Propensity (eğilim) for a product to perform
consistently over its useful design life
by Ozlem Albayrak
Conformance

Is a particular dimension of a product is within
the allowable range of tolerance of the
specification, the product conforms.
by Ozlem Albayrak
Durability

The degree to which a product tolerates stress
or trauma without failing
by Ozlem Albayrak
Serviceability

Ease of repair for a product
by Ozlem Albayrak
Aesthetics

Subjective sensory characteristics such as
taste, feel, sound, look and smell.
by Ozlem Albayrak
Perceived Quality

As the customer perceives the quality.
QUALITY
by Ozlem Albayrak
Service Quality dimensions




More difficult to define than to define product
quality
Tangibles
Service reliability
Assurance
by Ozlem Albayrak
Differing Functional Perspectives
on Quality






Engineering
Operations
Strategic Management
Marketing
Financial
Human Resources
by Ozlem Albayrak
Engineering

Engineering is an applied science.
–
–

Product Design Engineering
Concurrent Engineering (simultaneous
performance)
Statistics + technical orientation
by Ozlem Albayrak
Design Life Cycle
Idea Generation
Preliminary Design
Iterations
Prototype Development
Final Definition
Product Design Evaluation
Implementation
by Ozlem Albayrak
Operations


Focus was initially on statistical process control
Uses system view underlying modern quality
management thinking
by Ozlem Albayrak
Strategic Management



Refers to the planning processes
Long term orientation
Goal: to achieve sustainable competititve
advantage
by Ozlem Albayrak
Generic
Strategic Planning Process (1)
Firm Mission and Goals
External analysis
Strategic Options
Business level strategy
Corporate level strategy
Operational Subplans
by Ozlem Albayrak
Internal analysis
Generic
Strategic Planning Process (2)
Operational Subplans
Organizational
Design
Conflict Politics and
Change
Strategic Alignment between
Structure and Goals
by Ozlem Albayrak
Organizational
Reward System
Marketing

Customer satisfaction
by Ozlem Albayrak
Financial

Risk of investment vs. Potential rewards
by Ozlem Albayrak
Deming Value Chain
Improve quality
Cost decrease
Productivity Improves
Capture the market
Stay in business
Provide Jobs and more jobs
by Ozlem Albayrak
Human Resources



Employee empowerment
Job analysis
Total quality human resources management
by Ozlem Albayrak
The Three Spheres of Quality
Quality
Management
Quality
Assurance
Quality
Control
by Ozlem Albayrak
Quality
Control
Quality Control

Scientific method includes:
–
–
–
Analysis ( a process is broken into fundamental
pieces)
Relation (relationships between parts)
Generalization (how interrelationships apply to the
larger phenomenon of quality being studied)
by Ozlem Albayrak
Quality Control Activities






Monitoring process capability and stability
Measuring process performance
Reducing process variability
Optimizing processes to nominal measures
Performing acceptance sampling
Developing and maintaining control charts
by Ozlem Albayrak
Quality
Assurance
Quality Assurance

Activities associated with guaranteering the
quality of a product or service.
by Ozlem Albayrak
Quality Assurance Activities







Failure mode and effect analysis
Concurrent Engineering
Experimental Design
Process improvement
Design team formation management
Off-line experimentation
Reliability/durability product testing
by Ozlem Albayrak
Quality
Management
Quality Management

Management processes that overarch and tie
toegther the control and assurance activities
make up quality management.
by Ozlem Albayrak
Quality Management Activities






Planning for quality improvement
Creating a quality organizational culture
Providing leadership and support
Providing training and re-training
Designing a system to reinforce quality ideals
Facilitating organizational communication
by Ozlem Albayrak
Other perspectives


Value Added Perspective
Cultural Perspectives
by Ozlem Albayrak
Contingency Theory

There is no theory or method for operating a
business that can be applied in all instances.
This provides flexibility to managers in pursuing
quality.
by Ozlem Albayrak
Summary


Different perspectives on quality management
Quality sphere is composed of:
–
–
–

Quality Control
Quality Assurance
Quality Management
Discussion Questions
by Ozlem Albayrak
Quality Example





Define high quality toilette paper?
Single (5 min)
In groups of two (5 min)
Reviews.......
One address for the class and registration to
the group.... Do not wait for invitation register
by Ozlem Albayrak
CTIS 494
Software Quality Assurance
by
Dr. Ozlem Albayrak
2007 Fall
by Ozlem Albayrak
A Practical Approach to Software
Quality
Book by
G. O’Regan
Slides by
Dr. Ozlem Albayrak
2006 Spring
by Ozlem Albayrak
Outline






Software Engineering Challenge
History of Software Failures
Background to Software Quality
History of Quality
Software Engineering
Modern Software Quality Management
by Ozlem Albayrak
Software Engineering Challenge

Challenge is to deliver:
High quality software on time to the customer

Focus on customer satisfaction
by Ozlem Albayrak
Software Engineering Challenge




Overestimation
How to estimate cost and schedule accurately
is a big challenge
Risk management is key
Example: Taurus project at London Stock
Exchange was 13,200% late and costed a lot,
and abondened
by Ozlem Albayrak
Software Engineering Challenge



Changing requirements
In civil engineering there are also problems
Software eng. Less mature than civil eng.
by Ozlem Albayrak
History of Software Failures

Y2K problem
–
–
–

Find and analyze all code with a Y2K impact
Plan and make necessary changes
Cost to UK is $32 billion!!!
Software quality needs to be a key driving
factor
by Ozlem Albayrak
Introduction to Software Quality(2)

History of Quality
–
–
–
–
–
Shewart
Deming
Juran
Crosby
Other Quality Gurus
by Ozlem Albayrak
Background to Software Quality



Customers have high expectations
In the market there are many high quality
companies
Late delivery leads to extra cost
by Ozlem Albayrak
Background to Software Quality (2)

What is software quality
–
“Conformance to the requirements” Crosby
“Fitness for use” Juran
–
ISO 9126 standard for information technology
–
by Ozlem Albayrak
ISO 9126 – Quality Characteristics






Functionality: extent to which functions are avaliable
Reliability: extent to which software reliable
Usability: users say easy to use
Efficiency
Maintainability: easy to maintain
Portability: easy to transfer do different environments
by Ozlem Albayrak
Total Quality Management

Modern approach to quality management is
TQM
–
–
–
Customer focus
Process improvement
Development of quality culture
by Ozlem Albayrak
Software Quality Control





Extensive inspections and testing
Formal reviews
Aim is to identify defects
White box and black box tests
Unit, function, subsystem, system,
performance, integration, acceptance tests
by Ozlem Albayrak
History of Quality

Shewhart
–
–
Statistian at AT&T, 1920s
PDCA Model




Plan
Do
Check
Act
by Ozlem Albayrak
History of Quality (2)

Deming
–
–
–
1945s, Japan
14 step programme
Consistent purpose and direction in the organization
is more important then everyone doing his best
alone
by Ozlem Albayrak
History of Quality (3)

Juran
–
–
–

Another giant in the quality movement
Fitness for use
10 step programme
Crosby
–
Has influenced CMM (Capability Maturity Model)
by Ozlem Albayrak
Other Quality Gurus



Taguchi
Ishikawa
Feigenbaum
by Ozlem Albayrak
Introduction to Software Quality(3)

Software Engineering
–
–
–
Requirements Analysis
Specification
Design




–
Architectural
Functional
Object-oriented reuse
Verification of Design
Implementation
by Ozlem Albayrak
Introduction to Software Quality(4)
–
–
Implementation
Testing






–
Unit
Integration
Sub-system
System
Performance
Acceptance
Maintenance
by Ozlem Albayrak
Maintenance


1st Malcom Baldridge Quality award to Motorola
Steps of Six Sigma:
–
–
–
–
–
–
Identify product or service
Identify customer and requirements
Identify needs to satisfy customer
Define process of doing work
Eliminate waste
Ensure continious improvement by:

Measuring, analyzing and controlling improved process
by Ozlem Albayrak
Modern
Software Quality Management




Software Inspections
Software Testing
Software Quality Assurance
Problem Solving
–
–
–
Fishbone diagrams
Histograms
Pareto Chart
- Trend Graph
- Scatter Graphs
- Failure Mode Effect Analysis
by Ozlem Albayrak
Modern
Software Quality Management(2)






Modeling
Cost of Quality
Metrics
Customer Satisfation
Assessments
Total Quality Management
by Ozlem Albayrak
Metrics

Measurement is important part of science and
engineering

Goal Question Metric by Basili

Data, Charts, Analysis, Trends, Action
by Ozlem Albayrak
Customer Satisfaction
Customer Saticfaction
Surveys
Customer Saticfaction
Measurement
Improvement
Planning
Execute Plan
by Ozlem Albayrak
Formal Methods

Advantage:
–
–
Utilization of mathematical models are being used
Lead to more robust(gürbüz) systems
by Ozlem Albayrak
Modern
Software Quality Management




Future of Software Quality
Organizational Culture and Change
Formal Methods
Quality and the Web
by Ozlem Albayrak
Quality and the WEB
Accesss from everywhere in the world
 May different browsers
 Usability, look and feel important
 Performance as key concept
 Security
 Many transactions at any time
 Avaliability (24x365)
 Relaibility
JAD/RAD with spiral development suggested

by Ozlem Albayrak
End of Musical notes

by Ozlem Albayrak


Review
Next class:
Search web for software quality assurance
Come to class to discuss
by Ozlem Albayrak