A Review/Analysis
of
Smartmatic-TIM Corp.’s “Mock Elections Summary Report”
Dated August 3, 2012
on the Mock Elections Conducted on July 24-25, 2012 at the
House of Representative’s Hearing of the Committee on
Suffrage & Electoral Reforms (CSER)
By: transparentelections.org.ph
transparentelections.org.ph
Four positions were audited
during the Mock Elections ...
• President
• Senators
• Party List
• House Member
Total
– 55(?) candidates
– 55
– 55
– 32
- 197 candidates
transparentelections.org.ph
Summary of Key Facts for the Audit, as presented on page 10 of
Smartmatic’s Report:
Total number of ballots used
1,000
Total number of ballots scanned
958
Total number of ballots rejected
42
Number of ballots with confusing marks
9
Number of positions audited
Total number of marks counted by the PCOS
(for positions subject to audit)
Total number of marks counted manually
(for positions subject to audit)
Variance
Percentage of Match Between Manual and
Electronic Count
transparentelections.org.ph
4
8,295
8,402
107
99.98710%
Summary of Key Facts for the Audit, as presented on page 10 of
Smartmatic’s Report:
Total number of ballots used
1,000
Total number of ballots scanned
958
Total number of ballots rejected
42
Number of ballots with confusing marks
9
Number of positions audited
Total number of marks counted by the PCOS
(for positions subject to audit)
Total number of marks counted manually
(for positions subject to audit)
Variance is
Percentage of Match Between Manual and
Electronic Count is
transparentelections.org.ph
4
8,295
8,402
We say
107
231
99.98710%
42.13198%
Further ...
… we say that Smartmatic’s error
rate is 557 times the TOR*-defined
requirement of 1 error in 20,000
marks!
*TOR – Terms of Reference in the 2010 ComelecSmartmatic-TIM contract
transparentelections.org.ph
Variance
To illustrate how they computed the Variance, let us use the results for
Presidential candidates #25 and #26. In this example, the Smartmatic
interpretation will say the variance is 0.
No.
Candidates
Manual
Count
PCOS
Count
Manual
Count –
PCOS
Count
25 Holliday, Darren (LP)
10
13
-3
26 Honnet, Guillaume (SJS)
18
15
3
28
28
0
Total
Using the logic of Smartmatic, then the accuracy rate is an
illogical 100%!!!!
The Smartmatic method covers up electronic Dagdag-Bawas!
transparentelections.org.ph
Variance
We all know, however, that the correct way for computing
the total variance is to use the absolute values of the
variances per candidate.
No.
Candidates
Manual
Count
PCOS
Count
Manual
Count –
PCOS
Count
Absolute
Value of
Manual
Count – PCOS
Count
25 Holliday, Darren (LP)
10
13
-3
3
26 Honnet, Guillaume (SJS)
18
15
3
3
28
28
0
6
Total
Yet, Smartmatic says that theirs is the “more
accurate way”. Huh?!
transparentelections.org.ph
Variance
Smartmatic says:
VARIANCE = 107
But using the absolute values of the variances for
all 197 candidates ...
… we get:
VARIANCE = 231
Smartmatic is therefore WRONG!!!
transparentelections.org.ph
Percentage of Match
Smartmatic says:
We say:
Percentage of
Match is
99.98710%
Percentage of
Match is
42.13198%
transparentelections.org.ph
Percentage of Match
Smartmatic says:
Total number of marks counted by the
PCOS (for positions subject to audit)
Variance
Percentage of Match Between Manual
and PCOS Counts
8,295
107
99.98710%
No known mathematical formula could result in their claim of
99.98710%, so we tried to speculate on how Smartmatic
might have arrived at the figure.
transparentelections.org.ph
Percentage of Match
This formula might have been used:
Percentage of Match = 100(?) – (Variance/PCOS Count)
so that,
Percentage of Match = 100 – (107 / 8,295)
Percentage of Match = 100 – 0.012899
Percentage of Match = 99.98710
Strangely, the percent symbol (%) was appended to the
computed result, so that:
Percentage of Match = 99.98710%
transparentelections.org.ph
Percentage of Match
Or perhaps this formula:
Percentage of Match = 1 – (Variance/PCOS Count)
Percentage of Match = 1 – (107 / 8,295)
Percentage of Match = 1 – 0.012899
Percentage of Match = 0.98710
Then prefix the result with “99” and append it with %,
Percentage of Match = 99.98710%
Weird? We agree!
transparentelections.org.ph
Percentage of Match
So, what should the Percentage of Match be?
The Percentage of Match is the ratio of the
(no. of candidates where the Manual Count
and the PCOS Count matched) over the (total
no. of candidates), expressed as a percentage.
Matched
Count
Unmatched
Count
transparentelections.org.ph
Percentage of Match
Based on the following results of the Manual Count and PCOS
Count (per candidate):
• Total no. of candidates in all positions audited = 197
• No. of candidates whose Manual Count and PCOS Count
matched = 83
• No. of candidates whose Manual Count and PCOS Count DID
not match = 114
Percentage of Match should therefore be
Percentage of Match = 83/197 x 100%
Percentage of Match = 0.4213198 x 100%
Percentage of Match = 42.13198%
Smartmatic is again WRONG!!!
transparentelections.org.ph
Accuracy
COMELEC, in its Request for Proposal for the
automation of the 2010 National and Local
Elections (NLE), required an Accuracy Rate of
99.995%, which means that only 1 error in
20,000 marks is allowed.
transparentelections.org.ph
Accuracy
Accuracy Rate = (1 – Variance/PCOS Count) x 100%
The correct Variance, as we have determined, is 231
The PCOS count presented by Smartmatic is 8,295
Therefore,
Accuracy Rate = (1 – 231/8295) x 100%
Accuracy Rate = (1 – .0278481) x 100%
Accuracy Rate = 97.21519%
transparentelections.org.ph
Accuracy
The Accuracy Rate of 97.21519% translates to
an error rate that is 557* times the TORdefined requirement of 1 error in 20,000 marks!
*(1-0.9721519) x (20,000) = 556.962
transparentelections.org.ph
Questions for ...
Smartmatic
• Why did they measure the performance of PCOS during the
Mock Elections in terms of Percentage of Match when there
is no mention at all of this term in the TOR; it should have
been Accuracy Rate.
• Smartmatic should explain their formulas for computing
Variance and Percentage of Match.
• Smartmatic should explain why they shouldn’t be
disqualified as a Comelec vendor when they failed dismally
to meet the accuracy rate requirement.
transparentelections.org.ph
Questions for ...
COMELEC
• Why is Comelec so fixated on Smartmatic, when the latter
can’t even meet the Comelec-defined Accuracy Rate
requirement of 99.995%?
• Shouldn’t the Comelec disqualify Smartmatic as a vendor,
not only for falling short of the requirements, but more
grievous, for trying to deceive the Comelec by using
incorrect and improper formulas to hide their failures?
• Comelec should release to the IT community the detailed
working papers of PPCRV in the conduct of the Random
Manual Audit in 2010 so that its failure, like Smartmatic’s,
can also be unmasked.
transparentelections.org.ph
Questions for ...
The Public
• Now that the results of the Mock Elections have been
explained, what does the public think? Was it
incompetence, or deception?
• Mukhang ginagago na tayo. Hanggang kailan ba natin
papayagan ito?
transparentelections.org.ph
Maraming salamat po.
transparentelections.org.ph