MathScope Handbook - Calc/Comp Arithmetic
Page 1 of 7
CALCULATOR AND COMPUTER ARITHMETIC
Click here to return to the index
1 Introduction
Most of the computational work you will do as an Engineering Student will be performed either
on a calculator or by means of a computer program. This section is intended to illustrate the
ways in which calculators and computers store numbers and perform arithmetic operations.
It is not intended as a guide to the use of such devices.
2 Decimal Systems - Base 10
In everyday life information are represented in the decimal system using the digits
0,1,2,3,4,5,6,7,8,9 and the base 10
3
2
1
0
-1
-2
2156.12 = (2x10 ) + (1x10 ) + (5x10 ) + (6x10 ) + (1x10 ) + (2x10 )
e.g.
Notice how the position of a digit within the number indicates the power of 10 by which it is
multiplied.
Here is another example:
4
3
2
1
0
-1
-2
32696.486 = (3x10 ) + (2x10 ) + (6x10 ) + (9x10 ) + (6x10 ) + (4x10 ) + (8x10 ) +
(6x10 )
-3
Having understood this representation it is clear that we can use any positive integer as the
base.
As you know, when using a calculator you key in numbers in the usual decimal system and
the answers are displayed in the decimal system.
However, the machine will actually store the numbers using a different base. Depending on
the type of machine the base will be either 2 or 16.
3 Binary System - Base 2
The digits are 0, 1
3
2
1
0
-1
-2
-3
E.g.
1011.101 = (1x2 ) + (0x2 ) + (1x2 ) + (1x2 ) + (1x2 ) + (0x2 ) + (1x2 )
The fact that only two digits are required was the attribute of the binary system, which was so
useful in the development of machines for storing numbers and doing arithmetic, since
‘switch off 0’ and ‘switch on 1’ provided an obvious code.
In decimal arithmetic the above number is: 8 + 0 + 2 + 1 + 1/2 + 0/4 + 1/8 = 11.625
1
MathScope Handbook - Calc/Comp Arithmetic
4
Page 2 of 7
Hexadecimal System - Base 16
Some modern machines use this system in which the digits are
0,1,2,3,4,5,6,7,8,9,A, B, C, D, E, F
In decimal terms A=10, B=11,C=12,D=13,E=14,F=15
(
) (
) (
) (
) (
) (
5 AD2 4C = 5 × 163 + A × 16 2 + D × 161 + 2 × 160 + 4 × 16 1 + C × 16 2
)
which, as a decimal number is : 5x4096 + 10x256 + 13x16 + 2 + 4/16 + 12/256 =
23250.296875
Example
As an example, the decimal number 330.125 can also be expressed
as: Binary:
Hexadecimal:
101001010.001
14A.2
To check this
8
6
3
-3
101001010.001 = 2 + 2 + 2 + 2 + 2
= 256 + 64 + 8 + 2 + 0.125
= 330.125
2
-1
14A.2 = 16 + 4x16 + 10 + 2x16
= 256 + 64 + 10 + 0.125
= 330.125
5 Conversion Between Bases
As the example above shows, to convert from binary or hexadecimal to decimal is a simple
computation.
The algorithm for performing the reverse operation is illustrated in the following example.
2
MathScope Handbook - Calc/Comp Arithmetic
Page 3 of 7
Example
Decimal number 330.125
Conversion to binary:
Integer part
Divide by the new base repeatedly
330
= 165,
2
165
= 82 ,
2
82
= 41 ,
2
41
= 20 ,
2
20
= 10 ,
2
10
= 5,
2
5
= 2,
2
2
= 1,
2
1
= 0,
2
The integer part is then
remainder
R1 = 0
remainder
R2 = 1
remainder
R3 = 0
remainder
R4 = 1
remainder
R5 = 0
remainder
R6 = 0
remainder
R7 = 1
remainder
R8 = 0
remainder
R9 = 1
R9 R8 R7 … R1
= 101001010
Fraction Part
Multiply repeatedly by the new base until the fractional part becomes zero (It may not
do in which case stop the process when enough accuracy is achieved)
0.125x2 = 0.25 = 0+0.25 = I1+F1
F1x2 = 0.25x2 = 0.5 = 0+0.5 = I2+F2
F2x2 = 0.5x2 = 1.0 = 1+0 = I3+F3
The fraction is then
Thus
I1, I2, I3 = 0.001
330.125
101001010.001
3
MathScope Handbook - Calc/Comp Arithmetic
Page 4 of 7
Conversion to hexadecimal:
Integer part
Divide by 16 repeatedly
330/16 = 20 remainder R1 = A
20/16
= 1 remainder R2 = 4
1/16
= 0
remainder
R3 = 1
The integer part is: R3 R2 R1 = 14A
Fraction part
Multiply repeatedly by 16 stopping as described above
0.125 x 16 = 2.0 = 2 + 0.0 = I1 + F1
The fraction part is 0.2 and so 300.125 = 14A.2
6 Addition and Multiplication in Binary Arithmetic
The basic rules are:
0+0 = 0, 1+0 = 0+1 = 1, 1+1 = 10
0x0 = 0, 1x0 = 0x1 = 0, 1x1 = 1
Example
Addition:
1011.101 +
101.1011
---------------10001.0101
Multiplication:
1011.101 x 101.1011
= 0.1011101 +
1.011101 +
101.1101
+
1011.101
+
101110.1
-------------------------1000011.0001111
The rules for hexadecimal arithmetic are omitted.
4
MathScope Handbook - Calc/Comp Arithmetic
Page 5 of 7
7 Normal Form and Floating Point Arithmetic
Even though your calculator or computer will be storing numbers and operating on them in
non-decimal form this is not visible to you as the user. You input data in decimal form and the
machine gives you the answer in decimal form.
In the following description of how computing machines store and operate on numbers we
can without any loss of validity ignore the conversion process and we shall use the decimal
system in all examples.
A computing machine has only a finite ‘space’ in which to store a number. If you compute the
quantity 2/300 the exact answer is 0.006666.... recurring. This is beyond the storage capacity
of any machine.
Your computer would actually store this as three pieces of information, which we call the
normal form of the number. In this case
-2
+ 0.66666667 (10 )
(The actual number of ‘sixes’ stored would be machine dependent, typically equivalent to
about 12 decimal digits).
The normal form of a number then consists of three parts:(sign) (mantissa) (bn)
sign :-
plus or minus
mantissa :-
a fraction with a fixed number of digits
b
:-
the arithmetic base (2 or 16 in a machine, 10 in our examples)
n
:-
an integer, positive or negative with a maximum value which is
machine dependent.
The integer n is chosen so that
1
b
mantissa < 1
Example
With b=10 we have
0.1 mantissa < 1
so that 245.673 in normal form
3
4
2
is + 0.245673 (10 ) but not + 0.0245673 (10 ) or +2.45673 (10 )
Computer arithmetic is called Floating Point Arithmetic. In essence it is as follows
5
MathScope Handbook - Calc/Comp Arithmetic
Page 6 of 7
Addition/Subtraction:
To add/subtract two numbers: Adjust the value of n (exponent) of
the smaller number to be equal to the exponent of the larger number. Add/subtract the
numbers. Adjust (round off) the mantissa to fit the available number of digits.
Multiplication/Division: Multiply/divide the two mantissas. Combine the exponents. Adjust
and round off the product mantissa.
Addition (Five digit mantissa)
2
-1
+ 0.41263 (10 ) + 0.63211 (10 )
2
2
=
+ 0.41263 (10 ) + 0.00063211 (10 )
=
+ 0.41326211 (10 )
=
+ 0.41326 (10 )
2
2
Multiplication (Five digit mantissa)
2
-1
[+ 0.41263 (10 )] x [0.63211 (10 )]
1
=
+ 0.2608275493 (10 )
=
+ 0.26083 (10 )
1
As can be seen in the above examples, each of the final answers is subject to a small error
due to the restriction on the number of digits in the mantissa. In some calculations the
accumulated effect (accumulated rounding error) can be quite serious.
Example
Consider the simple computation
10000
3
10000
÷ 0 0001
3 0001
The exact value is 1111.07....
We will now perform the computation as a hypothetical
calculator with a 5 digit mantissa would
10000
= 0 33333(10 4 )
3
10000
= 0 33332(10 4 )
3 0001
10000 10000
÷ 0.0001 = 1000
3
3 0001
An error of over 10%
All calculators and computers have rather better than 5digit mantissa but the example
nevertheless illustrates possible pitfalls.
6
MathScope Handbook - Calc/Comp Arithmetic
Page 7 of 7
Click here to return to the main document
7