Linear Sequential Machines
1
Zvi Kohavi and Niraj K. Jha
Introduction
A linear sequential machine is a network which has a finite number of input
and output terminals and is composed of interconnections of
three types of basic components (unit delays, modulo-p adders
and modulo-p scalar multipliers)
• The input signals are elements of a finite field GF(p) = {0,1,…,p-1}
• The operations performed by the basic components on their inputs are
carried out according to the rules of GF(p)
Block diagram: number of delay elements -- dimension
x1
x2
xl
z1
z2
zm
Adders
and
multipliers
y1
Y1
y2
Y2
yk
Yk
2
Linear Machine
For a machine to be linear: its response to a linear combination of inputs
must preserve the scale factor and the principle of superposition
•
•
•
Thus, each basic component must be linear
Cannot use an AND gate: z = x1x2 modulo 2
Or an OR gate: z = x1 + x2 + x1x2 modulo 2
Linear components:
1. Unit delays: y(t) = Y(t-1)
2. Modulo-p adders: z = x1 + x2 + … + xl (modulo p)
3. Modulo-p scalar multipliers: z = cx (modulo p), where c is an element of
GF(p)
Y(t)
y(t) = Y(t--1)
x1
x2
+
xl
(a) Unit delay.
x1 + x2 + … + xl
(modulo p)
(b) Modulo-p adder.
x
c
cx (modulo p)
(c) Modulo-p scalar multiplier.
3
Example
Example: A four-terminal, four-dimensional linear machine over GF(3)
2
x1
+
y1
+
y2
+
y3
+
2
y4
+
z1
z2
x2
Linear machine over GF(2): binary machine
Inert (or quiescent) machine: a linear machine whose delay elements are
initially in the zero state
• Used as encoding and decoding devices
4
Feedforward Shift Registers
Simplest type of inert linear machines: two-terminal shift register which
contains only feedforward paths and whose output is a modulo-p
sum of selected input digits
x
Yk
a1
ak-1
ak
yk
+
Y2
y2
+
a0
Y1
y1
+
z
• z = a0x + a1Dx + … + akDkx
– D0 = 1: identity operator
– z = D2x: for all t >= 2, z(t) = x(t-2)
– Initial conditions: y1(0) = y2(0) = … = yk(0) = 0
• Rewrite equation as: z = (a0 + a1D + … + akDk)x, or
– T(D): transfer function
5
Example
Example: Consider the inert linear machine over GF(2) below where z(t) =
x(t) + x(t-1) + x(t-3)
•
•
Corresponding polynomial: z = x + Dx + D3x
Transfer function:
x
+
+
z
6
Serial/Parallel Connection of Machines
Consider two linear machines with input x1 (x2), output z1 (z2) and
transfer function T1 (T2)
•
Transfer function T3 of the serial connection:
–
•
Since x2 and z1 are identical:
Transfer function T4 of the parallel connection: T4 = T1 + T2
Example: Let T1 = D2 + 2D + 1 and T2 = D + 1 over GF(3)
•
•
Then T3 = (D2 + 2D + 1)(D + 1) = D3 + 1
T4 = (D2 + 2D + 1) + (D + 1) = D2 + 2
7
Impulse Response and Null Sequences
Impulse response h of an inert linear machine: its response to 100…0
•
Impulse response of the feedforward shift register: a0a1…ak0…0
x
Yk
•
•
a1
ak-1
ak
yk
+
Y2
y2
+
a0
Y1
y1
+
z
After at most k+1 time units: the output of the k-dimensional feedforward
shift register is a sequence of 0’s
Response of an inert linear machine to an arbitrary input sequence: can
be obtained from its impulse response through discrete “convolution” in
GF(p)
8
Example
Example: Impulse response of T1 = 1 + D + D3 is h = 110100…0
• Response of T1 to input sequence 1011: obtained by addition (modulo 2)
+ D2h + D3h of sequences as follows
h
9
Null Sequence
Null sequence X0: nonzero input sequence that generates an output
sequence consisting of 0’s
• Thus, TX0 is a sequence of 0’s
• Any linear combination of null sequences is also a null sequence
Example: Null sequence for T1 = 1 + D + D3 determined as follows:
0 = X0 + DX0 + D3X0
X0 = DX0 + D3X0
• Select an arbitrary (nonzero) sequence of length 3 (in general, equal to
dimension k) and specify subsequent digits
– Selection of 001
X0 = (001)1101001
– After seven digits: the null sequence repeats itself
10
Example
Example: Null sequence for polynomial T = 1 + 2D2 + D3 over GF(3)
0 = X0 + 2D2X0 + D3X0
Adding 2X0 to both sides and recalling that 2X0 + X0 = 0 in modulo 3:
2X0 = 2D2X0 + D3X0
Multiplying both sides by 2 yield:
X0 = D2X0 + 2D3X0
Starting with 111, null sequence:
X0 = (111)00202122102220010121120111
Maximal sequences: null sequences that contain pk-1 digits
• All possible k-tuples except 00…0
11
Inverse Machines
A polynomial T(D), where z = Tx, has an inverse, denoted by 1/T(D), if
there exists a network which realizes x = (1/T)z
Example: Machine and its inverse:
x
Yk
--ak
x
•
•
a1
ak-1
ak
yk
+
Y2
y2
+
a0
Y1
y1
z
+
+
+
+
--ak-1
--a1
1/a0
wi
z
w0
Note that (-a) modulo p = (p-a) modulo p
When the inverse machine is fed the impulse response of the original
machine, i.e., a0a1…ak00…0: its response is the original message
x = 100…0
12
Inverse Machines (Contd.)
Inverse is realizable only if a0  0
• An inert linear machine described by delay polynomial T has a linear
inverse described by T-1, which decodes without delay: if and only if T
contains a nonzero constant term which is prime to modulo p
• For a binary machine: a0 must be 1
• If T does not contain a nonzero constant term: no instantaneous inverse
can be found
– However, an inverse, which decodes the original input after a finite
delay, can be found
– Let ai be the scalar associated with the lowest power of D for which
ai  0, i.e.,
T = Di + ai+1Di+1 + … + akDk
– The inverse is given by:
– Or:
13
Example
Example: A machine with T1 = 1 + D + D3 and its inverse with
T1-1 = 1/(1+D+D3)
x
+
x
z
+
w0
+
+
wi
z
14
Linear Machines with Nonzero Initial
Conditions
The inverse of an inert linear machine might not be inert
• Hence, its response to a sequence of zero input symbols is not
necessarily a sequence of zero output symbols: but may be null sequence
X0, whose starting digits are determined by the initial state of the inverse
Example: Let the input symbols of T1 and T1-1 be 0’s
• If they are inert: their respective output symbols will be all 0’s
• If they are not inert: output symbols will depend on their initial states
• Autonomous behavior: behavior of a noninert linear machine whose input
is a sequence of 0’s
• Cycle sets: the two loops in the state diagram of T1-1
100
111
110
101
111
101
110
011
010
001
011
001
010
100
000
000
15
(a) T1 = 1 + D + D3.
(b) T1--1 = 1/(1 + D + D3).
Inert Linear Machines and Rational
Transfer Functions
Realization of rational polynomials: consider the inert linear machine
whose output z is given by
z = x + Dx + D2x + D4x + Dz + D3z
This can be rewritten as:
z(1 + D + D3) = x(1 + D + D2 + D4)
The transfer function:
Trivial, but inefficient, realization: Serially connect two inert linear machines
given by polynomials 1 + D + D2 + D4 and 1/(1 + D + D3)
• Requires seven delay elements
Minimal realization: determined by the highest degree in either the
numerator or denominator of the transfer function
• Only four delay elements necessary
16
Example (Contd.)
Example (contd.): Rewrite equation as
x + z = D(x + z) + D2x + D3z + D4x
Or
x + z = D{(x + z) + D{x + D(z + Dx)}}
Chain realization of T2 and T2-1
x
+
+
+
x+z
+
z
1 + D + D2 + D4 .
(a) Realization of T2 =
1 + D + D3
x
+
+
+
T2--1
1 + D + D3
.
=
1 + D + D2 + D4
x+z
+
z
(b) Realization of
17
Chain Realization
Chain realization of an arbitrary transfer function over GF(2):
x
x
k
+
k-1
2
+
+
1
x+z
+
z
x+z
z
Realization of
x
z
ak
ak-1
a1
a0
+
+
+
+
--bk
--bk-1
--b1
18
Example
Example: Realize the following transfer function over GF(3)
x
2
2
+
+
2
2
+
+
z
19
Impulse Response and Transfer Function
Synthesis of an inert linear machine from its impulse response:
• If the impulse response is realizable: it consists of two components –
transient component ht and periodic component hp
• For a k-dimensional inert linear machine: the period of the response to a
sequence of 0’s is at most pk-1 = n-1, where n is the number of states
• Thus, a necessary condition for an impulse response h to be realizable: it
ultimately becomes periodic
• Since the length of the transient response is at most k + 1: the transfer
function of a realizable two-terminal k-dimensional inert linear machine
can be specified uniquely by observing the first k + pk symbols in the
impulse response
20
Example
Example: Consider impulse response 1010100, 1110100, 1110100, … of
an inert linear machine over GF(2)
• The impulse response can be separated as h = ht + hp
Impulse: 10000000000000000...
Tp
h: 10101001110100111...
ht: 01000000000000000...
hp: 11101001110100111...
,
,
(a) Impulse response and
its components.
x
+
z
Tt
(b) T = Tp + Tt .
• From ht: Tt = D
• Periodic component hp described by: 1 + D + D2 + D4
• Since the period is 7:
Tp = (1 + D + D2 + D4)(1 + D7 + D14 + …) =
• Hence:
21
Multiterminal Machines
A multiterminal inert linear machine with l input terminals and m output
terminals can be characterized by a set of lm transfer functions:
•
Tij is evaluated when xi = 0 for all i  j
One possible realization (not necessarily minimal):
x1
T11
T12
T1m
T21
T22
T2m
x2
z1
+
z2
+
+
zm
xl
Tl1
Tl2
Tlm
22
The General Model
The matrix formulation: Consider a k-dimensional linear machine over GF(p),
with l inputs and m outputs
•
Next-state equation for delay Yi:
•
Matrix form:
•
Or
•
•
Y(t) = y(t+1) = Ay(t) + Bx(t)
A, B, C, D: characterizing matrices
A: characteristic matrix
23
The General Model (Contd.)
Output function zi:
•
Matrix form:
•
Or
z(t) = Cy(t) + Dx(t)
24
Definition
A machine is said to be linear over a finite field GF(p) if its states can be
identified with the elements of a vector space, and its next state
and output functions can be specified by a pair of matrix
equations GF(p):
Y(t) = Ay(t) + Bx(t)
z(t) = Cy(t) + Dx(t)
• Dimension of the machine: dimension of the state vector
• The machine is Moore or Mealy: depending on whether D is or is not
identically zero
Example: A four-terminal machine and its characterizing matrices
2
x1
x2
+
y1
+
y2
+
y3
+
2
y4
+
z1
z2
25
The Response of Linear Machines
Relationship between the input sequence to machine {A,B,C,D} and its
corresponding output sequence:
or
autonomous response
forced response
where
26
Autonomous Response
Internal circuit: that part of the circuit that can be specified by A alone
• It contains only the delay elements and their interconnections, while the input
and output lines have been deleted
Autonomous response: generally determined from the analysis of the internal
circuit
Y(t) = y(t+1) = Ay(t)
• Since the internal circuit is autonomous: the  -successor Sj of state Si
[where Si = yi(t)] is given by:
yj(t) = A yi(t)
where  denotes the number of state transitions
• The sequence of predecessors of a given state is established by constructing
the inverse internal circuit:
– Such an inverse exists: only if each state has a unique predecessor
– For an internal circuit given by A: the inverse is given by A-1
– Thus, the inverse circuit exists: if and only if A is nonsingular
27
Reduction of Linear Machines
Let L be a k-dimensional linear machine over GF(p)
• To describe an experiment of length k: compactly represent earlier
equations as
Z(k) = Kky(0) + VkX(k)
• Kk: diagnostic (or distinguishing) matrix
• For initial states Sa and Sb: corresponding state vectors are ya(0) and
yb(0)
28
Reduction of Linear Machines (Contd.)
If Sa is equivalent to Sb: then
Kkya(0) = Kkyb(0)
Since the second term VkX(k) is independent of the initial state
• To simplify the computation: X(k) may be selected as the all-zero sequence
X(k) = 0
• Thus, the output equation reduces to:
Z(k) = Kky(0)
Theorem: A k-dimensional linear machine {A,B,C,D} is definitely
diagnosable of order k if and only if diagnostic matric Kk has k
linearly independent rows
• A linear machine is in reduced form: if and only if the rank of Kk is k
• Every reduced k-dimensional linear machine is definitely diagnosable of
order k, and is finite memory of order less than or equal to k: observability
and predictability properties
29
Example
Example: Consider linear machine L1 over GF(2) given by
• Diagnostic matrix K3 is obtained and Z(3) becomes:
• Since the rank of K3 is 3: the dimension of L1 cannot be reduced
• For a given initial state: the values of y1(0), y2(0), y3(0) are specified
• Matrix Z(t) yields the response of L1 to distinguishing sequence 000
– E.g., if the initial state is (111): then in response to 000, sequences z1
= 010 and z2 = 100 are produced
30
The Minimization Procedure
Let L be a k-dimensional linear machine {A,B,C,D} over GF(p), and let r be
the rank of the diagnostic matrix, r < k
• Define an r x k matrix T consisting of the first r linearly independent rows
of Kk, and a k x r matrix R denoting the right inverse of T, s.t. TR = Ir
• Define an r-dimensional machine L* with characterizing matrices
{A*,B*,C*,D*}, s.t. A* = TAR, B* = TB, C* = CR, and D* = D
Theorem: State y of L is equivalent to state y* = Ty of L*. Machine L* is a
reduced machine equivalent to L
31
Example
Example: Consider linear machine L2 over GF(2), defined by
• The rank of K3 is 2: thus L2 is reducible
• The first two rows of K3 are linearly independent: therefore
• The right inverse R of T is constructed by selecting a set of r linearly
independent columns from T
– Since the rank of T is r and column rank equals row rank: such a set
always exists
– Form an r x r matrix Q from these columns and find its inverse Q-1
32
Example (Contd.)
Example (contd.): The right inverse R, which is a k x r matrix, is formed by
placing in it the rows of Q-1 in positions corresponding to the
columns selected from T, and where all other rows are set to zero
• In our case:
• From definitions of characterizing matrices of L2*:
33
Example (Contd.)
Circuit diagram of reduced machine L2*:
x
+
y2
+
y1
+
z
34
Example
Example: Consider linear machine L3 given by {A,B,C,D} over GF(2) and
shown in the figure
x2
+
x1
+
+
y1
+
y2
+
+
y3
+
+
z1
y4
+
+
z2
35
Example (Contd.)
Example (contd.):
• Q occupies the first three columns of T and Q-1 the first three rows of R,
since the linearly independent columns in T have been selected from
positions 1, 2, and 3
36
Example (Contd.)
Example (contd.):
37
Example (Contd.)
Example (contd.): The reduced circuit corresponding to {A*,B*,C*,D*} is:
x2
x1
z1
+
+
+
y1
+
y2
y3
+
+
+
z2
38
Example (Contd.)
The first three linearly independent rows of K3* are the rows of I3 in natural
order:
• Matrix (A*)t is related to At by:
(A*)t = TAtR
and diagnostic matrix K* is related to K by
K* = KR
• Thus, for initial state ya* = [y1*,y2*,y3*] and the all-0 input sequence: the
output corresponding to the unit vector rows of K3* are identical to values
y1*, y2*, y3*
39
Identification of Linear Machines
Let sequential machine M have pk states, denoted by Sa, Sb, .., Spk
• Let the l-dimensional vector x and m-dimensional vector z denote the
input and output vectors, respectively
• Construct a distinguishing table, which contains the output symbols
generated by M in response to a sequence of 0’s
• The table contains pk columns corresponding to the states of M
– It is formed block by block: the ith block corresponds to z(t) at t = i
– The table thus contains at most k blocks of m rows each:
corresponding to z(0), z(1), …, z(k-1)
– The process of adding blocks to the table is terminated: when, for
some t, the set of rows contained in block z(t) is linearly dependent on
the rows in preceding blocks
Example: Machine M4 and its distinguishing table U
40
First Test
First test: based on the fact that, for every linear machine, the all-0’s
sequence is a distinguishing sequence
• If M is reduced: the columns of U must be distinct, since otherwise there
would be two or more states in M which are indistinguishable under the
all-0’s sequence, and M is not linear
– The U in the example satisfies the test
Let U* be the table consisting of the first r linearly independent rows of U,
and let Si denote the ith column of U*
• Assuming that a linear realization of M is possible: let states A, B, … of M
correspond to ya, yb, … of its linear realization L
• This is accomplished: by selecting the pk columns of U* as the state
assignment for the pk states of L
• For machine L4, the linear realization of M4, we have
41
Identification (Contd.)
To obtain {A,B,C,D} of L: we select r linearly independent columns from
U*, corresponding to r state vectors of L, and form an r x r matrix
v s.t.
v = [ya,yb,…,yr]
Hence, the next-state function of L under 0 input symbols is:
[Ya0,Yb0,…,Yr0] = Av
where Yi0 denotes the 0-successor of yi. Since v is nonsingular:
A = [Ya0,Yb0,…,Yr0]v-1
If all r unit vectors appear in U*: v can be chosen as Ir, which yields v = v-1
Thus,
A = [Ya0,Yb0,…,Yr0]
Whenever the number of states pk = pr, i.e., k = r: v can be specified as Ir
42
Identification (Contd.)
Similarly, for x(t) = 0:
[za0,zb0,…,zr0] = Cv
where zi0 denotes the output symbol produced by L when in state yi and
excited by x = 0. Thus:
C = [za0,zb0,…,zr0]v-1
and when v = Ir:
C = [za0,zb0,…,zr0]
In order to obtain B and D: let us denote a unit input vector as ui, where
the ith component of ui is 1 and all other components are 0’s
We know that:
Bx = Y – Ay
In order to obtain B: we select some state yi (preferably the zero state if it
exists in U*) and specify B in terms of the constraints imposed on
it by yi and the unit input vectors
43
Identification (Contd.)
Let the input consist of unit vectors:
u = [u1,u2,…,ul]
The next-state vector, Yiuj, denotes the uj-successor of yi. Thus:
Yiu = [Yiu1,Yiu2,…,Yiul]
and
Bu = Yiu - Ayi
or
B = [Yiu – Ayi]u-1
Since u generally consists of unit vectors, and y is the zero state:
B = [Yiu1,Yiu2,…,Yiul]
Similarly:
D = {[ziu1,ziu2,…,ziul] - Ayi}u-1
where ziuj is the output vector associated with the transition from yi under
uj. Thus, the reduced equation is:
D = [ziu1,ziu2,…,ziul]
44
Example
Example: Returning to machine M4, we specify v to be
We obtain:
The only unit input vector is u = [1]: hence Yi1 is the 1-successor of yi
Since the zero state is contained in U*: let yi = yd. Thus,
The state and output equations are:
45
Example (Contd.)
The final test is to verify that the above equations indeed represent M4
under all input and state combinations: this is accomplished by
verifying each state transition and its corresponding output
symbol
• E.g., substituting ya for A and 0 for x(t): the machine should go to state yb
and produce output symbol 11, corresponding to entry B,11 in column 0,
row A
• Indeed:
• The characterizing matrices are thus verified. Linear realization:
+
x
+
y2
+
z1
y1
z2
46
Example
Example: Machine M5 and its distinguishing table are given below
• Checked rows are linearly independent
• Since U* contains all possible eight 3-tuples: identification procedure is
continued
47
Example (Contd.)
Example (contd.): Select
We obtain:
Setting u = [1] and yi = ya = 0:
Thus:
Matrices are verified to correspond to M5:
x
y3
+
y2
y1
z1
+
z2
48
Example
Example: Consider the four-state up-down Gray-code counter and its
distinguishing table
The state assignment is given by:
• Though M6 has only four states: its minimal realization is of a third
dimension
• v cannot be chosen as the identity matrix: and zero state yi = 0 is not
contained in the state assignment
– Hence, simplified equations cannot be used: matrix inversion cannot
be avoided
49
Example (Contd.)
Let
Hence, we obtain
Let yi1 = ya. Then
x
Minimum-dimension
linear circuit:
+
y3
+
y2
+
y1
50
z1
z2
Application of Linear Machines to Error
Correction
Consider the communication-system model shown below
• Message, denoted X: consists of a sequence over GF(p) of length n
• Encoder, with transfer function T: transforms the message into another
sequence over GF(p) of length n
– This sequence is referred to as the transmitted sequence, designated
Z: where Z = TX
• Sequence Z is transmitted through a noisy channel: whose output
sequence Z is called the received sequence
• In the channel: a noise sequence over GF(p), denoted N, is added to the
transmitted sequence, s.t. the received sequence is equal to
Z=Z+N
= TX + N
X
Message
Encoder
Z
Transmitted
sequence
Channel
+
Received
sequence
Z
Decoder
Received
message
X
Noise sequence, N
51
Decoder
Decoder, with transfer function T-1: processes the received sequence and
produces X s.t.
X = T-1Z
= T-1(TX + N)
= X + T-1N
Example: Consider the communication system shown below
•
•
•
•
Information digits: the first four digits
Checking digits: the remaining three digits
Checking digits in X: always 0
If X is received with three 0’s in the last three positions: no noise present
– Else, error correction is necessary
X:
1010000
T = 1 + D2 + D3
Z:
1001110
+
Z:
1101110
T--1 =
1
1 + D2 + D3
X:
1111110
N: 0100000
52
Example (Contd.)
When an error occurs: obtain T-1N and subtract from X
• Since the last three digits of X were originally 0’s: the last three digits of X
must consist only of digits of T-1N, without any contribution from X
• If only a single error occurred at time t: T-1N is simply the response of
decoder T-1 to a unit impulse at t
• Hence, the checking digits of X consist of a subsequence of three digits of
the impulse response of T-1
• The decoder is chosen s.t. its impulse response has a maximal period of
seven digits: this ensures that by observing the subsequence contained in
the last three digits of X, we can determine T-1N uniquely
• Suppose 1010000 is to be transmitted:
– Impulse response of the decoder: 1011100
– Checking digits of X: identical to fourth, fifth and sixth digits of the
impulse response
» Thus, we conclude that noise impulse occurred in the second
information digit
X:
1010000
T = 1 + D2 + D3
Z:
1001110
+
N: 0100000
Z:
1101110
T--1 =
1
1 + D2 + D3
X:
1111110
53
Single-error Correction
To correct single errors in messages over GF(2) containing m information
digits and k checking digits
• We need a decoder whose impulse response is of length m + k: with each
string of k successive digits different from every other subsequence of
length k
• Such an impulse response can be obtained from: a decoder whose
transfer function is of degree k and whose impulse response is maximal,
i.e., of length m + k = 2k -1
• If the last k digits of X are not zeros: T-1N must be subtracted from X
– Accomplished by shifting X over the decoder’s impulse response: until
the last k digits of X match a corresponding subsequence of the
impulse response
– This is always possible: since the impulse response contains every
nonzero subsequence of length k
– The modulo-2 sum of X and the digits of the impulse response
appearing directly below it: yield X
54