From: ISMB-93 Proceedings. Copyright © 1993, AAAI (www.aaai.org). All rights reserved.
1Protein Structure Prediction System based on Artificial
Neural Networks
J. Vanhala and K. Kaski
TampereUniversity of Technology/Microelectronics Laboratory
P.O.Box 692
33101 Tampere, Finland
email jv@ee.tut.fi, tel. +358-31-161407,fax +358-31-162620
Abstract
Methodsbased on the neural network techniques are
among
the mostaccurate in the secondarystructure prediction of globular proteins. Here the sameprinciples have
beenusedfor the tertiary structurepredictionproblem.The
mapof dihedral dp and V angles is divided into 10 by 10
squareseach spanning36 by 36 degrees. Bypredicting the
classification of eachresidue in the protein chain in this
mapa roughtertiary structure can be deduced.A complete
prediction systemrunningon a cluster of workstationsand
a graphicaluser interface wasdeveloped.Keywords:
artificial neuralnetworks,protein structure prediction,distributed computing.
Introduction
Neural networktechniques have been successfully used in
the prediction of the secondary structure of the globular
proteins. Although secondary structures are normally
defined by the presence or absence of certain hydrogen
bonds between amino acid residues, they could also be
described by the dihedral angles of the backbone. When
the protein chain adopts a certain secondary structure the
angles are forced to corresponding canonical values. Thus
predicting the secondarystructure is equivalent to predicting the classification of the backbonedihedral angles into
classes correspondingto helix, sheet, and coil -structures.
Each class include residues whose dihedral angles may
differ over 90 degrees. Thusthe secondary structure classification is not sufficiently accurate for deducingthe tertiary structure. Howeverby refining the classification the
network can be taught to predict more accurate dihedral
angles which in turn can be used as starting point for a
search of the three dimensionalstructure.
Thegeometryof a protein backboneis relatively rigid in
the sense that the bondlengths are practically constant and
the variations of the bondangles are relatively small, FigIThis paperdescribesthe researchbasedon the workdonefor
the MOTECC
computational chemistry software package described in Clementi1991and continuedat the Tampere
University of Technology.
4O2 ISMB-93
ure 1 (see e.g. Cantor & Schimmel1980 or Momany
et al.
1975). Since the backboneis a periodic linear structure of
the three atoms, (-N-C-C-),there are three dihedral angles
per residue. Oneof these, co, is normallyin trans-conformation, which leaves only two degrees of freedom, the dihedral angles ~ and tit. Notethat the conformationalflexibility of the side chain is muchhigher. To define the three dimensionalstructure of a protein backboneit suffices to list
the ~ and ~ angles. This makesthe prediction of the conformationrelatively well defined task: find a mappingfrom
the space of aminoacid sequencesto a space of dihedral angle sequences, which will correctly mapprotein sequences
onto their three dimensionalstructures. Anempirical way
of defining the mappingis to look at the proteins with a
knownsequence and a knownstructure and try to extract
the information required for the mappingfrom them. This
task can be approachedfrom different viewpoints, using
e.g. statistical methods,pattern matching,or as in this work
using artificial neural networktechniques.
Figure1. Protein backbone
structure. Bondlengthare in nanometers. Bondanglesare in degrees.¢ andVtorsionanglesdefinethe
backboneconformation.Torsionangle to of the peptidebondis
normallyin trans-conformation
thus the consecutiveCct-C’-N-Cct
atomsforma rigid planarpeptidegroup.R denotesthe side group.
Dashedlines separatethe aminoacid residues.
Anartificial neural networkis used to predict the main
chain conformation of globular proteins from their amino
acid sequence. The dihedral ~ and V angles of the protein
mainchain are calculated from the X-ray coordinates of a
set of proteins from the BrookhavenProtein Data Bank
(Bernstein et al. 1977) and has been given to the network
during the learning phase. In prediction phase an amino
acid sequencenot included in the training set is input to the
networkand the networkgives its prediction for the (or
set of) dihedral ¢ and ~ angles for each residue. Theresults
can be used as a starting point for computationallyintensive energy minimization calculations. Therefore neural
networkapproaches mayserve as a significant time saving
by suggesting conformationsrelatively close to an energy
minimum.
Neural networks in globular protein secondary
structure prediction
Several research groups have used neural network techniques to predict the secondarystructure of globular proteins from their amino acid sequence e.g. (~lan and
Sejnowsld (1988), Holley and Karplus (1989), Bohr et al.
(1988), and Mejia and Foge!man-Soulie(1990). Although
the groups worked independently they used basically the
same network architecture. The network was a feedforward layered network with full connection topology
betweenlayers and with sigmoid type nonlinear activation
function and error back-propagation learning rule i.e. a
standard back-prop network. The input to the networkis a
fragmentof the aminoacid sequence of a protein¯ The output gives the secondarystructure classification for the residue in the middle of the fragment, see Figure 2.
Secondarystructure for the wholeprotein can be produced
by sliding a windowover the whole length of the amino
acid sequence. The input to the network is represented
using singular coding where each residue is coded with 20
input nodes. Eachof these nodes correspond to one of the
20 naturally occurring aminoacids. Thusthe active unit in
a input group flags the presence of a given residue type in
that sequence position. For a windowof n residues the
input layer has the total of 20*nunits.
In the output layer there is a separate unit for eachof the
secondarystructure class used e.g. ot helix, 13 sheet, random
coil or turn. Thelevel of activity of a given output unit can
be interpreted as a probability of the correspondingclass.
Classification is then done by choosingthe class with the
highestprobability i.e. activity. In addition to the input and
output layers there is also a hiddenlayer¯ This is neededbecause a networkwith only input and output layers is capable of extracting only the first order features fromthe training set. Theseare the features that are causedby each input
unit individually. Since the secondarystructure of the proteins clearly dependson the local context of the aminoacid
Secondaryclass
Helix
Output
layer
Hidden layer
~i~i~i~
Gr iI
S [~
N~
+1~
J~p
¯
V
[H
i Illl~l
i i ii
|1
IIII i I i I i I
Coil
~ I
I I ~ ~ I I ~ I I ~ I I
i I’1i ii IIIi 1~I i IIIi i Ii i I i IIIi[i IIIllllll]l
i[i i i ii i i Ii
i I~E i iI i ii ii i i i lmB i i i i i i ii ii
I I I I1[i i i i i i i i i i i i i mlB
I1|
IIII
I III1~1
II
I~ ~illllllllE~
I
II I$~-I I I I I I I1-1 II IT[I I I I I i i i i ~ i i i[I
~111111
Illll
I I I I~1~111111111111
I
III I I I I[I I I I I I I I I I I I I I I I I ~) 1~ I I I
IIIIIII
IIII
$111 II1[1111111
II
IIII1~11111 I IIII I II III
IIIII
IIIIII1~11
III I I I I I I I I I ITIITI i]fl’|l
I I I I I I I I I ~
II I11 III I I I I I I I I I I I I I IEiS$1 I I I~ I I
tSRI I I I ~ IN’I I I I I I I I I I I I I I I I I I I I I I I I
IIIIIII
IIIII
I I.~ + I lllllmllllllll
I
II II I I I I I I I I I I I ~IE I I I I I I I I I
I II I
lllllll]llll
IIIIIIIIIIIIIIIIIIIII
I
III I I I I I I I BI III I I I I I I i i i i i i i i i i i i I
IIIIIIII1~|
IIIII
IIIIIIIIIIIIIIII
I
Ii
i
I
+lrrw-I .....................
~-4
m ,,,,
Ei
Sheet
I ~ ~l
-N/2
IIIIII
fill ....
.....................
0
N/2-1
Input layer
...PFANGHFSFYNKCTEOPICPNSMEMVDAVHCL...
Aminoacid sequence
Figure2. Network
architecturefor the secondarystructurepredictiorL Theinput layer has one active node(shownin black)coding
the residue type at eachsequenceposition. Thewindow
widthis
N. Everynodein the input layer is connected
to everynodein the
hiddenlayer troughthe lowerset of weights. Alsoeverynodein
the hiddenlayer is connectedto everynodein the output layer
trough the upperset of weights. Amino
acid types are given in
one-lettercode.Theclassificationfor the prolinein the middleof
the window
is sheet, whichhas the highestactivity level in the outsequence a hidden layer is needed. This gives the network
the ability to represent also secondorder features i.e. features that dependon morethan one input unit at a time.
Both Qian and Sejnowski and Holley and Karplus used
also alternative waysof representing the input to the network. They used physicochemical properties of the amino
acid residues such as hydrophobicity, charge, side chain
bulk, and backboneflexibility. Qian and Sejnowski also
provided the networkwith global information such as average hydrophobicityof the protein and the position of the
residue in the sequence. All these attempts apparently
failed to improvethe prediction reliability over the basic
scheme.
Bohret al. used a separate networkfor each secondary
structure class. Also they used two nodes to represent the
output class. One node gives the probability p for belonging to the class and the other gives the probability(l-p) i.e.
Vanhala
403
the probability for not belongingto the class. In this wayit
is possible to recognize those cases where the network is
not able to give a valid prediction.
Neural networks in globular protein tertiary
structure prediction
Bohr et al. (1990) have developed a method where they
predict the distance matrix (or rather a contact matrix) of
globular protein from its aminoacid sequenceusing a feed
forward neural network. The networktells if the distance
betweentwo ix- carbons is less than a preassigned threshold value, e.g. 8 angstroms. All pairs of ct- carbons up to
30 residues apart are considered, thus the network gives
distance constraints for a diagonal band whichis 30 residues wide. To generate the full distance matrix they use a
deepest decent optimization to satisfy as manyof the distance constraints as possible. Since the distance matrix can
be generated from the back bone internal coordinates and
vice verse, this method bears resemblance to the work
described below, but uses a different wayto represent the
three dimensionalstructure of the protein backbone.
Their networkis large compared
to the size of their training set. Indeed the networkis capable of reproducing the
training set 100percent correct. This will reduce the ability
to generalize to unseen examples. Howeverthey use the
networkto predict the structure of a protein that has highly
homologouscounterparts in the training set obtaining good
results. Theydo not report results for proteins that do not
have homologousproteins in the training set. Wilcoxand
Poliac (1989) report even bigger network and use fewer
proteins in the training set. Their networkis capableof correctly recalling the distance matrices of the proteins in the
training set whengiven the hydrophobicities of the amino
acid sequence; howeverthey do not test the network with
unknownproteins. Fiedrics and Wolynes (1989) use
methodclosely related to Hopfield type neural networks to
predict the tertiary structure of globular proteins. Theycall
their methodassociative memoryhamiltonian.
Methods
Our approach draws ideas from two lines of development
in protein structure prediction. The first is the workdone
with artificial neural networksaimedat predicting the secondary structure. This has given us the tool. The second
source of inspiration has been the methodsdeveloped by
Lambert and Scheraga (1989a, b, c) to predict tertiary
structures. This has given us somepractical guide-lines
and a reference for comparingour initial results.
Our training set is the sameas the one used by Lambert
and Scheraga in their pattern recognition-based importance-sampling minimization (PRISM) method. This
choice offers the advantagefor fair comparisonof results.
404
ISMB-93
Torsion angles
Output layer
I I II |
~
n lnnl
I I I I I
-rm 0 I--TT-I
II I I II
I I I F.~:: I I I I
IIIII
I
I VI I I
-180 I ] n I I
-180
O
180
@
Hidden layer
G
S
-~
¯ ~
N
iii
i RI~ i i i i iiiiiiiiIiIIiiI
iiilllil
A ...........................
~.~
V
O
f-I
H
]~I
N
IIII
]IIIIIIIIIIIIIIIIIIIIIIII~I~III
I|I
I I I I I I lllllll~mlllll]
IIII
III I I I
fTI
IIIIII I I II iIII1~11
i
III I II II$I I I I II1 Illlllllll
I I I III I I III
i i i ii ~R
I I I I I I I I I I I I I I I I[] I I I I I[~l ~I[U I I I I
III
I I I I IIIIIIII1~11~111111111111
I I I I I I I m~i I I I I I I I ~1 I I I Kill
I |1[I I
III
I
II I1 IIIIIIIJI
I |llllllllllllllllll
I
I I I I I I I I I I $I I I I I I I I I I I I I I I I I I I I I I
III
I I I[I I I~1 I I I I I I I I I I I I I I I I I | I I i
I
~1K ,,,,,,,,,,
E
i
IIIIII
III
III1~1111111111
Illl
[l|ll
III
III1[11111111[[11~1111111111
I
I I I I I I I I I I I I I I I I I I I I I I I I I I
I I I I I I
I I li~ I [ I II lI l~i
I ! III
IIII
I~ III I III III II I III III III III II~II II II I I Im~l
[I I I~g
i i ii i i i i i I ~ II
-N/2
............
,, .........
O
N/2-1
Input layer
...PFANGHFSFYNKCTEOPICPNSMEMVDAVHCL...
Aminoacid sequence
Figure3. Thenetworkarchitecturefor the tertiary structureprediction. Theoutputlayer in Figure2 has beenreplacedwitha map
of 10 by 10 output nodes. Eachnodecorrespondto 36 by 36 degreessquareof the d?--Vdiagram.Everynodein the hiddenlayer
is connectedto everynodein the output maptroughthe upperset
of weights.In this case the outputfor the prolinein the middleof
°
the window
can be interpreted approximately
as tl)=-100°, V=100
Also we could skip the time consumingphase of studying
the protein structure data banks and makingfair and unbiased selections from them. Wecould as well have took either of the training sets collected by Kabschand Sander
(1983) or by Qian and Sejnowski (1988). The training
is collected fi-om the BrookhavenProtein Data Bank by
Bernstein et al. (1977) and it contains 46 strands from
proteins and has 6964residues.
Because of the differences in the methods we had to
makesomeslight modificationsto the training set. ’Lambert
and Scheraga calculated conformational probabilities for
tripeptides i.e. the windowwidthis three. Becauseof this
choice they had to drop the first and the last residue from
each amino acid sequence. Instead we have used a much
larger windowof residues. The width of the windowvaries
from case to case from 7 to 51 residues. Since neglecting
one half of the windowsize of the terminating residues at
both ends woulddecrease the training set too much,we define a newnull residue that can be appendedto both ends
of the aminoacid chain (as is also doneby e.g. Qianand Sejnowski). In addition we could not simply removefrom the
training set those residues that are in cis-conformationor
those that are of nonstandardtype. Instead we define another new null residue which is skipped during the training
phase. Note that in the prediction phase all amidebondsare
also considered to be in the planar trans-conformation. All
protein sequences have been concatenatedinto one long sequencewith the null spacer residues betweenthe proteins.
Thefile that contains the training set has the residue identifiers together with the corresponding ¢ and ~g angles.
Theseangles were calculated from the protein X-ray chrystallog,~c data in the protein structure data bank. Notethat
although the training set file contains the actual angles,
they are classified into discrete output classes before being
given to the networkin the training phase.
Wehave used a feed forward layered neural network and
the error back propagation algorithm. Lambertand Scheraga (1989a, b, c) did not report on prediction for the d~ and
~g angles. The four classes is the upper limit for the Lambert’s and Scheraga’sapproach since they collect the statistics for each of the 64 (--43) possible tripeptide
conformational combinations. For even four classes the
frequency of more rare combinations is not sufficient to
obtain good prior probabilities. In our case in order to
makemorerefined predictions for the angles we divide the
¢-~g mapinto a 10 by 10 grid. Eachsquare spans an area of
36 degrees by 36 degrees, see Figure 3. As the output layer
in the network mayhave several active units, we need a
postprocessing phase that identifies the most probable
answersand also gives the actual angles instead of a classification into the hundredoutput classes. First the output
is low pass filtered to smooth out excessive roughness.
Thenall peaks are identified and their relative strength is
calculated to give them a rank. At this point the output
Sequence
G P
S QP
TY PGDDA
180 E
90O[:g
0O~
-90 -
-180
-180
I
-90
I
90
0
180
Figure3. Conformational
quadratures.Regiona containsIx helices, x helices, andisolated turns. Regione contains[~ bridges,
sheets, andprolinerich strands.RegionIx* containsleft handedIx
helices. Regione* containsglycinestrands.
mapis still in discrete form.Theactual t~ and ~g angles are
produced by computingthe "center of mass" for a neighborhood of each peak.
Results
Predicting
conformafional
quadratures
of PPT
To be able to make comparison to work done previously
we choosedto try to predict the conformationalclasses of
the avian pancreatic polypeptide as have been done by
Lambert and Scheraga (1989a). They divide the two
dimensional ~¥ map into four regions as shownin Figure
4. Their prediction methodPRISM
gives the classification
PVEDL
I
R F YDN LQQY
LNVV
TRIIRY
Correct
e e E e e e o~e*
oco~ e Iz o~ cx cx ~ ~x Ix oc cx Ix ix oc o~ Ix ix o~ ot Ix Ix (xlx*E Ix
Neural net
e e e
e e
e* ot cx e e ~ oc cx ~ o~ ,’ ,’ cx cx oc ~ o~
E
Et IX, E
a*
~k_lal._Acx!._!
Figure4. Secondary
structure predictionfor avianpancreaticpolypeptideI PPT.Thetop line gives the aminoacid sequencein one-letter
codeand belowthat is the correct classification into the four conformational
quadratures.Thepredictionswiththe neuralnetworkand
with the PRISM
methodare on the bottomtwolines. Harderrors are shownin solid boxesand soft errors in dashedboxes.Thedots at
the bothendsof the classificationdenotethe unusedresidues.
Vanhala
405
into one of the conformationalclasses for each residue in
the sequence. This approximatedescription can be used as
an input to a subsequent energy minimization step. In the
similar mannerwe created a neural networkwith four output nodes each correspondingto one of the conformational
quadratures. Using the sameprotein structure data base we
taught the networkto mapthe aminoacid sequence to output classes. The results for both methodsare shownin Figure 5. The four conformationalclasses a, E, c~* and ~* are
shown.The first line gives the aminoacid sequence in one
letter code. Thesecondline gives the correct classification
derived from the X-ray coordinates. Belowthose there are
the prediction results from the neural networkalgorithm
and from the PRISMmethod. The neural network does not
simply give one class as an answerbut rather a probability
of the correct classification of being in somequadrature. In
this way one can have a multiple choices in situations
were the networkis not able to makeits mind. Errors are
classified in twoclasses: a boxwith thin edgesis an error
wherethe alternate answer is correct and a box with thick
edgesis an hard error wherefor all alternatives the prediction is wrong. Both the neural network and the reference
has 6 hard errors, but the network gives also two soft
errors. The performanceof the networkis a bit worsethan
that reported for PRISM.One explanation for this is that
they used elaborate statistical meansin dealing with rare
cases in the aminoacid sequence data, somethingthat the
neural networkin its extremesimplicity is not able to do.
Sincethe classification is not fully correct, it is not possible to obtain a tertiary structure whichis relatively close to
the native one. In PRISM,the problemis solved by generating a multitude of dihedral angle classification
sequences and choosing the most probable ones for further
processing. This tends to be a very time consumingprocedure with an exponential run time complexity with respect
to the chain length. Indeed it becomesunpractical to process proteins with morethan 100 residues. The neural networkapproachdoes not have this limitation, since it gives
multiple answers simultaneously and it is easy to sort
these accordingto their probabilities.
gions. The predicted structure for BPTIis not globular but
elongated. Howeversince the cysteine residue pairs that
form the sulphur bridges are normally knownit "should be
possible" to fold the structure into moreglobular shape by
forcing the correct cysteine pairs together and at the same
time trying to maintain the predicted dihedral angles. The
native and the predicted structure of BPTIis shownin Figure 7.
The distance matrices shownhere display all distances
of the a -carbons of the protein chain. The matrix is symmetric, thus only one half of it is shown.This makesit easy
to comparetwo structures by placing themon each side of
the diagonal. The distances are gray scale coded. The
change of colour correspondsto a distance difference of 2
A, ngslr6ms. The 16 shades cover a region from 0 to 32 A.
The distance matrix representation has some advantages
over displaying the structures on the computerscreen: It is
rotation invariant so that the structures do not have to be
aligned. It can showboth local small scale details (close to
....,., S
1
Predicting ~b and V angles of BPTI
Bovine Pancreatic Trypsin Inhibitor (BPTI, 5PTI) is
muchstudied protein that has 58 residues. The prediction
by the networkhas been analysed by finding the fragments
of the protein chain that has (near) correct ~ and ¥ angles.
In the Figure 6 the predicted backbonestructure of BPTIis
showntogether with fragmentsof the native BPTIstructure
from the X-ray data. Abouta half of the residues fall into
correctly predicted regions. Both a helices (residues 2-7
and 47-56) and the 13 sheet fragment (residues 29-35)
correct. Also four ends (residues 5, 14, 38 and 51) of the
three sulphur bridges (S1, S2, and $3) are inside these re406 ISMB--93
......... 2
’i’°
14-
s2 or-Figure 6. Fragmentsof the BPTIbackbonefrom X-ray data
alignedwiththe predictedstructure. Solidline showsthe correctly
predictedregions. Thenumbersrefer to the aminoacid sequence
positions. Thethree sulphurbridgesSI-S3 are also shown.
a)
Figure8. Thedistancematrixof BPTI.
b)
perpendicularto the diagonal) is predicted. Onecould also
imagine to see parts of the two spherical "eyes" on both
sides of the I~ sheet. Theoverall structure is elongated,thus
the right uppercomeris black whichtells that the residues
far from each other in the sequenceare also far from each
other in the three dimensionalstructure.
Prediction of the ~ and ¥ angles of LZ
Figure7. Thenativestructure(a) andthe predictedstructure(b)
of BPTI.
the diagonal) and the overall conformationin the large (the
overall pattern). The scale dependson from howfar you are
lookingat the picture; It is insensitive to fewbig errors in
dihedral angles. Inspecting visually the three dimensional
structures on the computer screen can be very difficult
since although most of the predicted angles are relatively
close to correct ones, there are few angles that are predicted
with almost the maximum
error, 180degrees. This distorts
the overall structure and the corresponding residues may
end up in completelydifferent places and rotations.
In Figure 8 is the distance:matrix of BPTI.The two sides
close to the diagonal match rather nicely. The tx helices
whichare shown~.n light colour are predicted correctly otherwise but there is an extra piece of helix in the lowerfight
part. Onlythe very start of the [~ sheet structure (light strand
HumanLysozyme (1LZ1) is an example of a bit longer
protein chain. It has 130residues. Againthe overall structure of the prediction seems to be nonglobular, Figure 9a.
It seems to be put together from smaller regions that are
connected by long strands of randomcoil. The distance
matrix in Figure 10 showsthe same interesting features.
The closely packed regions have good correspondence to
the native protein. Specially the upper left corner of the
distance matrix, whichshowsthe start of the protein chain,
makesalmost an exact match, tx helices are correctly identiffed. The I]- structures in the middleof the matrix seem
to have somerelation to the native side.
Prediction of the ¢~ and ~ angles of PPT
Avian Pancreatic Polypeptide (1PPT) is an example of
small protein. It has only 36 residues. The native structure
of 1PPTis a fold of one long ~t helix and a shorter strand
of 1~ sheet. This can also be seen fromthe lowerleft part of
the distance matrix, Figure 11. The prediction has correctly producedthe helix and the I~ sheet but their relative
Vanhala
407
Figure10. Thedistancematrixof ILZI.
Figure9. Thepredictedstructure of I LZ1(a) and 1PPT(b).
the Insert button. First a structure nameis highlighted by
clicking the mousebutton on that entry and then clicking
on the Insert button. The system will copy the structure
nameon the Training set list. If a wrongstructure is accidentally inserted into the training set, it can be removedby
highlighting the structure nameand clicking the Delete
button. The PDBdataset is first loaded by giving its location on the computerfile system(i.e. the directory containing the files) on the line next to the Loadbutton and then
clicking the button. Training set list showsthe namesof
the structures that will be used whentraining the neural network. The list can be saved for a later use with the Save
positions are wrong. The secondary structures are correct
but their packingis not. This can also be seen in Figure 9b.
Theuser interface
The arrangements of the elements in the graphical user
interface for the protein tertiary structure prediction system is shownin Figure 12. This version is build using the
OpenWindowsGraphical User Interface Design Editor,
GUIDE(SunSoft 1991).
The Protein structures list showsthe structures that are
available on the computersystem. Each structure is in a
separate file in PDB-format.The system uses only the SEQRESand ATOM
records. The training set for the neural
networkis created by copying structures from the Protein
structures list to the Training set list. This is done with
4O8 ISMB-93
Figure1 I. Thedistancematrixof 1PPT.
Log
Training set
Protein structures
Tra|nlng Um~%
)
Pl. .........
0
100
(r~-)
~let’work file
Type
1
Sequence
file
In
S~’uc~urefile out
Figure 12. The graphical user interface.
q
button. The saved set can. be retrieved from the saved fde
with the Loadbutton. In this wayseveral different training
sets can easily be maintainedfor e.g. predicting the structures in a knownprotein family.
Whenthe training set has been created the network can
be trained simply by clicking the Train button. Since the
training maytake longer than one is willing to wait at the
keyboardthere is a gaugethat showshowfar the training
has proceeded. The trained network is saved into a file
whose name must be given on the line below the Network
f’de heading. In case the networkhas been trained before
with the sametraining set there is no sense to train the network again. Thusthe networkcan be retrieved from a previously saved file with the Loadbutton. If the networkis
already trained or loaded froma file before the train button
is clicked the training is continuedfromthis old state.
A new structure can be generated by first giving the
nameof the file containing the sequence information. The
file maybe in either PDB-format
or written with the I-letter code or the 3-letter code. Thetype must be indicated by
clicking the appropriate button. The nameof the prediction
result which is always in the PDB-format(with only SEQRESand ATOM
reco~s) must also be given. The structure is generated by clicking the Predict button. The prediction phase is very. fast comparedto the training phase,
thus there is no gaugeto showthe training time.
Everything that is done is echoed on the Log window.
Every time a button is pressed or a file nameis given the
log is updated. Youcan also makeyour ownnotes on the
log. Thelog can be savedinto a file for later referencewith
the Save button.
Discussion and future development
Werecall that Holley and Karplus and Qian and Sejnowski have attempted a coding based on physicochemical properties of aminoacids They were not successful in
improving over the simple method, using singular coding.
Perhapsthis has blurred the networkvisibility and the identity of residues is no longer visible to the network.Onthe
other hand we knowthat the stereochemical structure is of
importanceespecially in the short range interactions e.g. in
forming the hydrogenbonds, as has been shownby Presta
and Rose (1988). Another feasible explanation might
that the performanceof the networkis not limited by the
net-worksability to extract the informationfromthe training set whateverthe input codingis but rather by someother factor inherent in the approachor in the training set.
Usinga neural networkmethodsfor predicting the tertiary structure of a globular protein has its limitations. Some
of these are inherent to the neural networkapproachitself
and someraise fromthe nature of the application. Themain
limitation is probably the numberof currently available
protein structure data. Roomanand Wodak(1988) show
that some1500 proteins with average length of 180 residues wouldbe neededin their approach to obtain goodresuits in structural prediction. It will take several years to
collect the required structural data and eventhen there is no
guarantee that the newproteins contain the missing inforVanhala
409
marion of the sequence-structure mapping.
Also for a neural networkthere is alwaysa trade-off between the memorizationand the ability of generalization.
The size of the networkplays a majorrole. Asa rule of the
thumb(which also has theoretical foundations as described
by Baumand Haussler 1989), in order to get a good degree
of generalization, there should be about 10 times as many
training examplesin the training set as there are weightsin
the network.In practice this rule holds for every fitting of
parameters, if they are nearly linearly independent. The
smallest network we are currently using has 2540 weights
(154 input nodes, 10 hidden nodes and 100 output nodes).
Thus there should be about 25000 training examples. The
training set with its 7000examplesis only one third of the
theoretical minimum.
Weconclude that the back propagation neural network
has had limited success in predicting the protein backbone
dihedral angles. Prediction can be quite close to the correct
one over a short sequenceof residues. For examplect helices that are energetically the mostfavourablestructures are
often predicted in their correct places. By nowit has becomeapparent that attempt to reproducethe wholethree dimensionalstructure from the backbonedihedral angles will
be too sensitive
to errors in them. Whatis neededis a rep#
resentation that is more robust and contains more redundancy. In this respect the distance malrix representation
seems to be promising.
As a bottom line we would like to emphasize tha~ the
neural network(or in a broader context the artificial intelligence) approachis not per se unsuitable for the prediction
of molecular phenomena.Our research is only taking its
first steps and it mayhave a long wayto go before it will
proveits feasibility andlater yield significant results.
References
Baum,E. B., and Haussler, D. 1989. WhatSize Net Gives
Valid Generalization?. Advancesin neural information
processing systems L, Touretzky, D.S. ed., Morgan
KaufmannPublishers, San Mateo, CA.
Bernstein, F.C., Koctzle, T.F., Williams, G.J.B., Meyer,
E.F.Jr., Brice, M.D., Rodgers,J.R., Kennard,O., Shimanouchi, T., and Tasumi, M. 1977. The Protein Data
Bank: a computer-basedarchival file for macromolecular
structures. J. Mol. Biol. 112: 535-542.
Bohr, H., Bohr, J., Brunak,S., Cotterill, R.M.J., Lautrup,
B., Norskov,L., Olsen, O.H., and Petersen, S.B. 1988. Protein secondary structure and homologyby neural networks.
FEBSLett. 241: 223-228.
Bohr, H., Bohr, J., Brunak,S., Cotterill, R.M.J., Fredholm,
H., Lautrup, B., and Petersen, S.B. 1990. A novel approach
to prediction of the 3-dimensionalstructures of protein
backbones by neural networks. FEBSLett. 261: 43-46.
410
ISMB-93
Cantor, C.R., and Schimmel,P.R. 1980. Biophysical
Chemistry, Part 1: The conformationof biological macromolecules. W.H. Freeman and Company.
Clementi, E. ed. 1991. MOTECC
ModernTechniques in
Computational Chemistry. ESCOM,
Leiden.
Friedrics, M.S., and Wolynes,P.G. 1989. TowardProtein
Tertiary Structure Recognition by Meansof Associative
MemoryHamiltonians. Science 246:371-373.
Holley, L.H., and Karplus, G. 1989. Protein secondary
structure prediction with a neural network. Proc. Natl.
Acad. Sci. U.S.A. 86: 152-156.
Kabsch, W., and Sander, C. 1983. Howgood are predictions of protein secondarystructure?. FEBSLett. 155:179182.
Lambert, M.H., and Scheraga, H.A. 1989a. Pattern Recognition in the Prediction of Protein Structure. I. Tripeptide
Conformational Probabilities Calculated from the Amino
Acid Sequence. Journal of ComputationalChemistry 10:
770-797.
Lambert, M.H., and Scheraga, H.A. 1989b. Pattern Recognition in the Prediction of Protein Structure. II. ChainConformation from a Probability-Directed Search Procedure.
Journal of Computational Chemistry 10:798-816.
Lambert, M.H., and Scheraga, H.A. 1989c. Pattern Recognition in the Prediction of Protein Structure. 111. AnImportance-Sampling Minimization Procedure. Journal of Computational Chemistry 10: 817-831.
Mejia, C., and Fogelman-Soulie,F., 1990. Incorporating
knowledgein multilayer networks: The exampleof proteins secondarystructure prediction. Rapportde recherche,
Laboratoire de Rechercheen Informatique, Univ. de Parissud.
Momany,F.A., McGuire, R.F., Burgess, A.W., and Scheraga, H.A. 1975. EnergyParameters in Polypeptides. VII.
Geometric Parameters, Partial AtomicCharges, Nonbonded Interactions, HydrogenBondInteractions, and Intrinsic
Torsional Potentials for the Naturally Occurring Amino
Acids. The Journal of Physical Chemistry 79: 2361-2381.
Presta, L.G., and Rose, G.D. 1988. Helix Signals in Proteins. Science 240: 1632-1641.
Qian, N., and Sejnowski,T.J. 1988. Predicting the Secondary Structure of Globular Proteins Using Neural Ne 1 work
Models.J. of Mol. Biol. 202: 865-884.
Rooman,M.J., and Wodak.S.J. 1988. Identification of predictive sequencemotifs limited by protein structure data
base size. Nature335: 45-49.
SunSoft 1991. OpenWindowsDeveloper’s GUIDE3.0,
User Guide. SunSoft, MountainView, CA.
Wilcox, G.L., and Poliac, M.O.1989. Generalization of
Protein Structure From Sequence Using A Large Scale
Back propagation Network.Proceedingsof the International Joint Conference on Neural Networks, IJCNN1989,
Washington, D.C.