561-70 ADAPTIVE CONTROL ODYSSEY By E. H. Bristol The Foxboro Company Foxboro, Massachusetts 02035 Copyright 1970 Silver Jubilee International Conference & Exhibit Oct. 26-29, 1970 Philadelphia, PA Instrument Society of America desired, it is necessary to provide means for predicting the effect of changes in parameters on performance. ABSTRACT The paper describes a 10-year pilgrimage in industrial adaptive control that began with the publications of Caldwell1 and Pessin,2 Reswick and Goodman,3 and from some theoretical thinking on Bayesian statistics and functional analysis. The work ended with some recent papers by the author on functional adaptive systems. A more complex and formal approach to the problem, and one which leads to greater understanding, involves giving the computer the ability to use the process measurement information to identify the control characteristics of the process and control disturbances. This information permits computation of the settings which will give the best performance. It turns out, however, that this identification problem is not solvable as it stands; for any set of measurement data, there will always be many process disturbance combinations which could explain the data. And yet controllers are easily tuned manually. Discussed are the philosophical underpinnings of adaptive control, theoretical promise and practical problems of identification, and the problems of defining workable assumptions about the control environment to give an adaptive system a starting point. The consequences of failure of these assumptions are detailed, along with the problems of guaranteeing safety, even when the adaptive system is misused. In practical adaptive systems this paradox is resolved by supplying the missing information in the form of explicit or implicit assumptions about the process or the disturbances. These assumptions may take a number of forms: Where possible, the comments are illustrated by experimental data, but since much of the data is old and sketchy, the emphasis is on ideas. At the odyssey's end, the important technical and economic implications of adaptive control are discussed. 1 2. INTRODUCTION One may make statistical assumptions about either the nature of process or the disturbances. One may assume a model form (second order process or deadtime and lag) for process or disturbance.4 It would be a mistake to generally define adaptive control. This paper will discuss adaptive control in terms of automatic tuning of feedback controllers. Many people in the instrument industry contend that few users succeed in keeping controllers anywhere near perfectly tuned, and many do not even try. It is clear that if someone were able to provide an inexpensive way of automatically tuning controllers and keeping them tuned, it would be of value to everyone. The paper discusses an extended effort to solve this problem. One may assume a closed-loop model form.5 One may assume a frequency spectrum form for either open or closed-loop response. Each of these models or forms may be expressed in terms of parameters which must be computed from process data. 3. 4. 5. PHILOSOPHY A simple approach to the problem involves allowing the adapting system (the computer in most cases) to continually adjust the tunings to improve performance. A means of recognizing the quality of response under varying control disturbances is necessary to the computer. If rapid adaptation is Finally, assumptions may be made about process linearity, maximum settling time, or about disturbance settling times and the likelihood of occurrence of multiple disturbances within short spans of time. SOME EXPERIMENTS ON IDENTIFICATIONI The following experiments spanned the years 1962 561 In each of these experiments, to start with, K = K’ = 1, τ1 = τ2 = 10, τ1’ = .625, τ2’ = 17.18 and 1964. The method used for identification is similar in spirit to Kalman's method4,7 except that an explicit attempt is made to identify load. The method is described in Appendix I. It is assumed the process is linear. K -------------------------------------------- ]. The second has eight lags where ( τ1 s + 1 ) ( τ2 s + 1 ) seven of the eight time constants are set identical. The remaining time constant and the process gain may be set independently. [The model is of K′ form -------------------------------------------------- ]. Thus, each process has 7 The control situation is shown in Figure 1. A hypothetical process was selected and controlled with a digital controller to a zero set-point. A hypothetical upset was introduced to the process and the values of manipulated variable M and controlled variable C following the upset were recorded. Table I shows a tabulation of the sampled values of C going into the computer and the successive levels of ∆M outputted to the process. Figure 2 is a closed-loop analog record of C under the controlled upset. From this data, the writer attempted to calculate the process response to step control action and the upset response, which is the set of values that C(t) would have taken under the upset in absence of control. Figure 3 shows measured analog records of the step response and upset response recorded at a later date. The small x's on the record show values calculated from the original closed-loop data of Table I and Figure 2. Most of these calculations are precise. The exception is the last value in time on the record of the upset response which is off by about ten percent. However, later examination showed that there was a mistake in the gain settings for these recordings and that the calculations were more accurate than the measured record! ( τ′ 1 s + 1 ) ( τ′ 2 s + 1 ) three degrees of freedom. The two process outputs can be compared in a mean square sense by the summing amplifier, squares, integrator combination. Also switching allows tuning each process for optimum integral square error set-point change. The purpose of the experiment is to demonstrate the consequences of using models with limited degrees of freedom to represent a process of differing structures. This is important since none of the usually recommended models (two lag, deadtime and lags, etc.) is likely to give anywhere near exact feedback control tuning results. Three consequences will be apparent: 1 Modelling or identification arrived at under one set of tunings or operating conditions may well give an extremely poor fit under other tunings or operating conditions. 2. Repeated identification under successive adapted settings will often give convergent control settings and behavior which is reasonably good even when the initial identification gave poor control fit, as discussed previously. 3. True optimum settings cannot generally be inferred from an approximate model under any circumstances, even though the resulting control is near optimum and the optimum response itself can often be inferred from the model. In the experiment, one of the simulated processes is considered to be the "real" process, the other is considered to be the "model" for identification and tuning purposes as if an on-line adaptation were going on. For each of the experiments the twolag process was set initially with unity gain and identical time constants, and the eight-lag process was set to give a best mean square fit to the twolag process under an open-loop step. In the odd numbered figures showing experimental results, the first and third responses from the top correspond to the two-lag process and the second and fourth correspond to the eight-lag process. The even numbered figures show the results in the preceding figure compared by superimposing the two- and eight-lag responses. The same method was used as a continuous adapting scheme. Since the adaptation was continuously active it was necessary to differentiate automatically between circumstances when data was meaningful and circumstances when the data could not give valid identifications. The clearest case of the latter occurs when the measurement is steady on set-point, but there are instances where transient measurements fail to give valid identification. The method used was to calculate the sensitivity of the identification and use this result to control the use of information in the scheme. Figure 4 shows the structure of the control system. The sensitivity calculation consists of a worst case analysis of the effect of a unit amount of controlled variable noise on the identification. Readers interested in this type of technique might consult any of the literature on the circuit analysis program ECAP6. The identification is most credible when it is least sensitive. Controller constants were averaged with past values weighted according to sensitivity. The effects of this are shown in Figure 5 where control measurements under set-point changes are shown together with the identified process gain (actually states as an unaveraged controller gain K), the credibility estimate and the averaged gain K (reflecting control settings used). This data clearly shows the correlation between active signal, identification, and sensitivity-derived credibility calculation. Later experiments dealt with situations in which the operating assumptions were increasingly violated. (See Appendix I.) The results gave good control even when the identification became erratic! This suggests that the model may not be all that important. In the first experiment (Figures 7 through 12), the two-lag process is considered to be the "model." The controller corresponding to it is tuned for best performance and the "real" process (the eightlag) controller is set to the same settings. The open-loop step responses are shown at the top of Figures 7 and 8, and visually the fit is good, with less than three percent error. But, the closed-loop set-paint change responses, shown at the bottom of Figures 7 and 8, resulted in unstable control of the "real" process. A second set of experiments amplify this position (See Figure 6). In this experiment two independent simulated control loops are shown each with two term controller. The first is a two-lag simulated process where the gain and the two time constants are adjusted independently. [The model is of form The data from the "real" process are then used to arrive at a better "model" in a mean square sense. The two-lag “model” is readjusted under the same controller settings to give a best mean square fit to this data as shown in the top of Figures 9 and 561 -3- with the desired response shape and the adjustment of the controller settings to improve this transient. The scheme assumes no prior knowledge of the process characteristics. Since the system is capable of adjusting the controller gains to achieve the desired response, nonlinear systems can be safely controlled under large control changes (even where nonlinearity is apparent in one response). In the following discussion the two-term controller is in control all the time. 10. This is done by radically increasing the openloop model gain as shown in the bottom of Figures 9 and 10. The controller settings can now be readjusted to the new model giving better control to both "model" and "real" process in the sequence shown in the top of Figures 11 and 12. Continuing in this vein, the experiment would converge to a final set of settings. The results made the identification look unnecessarily bad and show what happens if a particularly naive modelling approach is taken. Any sophisticated designer would know of the danger in using a second order model for these processes. But remember that the initial open-loop fits were good. Generally for process control, one would use a more complex model (such as a deadtime on lag model) here represented by the eight-lag process. The user selects his desired response by inserting only two values, B and C, which serve as comparison levels during gain computations (Figure 18) and serve to specify the desired response shape. (These B and C levels can be set to correspond to any desired control pattern, overdamped or underdamped.) The pattern is evaluated by calculating the integrated difference of measurement normalized by setpoint change or upset magnitude from the B or C level (Figure 18) over appropriate intervals of time corresponding to overshoot and undershoot. (The two intervals of time [one for B and one for C] are chosen, based on time measurements for the initial control recovery to correspond to the first two half cycles in the dominant eigenfunction [assuming that it is cyclic) after the initial recovery. For most purposes, the three dominant eigenfunctions accurately determine the pattern, but not the control characteristics.) Thus, the relationship between the initial recovery and the position of these half cycles is sufficiently independent of deadtime, process order, and reasonable degrees of nonlinearity that it does not adversely affect pattern characterization. The pattern areas thus evaluated are used to adjust control settings to improve response using simple feedback. The scheme works in a trouble-free and insensitive manner independent of process, initial settings, disturbances, or set-point magnitude. In the second experiment (Figures 13 through 16) the eight-lag process is the "model" and the twolag process is the "real" process. The tops of Figures 13 and 14 again show the open-loop responses. The bottom of Figures 13 and 14 show the set-point change responses resulting from optimum settings derived from the model as before. Note that the set-point response of the "real" process is now fairly good, even though it differs noticeably from the response of the "model." If the model is now readjusted for best closed-loop fit, the resulting closed- and open-loop responses are shown in Figures 15 and 16. By generating conservative settings, the eight-lag model is a safer basis for adaptation. It still requires successive iterations to reach a final "model" and each model is inaccurate except for the control settings under which it was taken. The quality of control is good because the control criterion is very insensitive near its optimum. This is best seen by comparing the true optima closedloop responses where the two processes are matched for open-loop step response (as in Figures 7, 8, 13, and 14). These are shown overlapped in Figure 17. Although the responses are similar in shape, if not in time scale, the optimum controller gain for the two-lag process is 13.16 as opposed to 3.85 for the eight-lag process: a ratio of more than 3 to 1. The reset times were respectively 34.12 and 27.7. A slight modification permits adaptation under certain transient load conditions. Several possibilities exist to permit adaptation of the three-term controller. The adaptive feedback equations are given in Appendix II. This scheme has performed safely and effectively in pilot plant tests and in limited field tests. In summary, the experiments indicate that an adaptation scheme based on such a modelling point of view will give good practical results. There is good cause to believe that much simpler methods would work nearly as well. Even so, the identification approach should still be kept in mind for theoretical studies or situations where a formal characterization of process is desired. The experiments took an extreme situation (two-lag versus eight-lag), but this kind of difference does exist in the real world. A more exact model with more degrees of freedom in parameters would give better results, but would be more sensitive to noise. OTHER TECHNIQUES The techniques presented have been those developed by the author. Other authors have covered identification in equally effective ways. The techniques using auto-correlation and cross-correlation3 filter out noise permitting data to be preprocessed allowing methods like those already given without requiring explicit identification of long noise signals. These methods also provide one way of dealing with nontransient signals. The identification methods previously discussed, as well as the pattern scheme, all assume a transient disturbance. The correlation methods3 can transform a continuously varying signal into a transient in an artificial time domain, thus permitting use of the previously mentioned techniques, but they do so at the cost of increased computation and signal time. A PATTERN SCHEME Note again the similarity of shapes in Figure 17. Similar observations of the author and the observation by Paynter8 that different criteria usually can be related to a fixed percentage of overshoot as in Figure 17 (Integral square criterion corresponds to 20 percent overshoot, integral absolute to 10 percent overshoot) led to the use of shape as a basis for adaptation. A scheme was developed based on the comparison of the actual process transient response under conventional two-term control A much simpler technique due to Bakke for dealing with continuously varying signals assumes that the disturbances can be defined in terms of their spectrum. The scheme applies simple feedback to enforce certain pretuned relationships on the closed-loop 561 -5- spectrum and is said to work well. The literature9 and work by Bakke and Inagaki indicates that the adaptation takes a considerable amount of time to converge. 2. The adaptation will then process automatically on any set-point or load change occurring in the loop. 3. Adaptation can prevent instability from ever occurring in any loop. 4. For a loop with adaptation, storage far the two or three additional adaptation parameters is required. Optionally, other specification parameters may be stored. 5. For any loop actually undergoing adaptation, the computational resources needed may be as much as two or three times of the unadapted two-term control algorithm itself. However, adaptation is only required when an appropriate transient has occurred. This may occur infrequently. Thus the additional load on the system may be slight. And in a crisis decreased sampling may be used. This is not to say that initial applications should not be approached carefully until operational characteristics are fully understood. But with reasonable care, it will be found that adaptation simplifies, rather than complicates life. This is to be expected since accurate experimental evaluations of spectra, or other statistical parameters, require large amounts of data. One disadvantage with any scheme which requires data taken over a long period of time is the increased likelihood of unaccounted changes in surroundings during adaptation. One of the earliest ideas in adaptation is that patented by Caldwell and heavily experimented with by Pessin.1,2 This scheme used spectrum-related methods to evaluate natural frequency and damping. Generally, schemes which calculate parameters of the spectrum must infer their parameter from a grossly sampled spectrum because every sample usually requires a filter. This limits their effectiveness. CONCLUSIONS Each of the schemes mentioned has its own special assumptions which determine the applicability of the scheme. The problem in designing adaptive control systems has been to come up with a scheme which involves a minimum amount of computation, requires little special understanding, and which will adapt most processes under as many disturbance situations as possible. It should not require pretuning or introduction of special tuning disturbances. Existing evidence indicates that the pattern scheme should meet most of these criteria. Some of the other schemes are also candidates for generally or special purpose practical adaptive control systems. Therefore, it follows that from the point of view of control technology, adaptive control is practical. Adaptation will be an essential future characteristic of the fully automated plant. In addition to giving the control system the means of monitoring control performance for adaptation, it is also giving it a more sophisticated basis for alarming. Thus instability or other dynamic conditions can be alarmed. The simple improvement in control can be significant in two ways: loops which we have been unable to keep tuned will now be kept in tune. But loops which would be unsafe if tightly tuned manually may now be kept tight since the adaptive system can prevent instability. With respect to the field of adaptation itself, it is now necessary to evolve means for adapting larger and more complex process control systems as a unit, automatically taking into account interactions between sections of the plant. The modeling techniques and human set-up methods presently available from the academic world are still too cumbersome to be considered as useful to practical operating personnel. The control system of the future must automate all aspects of the adaptation including any required set-up procedures. What then stands in the way of its application in industry? To begin with, most adaptation requires digital computation. Since it has not been practical to date to produce digitally tuned analog controllers, it follows that adaptation presumes DDC. This is likely to continue until adaptation or other techniques justify development of more specialized controllers or adapting equipment. But even this must await extensive adaptive experience. This same experience is required to develop more detailed design requirements for adaptations. Within the context of DDC, both for the short and long range, the industry will need to consider what are the best operational philosophies to follow. Taking the pattern scheme as a base, it is now possible technically to offer an adaptive DDC system with the following characteristics: 1 TABLE I Values of M, and C, in decimal equivalents of digital units Sample times i ∆Mi in dig. 0 1 2 3 4 5 6 7 8 9 10 11 Scale 2047 dig. = 10 V Human interference will not normally be necessary in any loop except for an initial pattern specification (two parameters specifying the desired quality of control) and adaptation condition specifications (whether or not set-point or load adaptation is allowed for the given loop). Occasionally measurement filtering may become necessary. 561 -7- 0 -225 -122 88 13 -56 4 12 0 -1 0 -1 Ci in dig. 0 307 321 39 22 77 33 -1 7 7 7 7 REFERENCES where pj = ps(jT) with T the sampling time. 1. W. L Caldwell. "Control System with Automatic Response Adjustment." U.S. Patent No. 2,517,081. (Note that better model forms such as those of Kalman and others would be more efficient but less clear for our results.) 2. D. W. Pessin. "Investigation of a Self-Adaptive Three-Mode Controller." ISA September 1963. Preprint 30.3.63. This equation is exact as long as M(t) changes values only at each sampling time. If Ci and Mi-j are known for all successive values of i and i-j the equation becomes a set of linear equations in pj and Ui: 3. T. P. Goodman and J. B. Reswick. "Determination of System Characteristics from Normal Operating Records." ASME No. 55-IRD-1. ASME-IRD Conference. April 1955. C0 = ∆M0p0+ U0 4. E. Mishkin and L. Braun. Adaptive Control Systems. McGraw-Hill Book Company, Inc. 1961. C1 = ∆M1p0 + ∆M0 p1+ U1 5. T. V. Osburn, H. P. Whitaker, A. Kezer. "New Developments in the Design of Model Reference Adaptive Control Systems." IAS 29th Annual Meeting, January 1961. IAS Paper No. 61-39. C2 = ∆M2 p0 + ∆M1p1 + ∆M0p2+ U2 n 6. R. W. Jensen and M. D. Lieberman. IBM Electronic Circuit Analysis Program. Prentice-Hall, Inc. 1968. pp 322-330. Cn = ∑ ∆Mn – j pj + Un j=0 7. R. E. Kalman. "Design of a Self-Optimizing Control System." Transactions of the ASME. February 1958. There are n + 1 equations and 2(n + 1) unknowns. We need n + 1 more equations which follow from experience with practical processes. In real life, all processes have a finite settling time. This is equivalent to a set of (n-k) equations for some k of the form: 8. H. Paynter. "Self-Tuning Controller Studies." December 1964. Unpublished Notes. 9. R. M. Bakke. "Adaptive Gain Tuning Applied to Process Control." ISA October 1964. Preprint 3.2.-1-64. pi = pk for k ≤ i ≤ n 10. E. H. 8ristol, G. F. Inaloglu, J. F. Steadman "Adaptation of Process Controllers by Pattern Recognition." JACC, August 1969. pp 605-606, Similarly many real nonnoise disturbances and upsets are isolated in time and of finite duration giving n-l equations of form: 11. E. H. Bristol et al. "Adaptive Process Control by Pattern Recognition." Instruments and Control Systems. March 1970. pp 101-105 uj. = ul for l ≤ j ≤ n Also for most practical processes p0 = 0. APPENDIX I Applications of these equations to the original set leaves n equations and k and l unknown. As long as k + l is less than n, the system is generally soluble. It is possible to separate this set easily into a set of k equations on pi, in which case the values of Ui can be later calculated directly. In the actual experiment l was assumed as 3, k as 5 and n as 8. This assumption was based on the physical feel of the calculator from looking at Figure 2 and will be natural to experienced persons. The assumption is, of course, justified by the results and it was made without collaboration of any kind between experimenter and calculator. Such an assumption is not critical and can be verified by taking several values of k and calculating the matrix resulting. The choice of l will be justified if changes in l do not change the calculated values of pi and Ui appreciably. The results may require that control be poor enough so that meaningful measurements occur for n greater than a reasonable l + k. This limitation can be avoided by calculating on the basis of several upsets or by recognizing that identification is necessary only under poor control. It should be noticed that changing the sampling time changes the definition of the identification but not the time necessary for the identification. Thus n•T is a constant time chosen appropriately to the process and upset. Under continuous adaptation these factors will not necessar- The process of Figure 1 is defined by the convolution integral relationship where the upset and control begin at time t=0, C(t) = t ∫0 M(t – τ)p(τ) dτ + U(t) where p(t) is the impulse response of the process and U(t) is the upset response or disturbance defined before. This same equation can be rearranged in terms of the step response and the derivative of M. C(t) = t ∫0 ddξ M(ξ) ξ = t – τ Ps(τ) dτ + U(t) where Ps(t) is the step response of the process. The sampled data equivalent of this equation is i Ci = ∑ ∆Mi – j pj + Ui j=0 561 -8- ily cause trouble because any seriously incorrect adaptation will cause control changes which degrade control, but which lead to better identification and thereby to better later control. This phenomenon was used in later adaptive experiments in which k was given a value much smaller than that of the process (up to 2 to l too small). Under these circumstances, identification was extremely erratic but control was acceptable. where: ∆S is the set-point step size, kp and kr are preset scale factors which can be manually or automatically adjusted, if needed, to speed up the adaptation, ρ is a deadtime compensation factor based on TD and is normally equal to unity. Note that the error signal has been normalized by ∆S so that the calculation depends not on set-point amplitude, but only on response shape. APPENDIX II10,11 At the end of T3 the scheme replaces the previous two-term controller gains with the computed gains P` and R'; adaptation stops and conventional twoterm control continues with these updated gains. Consider Figure 18. Any set-point step change of sufficient magnitude and duration applied by the operator, or generated by the program, may trigger the adaptation system. The respective times required for the response to go from 0 percent to 25 percent and from 25 percent to 75 percent of the set-point step are determined as TD and TL. The latter is established as the time scale for the adaptation and is used to establish three consecutive adaptation times: If, on the next set-point change, the scheme still sees a deviation from the desired B and C levels during the adaptation intervals, the controller gain values are updated again according to the above formulae. Procedures are included in the scheme to guarantee universal convergence in any special situations. The scheme considers only one transient at a time, but safe provision is made in case of simultaneous set-point or load changes. Extensions of the scheme permit load adaptations for transients. T1 = T D + T L + α • T L T2 = T 1 + β • T L T 3 = T2 + γ • T L BIBLIOGRAPHY where: Bakke, R. M. "Adaptive Gain Tuning Applied to Process Control." ISA, October 1964. Preprint 3.2.-1-64 α, β, γ are preset constants selected on the basis that the main transient is fully developed someβ + γ) • TL interval, and the first where during the (β half cycle in a resonant response develops during the β • TL interval. This ensures that the relationship between the controller gains, and the corresponding transient pattern comparisons in the adapting formulae are monotonic, assuring negative adaptive feedback only. Bellman, R.; Kalaba, R. E.; Lockett, Jo Ann. Numerical Inversion of the Laplace Transform: Application to Biology, Economics, Engineering, and Physics. American Elsevier Publishing Company, Inc. 1966. Bristol, E. H. "A Simple Adaptive System for Industrial Control." Instrumentation Technology. June 1467, pp 70-74. The two-term controller has the DDC algorithm: Bristol, E. H. et al. "Adaptive Process Control by Pattern Recognition." Instruments and Control Systems. March 1970. pp 101-105. ∆V = P•∆ ∆E + P•R•E where: Bristol, E. H.; Inalaglu, G. F.; Steadman, J, F. "Adaptation of Process Controllers by Pattern Recognition." JACC, August 1969. pp 605-606. ∆V is the incremental output position, P and R are initial proportional and reset gains, E is the instantaneous error, and ∆E is the incremental error. Caldwell, W. I. "Control System with Automatic Response Adjustment." U. S. Patent No. 2,517,081. Goodman, T. P, and Reswick, J. B. "Determination of System Characteristics from Normal Operating Records." ASME No. 55-IRD-1. ASME-IRD Conference. April 1955. During the intervals from T1 to T2 and from T1 to T3, gain computations are made: 1 P′ = P 1 + ------------β ⋅ TL T2 ∑ Time = T 1 1 R′ = R 1 + ---------------------------〈 β + γ 〉 ⋅ TL Inagaki, Tohru. "An Adaptive Direct Digital Control." Case Western Reserve University Systems Research Center Report SRC-68-6. E k p 〈 ------ – B〉 ∆S T3 ∑ Time = T 1 Kalman, R. E. "Design of a Self-Optimizing Control System" Transactions of the ASME. February 1958. E k r 〈 ------ – ρ ⋅ C〉 ∆S Jensen, R. W.; Lieberman, M. D. IBM Electronic Circuit Analysis Program, Prentice-Hall Inc. 1968. pp 322-330. 561 -9- Mishkin, E. and Braun, L. Adaptive Control Systems. McGraw-Hill Book Company; Inc. 1961. Osburn, T. V.; Whitaker, H. P.; Kezer, A. "New Developments in the Design of Model Reference Adaptive Control Systems." IAS 29th annual meeting January 1961. IAS paper No. 61-39. Pessin, D. W. "Investigation of a Self-Adaptive Three-Mode Controller." ISA September 1963. Preprint 30.3.63. 561 -10-