BUSINESS STRATEGY FORMULATION MODELING VIA HIERARCHICAL DYNAMIC GAME Celso Pascoli Bottura; Eliezer Arantes da Costa Laboratório de Controle e Sistemas Inteligentes – LCSI Faculdade de Engenharia Elétrica e de Computação – FEEC Universidade Estadual de Campinas – UNICAMP <bottura@dmcsi.fee.unicamp.br>; <eliezer@futuretrends.com.br> Campinas, São Paulo, Brazil Abstract – Stochastic infinite dynamic games can be used as a conceptual basis for the development of strategic business game models for cooperative and competitive environments and under turbulent conditions. This study presents an analytical framework to develop a challenging and motivating tool for the improvement of executive strategic management capability, using discrete time stochastic strategic hierarchical dynamic games. The dynamic game is organized in three hierarchical levels: at the upper one lies the coordinating module, representing the market actions, at the intermediate level are the competing companies, and at the lower level those responsible for each company’s management units. Each company’s team has a computational model to simulate the company’s dynamic behavior. Some classic game theory equilibrium strategies are presented and applied to the proposed strategic game. Stackelberg, Nash and Pareto equilibrium strategies, duly combined in the proposed hierarchical structure, allow the attainment of theoretical lower-bound solutions for didactic games. Keywords – Business strategic games – hierarchical dynamic games – strategic management training Introduction Several efforts have been made to model and to solve problems of optimization for large-scale systems with hierarchical structures, as much for tactical as for strategic use, in productive systems [1, 2] and in decision support systems [3]. The formalization of non-cooperative dynamic game theory in [4] serves as the basis for a series of interesting applications, among them the treatment of an optimization problem utilizing distributed and parallel processing in hierarchical structures [5]. Non-cooperative dynamic game theory is a field that can be explored in educational applications for development and training of executives, which is the purpose of this study. Concepts about company strategy have been consolidated in [6], reviewed and expanded in [7], and applied to manufacturing for instance, in [8]. A tutorial consolidation of the concepts of strategic management is presented in [9]. More recently, an effort toward strategic planning was developed in [10], in which a strategic business game is employed to generate turbulent scenarios. Such research creates a business game using dynamic simulation resources, applying principles originally proposed in [11], revised and expanded to apply in the business arena in [12]. The work presented here, expanding the application of [10], employs competitive and turbulent scenarios, in which companies are simulated vying for markets and clients and, at the same time, disputing limited resources and supplies. A stochastic strategic hierarchical dynamic game for application in executive training programs is proposed here as a model for a competitive and turbulent business environment 1. Executive capability development Experiments in the development of human resources and specialized studies about adult education – also called ‘andragogy’, by some authors – show that traditional teaching methods, based on presentations, lectures, readings and seminars, lack something as they do not motivate, nor involve participants on their real day-to-day problems. In contrast, playful activities such as business games and firm simulations in cooperative or competitive conditions and under turbulent environments are methods that, if well applied, can stimulate emulation, enthusiasm, and motivation – essential elements for efficient learning. Some modern pedagogical applications for the development and improvement of companies’ future managers and decision-makers have used business games, mainly implemented as computer models for companies’ operational programming in short-term time frames, usually based on spreadsheets concepts. It is a fact that a vast amount of literature on this subject [6, 8] – even some of the most recent – [7, 9], treat the concepts of strategic management from a more qualitative than quantitative point of view. Nevertheless, some efforts have been made to quantify strategic actions and their long-term consequences through mathematical models [10, 12]. Company modeling using optimization and simulation concepts and methods may reasonably represent these business environments, at least for educational purposes. Concepts and resources for modeling and simulation of continuous systems can be found in [11, 12]. For the present work, important contributions are in [4, 5, 13, 14, 15, 16, 17]. y Business games were greatly developed mainly from the 1950s on as an adaptation, for the company’s environment, of war games concepts, methodology and results that had long been used in the military field. Experiments have shown that participants of business games end up enthusiastically motivated to increase their capabilities, including abilities to use more quantitative tools. Well-elaborated games are able to integrate segmented knowledge and concepts from various fields or disciplines, thus enabling participants to form a global vision of a company and its market as a whole. The team decision-making interactive process fosters interpersonal relationships; it provides incentive for the search for objective data, for the ability to seek out consensus and for solutions by negotiation. Leadership, clear and assertive communication, and giving and receiving feedback capabilities are also highlighted. Otherwise, competition among distinct teams encourages the ability to make decisions under pressure, uncertainty, conflicting situation and tension, seeking consensual solutions internally, while under intense external competition. 3. Non-cooperative deterministic game In order to implement a conceptual platform for business games, we begin at the concepts and formulations of dynamic games theory. With a sufficiently broad meaning for this purpose, we can define a noncooperative deterministic dynamic game [NDDG] with several participants and multiple stages as a systems optimization problem with multiple decentralized and autonomous decisions. Thus, from the point of view of systems control theory, a [NDDG] can be associated with a particular problem of optimal control with multiple controllers. In this type of game, each of the N participants – or players – receiving information progressively disclosed by the structure of the game, makes a sequence of decisions, stage by stage, attempting to optimize one’s objective function (1) – while obeying the game constraints. For a formal presentation of the optimization problem introduced above, we adopt the following notation, derived from the terminology of systems theory [4]. For the basic model, we call: x k the vector that represents the state of the game at stage k; (1 ) We always make use of the concept that the players must minimize their objective function. k an observation of the state x k for the player Pi at stage k; η ki the information available to the player Pi at stage k; u 2. Educational uses of business games i i k the action decided by the player Pi at stage k. So, we can define an infinite non-cooperative deterministic dynamic game [NDDG] as a structured set of logical-mathematical conceptual elements, whose relationships are graphically represented in Figure 1. In fact, the above-referred definition assumes some simplification hypotheses: (a) the duration of the game is fixed; (b) time is assumed to be discrete, by stages; and (c) all equations, functions and variables are admitted to be deterministic. Restrictions (a) and (b) of the [NDDG] do not limit its applicability to strategic business games for educational purposes, but the item (c) will be relaxed in Section 4 below. 4. Non-cooperative stochastic game The [NDDG] described above does not adequately represents a business environment due to the failure to encompass random possibilities in the process, which are common in the business world, full of surprises brought about by both external and internal factors. An extension of the [NDDG] concept to deal with these stochastic aspects can be accomplished with small changes in some of the formal elements described above. A simple way to understand and to treat the randomness of the game is to add into the model a new ‘player’, the (N+1)th, called nature, S, whose decisions and actions influence the state of the game evolution and the objective function value calculation. Nature’s actions can be assumed to obey a probabilities law established a priori and thus, the state transition equation in [NDDG] is replaced by a function of a conditional state probability distribution, given the previous actions of the players and the previous values of the state [4]. Keeping these considerations in mind, we can define an infinite non-cooperative stochastic dynamic game [NSDG] discrete in time, with N players, with a pre-defined fixed duration, as a structured set of logical-mathematical conceptual elements, whose relationships can also be illustrated graphically in Figure 1, by merely introducing one more player, S, the nature, and replacing the deterministic character by a stochastic one for some variables and functions. From the point of view of systems control theory, a [NSDG] non-cooperative stochastic game can be associated with a particular problem of optimal stochastic control with multiple controllers [1, 4]. Considering the sufficient generality of the [NSDG] game, it is adopted as the conceptual reference platform for the strategic business games treated in this work. U i k : u k ∈U k i U Γ Γ :γ i i k k ∈ Γk i u i k η Informat ion Space Ν :η i i k k Ν i = γ (η i i k k u η ∈ Νk i k u ⊂ { y ,..., 1 k u i i i 1 y ,u k (z zi* = imin ∈N f 1 x 1 k k +1 k ..., u k ,..., u k ) x k ,..., y N i y k = i h (x k k x ⇐ k x Observat ion Funct ion ,..., u1 ,..., u k −1} yes K +1 ? no k −1 i = x N i )= i z ( xk , u k , i k u : N k −1 xk +1 = Delay i i X St ate Transit ion i ) k k ∈ X i η k k Informat ion i St ruct ure k x i Decision / Control k : X i St rat egy Space End of Game: * The Winner is i for which * St ate Space Control Space K +1 k +1 k +1 x k Object ive Funct ion z : J ( x ,..., x u ,..., u ,..., u ) i z i = K +1 1 i 1 i N 1 k K , Init ializat ion: Init ial st at e is given k ) x k Y z i k ⇐ x 1 i k Observat ion Set Y i k : y i k ∈Y k i Figure 1 – Sche mat ic representation of a non-cooperative dynamic ga me – [NDDG] / [NSDG] 5. Strategic reference game Applying the concepts and formulation of [NSDG] to model business games for educational purposes requires several simplifications and specifications without loss of the generality and applicability intended for the given purpose. We will work with a special class of games, which we call strategic reference game [SRG], defined as a particular case of [NSDG], which is, simultaneously, strategic, stationary, equitable, and closed-loop, encompassing the following concepts: (a) A strategic game can be defined as a [NSDG] for which the objective function to be minimized is an exclusive function of the final state of the game, x K +1 , that is, z i = J (x i (i ) K +1 ) ; i ∈ N ; ( 2) (b) A stationary game can be defined as a i[NSDG] for (i ) i which the functions f k (...) , hk (...) and γ k (...) and the space Ν k can be rewritten as i and f (i ) (...) , i h (...) and γ i (...) Ν , respectively, discarding, thus, the subscript k ; i (c) An equitable game can be defined as a [NSDG] for which the superscript i, which differentiate the characteristics of (ithe players, cani be discarded. Thus, ) i the functions f k (...) , hk (...) , γ k (...) , J i (...) and the space J (...) Ν can be rewritten as and Ν , respectively; i k f k (...) , h (...) , γ k k (...) , k (d) A closed-loop game can be defined as a [NSDG] i for which η k = { x1 ,..., xk} , for every k ∈ K , that is, the players have access to all information about the past states of the game. 6. Games with hierarchical structure Proceeding the work with [SRG], we further specify its structure, in order to obtain a more useful formal model for the intended purpose. 6.1. Hierarchical game in two levels A non-cooperative hierarchical game in two levels is modeled through a process of forming a group of subsystems, each one representing a competing company, with its own operative rules for state transition, information, decision, and objective function. Each company, here represented by a subsystem [CS]i , vies in the market for raw materials, specialized production man power, managerial resources, financial resources, technology, and other supplies. On the other hand, they also compete in the market for clients’ preferences. The market, in the broader sense, also interferes on the game, acting on prices and quantities transacted by the N competitors, their clients and providers. A method of representing the market actions is to introduce into the game a new player, the (N+2) th, which we will call ‘the market’, symbolized by Q, which has the duty of seeking out the balance between the aggregate demand and the aggregate offer for both supplies and products. The [SRG], organized as described above, is defined as a hierarchical game in two levels, [HG2]. The formulation of this concept can be obtained through a convenient partition and segmentation process of the [SRG] game, resulting in two types of subsystems described below: I. Company subsystems [CS]i – The mathematical formulation of a company subsystem [CS]i, for k ∈ K and i ∈ N can be written as follows: (a) i x k +1 The problem of optimal stochastic control with a strategic objective function such as defined here can be seen as a specific application of predictive control [19]. The goal is to optimize the desired ‘final result’. That is, the ‘closer’ the players are able to place their companies to the target final state, the better their game performance is graded. f ( x ,u , λ ) , i i i i k k k k x given the initial state i 1 , for each subsystem [CS]i , and (b) (2 ) = z =J i y = h ( x ) , η k ⊂ { yk , uk −1} , i i i k k k i K +1 i i i u = γ (η ) i i i k k k and i ( xK +1) . II. Market Coordinator Subsystem [MCS] – The mathematical formulation of the market coordinator subsystem [MCS], k ∈ K and i ∈ N , similarly to the those prices. [CS]i subsystem (3) can be written, respectively, as: Coordination by prices is what mostly resembles the situation prevalent in a free competitive market, with multiple suppliers and multiple buyers, without any of them regulating or controlling the market prices and/or quantities. (a) m = k +1 g (m , λ ,..., λ 1 k k k v = ρ (m ) , µ δ = R (m ) . (b) k k k k k k ,..., λ k , u k ,..., u k ,..., u k ) , and k i N 1 = {vk , λ k −1} , i N λ = β (µ ) i i i k k k and k The [CS]i modules communicate only with the market coordinator subsystem, [MCS], which informs to each one of them, at the beginning of each new i stage, its decision parameter, λ k . The [CS]i, in turn, i informs the [MCS] about their decisions uk . 6.2. Hierarchical game in three levels By analogy with the treatment given to the [SRG] game, we can additionally expand the dynamic hierarchical game [HG2], applying a further segmentation process to each company subsystem [CS]i: each of the competing i companies is assumed to consist of G Managerial Units, [MU]ij, where j ∈ {1,2,..., G} , introducing G new players for each company. These managerial units, [MU]ij, represent the main functional or managerial areas of the company. In this sense, each [MU]ij has its own state transition equation, information structure, strategies, and decisions, and a specific objective function to be minimized. However, leaders of managerial units should seek out consensus with their peers to reach the optimal objective function for their company as a whole. Therefore, by analogy with the segmentation presented at Section 6.1, the model for [HG2] can be expanded to obtain [HG3] wherein the coordination of a second level is achieved by a new module called [CSC]i, representing the coordination of all the [MU]ij, by the company’s chief executives. 7. Coordination of hierarchical game Resuming the problem of multi-level coordination, we choose, for simplicity sake, the [HG2] as a basic reference. Two classic ways to treat the matter of multilevel coordination have been proposed: coordination by quotas and coordination by prices. We will only describe the latter, which is the one utilized in this study. (4) Within this option, player Q i establishes market prices, λ k , as much for supplies as 8. Equilibrium strategies To formulate a dynamic hierarchical game, such as the game [HG2], or even better, as [HG3], the best strategies for the player ‘market’, Q, for the company directors, Pi, for the company coordination the [CSC]i and for those in charge of the Managerial Units [MU]ij should be those ones that optimize their respective objective functions. In strategic hierarchical dynamic games, the purpose is to find a sequence of decisions that brings the system to the final desired state– or as close as possible to it. The classic ways of solving this problem would be to treat it as an optimal control problem, solvable, in theory, by Pontryagin’s Minimum Principle, by the Calculus of Variations or by Dynamic Programming [4, 13, 18, 19, 20], depending on the case. These theoretical methods, however, are not adopted in this study, as they do not make use of the specific knowledge of the problem and its topology, on the way we are implementing and exploring, with multiple heuristic human decision-makers. From a careful analysis of possible alternative strategies for each configuration of the relationship between players in [HG3] results the following conclusions: (i) The competitive relationship among players Pi can be treated as a typical Nash equilibrium strategy, (5) since they act independently from each other and are prevented from sharing information and from cooperating among each other for making coordinated decisions in order to optimize together their objective functions; (ii) The relationship among those responsible for the Managerial Units [MU]ij of the same company i can be characterized as a game to which the Pareto equilibrium strategy (6) applies, since it is a game of (5 ) A Nash equilibrium point, The symbols m, g , λ , v, ρ , µ , β , δ , R used for the [MCS] subsystem are defined accordingly by analogy to [CS]i. (4 ) In coordination by quotas, the market establishes quantities of supplies and of products for each player. The players offer prices for supplies and products they wish to buy and to sell. 1 i N for a non- cooperative game (K=1) of variable sum, with N players, may be i i defined if, for every u ∈U , i ∈N , the following N inequalities are simultaneously true: 1 i J (u ,..., u ,..., u N 1 1 fori final products, and players Pi make their decisions uk about the quantities they want to buy and sell at (3 ) * u = (u ,...,u ,...,u ) ∈U i J (u ,..., u ,..., u N i 1 i J (u ,..., u ,..., u N )≤ 1 i J (u ,..., u ,..., u N 1 1 ) , ..., ) ≤ J i (u ,..., u ,..., u ) , ..., N )≤ i 1 N i J (u ,..., u ,..., u N N ). (6 ) A Pareto optimum in a variable sum cooperative game (K=1), with N players, if it exists, can be defined as the point * 1 i N for which there does not exist any other point = ( ,..., ,..., ) ∈ u u u u U u = ( u ,..., u ,..., u 1 i J (u ) ≤ J (u ) , i i i i N ) ∈U ∀i ∈ N , that meaning satisfies that the J (u i inequalities i )≤ J (u i i ), variable sum, in which there should be cooperation among the managers in charge in order to optimize the objective function for their company as a whole; (iii) The relationship between the player Q, coordinator, representing the market, and each player Pi can be interpreted as a typical Stackelberg equilibrium strategy (7), in which the market is the leader and each player Pi acts as a follower; (iv) The relationship between the internal coordinator of each company, called the [CSC]i, and each [MU]ij can also be considered as a typical Stackelberg equilibrium strategy, since the strategic decisions of the coordinator are assumed to be known a priori by each unit manager. Despite these considerations, the direct application of the conditions mentioned above – the Stackelberg, Nash and Pareto equilibrium strategies – to the game in question ends up handicapped for practical reasons because, with the exception of Q, the other players are human beings making heuristic decisions. Nonetheless, these conclusions are certainly useful to the conceptual organization of the games, considering that: (a) they enable the creation of formal models for business games, which expand our understanding of alternative strategies available to the players; (b) they enable the creation of a correct taxonomy for business games within the concepts of classic game strategies; and (c) they enable, under some specific conditions, the generation of algorithms for the calculation of a lower-bound theoretical solution, (8) which could serve as the lower limit to the heuristic solutions human players can produce. Figure 2 illustrates the application of the concepts presented here, for a hierarchical game in three levels, indicating applicable classic equilibrium strategies for each case. 1st Level Market Coordinator Stackelberg Strategy 2nd Level Co mpany 1 ... Co mpany i ... Co mpany N Stackelberg Strategy Nash Strategy 3rd Level Managerial Unit i,1 ... Managerial Unit i,j ... Managerial Unit i,G Pareto Strategy Figure 2 – Classic strategies applicable to a hierarch ical ga me on three leve ls - [HG3] 9. Implementing business games A business game in three hierarchical levels, such as [HG3], can be computationally implemented through the interconnection of the following modules. (9) 9.1. Implementing subsystems [CS]i The computational implementation of a subsystem [CS]i is achieved in such a way that the player Pi can ‘simulate’ as many times as he/she deems necessary, within the allowed time limit, within each stage, the dynamic behavior of its company, within the planning horizon span. Among the computational environments that generate dynamic simulation tools available on the market, the software Vensim® DSS32 5.1 (10) can be adopted, utilizing the concepts of level variables (11) and of flow variables, (12) duly inter-related. The software Simulink® (13) can also be used. 9.2. Implementing subsystems [MU]ij ∀i ∈ N , only if J (u ) = J (u i i i inequality written for at least one i∈N i ), ∀i ∈ N , with the . (7) Let a hierarchical game between player M, called the leader, and a player P, called the follower, be with strategic decisions λ and u , J (λ , u ) , respectively, in which, the player M, opts first for his decision λ and, following and objective function R (λ , u ) that, P opts for his decision u and , knowing λ beforehand. A Stackelberg’s equilibrium point, if it exists, is defined as equal to ( , u ) ∈ ( L ,U ) for which: λ T : L →U such that, for any J (λ , T λ ) ≤ J (λ , u ) for every The computational implementation of the [MU]ij modules uses the same methodology described above, with dynamic models in Vensim or Simulink. In order to implement a strategic cooperation among those in charge of the [MUs]ij, they are encouraged to share information and heuristically seek the optimization of the objective function of the company as a whole. The players – the executives of each company – make their decisions in a cooperative way, with the purpose of minimizing a multi-criteria global function, which should take into account the ‘strategic health of the company’, (14) assuring the state trajectory of their a) There is a transformational relation fixed λ ∈L , u ∈U b) There is for every (8 ) λ ∈L λ ∈L such that , where R (λ ,T λ ) ≤ R (λ ,T λ ) u =T λ . We call the lower-bound solution that set of decisions for a strategic game, which optimizes all the objective function for all the players at all levels. (9 ) The [HG2] game can be treated as a particular case of the [HG3]. Ventana® Simulation Environment, of Ventana Systems, Inc. (11) The level variables represent the quantities that can be modeled as results of an accumulation process. For example: raw materials in stock, orders backlog, employees level, etc. (12) The flow variables represent quantities that can vary over time as functions of decisions made by managers, clients, providers, competitors, etc. (13) Simulink® is a module of MATLAB 6.0, of MathWorks, Inc. (14) A successful implementation of a solution for this problem [5] is the one that adopts a linear quadratic objective function, associated (10) companies is maintained within a hyper-tube feasibility region. (15) 9.3. Implementing modules [CSM] and [CSC]i The algorithm for the subsystem coordination [CSM] module generates the prices for supplies and products that are informed to the players – executives of competing companies. For the aggregate supply, the algorithm forces the prices to rise whenever the aggregate demand has a tendency to overtake its supply availability; and it forces prices to fall, in the opposite situation. For the aggregate supply of products to the market, the behavior of the coordinator is the reverse. The [CSM] module also generates randomly, according to specific probability distributions, the i numeric sample values for the variables θ k ∈ Θ and ϕ ∈Ψ i k influencing the game, thus creating probabilistic scenarios with graded rigor levels, both externally and internally. The module that coordinates all the [MU]ij, called [CSC]i , implemented at the level of the [SSE]i, follows the same principles and mechanisms of the [CSM] module, with appropriate adaptations. 10. Conclusions Some comment and conclusions can be summarized as follows: • The classic equilibrium strategies from game theory are applicable to adequately represent the optimal cooperative and/or competitive players’ strategic decisions in hierarchical business games; • Game theory concepts and models can be used for computation of a solution to be used as a lower-bound for the heuristic solution human beings can produce; • Complex competitive and turbulent business environments can be adequately modeled as stochastic hierarchical strategic games; • Dynamic non-cooperative game theory can be used as a conceptual and formal platform for the development of strategic business games for educational purposes; • Business games constitute a support decision tool for good strategic decision formulation via heuristic teamwork and game theory equilibrium strategies concepts utilization. • The educational experiment under implementation using the model here presented is sufficient to show that the framework and modeling proposed can be efficiently used as a didactic tool for the education and of decision makers and for fostering their capability for planning and strategic management, as intended. to the ‘distance’ – within a given metric – between the values obtained by the players and the established target-value. (15) We call feasibility hyper-tube that region of the space X consisting of all the possible state trajectories of the system, x ik ∈ X , obeying the rules of the game. Bibliografical References [1] A.E. Bryson, Jr. & Y.C. Ho, Applied optimal control (Washington, DC: Hemisphere, 1975). [2] E.A. Costa, Otimização de operação de redes de escoamento (Campinas, SP: Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica, Dissertação de Mestrado, 1979). [3] A.P. Sage, Decision support systems engineering (New York, NY: Wiley, 1991) [4] T. Başar & G.J. Olsder, Dynamic non-cooperative game theory (Philadelphia, PA: SIAM, Series in Classics in Applied Mathematics, Philadelphia, 1999). [5] J.T. Costa Filho, Proposta para computação assíncrona paralela e distribuída de estruturas especiais de jogos dinâmicos (Campinas, SP: Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica, Dissertação de Doutorado, 1992). [6] H.I. Ansoff, Implanting strategic management (Englewood ClifCS, NJ: Prentice-Hall, 1984). [7] S.B. Zaccarelli, Estratégia e sucesso nas empresas (São Paulo,SP: Saraiva, 2000). [8] M. Porter, Competitive strategy (New York, NY: Free Press, 1980). [9] E.A. Costa, Gestão estratégica (São Paulo, SP: Saraiva, 2002). [10] R. Colombo, Aplicação de jogo de empresas: Um experimento com geração randômica de cenários em sistemas dinâmicos (São Paulo, SP: Fundação Getulio Vargas, Dissertação de Doutorado, 2003). [11] J.W. Forrester, Industrial dynamics. (Cambridge, MA: The MIT Press, 1961). [12] J.D. Sterman, Business dynamics: Systems thinking and modeling for a complex world (New York, NY: McGraw-Hill, 2000). [13] R.E. Bellman, Dynamic programming (Princeton, NJ: Princeton University Press, 1957). [14] C.P. Bottura, H.M.F. Tavares & E.A. Costa, Hierarchical control of a production-transportation network with buffer storage, Proc. International Conference on Cybernetics and Society - IEEE, Denver, 1979, 280-285. [15] J.T. Costa Filho & C.P. Bottura, Hierarchical multidecision making on a computer network with distributed coordination and control. Proc. 39th Annual Allerton Conference on Communication Control and Computing, 10/1991, Urbana, 1991, 703-704. [16] Y.Y. Haimes & D. Li, Hierarchical multiobjective analysis for large-scale systems: review and current status, Automatica, v. 24 (1) 1988, 53-69. [17] J.B. Cruz, Jr, Leader-follower strategies for multilevel systems, IEEE Transactions on Automatic Control, v. AC-23.(2) 1978, 244-255. [18] M. Athans & P.L. Falb, Optimal control: An introduction to the theory and its application (New York, NY: McGraw-Hill, 1966). [19] R.R. Bitmead, M. Guevers & V. Wertz, Adaptive optimal control (New York, NY: Prentice Hall, 1990). [20] A.P. Sage & C.C. White, Optimal system control (Englewood Cliff, NJ: Prentice-Hall, 1977).