1 Pricing and Power allocation in Femtocells using Stackelberg Game Theory V. Udaya Sankar and Vinod Sharma Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore 560012, India Email: {uday,vinod}@ece.iisc.ernet.in Abstract—We consider a system with multiple Femtocells operating in a Macrocell. The transmissions in one Femtocell interfere with its neighboring Femtocells as well as with the Macrocell Base Station. We model Femtocells as selfish nodes and the Macrocell Base Station protects itself by pricing subchannels for each usage. We use Stackelberg game model to study this scenario and obtain equilibrium policies that satisfy certain quality of service. Keywords- Femtocell, cellular communication, power control, QoS, Game theory, Decentralized algorithms, Stackelberg Game. I. I NTRODUCTION Carbon footprint of Information and communication Technology (ICT) industry has been a cause of concern in recent years [15]. The energy consumed in cellular networks is a significant part of it. Small cells and Femtocells (FCs) ([5], [6]) are an important part of the overall strategy to reduce the RF transmission in a cellular system. These cells have small, low power Access Points (APs) deployed indoors. These are connected to DSL lines or cable TV and are used to provide service to indoor users. Since data generated indoors is about 60% of the total data carried in a cellular network, these APs can divert a substantial part of the network traffic from Base Station (BS) and improve the coverage and capacity of the indoor users. Also, then the BS has to transmit at a much lower power. As envisaged, there will be many FCs/small cells in a (macro)cell, each servicing a building. These small cells will share the spectrum with each other and also with the macro cell (MC). Thus the users in different FCs and MC will interfere with each other. Therefore, to be able to utilize the channels efficiently the FCs and the Macro Base Station (MBS) will need to coordinate with each other such that their users get their Quality of Service (QoS). We address this problem in this paper. In the following we provide related literature survey. Game theoretic approaches for multiple access are surveyed in [1]. The studies on Interference management in FCs are summarized in ([9], [10], [18]). In [17], authors studied distributed multichannel power allocation for spectrum sharing cognitive radio networks with QoS guarantee. Down link power allocation in FC based cellular network was considered in [7] using Stackelberg game theory. Spectrum sharing between MBS and FBSs is studied in [2]. Cross tier interference mitigation using power allocation in two tier FC networks is studied in [8]. In [3], authors studied the problem of pricing uplink (UL) power in wideband cognitive radio networks under the objective of revenue maximization for the service provider while ensuring incentive compatibility for the users. The distributed power allocation strategies for spectrum sharing FCs deployed in a MC using Stackelberg game was investigated in [11]. In [19], authors studied the joint pricing and power allocation for dynamic spectrum access using a Stackelberg game. In this paper we consider the problem of UL/Downlink (DL) subchannel power allocation to MSs in a multi FC multi channel environment by considering interference at the MBS using the Stackelberg Game model. The MBS computes the price per subchannel usage under the constraints of QoS to MC users and maximum interference limit at each subchannel. At each FC, joint power allocation is done under the QoS constraints to its users. In the previous studies, authors considered only single channel case UL subchannel allocation with no QoS guarantee. Furthermore, in our case the followers’s game becomes a potential game and we use regularizaton to obtain a unique equilibrium (NE) for the follower’s game resulting in algorithms providing a Stackelberg equilibrium under much weaker conditions than in literature [11], [19]. We also guaranteed a minimum QoS (rate) to the FCs and MBS and use minimum power at the MBS and FCs. This paper is organized as follows. Section II describes the system model and formulates the problem. Section III provides algorithms to obtain a Stackelberg equilibrium. Section IV shows the efficacy of the algorithms via simulations. Section V concludes the paper. II. S YSTEM M ODEL AND P ROBLEM FORMULATION We consider a two tier cellular system in which within a MC there are K FCs (Fig 1). The cellular system has N subchannels, perhaps using OFDMA (e.g., LTE or WiMAX) and the subchannels are shared by the FCs and the outdoor users in the MC. The transmission between a Femto Access Point (FAP) and its Mobile Stations (MSs) may be in the uplink or downlink or in both directions. Each of the FCs would like to ensure a minimum rate for its users. The transmission considered in the MBS is only in the uplink (this is because we will consider a constraint on the interference at the MBS). Also, the MBS wants to ensure that its users get their rate requirements. To ensure that the different users in the FCs and in the MC use channels in such a way that 2 on its channels, i = 1, 2, ..., N . The interference limit Qi is fixed such that (1) is satisfied with a power P̂i ≤ P̂i,max when Ii = Qi . If FC k uses power Pik on channel i and has channel gain k hi then its interference to MBS on channel i is Pik hki and it pays βi Pik hki to the MBS as interference price. Thus the FC k would like to use powers Pk = (P1k , P2k , ..., PNK ) on the N channels such that it minimizes N X βi hki Pik , (4) i=1 while ensuring that it can have at least total rate R̄k in the FC, i.e., N X Cik ≥ R̄k , (5) Fig. 1. i=1 Block diagram of the system P k Gk from the where = log2 1 + σ2 +αk P̂ +iP i Gk,l P l l6=k i i i i k Shannon capacity formula and Gi is the channel gain at FC k for channel i, αik is the channel gain from the MBS user to FC k for channel i and Gk,l i is the channel gain from FC l to Cik every one’s QoS is satisfied without a centralized control, the MBS fixes a price for the usage of each channel. Then each FC decides its channel usage. We use Stackelberg game frame work to use this approach for the FCs and the MBS to utilize the channels. Let P̂i be the power used by a user in the MC on the ith channel and let ĝi be its channel gain. Let Pik be the transmit power used in FC k on channel i. Also, let hki be the channel gain of the ith channel from the k th FC to the MBS. All the channel gains will be assumed constant during a slot for which decisions need to be made. Also we assume that all channels are Additive White Gaussian Noise (AWGN) channels with receiver noise σ 2 . The receiver noise power can be different for different receivers and it will not make any difference in our power management algorithms. The MBS charges βi ≤ β̄i < ∞ as the interference price per unit PKof interference from the FCs. Thus, the MBS charges βi k=1 hki Pik as interference price from all the FCs together for channel i. A user in the MBS using channel i requires a minimum rate R̂i and uses power P̄i in the absence of any interference from FC users, P̄i ĝi i.e. , R̂i = log2 1 + σ2 where σ 2 is the receiver noise FC k on channel i. Also, in FC k there is a maximum power constraint, Pik ≤ P̄ik , ∀i = 1, 2, ..., N. (6) We will take the utility of FC k as negative of (4) and denote by Uk . To obtain an equilibrium solution for the system at which all the users can agree to operate in a distributed manner while satisfying all the constraints, we formulate it as a Stackelberg game where the MBS is a leader while the FCs act as followers. The MBS fixes a price for the channel and the FCs (and the MBS) adjust their power allocations accordingly so as to satisfy their rate requirements. By pricing, the MBS influences the equilibrium point obtained so that it satisfies the constraints (3). For this we divide the optimization problem at the MBS into two parts, namely power allocation problem and interference price computation problem: Interference Price Computation Problem: For each P̂ and P , power and uses P̂i ≤ P̂i,max in the presence of interference which satisfies, log2 (1 + σ2 + P̂i ĝi PK k k k=1 hi Pi ) ≥ R̂i , ∀i = 1, 2, ..., N. (1) For this the MBS has to pay price A(P̂i − P̄i ) where A is a constant representing a marginal power cost. Thus MBS has a utility UM BS (β, P̂ , P ) = N X K X βi hki Pik − A i=1 k=1 N X (P̂i − P̄i ) (2) i=1 where P̂ = (P̂1 , P̂2 , ..., P̂N ), P = (P1 , P2 , ..., PN ) the powers at the FCs and β = (β1 , β2 , ..., βN ). The MBS would like to choose the price β to maximize its UM BS while satisfying QoS constraints (1) and interference limit Ii = K X k=1 hki Pik ≤ Qi , (3) N X K X max 0≤βi ≤β̂i i=1 k=1 βi hki Pik ∀i = 1, 2, ..., N, (7) such that Ii = K X hki Pik ≤ Qi , ∀i = 1, 2, ..., N. (8) k=1 Power Allocation Problem:For each interference price β min A 0≤P̂i ≤P̂i,max N X (P̂i − P̄i ) ∀i = 1, 2, ..., N, (9) i=1 such that log2 (1 + σ2 + P̂i ĝi PK k=1 hki Pik ) ≥ R̂i , ∀i = 1, 2, ..., N. (10) The strategy space for MBS ( allocated power per subchannel) for the power allocation problem is ΩM BS−C (Pk ) = {P̂ : P̂i ≤ P̂i,max , 3 log2 (1 + σ2 + P̂i ĝi PK k k k=1 hi Pi ) ≥ R̂i , ∀i} (11) which is convex and compact. For each price β = (β1 , β2 , ..., βN ) announced by the MBS, the FCs and the MBS adjust their powers to arrive at a Nash point which satisfies (if possible) each player’s rate requirements. Those powers are announced to the MBS. Then the MBS again modifies its prices (according to an algorithm in Algorithm 1 below). The game may eventually converge to an equilibrium, called Stackelberg equilibrium. At this equilibrium point we obtain the highest utility for MBS at which all the rate requirements at FCs and MBS are satisfied, the interference constraints at the MBS are met. In the following we discuss conditions which may guarantee the existence of an equilibrium point and then provide distributed algorithms to converge to an equilibrium point. III. S OLUTION CONCEPTS AND D ISTRIBUTED A LGORITHM Let Ωl and Πl be the strategy space and utility function of the leader and Ωk and Πk of the k th follower for k = 1, 2, ..., K. Let xk ∈ Ωk be a strategy of follower player k, x−k be the strategy of other followers and xl ∈ Ωl be a strategy of player l. Then xk is called the best response of player k for (x−k , xl ) if Πk (xk , x−k , xl ) = supx Πk (x, x−k , xl ) and then we write xk ∈ BRk (x−k , xl ). Similarly we define the best responses of the leader and the other followers. Definition 1. The strategy (xl , xk , k = 1, 2, .., K) is a Stackelberg equilibrium if (1) xl ∈ BRl (xk , k = 1, 2, .., K) and (2) xk ∈ BRk (x−k , xl ), ∀k = 1, 2, .., K From [4] we obtain conditions for existence of a Stackelberg equilibrium. Proposition 1. There exists at least one Stackelberg NE if: (a) Ωk is nonempty, compact and convex, ∀k = 1, 2, ..., K, (b) Ωl is nonempty, compact and convex, (c) Πk is once continuously differentiable and pseudo concave with respect to xk given any (xl , x−k ), ∀k = 1, 2, ..., K, (d) Πl is continous. We assume that Rate requirements at the leader and the follower’s are feasible within the given constraints. Then, it is easy to see that our problem in section II satisfies the above assumptions and hence we have existence of a Stackelberg NE. Now we develop distributed algorithms using which the system can converge to a Stackelberg equilibrium. A. Convergence condition To develop our distributed algorithm further, we will use the concept of a Potential non-cooperative game. Definition 2. A strategic game GQ = {I , X , (Φk )k∈I } where I is the set of players, X = k Xk is the strategy space and Φk is the utility of player k, is called an exact Potential game if there exists a function f : X → R such that ∀k ∈ I , xk , yk ∈ Xk and x−k ∈ X−k , Φk (xk , x−k ) − Φk (yk , x−k ) = f (xk , x−k ) − f (yk , x−k ). (12) Theorem 1. ([14]) Let G = {I , X , (Φk )k∈I } be a potential game, with a potential function f , and let Pmax denote the set of maxima of f (assumed non empty). Then each point x ∈ Pmax is a NE of G . If f is strictly concave then the game has a unique NE. It is easy to verify that the power allocation game played by FAPs and MBS is a potential game with potential function f (x) = − K X N X k=1 i=1 βi hki Pik − A N X (P̂i − P̄i ). (13) i=1 which they try to maximize. The strategy spaces of FCs and the power game of the MBS are compact and convex sets and the Potential function f is continuous. Hence there exists at least one NE for this game for each price β. Also, the best response iteration converges to a Nash Point ([12]). For the best response iteration, each user in turn responds with a strategy that maximizes its utility, given the strategies of the other players. This can be done by solving the optimization problem using Karush-Kuhn-Tucker (KKT) conditions. Since our utility function is linear we will get a global optimal solution. Thus for the k th FC, the best response power allocation is 1 λk − k ]+ (14) Pik = [ βi hki + µik di where λk , µik are Lagrange multipliers such that eq(5)-(6) are Gk satisfied, (x)+ = max(0, x) and dki = σ2 +αk P̂ +Pi Gk,l P l . k i i l6=k i i At this point if feasible the total rate achieved by FC k is R̄k . Similarly, for the MBS, PK σ 2 + k=1 hki Pik + λi ] , ∀i = 1, 2, ..., N, (15) P̂i = [ − A ĝi where λi ’s are the Lagrange multipliers. The FCs do not explicitly need to exchange their best responses. Rather, each FC at its turn, measures the interference and nose power at each channel and then computes its best response powers (14), (15) and transmits in each channel at that power. The overall distributed algorithm to compute the Stackelberg NE works as follows. The MBS starts with an initial price vector β0 = β̂ = (β̂1 , ..., β̂N ) and announces it to all FCs. The FCs and the MBS iteratively update their power strategies via best response strategies (14) and (15) to eventually converge to a NE. Existence and convergence to a NE corresponding to any price β is obtained because we have a potential game corresponding to each β. This NE is sent to the MBS which updates its price according to an algorithm in Algorithm 1 below and announces it to the FCs. Basically, we start with the maximum possible prices βi = β̂i , ∀i (to maximize revenue). If from the corresponding Nash Point Interference Ii in channel i exceeds Qi , lower the price of channel j with Ij < Qj − by a small amount ∆. This will make the FCs increase their powers in these channels and reduce in others. The FCs and the MBS now again play the non cooperative game to converge to another NE and send to the MBS. This is repeated till we obtain convergence of the overall game to a Stackelberg NE. At equilibrium if feasible the rate constraints 4 will be satisfied with equality everywhere if possible. This distributed algorithm is summarized in Algorithm 1. We now discuss the issue of uniqueness of a NE for a given β. In general a potential game may have multiple NE. Therefore, corresponding to each price β, our algorithm may pick one of the NE. It is desirable to have a unique NE. It is shown in [16] that if the set of channel gains satisfies k Gi k2 < 1, ∀i = 1, 2, ..., N where k . k2 is the L2 norm and ( k,l Gi , i Gk [G ]l,k = i 0, (16) if k 6= l otherwise, IV. N UMERICAL R ESULTS then there exists a unique NE for the follower’s game. Conditions (16) are satisfied if ∀i = 1, 2, ..., N , P a. b. k Gk,l i +αi < 1, ∀k ∈ K, k Gi PK k k=1 hi < 1, ∀i = 1, 2, ..., N . ĝi l6=k Algorithm 1 Distributed Algorithm a. Initialize β 0 = (β̂1 , β̂2 , ..., β̂N ) and P̂ = (P̂1 , P̂2 , ..., P̂N ) and broadcast to all FAP’s, n = 1. b. Power Iteration ( Do until convergence of power) 1) solve (4)-(6) using KKT for all FC’s for getting Pk = (P1k , P2k , ..., PNk ), ∀k = 1, 2, ..., K using eq (14). 2) By using PK , solve for P̂ = (P̂1 , P̂2 , ..., P̂N ) at MBS using eq (15) iteratively until convergence.( can use fmincon function to compute power) PK c. From eq (3) measure Iin = k=1 hki Pik , ∀i = 1, 2, ..., N at MBS and update the interference price based on the following algorithm. • Take a small positive constant. This controls the algorithm accuracy. Let NS = {i : Iin > Qi + } and S = {i : Iin < Qi − }. n • if (Ii ≤ Qi + , ∀i) then stop and return β n • else if (| S |6= ∅ and | NS |6= ∅) then βin+1 = βin − ∆, ∀i ∈ S • end if • n=n+1 d. Repeat (b),(c) until | Ii − Qi |≤ , ∀i = 1, 2, ..., N . The above conditions are quite restrictive. Thus we also use another method [13] by adding regularization terms to PN the utilities of the players as Φk (x) = − i=1 βi hki Pik − PN PN c i=1 (Pik )2 and ΦM BS (x) = −A i=1 (P̂i − P̄i ) − PN c0 i=1 (P̂i )2 , where c and c0 are small positive constants. The resulting Potential function is given by g(x) = − K X N N X X ( βi hki Pik + c (Pik )2 ) k=1 i=1 −(A N X i=1 (P̂i − P̄i ) + c0 i=1 N X i=1 (P̂i )2 ). It is observed that g(x) is strictly concave. Hence, the corresponding game has a unique NE. We use this NE in our algorithm. This NE will be close to a NE of the original problem which has low total power requirement. In the next section we will run the algorithm and show convergence of our algorithm to a Stackelberg NE (SNE) on an example. We have seen the convergence in several other examples also. The proof of convergence of the overall algorithm to a SNE is still under investigation although at an intuitive level we can see why it converges. (17) We consider two FC’s with 4 subchannels and one scheduled user deployed in MC environment. We consider the following parameters for simulation. We are simulating for uplink only. Bandwidth of each channel is : B = 1KHz, Background Noise σ 2 = 0.02, = 1.5 × 10−2 . MBS: channel gain from MMS to MBS is : ĝ = 0.9, 0.7, 0.8, 0.9 . Channel gain from FAP-1 user to MBS is : h1 = 0.04, 0.03, 0.05, 0.01 . Channel gain from FAP-2 user to MBS is : h2 = 0.02, 0.01, 0.04, 0.05 . Maximum constraint is (in watts) : subchannel power P̂max = 10, 4, 7, 8 . Rate requirement per subchannel in the MBS is (in Kbps): R̂ = 3, 4, 2, 3 . Maximum tolerable interference limit is : Q = 0.02, 0.005, 0.033, 0.05 . Interference price initial value is : βinit (= β̂) = 2, 3, 4, 5 . Price increment is : ∆ = 0.07. FAP-1: 0.8, 0.5, 0.6 . Subchannel Gain is :G1 = 0.7, Cross channel gain is : G1,2 = 0.2, 0.4, 0.1, 0.3 . = Channel gain from MMS to FAP-1 is : α1 0.05, 0.1, 0.2, 0.03 . Maximum rate requirement (in Kbps) is : R̄1 = 5, Power per subchannel (in watts) is : P̄1 = 5, 5, 5, 5 . FAP-2: 0.6, 0.4, 0.7 . Subchannel Gain is :G2 = 0.9, Cross channel gain is : G2,1 = 0.1, 0.3, 0.2, 0.4 . 2 Channel gain from MMS = to FAP-1 is : α 0.02, 0.2, 0.03, 0.01 . Maximum rate requirement (in Kbps) is : R̄2 = 7, Power per subchannel (in watts) is : P̄2 = 5, 5, 5, 5 . For regularization we considered c = c0 = 0.05. With regularization case: Total Powers allocated at FAP1 is 1.2964, FAP2 is 2.0642 and MMS is 1.6703 and the total utility for MBS is 0.1675 with exact rates achieved at both Femtocells as 5 and 7. Similarly the Rate requirements at the MBS also satisfied with equality for each user. Without regularization case: Total Powers allocated at FAP1 is 1.2348, FAP2 is 2.1937, MMS is 1.6887 and total 5 Fig. 2. Variation of interference price, without regularization Fig. 3. Variation of interference price using reqularization utility for MBS is 0.2079 with exact rates achieved at both Femtocells as 5 and 7. Similarly Rate requirements at the MBS also satisfied with equality for each user. Fig 2 shows the variation in Interference price when we do not use regularization (condition (16) is satisfied) in our example. Fig 3 shows the price evolution for the regularization case. In Figs 4 and 5 we show the power allocation in the four channels at the FC 1 and FC 2 respectively for regularization case. We see that as the price of a channel in Fig 3 reduces, the power in that channel at FC 1 and FC 2 is increases. Also, if a channel has lower Qi , this channel uses less power at FCs. V. S UMMARY We have considered the problem of interference management in a Macrocell with multiple Femtocells and multiple channels. Each Femtocell and the users in the Macrocell need guaranteed minimum rate and the interference at the MBS needs to be below a threshold. Using Stackelberg Game theory we have developed a distributed algorithm which achieves the above aims. R EFERENCES [1] K Akkarajitsakul, E Hossian, D Niyato, D I Kim, “Game theoritic approach for multiple access in Wireless Networks: A Survey”, IEEE communication surveys and tutorials, Vol. 13, No. 3, Third Quarter, 2011. Fig. 4. Allocated Power with regularization in FC 1 Fig. 5. Allocated Power with regularization in FC 2 [2] M Bennis, M Debbah, “On spectrum sharing with underlied Femtocell Networks”, IEEE 21st International Symposium on PIMRC workshop, 2010. [3] A A Daoud, T Alpcan, S Agarwal, M Alanyali, “A Stackelberg game for Pricing Uplink Power in Wide-Band Cognitive Radio Networks”, IEEE conference on Design and Control, 2008. [4] S. C Choi, W S. Dasarbo, P T. Harker, “Product positioning under price competition”, INFORMS, Management Science, Vol. 36, No. 2, pp. 175199, February, 1990. [5] H Claussen, L T.W.Ho, and L G.Samuel, “An Overview of the Femtocell Concept”,Bell Labs Technical Journal 13(1),221-246,2008. [6] ‘Femtoforum’, www.femtoforum.org. [7] S Guruacharya, D Niyato, E Hussain, D I Kim, “Hierarchical compitation in Femtocell based cellular networks”, IEEE Globecomm, 2010. [8] Z Huang, Z Zeng, H Xia, “Hierarchical power game with dual-utility in two-tier Femtocell networks”, IEEE WiCOM, 2011. [9] Interference management in UMTS femtocells, Femtoforum white paper. [10] Interference management in OFDMA femtocells, Femtoforum white paper. [11] X Kang, Y-C Liang, H K Garg, “Distributed Power control for SpectrumSharing Femtocell Networks using Stackelberg game”, IEEE ICC, 2011. [12] A B. MacKenzie, L A. DaSilva, “Game theory for wireless engineers”, Morgan and Claypool Publishers. [13] J Morgan, F Patrone, “Stackelberg Problems: Subgame Perfect Equilibria via Tikhonov Regularization”, Annals of the International Society of Dynamic Games, Volume 8, 2006, pp 209-221 [14] G. Scutari, S. Barbarossa, D. P. Palomar, “Potential Games: A framework for vector power control problems with coupled constraints”, ICASSP 2006 Proceedings. [15] M Z Shakir, k A. Quaraqe, “Green Heterogeneous small cell netowrks: toward reducing the CO2 emissions of mobile communicatins industry 6 using uplink power adaptation”, IEEE Communicatin Magazine, June, 2013. [16] K. Shum, K. Leung, and C. Sung, “Convergence of iterative water filling algorithm for Gaussian interference channels,” IEEE J. Sel. Areas. Commun. , vol. 25, no. 6, pp. 1091-1100, Aug. 2007. [17] Y Wu, D H. K. Tsang, “Distributed Power Allocation Algorithm for Spectrum Sharing Cognitive Radio Networks with QoS Guarantee”, IEEE Infocom, 2009. [18] T Zahir, K Arshad, A Nakata, K Moessner, “Interference management in Femtocells”, IEEE communication surveys and tutorials, Vol. 15, No. 1, First Quarter, 2013. [19] Y Wu, T Zhang, D H.K. Tsang, “Joint Pricing and Power allocation for Dynamic spectrum access networks with Stackelberg Game Model”, IEEE Transactions on Wireless Communications, Vol: 10, January, 2011.