COGNITIVELY INSPIRED RADIO ENGINE FOR IEEE 802.22 WRAN Lizdabel Morales-Tirado Dr. Jeffrey H. Reed, advisor Mobile Portable Radio Group, Virginia Tech, Blacksburg, VA 24061 Abstract Cognitive radio (CR) is a promising tool in the field of wireless communication research. Although algorithms and techniques envisioned in CR research can reside in any of the layers of the protocol stack, current investigations in CR have been focused on the physical layer functionality. Research in radio resource management (RRM) of a distributed, decentralized CR network has begun recently. IEEE 802.22 WRAN system claims to be the first wireless communication standard adopting cognitive radio functionalities. However, current specifications seem to be more like “a frequency agile radio” system. In order to be a cognitive radio system, the system needs to have the capabilities of recognizing its system operating scenarios on top of its awareness of surrounding environment, choosing intelligently efficient strategies and algorithms under, supporting coexistence of other CR systems, and learning from its experiences and predefined rules. During the last year, we investigated and developed a cognitive engine, which is the first application of cognitive radio concept to more efficient radio resource management (RRM) in a centralized network. We have designed the architecture of a cognitive engine (CE) which is generic and flexible so that it can be applied to general cognitive radio systems. Along with a general framework that will allow for future design, development, and testing of more enhanced cognitive engine, we have implemented a cognitive engine which provides means for 802.22 base stations to have scenario-classification and optimization capability by tailoring the developed CE architecture to fit 802.22 base station operation scenarios. the cognitive radio (CR) technologies. IEEE 802.22 will be the first worldwide CR based standard to support the unlicensed operation in TV bands (54-862 MHz), which is to coexist with incumbent users and provide wideband internet access to rural and suburban areas. According to US FCC’s recent public notice, products refarming TV bands are scheduled to be available to the market by February, 20091. The IEEE 802.22 Work Group kicked off in November 2004 and approved the functional requirement document for WRAN systems in September 20052. Ten initial proposals were merged into a single one in March 2006, and the draft standard (D0.1) was developed in May 20063. The complete 802.22 standard is expected to be approved by May 2007. For 802.22 WRAN systems, the primary users (PU), those with priority rights, mainly include incumbent analog and digital TV stations, TV translators, TV boosters, TV receivers, and wireless microphones. WRAN systems, including base stations (BS) and customer premise equipment (CPE), are secondary users (SU), and therefore, should avoid generating harmful interference to the PUs. In this standard, the “intelligence” resides in the 802.22 WRAN base stations. The base station is aware of its environment, and also makes the best decision in terms of spectrum management and maximization, as well as, network optimization. The CPEs help the base station obtain current radio environment information by scanning the spectrum in their vicinity and reporting the results. Figure 1 shows the typical IEEE 802.22 WRAN scenario. WRAN Base Station Introduction CH3 On December 19, 2005, the House of Representatives of the United States approved legislation to complete the country’s transition to new, higher-quality digital television (TV) by February 17, 2009. Refarming the analog TV spectrum promotes new wireless communication technology developments, most notably CH2 CH1 CH1 CPE 4 CPE 3 CPE 1 CPE 2 TV Station (Primary User) Figure 1: IEEE 802.22 WRAN Scenario Morales-Tirado 1 Cognitive Radio Engine Architecture As illustrated in Figure 2, the proposed 802.22 WRAN BS consists of three primary entities: the spectrum sensing module, radio environment map (REM) and the core CE. Both the spectrum sensing module and REM database continually run to keep information current. The core decision-making process exists in the CE module and relies on information stored in the REM. The CE module acts both as a high-level spectrum management entity - allocating television channels and sub-carrier bands as necessary - and as a MAC/PHY layer controller by adjusting coding, modulation, and interleaving levels. Since the 802.22 relies on a very strict policy it is necessary to combine all these levels of decision in order to maximize spectrum usage and minimize interference to primary users. Main Controller Sensing Module Utility REM Core Learning Agent Channel Modeler and Predictor Multi-objective Optimizer Spectrum Manager WRAN Cognitive Engine Figure 2: IEEE 802.22 WRAN Cognitive Engine Architecture Furthermore, cross-layer optimization can provide significant performance improvements over systems which treat each layer as a separate problem. The drawback to this method is producing a potentially large, complex solution space over which searching becomes an issue. To mitigate this problem, we propose a multistep search process as a compromise between the accuracy and complexity of searching. In the first stage the cognition focuses on assigning an appropriate TV channel and initial power setting to new service requests; on the second stage the cognition focuses on assigning the resources to the node, such as: modulation scheme, sub-carriers, time-slots, coding, etc. Learning algorithms in CE Design In this section we discuss several artificial intelligence techniques that can be applied in the design of a cognitively inspired engine for an IEEE 802.22 WRAN network. It is worth mentioning the key characteristics of each technique and their major role in the cognitive radio engine implementation. There are several cognitive tasks that the IEEE 802.22 WRAN base needs to perform. The WRAN system is a secondary user in the TV spectrum thus protecting the incumbent users is a Morales-Tirado primary goal for the base station. In order to accomplish this task, periodic sensing of the radio spectrum is performed by the CPEs in the network. The base station gathers sensing data from the CPEs in the network, and then manages the spectrum accordingly in order to avoid harmful interference to primary users. The second cognitive task is to maximize the available radio spectrum. For this task, the base station relies on maximizing the spectrum usage via effective radio resource management, by predicting the detection of incumbent users and by exploiting past experience stored in the REM. There are many benefits to the addition of cognitive tasks to the WRAN base station, such as: flexibility in the network, adaptability to the current and future situations, increased network optimization, increased network capacity, etc. On the other hand, the addition of these capabilities leads to more complexity in the design and implementation of the WRAN base station, thus creating the need for “intelligent” agents that manage the radio resources in the network. In recent years, there has been lots of research focused on merging intelligent agent technology and telecommunication systems. Wireless technologies have expanded to provide countless services to users, and information is required anytime, anywhere and in any form 4, therefore the need for communication systems that are flexible, and adapt to the needs of the user and the dynamically changing radio environment. In reference 4, the author discusses how artificial intelligence techniques can be applied to wireless communication systems and suggests a functional scheme for implementing an intelligent system to manage communication networks. He also suggests that “intelligence” can be dispersed throughout the OSI layers in the protocol stack. In reference 5, the authors surveyed various machine learning techniques and discussed possible “intelligent” implementations for the design of cognitive networks. They also identify various cognitive networking tasks which include: anomaly detection and fault diagnosis, responding to intruders and worms, and rapid configuration of networks. If we evaluate the IEEE 802.22 WRAN system it can include similar cognitive tasks as those described above, including incumbent detection, network optimization, spectrum radio resource management, etc. Joseph Mitola in his dissertation6 also describes the different cognitive tasks, and furthermore divides the “radio world” of the cognitive radio CR1 in “micro-worlds”, or in other words knowledge domains. It is sometimes difficult to apply a single artificial technique that will provide an accurate solution to all the 2 various problems in a network7, it is necessary to determine which techniques can provide a more accurate and quick solution given the type of domain problem and radio scenario. In the next paragraphs we discuss several of these techniques and describe the characteristics that make them suitable for cognitive radio engine development. In the next paragraphs we survey: knowledge-based (or rule-based) systems, case-based system, artificial neural networks, fuzzy logic, and hidden Markov models. These techniques are discussed in the context of “expert systems”, and the expert system in this case has the goal of managing the 802.22 WRAN. The most common expert system is the Knowledgebased (or Rule-based) expert system. Knowledge-based systems rely in a simple architecture: a set of rules included in a module known as the knowledge base, an inference engine and a working memory. The inference engine processes the rules and the working memory contains the information of the current problem. The accuracy of this technique depends upon the available domain knowledge. Another key issue is the amount of rules that some systems may require, as the complexity of the system increases so as the domain knowledge. The 802.22 scenario is suitable for the use of knowledgebased systems, especially when complemented with inductive learning techniques (such as case-based reasoning) to account for the imperfect knowledge in the system such as interference, location of incumbents, etc. As an example, the authors in reference 8 implemented a generic cognitive radio using a knowledge-based system. Another common technique used to build expert systems is Case-based reasoning (CBR), which main focus is experience rather than knowledge. CBR is analogous to an old physician; the experienced physician can quickly diagnose a patient since he has seen many similar cases before. CBR is an area of machine learning that concentrates on using previous similar cases to guide the problem solving process and to achieve a solution. This technique has been successfully applied in expert systems in the areas of law, medicine, manufacturing, design, education and many others. The author in reference 9 describes in detail a case-based reasoning approach to network management. The author divides network management in various tasks: network design, planning, and implementation, management of customers, and finally operation and maintenance budgeting. CBR is suitable for the development of a cognitive engine as it performs very well in dynamically changing environments, and can provide solutions even in the absence of perfect domain knowledge. Morales-Tirado Another technique not frequently included in expert systems is artificial neural networks (ANN). ANNs form powerful generic modeling techniques, well-suited for use within cognitive radios, particularly due to their learning capabilities, scalability, and dynamic flexibility in describing numerous situations7. By modeling the biological plexus in organisms, ANNs are nothing more than a set of nonlinear functions with adjustable parameters to give a desired output. Each artificial neuron produces a single output value by accumulating that from other neurons, thus forming a network. Networks can be classified by their neuron configuration, interconnections, and training methods, allowing for a multitude of applications. Because of their ability to dynamically adapt and be trained at any time, ANNs are able to “learn” patterns, features, and attributes of systems they describe. For this reason they have long been used to describe functions, processes, or classes that are difficult to formulate analytically, particularly within device control, but only recently have they been proposed for use within cognitive radios7. The authors in reference 10 implemented a cognitive radio using an ANN. The network was used only to assist the engine in its decision for parameters to be used in the next block of transmissions. The engine used the network to determine how closely a set of parameters met a set of goals, and then chose the set of parameters which maximized the utility functions. The engine re-trained the network based on the decisions made and the resulting outcome. Fuzzy logic is derived from fuzzy set theory; it refers to the process of reasoning that is approximate rather than precisely deduced from classical predicate logic. Fuzzy logic uses a “qualitative” approach rather than a “quantitative” one. Furthermore, fuzzy logic allows for set membership values between and including 0 and 1, and in its linguistic form, imprecise concepts like "slightly", "quite" and "very". Specifically, it allows partial membership in a set. It is related to fuzzy sets and possibility theory. Fuzzy set theory has been applied successfully to computing with words or the matching of linguistic terms for reasoning11. In terms of our current implementation fuzzy logic can be used together with CBR to aid in the reasoning of non-numerical parameters as suggested by the authors in reference 11. Also using fuzzy set theory to measure similarities between cases may allow for simpler comparisons, multiple indexing schemes, and term modifiers to increase flexibility in the case retrieval process. Hidden Markov Models are convenient and mathematically tractable tools, and useful to describe and analyze the dynamic behavior of complicated random phenomena. In general, a real world process that can or cannot be expressed as a random process produces a sequence of observable symbols or pattern. The symbols 3 or patterns can be discrete or continuous depending on the process of specific applications. If we build a signal model that explains and characterizes the occurrence of the observed symbols or patterns, then we can use that model later to identify or recognize other sequences of observations by choosing most likely model close to obtained model. In an 802.22 network the HMM is a suitable learning technique for classification, detection and possibly channel prediction. In this section, we focused on the discussion of algorithms that are suitable for the development of a cognitive radio engine, and their key characteristics. Due to time constraints, only a few algorithms were implemented. Learning Agents In this section we discuss the learning agents implemented in the current cognitive radio engine design for the IEEE 802.22 WRAN. We first discuss the casebased reasoning engine and follow with a discussion of a hybrid design which includes knowledge and case-based engine. Case-based Reasoning (CBR) Engine For this initial implementation our team chose to implement a simple case-based reasoner, as a proof of concept that this learning technique is suitable for cognitive radio engine implementation. Case-based reasoning was selected since its implementation didn’t require complete domain knowledge. Also, surveyed literature presented various successful implementations of CBR systems for network management 9,12 and radio resource management in telecommunication systems 13,14 . In the next paragraphs we briefly discuss the CBL process. Case-based learning (CBL), also known as case-based reasoning, focuses on using previous similar experiences; or in other words, cases to guide the problem solving process and to achieve a solution. In CBL, a solution to the problem is created by selecting the cases that are relevant to the problem, and then the best match is chosen and adapted to fit the current case or instance. The case selected by the retrieval process may not match exactly with the current problem; the relevant case may need to be adapted to meet the requirements of the current situation. This process is called case adaptation. In summary, the case-based learning approach can be described by four action words: retrieve, reuse, revise and retain. Figure 3 shows the basic case-based reasoning approach. Morales-Tirado Retrieve Reuse Revise (Relevant cases) (useful experience) (adapt) Retain (for future cases) Solution New Case Figure 3: Case-base reasoning approach For this implementation in Step 1 we created a simple case memory implemented as a text file with a few typical scenarios. Step 2 and Step 3 were executed by performing a single nearest neighbor search and returning the closest match to the input case. By default, the case library search process assigns equal value to all the parameters, but the user can easily modify the search by changing the weight vector. For example, if the matching has to be performed solely of the amount of radio resource units requested, the weights vector will have a 1 for that parameter and 0 for the rest. One important key issue to note is that currently the nearest neighbor algorithm returns a single case; however, in the future the algorithm can be modified to return k cases. The case-based reasoner then can perform further evaluations on the relevant cases and develop a more accurate and tailored solution. Step 4 has been implemented by following a small set of rules as describe in Table 1; in the future it will be necessary to include sensing information in an attempt to uniquely describe a case. The key issue with this approach is that the case library can increase significantly as more cases are gathered, thus requiring faster searching mechanisms. Integrating Knowledge in the Reasoner As mentioned previously, one of the key issues of casebased reasoning is that the solution is only as good as the case library itself. When we have not compiled enough cases or there are no cases that are relevant to our current case, it is useful to combine both approaches: using case and knowledge based reasoning to achieve a solution. Multi-objective Optimizer The multi-objective optimizer of WRAN CE is used to maximize the system utility subject to multiple constrains, such as the interference to PUs, and CPE transmit power. In other words, CE needs make decisions and/or adaptations in multiple domains, such as frequency (channel selection), time (timeslot and frame scheduling) and power. Because of the high dimensionality the problem, however, its performance will be highly dependent upon its starting point, and therefore will rely highly on the output of the CBL entity. 4 The general objective of WRAN BS is to meet the ToS (type of service) and QoS requirements of CPEs with minimal costs and risk of interference to PU, subject to practical constraints of bandwidth (number of TV channels supported simultaneously) and transmit power (spectral mask specified by standard body). For 802.22 systems, to avoid/reduce harmful interference to incumbent PUs is always the primary concern. We have condensed the complexity of the search problem to a linear programming search over a finite set of values. Discretizing each parameter relaxes the programming restrictions and simplifies the problem. For this stage of development we are considering the following search agents: 1. 2. 3. 4. Exhaustive search Local dimension search Hill climbing search Genetic algorithm We have successfully implemented and verified the exhaustive, local-dimension, and genetic algorithm searches. We are in the process of using it to find the optimum solution to randomly generated scenarios such that we can populate the case library. This will provide a benchmark against which more efficient search agents can be tested. Simulation Approach A basic framework for testing algorithms for the proposed 802.22 cognitive base station has been developed. It has been written in C++ and provides a basic platform for analyzing performances of the modules proposed in this section. It is capable of generating multiple base stations, CPEs (nodes), and wireless microphones (Part 74 devices). Each node has a unique service request that the cognitive base station must satisfy before its activation. The framework has the capability of analyzing the performance while varying the following parameters: television channel (currently fixed at 6MHz, but easily changeable), transmission power, modulation scheme, FEC coding scheme, and the number of subcarriers on both the downlink and uplink for each CPE. The simulation starts the base station with a certain task (such as activating links to new users or vacating users from a television channel). The simulator analyzes the performance of the proposed cognitive engine in two main categories: CPU run time and resulting performance. Morales-Tirado Utility Function Performance of the cognitive engines is based upon spectrum utilization as well as individual CPE utilities. The global utility is defined as utotal = 0.5ucpe + 0.35uspectrum + 0.15 min {ucpe } where the spectrum utility is simply the ratio of candidate channels to total channels, and the individual node utilities are comprised of three simple sub-goals: a. Meet the application BER requirement (50%) b. Meet the application data rate (35%) c. Minimize transmit power (15%) These can be expressed mathematically as follows: (k ) k k ucpe 0.5uBER 0.35udata 0.15u kpower See Figure 4 for the individual utility functions for the CPEs. Results Two simple cognitive base stations were implemented with the framework; one which relies on knowledgebased rules to determine optimum parameter sets and allocate resources to the nodes, and one which relies completely upon a genetic algorithm search for the best solution. The simulation results can be seen below in Figure 5. Five basic scenarios were described using external files. In each scenario, both cognitive engines were required to add up to 40 nodes to an existing network. Table 2 describes the scenarios under test. Each scenario was repeated 20 times with each new CPEs having a random location with respect to the base station. The maximum range was set to 33km for any node, and the application requirements were chosen randomly among four levels with bit error rates ranging from 10-2 to 10-6, and data rates from 10kbps to 750kbps. According to Figure 5, CE that uses genetic algorithms produced a consistently better result in terms of end utility, but consumed several orders of magnitude more CPU clock cycles. Furthermore, as the number of CPEs added increases, the end utility between the two cognitive engines converge. 5 Data Rate Utility 0.9 0.9 0.8 0.8 0.7 0.7 0.6 0.6 data 1 utility u utility u BER BER Utility 1 0.5 0.4 0.5 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 -2 10 -1 0 10 1 10 BER/BER0 0 2 10 10 0 0.2 0.4 0.6 0.8 1 D/D0 1.2 1.4 1.6 1.8 2 Transmit Power Utility 1 0.9 0.8 utility u power 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -5 -4 -3 -2 -1 0 1 P - P0 [dB] 2 3 4 5 Figure 4 : Utility functions for individual CPEs 3 10 1 GA CKL GA CKL 0.95 Average Adaptation Time [ms] 0.9 Average Utility 0.85 0.8 0.75 0.7 0.65 2 10 1 10 0.6 0.55 0.5 0 0 5 10 15 20 25 30 Number of CPEs Added 35 40 45 10 0 5 10 15 20 25 30 Number of CPEs Added 35 40 45 Figure 5: Preliminary Results Morales-Tirado 6 Table 1 Rules Problem Types (1) Add new connection to BS Case Attributes [1] CPE_profile (id, ToS, location) [2] RRU_Req_New [3] RRU_Assigned [4] MaxAvailableRRU_CH [5] MinAvailableRRU_CH [6] Active_CH [7] Candidate_CH Candidate Solutions [1] Reject new connection [2] Activate a Candidate Channel with highest "channel reputation" [3] Use the lowest_RRU_assigned_CH [4] Use the highest_RRU_assigned_CH [5] Use the largest_Number of CPEs and reloacte lowRRU_CPE to other Active channel [6] Use the smallest_Number of CPEs [7] Relocate to MaxAvailble_CH (to get the max. available BW) [8] Relocate to MinAvailble_CH (for BE type of service) [9] Use the most matched Available_RRU CH Action [1] if Similarity > threshold_h1 && Est_Utility > threshold_h2, skip optimizer; else, initialize operation parameters, and run Optimizer [2] refer to “channel reputation” for candidate channel selection [3] Use lowest frequency Channel for CPE with large pathloss (e.g., remote CPE) [4] Use highest frequency Channel for CPE with low pathloss (e.g., close to BS) Table 2 Testing Scenarios Scenario # of existing CPEs # of CPEs to add # of initial candidate channels 3 # of initial active channels 1 1 2 2 3 10 5 2 8 10 10 2 8 4 10 20 3 7 5 10 40 3 7 Morales-Tirado 9 7 Conclusions and Future Work IEEE 802.22-05/0007r46, “Functional Requirements for the 802.22 WRAN,” Sept., 2005. 2 IEEE 802.22 WRAN is the first application of cognitive radio networks refarming the TV broadcast bands. The standardization and deployment of cognitive WRAN systems will have significant impact on the future advancement of cognitive radios. In this paper, we describe our research efforts in developing both a cognitive radio engine framework that integrates the spectrum management capabilities outlined in the IEEE 802.22 standard, as well as a few preliminary cognitive engine models. We have developed a solid and generic architecture and a general framework that will allow for future design, development and testing of various learning, decision-making and optimization techniques that will provide efficient performance and accurate solutions for the IEEE 802.22 problems. 3 IEEE P802.22 (D0.1) Draft Standard for Wireless Regional Area Networks Part 22: Cognitive Wireless RAN Medium Access Control (MAC) and Physical Layer (PHY) specifications: Policies and procedures for operation in the TV Bands”. 4 M. Kaiser and Z. Cucej, "Artificial intelligence in modern telecommunication systems: analysis and implementation," in Proceedings of Video/Image Processing and Multimedia Communications, 2003. 4th EURASIP Conference focused on, Volume 2, 2-5 July 2003 Page(s):805 - 810 vol.2. 5 Using this general framework we have the tools to implement and optimize specific modules for cognitive engines and test their performance and functionalities further for future development. Based upon these initial results, we can formulate which algorithms are suited for specific scenarios. By tailoring our engine to the strengths of each algorithm, we can optimize the performance for an efficient cognitive engine for 802.22 wireless base stations. Therefore, to enhance the functionality and performance of CE, we propose to refine utility functions and testing scenarios under possible real-world system operation scenarios, enhance the core engine by incorporating more sophisticated case and knowledge adaptation or decision fusion methods, extending case library with more sophisticated radio scenarios, and implement more multi-objective optimizers as fine optimizers. Acknowledgment I would like to thank Dr. Jeffrey H. Reed, my advisor, for his support during my graduate student career. Also, special thanks to my colleagues Joseph Gaeddert and Youping Zhao, who have worked on this research problem with me. Also, thanks to our sponsor, ETRI, for all their assistance in the development of this Cognitive Radio Engine. This project was also funded by the Virginia Space Grant Consortium and the Pratt Fellowship. References Federal Communications Commission, “Office of Engineering and Technology Announces Projected Schedule for Proceeding on Unlicensed Operation in the TV Broadcast Bands,” FCC Public Notice, ET Docket No. 04-186, Sept., 2006 1 Morales-Tirado T. Dietterich and P. Langley, "Machine Learning for Cognitive Networks: Technology Assessment and Research Challenges," Technical Report, May 2003. 6 J. Mitola, III, "An Integrated Agent Architecture for Software Defined Radio," PhD Dissertation, Royal Institute of Technology, Stockholm, Sweden, 2000. 7 J. Gaeddert, K. Kim, R. Menon, L. Morales, Y. Zhao, K. K. Bae, and J. H. Reed “Applying artificial intelligence to the development of a cognitive radio engine”, MPRG Technical Report, June 30, 2006. 8 E. Stuntebeck, T. O'Shea, J. Hecker, and T. Clancy, "Architecture for an Open-Source Cognitive Radio," SDR Forum Technical Conference, November 2006. 9 L. Lewis, "Managing Computer Networks: A CaseBased Reasoning Approach," Artech House, 2005. 10 C. R. Aguayo, J. Gaeddert, K. Kim, K. K. Bae, L. Morales, M. Robert, Y. Zhao, C. Dietrich, and J. H. Reed, "Desing and implementation of an open-source software-defined cognitive radio testbed," unpublished paper. 11 S. K. Pal and S. C. K. Shiu, "Foundations of Soft Case-Based Reasoning," John Wiley and Sons, Inc., 2004. 12 M. Stadler, "Case-based Reasoning for Network Management," in Proceedings from Topics in CaseBased Reasoning, First European Workshop, EWCBR93, pp. 414-423, 1993. 13 S. Chantaraskul and L. Cuthbert,"Using Case-Based Reasoning in Traffic Pattern Recognition for Best Resource Management in 3G Networks," in 8 Proceedings from 7th ACM international symposium on Modeling, analysis and simulation of wireless and mobile systems, pp. 252-259, 2004. 14 N. Yao and L. Cuthbert, "Resource Management in 3G Networks using Case-Based Reasoning," in Proceedings of the 8th ACM international symposium on Modeling, analysis and simulation of wireless and mobile systems, pp.307-312, 2005. 15 J. Mitola, III, "An Integrated Agent Architecture for Software Defined Radio," PhD Dissertation, Royal Institute of Technology, Stockholm, Sweden, 2000. 16 J. Gaeddert, K. Kim, R. Menon, L. Morales, Y. Zhao, K. K. Bae, and J. H. Reed “Applying artificial intelligence to the development of a cognitive radio engine”, MPRG Technical Report, June 30, 2006. Morales-Tirado 9