UNIVERSIDAD POLITÉCNICA DE MADRID DEPARTAMENTO DE AUTOMÁTICA, INGENIERÍA ELECTRÓNICA E INFORMATICA INDUSTRIAL ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES CENTRO DE ELECTRÓNICA INDUSTRIAL A novel methodology for planning reliable wireless sensor networks TESIS DOCTORAL Autor: Danping He Master of Electronics Engineering from Politecnico di Torino Directores: Teresa Riesgo Alcaide Doctora Ingeniera Industrial por la Universidad Politécnica de Madrid Jorge Portilla Berrueco Doctor por la Universidad Politécnica de Madrid en Ingeniería Electrónica 2014 Tribunal Tribunal nombrado por el Mgfco. y Excmo. Sr. Rector de la Universidad Politécnica de Madrid, el día de de 2013. Presidente: Javier Uceda, Universidad Politécnica de Madrid Vocales: Roberto Sarmiento Rodríguez, Universidad de las Palmas de Gran Canaria Celia López Ongil, Universidad Carlos III de Madrid Alan Mc Gibney, Cork Institute of Technology Secretario: José Ramón Casar, Universidad Politécnica de Madrid Suplentes: Ángel De Castro Martín, Universidad Autónoma de Madrid Marta Portela García, Universidad Carlos III de Madrid Realizado el acto de lectura y defensa de la Tesis el día de de 2013 en la Escuela Técnica Superior de Ingenieros Industriales de la Universidad Politécnica de Madrid. Calificación: EL PRESIDENTE EL SECRETARIO LOS VOCALES To my parents Acknowledgements I came to this beautiful country three years ago to pursue PhD study in CEI. I’ve been getting acquainted with those lovely professors and colleagues, who inspire me the best on scientific researches and bring a lot of happiness to my life. I do not feel alone on the way fighting for dreams, because deep in my heart, I know that there are someone always besides me and supporting me. At this important moment when I about to finish this doctoral thesis, I would like to express my sincerely appreciations to them. First and foremost, I would like to thank my advisor Professor Teresa Riesgo. I’ve been profoundly benefit from her advises and inspirations over the past three years, which are always important guiding lights leading me to the bright way towards scientific research. I deeply thank her for the unprecedented freedom she offered to explore my intellectual curiosity in my work, and for fostering my capacity critically as an independent researcher. I would also sincerely thank my co-advisor Dr. Jorge Portilla. I absorbed important knowledge from him through every discussion, not only on professional area but also on other aspects that will absolutely benefit my future careers. I learnt a lot from him and progressed fast. I will always appreciate him for all the patience to correct my work, and for the consistent support to eliminate my nervousness before presentations. I take this opportunity to record my gratitude to Professor David Symplot-Ryl and Dr. Nathalie Mitton, who offered me the exchange research stay at Inria Lille - Nord Europe. Their valuable advices and ideas not only make me successful in academic publications, but also illuminate another research direction and methodology to me. I would like to thank the members of my thesis committee, for generously offering their time, support, guidance and good will throughout the preparation and review of this document. Gabriel Mujica, my project partner in CEI. We work together for the great aspiration of achieving success in DPCM project. His knowledge on hardware programming perfectly compensates my shortage and it’s my fortune to have his collaboration. I would like to thank him for all the intelligent and patient works on establishing sensor testbed to allow us making real measurements efficiently and precisely. I would also express my acknowledgement to Edu, who gives me so many important advises and encouragements. From time to time, he concerns about my work and my progress on preparing documents for the thesis committee. Without his great efforts, the whole procedure would not be run smoothly.(!) All the lovely professors in CEI, José Antonio, Yago, Rafa, Jesús, Pedro and Roberto. I sincerely thank you all for the supports and encouragements on my study. I gain quite significantly from your teaching and supports. I enjoyed a wonderful time in CEI with all the colleagues (CEIers). They are earnest, friendly and hard-working, and I would like to appreciate them for all the help and happiness they gave me. My special gratitude should give to Marcelo who taught me on how to efficiently use Latex, Monica who translated for me the abstract of this thesis, and Nico who gave me so many helpful suggestions on multi-objective optimization and writing papers, thank you so much! I would like to express my deep appreciation to James Zhao for everything he has done for me. His working attitude inspire me so much along these years. His valuable advises, encouragements and generous supports bring warmth to me and shine my life, like the sunlight in severe winter. My acknowledgement will never be complete without the special mention of my Chinese friends. I would like to thank Tianjun Zhou, Cheng Xie, Yang Wang, Pengming Cheng, Wei He, Sisi Zhao, Zhi Wang and Ke Guan for being with me and trusting me. Their moral supports and motivations drive me to do the best and I find myself lucky to have friends like them. Finally, I would like to give my greatest appreciation to my parents Dedong He and Sujuan Qiu, for showing faith in me and giving me liberty to chase what I desired, for standing behind me with all their love. Sincerely thank you all. Danping ii Abstract Wireless sensor networks (WSNs) have shown their potentials in various applications, which bring a lot of benefits to users from both research and industrial areas. For many setups, it is envisioned that WSNs will consist of tens to hundreds of nodes that operate on small batteries. However due to the diversity of the deployed environments and resource constraints on radio communication, sensing ability and energy supply, it is a very challenging issue to plan optimized WSN topology and predict its performance before real deployment. During the network planning phase, the connectivity, coverage, cost, network longevity and service quality should all be considered. Therefore it requires designers coping with comprehensive and interdisciplinary knowledge, including networking, radio engineering, embedded system and so on, in order to efficiently construct a reliable WSN for any specific types of environment. Nowadays there is still a lack of the analysis and experiences to guide WSN designers to efficiently construct WSN topology successfully without many trials. Therefore, simulation is a feasible approach to the quantitative analysis of the performance of wireless sensor networks. However the existing planning algorithms and tools, to some extent, have serious limitations to practically design reliable WSN topology: • Only a few of them tackle the 3D deployment issue, and an overwhelming number of works are proposed to place devices in 2D scheme. Without considering the full dimension, the impacts of environment to the performance of WSN are not completely studied, thus the values of evaluated metrics such as connectivity and sensing coverage are not sufficiently accurate to make proper decision. • Even fewer planning methods model the sensing coverage and radio propagation by considering the realistic scenario where obstacles exist. Radio signals propagate with multi-path phenomenon in the real world, in which direct paths, reflected paths and diffracted paths contribute to the received signal strength. Besides, obstacles between the path of sensor and objects might block the sensing signals, thus create coverage hole in the application. • None of the existing planning algorithms model the network longevity and packet delivery capability properly and practically. They often employ unilateral and unrealistic formulations. • The optimization targets are often one-sided in the current works. Without comprehensive evaluation on the important metrics, the performance of planned WSNs can not be reliable and entirely optimized. • Modeling of environment is usually time consuming and the cost is very high, while none of the current works figure out any method to model the 3D deployment environment efficiently and accurately. Therefore many researchers are trapped by this issue, and their algorithms can only be evaluated in the same scenario, without the possibility to test the robustness and feasibility for implementations in different environments. In this thesis, we propose a novel planning methodology and an intelligent WSN planning tool to assist WSN designers efficiently planning reliable WSNs. First of all, a new method is proposed to efficiently and automatically model the 3D indoor and outdoor environments. To the best of our knowledge, this is the first time that the advantages of image understanding algorithm are applied to automatically reconstruct 3D outdoor and indoor scenarios for signal propagation and network planning purpose. The experimental results indicate that the proposed methodology is able to accurately recognize different objects from the satellite images of the outdoor target regions and from the scanned floor plan of indoor area. Its mechanism offers users a flexibility to reconstruct different types of environment without any human interaction. Thereby it significantly reduces human efforts, cost and time spent on reconstructing a 3D geographic database and allows WSN designers concentrating on the planning issues. Secondly, an efficient ray-tracing engine is developed to accurately and practically model the radio propagation and sensing signal on the constructed 3D map. The engine contributes on efficiency and accuracy to the estimated results. By using image processing concepts, including the kd-tree space division algorithm and modified polar sweep algorithm, the rays are traced efficiently without detecting all the primitives in the scene. The radio propagation model iv is proposed, which emphasizes not only the materials of obstacles but also their locations along the signal path. The sensing signal of sensor nodes, which is sensitive to the obstacles, is benefit from the ray-tracing algorithm via obstacle detection. The performance of this modelling method is robust and accurate compared with conventional methods, and experimental results imply that this methodology is suitable for both outdoor urban scenes and indoor environments. Moreover, it can be applied to either GSM communication or ZigBee protocol by varying frequency parameter of the radio propagation model. Thirdly, WSN planning method is proposed to tackle the above mentioned challenges and efficiently deploy reliable WSNs. More metrics (connectivity, coverage, cost, lifetime, packet latency and packet drop rate) are modeled more practically compared with other works. Especially 3D ray tracing method is used to model the radio link and sensing signal which are sensitive to the obstruction of obstacles; network routing is constructed by using AODV protocol; the network longevity, packet delay and packet drop rate are obtained via simulating practical events in WSNet simulator, which to the best of our knowledge, is the first time that network simulator is involved in a planning algorithm. Moreover, a multi-objective optimization algorithm is developed to cater for the characteristics of WSNs. The capability of providing multiple optimized solutions simultaneously allows users making their own decisions accordingly, and the results are more comprehensively optimized compared with other state-of-the-art algorithms. iMOST is developed by integrating the introduced algorithms, to assist WSN designers efficiently planning reliable WSNs for different configurations. The abbreviated name iMOST stands for an Intelligent Multi-objective Optimization Sensor network planning Tool. iMOST contributes on: (1) Convenient operation with a user-friendly vision system; (2) Efficient and automatic 3D database reconstruction and fast 3D objects design for both indoor and outdoor environments; (3) It provides multiple multi-objective optimized 3D deployment solutions and allows users to configure the network properties, hence it can adapt to various WSN applications; (4) Deployment solutions in the 3D space and the corresponding evaluated performance are visually presented to users; and (5) The Node Placement Module of iMOST is available online as well as the source code of the other two rebuilt heuristics. Therefore WSN designers will be benefit from v this tool on efficiently constructing environment database, practically and efficiently planning reliable WSNs for both outdoor and indoor applications. With the open source codes, they are also able to compare their developed algorithms with ours to contribute to this academic field. Finally, solid real results are obtained for both indoor and outdoor WSN planning. Deployments have been realized for both indoor and outdoor environments based on the provided planning solutions. coincide well with the estimated results. The measured results The proposed planning algorithm is adaptable according to the WSN designer’s desirability and configuration, and it offers flexibility to plan small and large scale, indoor and outdoor 3D deployments. The thesis is organized in 7 chapters. In Chapter 1, WSN applications and motivations of this work are introduced, the state-of-the-art planning algorithms and tools are reviewed, challenges are stated out and the proposed methodology is briefly introduced. In Chapter 2, the proposed 3D environment reconstruction methodology is introduced and its performance is evaluated for both outdoor and indoor environment. The developed ray-tracing engine and proposed radio propagation modelling method are described in details in Chapter 3, their performances are evaluated in terms of computation efficiency and accuracy. Chapter 4 presents the modelling of important metrics of WSNs and the proposed multi-objective optimization planning algorithm, the performance is compared with the other state-of-the-art planning algorithms. The intelligent WSN planning tool iMOST is described in Chapter 5. Real WSN deployments are prosecuted based on the planned solutions for both indoor and outdoor scenarios, important data are measured and results are analysed in Chapter 6. Chapter 7 concludes the thesis and discusses about future works. vi Resumen en Castellano Las redes de sensores inalámbricas (en inglés Wireless Sensor Networks, WSNs) han demostrado su potencial en diversas aplicaciones que aportan una gran cantidad de beneficios para el campo de la investigación y de la industria. Para muchas configuraciones se prevé que las WSNs consistirán en decenas o cientos de nodos que funcionarán con baterı́as pequeñas. Sin embargo, debido a la diversidad de los ambientes para desplegar las redes y a las limitaciones de recursos en materia de comunicación de radio, capacidad de detección y suministro de energı́a, la planificación de la topologı́a de la red y la predicción de su rendimiento es un tema muy difı́cil de tratar antes de la implementación real. Durante la fase de planificación del despliegue de la red se deben considerar aspectos como la conectividad, la cobertura, el coste, la longevidad de la red y la calidad del servicio. Por lo tanto, requiere de diseñadores con un amplio e interdisciplinario nivel de conocimiento que incluye la creación de redes, la ingenierı́a de radio y los sistemas embebidos entre otros, con el fin de construir de manera eficiente una WSN confiable para cualquier tipo de entorno. Hoy en dı́a todavı́a hay una falta de análisis y experiencias que orienten a los diseñadores de WSN para construir las topologı́as WSN de manera eficiente sin realizar muchas pruebas. Por lo tanto, la simulación es un enfoque viable para el análisis cuantitativo del rendimiento de las redes de sensores inalámbricos. Sin embargo, los algoritmos y herramientas de planificación existentes tienen, en cierta medida, serias limitaciones para diseñar en la práctica una topologı́a fiable de WSN: • Sólo unos pocos abordan la cuestión del despliegue 3D mientras que existe una gran cantidad de trabajos que colocan los dispositivos en 2D. Si no se analiza la dimensión completa (3D), los efectos del entorno en el desempeño de WSN no se estudian por completo, por lo que los valores de los parámetros evaluados, como la conectividad y la cobertura de detección, no son lo suficientemente precisos para tomar la decisión correcta. • Aún en menor medida los métodos de planificación modelan la cobertura de los sensores y la propagación de la señal de radio teniendo en cuenta un escenario realista donde existan obstáculos. Las señales de radio en el mundo real siguen una propagación multicamino, en la que los caminos directos, los caminos reflejados y los caminos difractados contribuyen a la intensidad de la señal recibida. Además, los obstáculos entre el recorrido del sensor y los objetos pueden bloquear las señales de detección y por lo tanto crear áreas sin cobertura en la aplicación. • Ninguno de los algoritmos de planificación existentes modelan el tiempo de vida de la red y la capacidad de entrega de paquetes correctamente y prácticamente. A menudo se emplean formulaciones unilaterales y poco realistas. • Los objetivos de optimización son a menudo tratados unilateralmente en los trabajos actuales. Sin una evaluación exhaustiva de los parámetros importantes, el rendimiento previsto de las redes inalámbricas de sensores no puede ser fiable y totalmente optimizado. • Por lo general, el modelado del entorno conlleva mucho tiempo y tiene un coste muy alto, pero ninguno de los trabajos actuales propone algún método para modelar el entorno de despliegue 3D con eficiencia y precisión. Por lo tanto, muchos investigadores están limitados por este problema y sus algoritmos sólo se pueden evaluar en el mismo escenario, sin la posibilidad de probar la solidez y viabilidad para las implementaciones en diferentes entornos. En esta tesis, se propone una nueva metodologı́a de planificación ası́ como una herramienta inteligente de planificación de redes de sensores inalámbricas para ayudar a los diseñadores a planificar WSNs fiables de una manera eficiente. En primer lugar, se propone un nuevo método para modelar de manera eficiente y automática los ambientes interiores y exteriores en 3D. Según nuestros conocimientos hasta la fecha, esta es la primera vez que las ventajas del algoritmo de image understandingse aplican para reconstruir automáticamente los escenarios exteriores e interiores en 3D para analizar la propagación de la señal y viii la planificación de la red. Los resultados experimentales indican que la metodologı́a propuesta es capaz de reconocer con precisión los diferentes objetos presentes en las imágenes satelitales de las regiones objetivo en el exterior y de la planta escaneada en el interior. Su mecanismo ofrece a los usuarios la flexibilidad para reconstruir los diferentes tipos de entornos sin ninguna interacción humana. De este modo se reduce considerablemente el esfuerzo humano, el coste y el tiempo invertido en la reconstrucción de una base de datos geográfica con información 3D, permitiendo ası́ que los diseñadores se concentren en los temas de planificación. En segundo lugar, se ha desarrollado un motor de trazado de rayos (en inglés ray tracing ) eficiente para modelar con precisión la propagación de la señal de radio y la señal de los sensores en el mapa 3D construido. El motor contribuye a la eficiencia y la precisión de los resultados estimados. Mediante el uso de los conceptos de procesamiento de imágenes, incluyendo el algoritmo del árbol kd para la división del espacio y el algoritmo polar sweepmodificado, los rayos se trazan de manera eficiente sin la detección de todas las primitivas en la escena. El modelo de propagación de radio que se propone no sólo considera los materiales de los obstáculos, sino también su ubicación a lo largo de la ruta de señal. La señal de los sensores de los nodos, que es sensible a los obstáculos, se ve beneficiada por la detección de objetos llevada a cabo por el algoritmo de trazado de rayos. El rendimiento de este método de modelado es robusto y preciso en comparación con los métodos convencionales, y los resultados experimentales indican que esta metodologı́a es adecuada tanto para escenas urbanas al aire libre como para ambientes interiores. Por otra parte, se puede aplicar a cualquier comunicación GSM o protocolo ZigBee mediante la variación de la frecuencia del modelo de propagación de radio. En tercer lugar, se propone un método de planificación de WSNs para hacer frente a los desafı́os mencionados anteriormente y desplegar redes de sensores fiables de manera eficiente. Se modelan más parámetros (conectividad, cobertura, coste, tiempo de vida, la latencia de paquetes y tasa de caı́da de paquetes) en comparación con otros trabajos. Especialmente el método de trazado de rayos 3D se utiliza para modelar el enlace de radio y señal de los sensores que son sensibles a la obstrucción de obstáculos; el enrutamiento de la red se construye utilizando el protocolo AODV; la longevidad de la red, retardo de paquetes ix y tasa de abandono de paquetes se obtienen a través de la simulación de eventos prácticos en el simulador WSNet, y según nuestros conocimientos hasta la fecha, es la primera vez que simulador de red está implicado en un algoritmo de planificación. Por otra parte, se ha desarrollado un algoritmo de optimización multi-objetivo para satisfacer las caracterı́sticas de las redes inalámbricas de sensores. La capacidad de proporcionar múltiples soluciones optimizadas de forma simultánea permite a los usuarios tomar sus propias decisiones en consecuencia, obteniendo mejores resultados en comparación con otros algoritmos del estado del arte. iMOST se desarrolla mediante la integración de los algoritmos presentados, para ayudar de forma eficiente a los diseñadores en la planificación de WSNs fiables para diferentes configuraciones. El nombre abreviado iMOST (Intelligent Multi-objective Optimization Sensor network planning Tool) representa una herramienta inteligente de planificación de redes de sensores con optimización multi-objetivo. iMOST contribuye en: (1) Operación conveniente con una interfaz de fácil uso, (2) Reconstrucción eficiente y automática de una base de datos con información 3D y diseño rápido de objetos 3D para ambientes interiores y exteriores, (3) Proporciona varias soluciones de despliegue optimizadas para los multi-objetivo en 3D y permite a los usuarios configurar las propiedades de red, por lo que puede adaptarse a diversas aplicaciones de WSN, (4) las soluciones de implementación en el espacio 3D y el correspondiente rendimiento evaluado se presentan visualmente a los usuarios, y (5) El Node Placement Modulede iMOST está disponible en lı́nea, ası́ como el código fuente de las otras dos heurı́sticas de planificación. Por lo tanto los diseñadores WSN se beneficiarán de esta herramienta para la construcción eficiente de la base de datos con información del entorno, la planificación práctica y eficiente de WSNs fiables tanto para aplicaciones interiores y exteriores. Con los códigos fuente abiertos, son capaces de comparar sus algoritmos desarrollados con los nuestros para contribuir a este campo académico. Por último, se obtienen resultados reales sólidos tanto para la planificación de WSN en interiores y exteriores. Los despliegues se han realizado tanto para ambientes de interior y como para ambientes de exterior utilizando las soluciones de planificación propuestas. Los resultados medidos coinciden en gran medida con los resultados estimados. El algoritmo de planificación x propuesto se adapta convenientemente al deiseño de redes de sensores inalámbricas, y ofrece flexibilidad para planificar los despliegues 3D a pequeña y gran escala tanto en interiores como en exteriores. La tesis se estructura en 7 capı́tulos. En el Capı́tulo 1, se presentan las aplicaciones de WSN y motivaciones de este trabajo, se revisan los algoritmos y herramientas de planificación del estado del arte, se presentan los retos y se describe brevemente la metodologı́a propuesta. En el Capı́tulo 2, se presenta la metodologı́a de reconstrucción de entornos 3D propuesta y su rendimiento es evaluado tanto para espacios exteriores como para espacios interiores. El motor de trazado de rayos desarrollado y el método de modelado de propagación de radio propuesto se describen en detalle en el Capı́tulo 3, evaluándose en términos de eficiencia computacional y precisión. En el Capı́tulo 4 se presenta el modelado de los parámetros importantes de las WSNs y el algoritmo de planificación de optimización multi-objetivo propuesto, el rendimiento se compara con los otros algoritmos de planificación descritos en el estado del arte. La herramienta inteligente de planificación de redes de sensores inalámbricas, iMOST, se describe en el Capı́tulo 5. En el Capı́tulo 6 se llevan a cabo despliegues reales de acuerdo a las soluciones previstas para los escenarios interiores y exteriores, se miden los datos importantes y se analizan los resultados. En el Capı́tulo 7 se concluye la tesis y se discute acerca de los trabajos futuros. xi Contents 1 Introduction 1.1 1 WSN applications and deployment experiences . . . . . . . . . . . . 2 1.1.1 Military applications . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.2 Environmental applications . . . . . . . . . . . . . . . . . . . 4 1.1.3 Health applications . . . . . . . . . . . . . . . . . . . . . . . . 7 1.1.4 Other applications . . . . . . . . . . . . . . . . . . . . . . . . 9 Challenges when deploying WSNs . . . . . . . . . . . . . . . . . . . . 10 1.2.1 Necessity of simulation . . . . . . . . . . . . . . . . . . . . . . 12 1.3 Introduction to the planning algorithms and tools . . . . . . . . . . . 13 1.4 Proposed methodology and work flow: Main contributions . . . . . 22 1.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 1.2 2 3D environment reconstruction method 2.1 2.2 2.3 27 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.1.1 3D environment reconstruction from Lidar systems . . . . . . 28 2.1.2 3D environment reconstruction from images . . . . . . . . . . 29 3D outdoor environment reconstruction . . . . . . . . . . . . . . . . 32 2.2.1 Proposed algorithm for outdoor environment reconstruction . 35 2.2.2 Image database . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2.2.3 Image understanding and segmentation algorithm . . . . . . 37 2.2.4 Performance enhancement . . . . . . . . . . . . . . . . . . . . 44 2.2.5 Shape matching and vectorization . . . . . . . . . . . . . . . 55 Indoor environment reconstruction . . . . . . . . . . . . . . . . . . . 64 2.3.1 Image calibration and classification . . . . . . . . . . . . . . . 65 2.3.2 Thinning and feature points extraction . . . . . . . . . . . . . 66 2.3.3 Smoothing and vectorizing . . . . . . . . . . . . . . . . . . . 66 Contents 2.3.4 2.4 Demonstration and analysis . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Ray-tracing engine and radio propagation modelling 68 69 73 3.1 Space division . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 3.2 Polar sweeping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.2.1 Direct path . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 3.2.2 Reflection path . . . . . . . . . . . . . . . . . . . . . . . . . . 81 3.2.3 Diffraction path . . . . . . . . . . . . . . . . . . . . . . . . . 82 Measurements and experimental results . . . . . . . . . . . . . . . . 83 3.3.1 Outdoor RF propagation verification . . . . . . . . . . . . . . 83 3.3.2 Indoor RF propagation verification . . . . . . . . . . . . . . . 88 3.3 3.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Planning the WSN 4.1 99 101 Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 4.1.1 Star network . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 4.1.2 Tree network . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.1.3 Mesh network . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 4.1.4 Cluster network . . . . . . . . . . . . . . . . . . . . . . . . . . 105 4.2 Heterogeneity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 4.3 Introduction and modelling of important metrics . . . . . . . . . . . 106 4.4 4.5 4.3.1 Preliminaries and assumptions . . . . . . . . . . . . . . . . . 106 4.3.2 The cost of WSN . . . . . . . . . . . . . . . . . . . . . . . . . 109 4.3.3 Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 4.3.4 Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 4.3.5 Lifetime, Packet latency and Packet drop rate . . . . . . . . . 118 The proposed multi-objective optimization methodology . . . . . . 121 4.4.1 Initialization of individuals . . . . . . . . . . . . . . . . . . . 124 4.4.2 Crossover and mutation . . . . . . . . . . . . . . . . . . . . . 125 4.4.3 Evaluation based on desirability models and constraints . . . 126 Experimental results and analysis . . . . . . . . . . . . . . . . . . . 127 4.5.1 The impact of maximum number of generation . . . . . . . . 127 4.5.2 Performance comparison with other heuristics . . . . . . . . . 130 xiii Contents 4.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 5 iMOST: an Intelligent Multi-objective Optimization Sensor network planning Tool 5.1 5.2 5.3 135 Menu bar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 5.1.1 Image Processing Module . . . . . . . . . . . . . . . . . . . . 136 5.1.2 Environment property configuration . . . . . . . . . . . . . . 136 Toolbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.2.1 Node deployment . . . . . . . . . . . . . . . . . . . . . . . . . 138 5.2.2 Network Planning Module . . . . . . . . . . . . . . . . . . . . 138 5.2.3 Ray-tracing Propagation Module . . . . . . . . . . . . . . . . 140 5.2.4 3D navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 6 Real measurements and results analysis 143 6.1 Aggregation mechanism of measured data . . . . . . . . . . . . . . . 143 6.2 Application interface . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 6.3 Indoor measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 6.4 Outdoor measurements . . . . . . . . . . . . . . . . . . . . . . . . . . 155 6.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 7 Conclusions and future works 163 7.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 7.2 Future works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 7.3 Publications based on this work . . . . . . . . . . . . . . . . . . . . . 166 7.4 Implementation of this work . . . . . . . . . . . . . . . . . . . . . . . 168 Bibliography 169 xiv List of Figures Fig. 1.1 A typical WSN architecture . . . . . . . . . . . . . . . . . . . . 2 Fig. 1.2 Environmental applications . . . . . . . . . . . . . . . . . . . . 5 Fig. 1.3 Health care applications . . . . . . . . . . . . . . . . . . . . . . 8 Fig. 1.4 Smartsantander . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Fig. 1.5 Libelium smart world . . . . . . . . . . . . . . . . . . . . . . . . 11 Fig. 1.6 An example of tool interface . . . . . . . . . . . . . . . . . . . . 17 Fig. 1.7 Demonstration of the single-hop solution . . . . . . . . . . . . . 18 Fig. 1.8 Demonstration of the multi-hop solution . . . . . . . . . . . . . 18 Fig. 1.9 The proposed methodology . . . . . . . . . . . . . . . . . . . . . 24 Fig. 2.1 Airborne laser scanning . . . . . . . . . . . . . . . . . . . . . . . 28 Fig. 2.2 DSM with the original image . . . . . . . . . . . . . . . . . . . 28 Fig. 2.3 The principle of visual hull reconstruction . . . . . . . . . . . . 30 Fig. 2.4 Example of space carving reconstruction . . . . . . . . . . . . . 31 Fig. 2.5 Example of image-based rendering . . . . . . . . . . . . . . . . 31 Fig. 2.6 Workflow of the work by Saxena et al. . . . . . . . . . . . . . . 32 Fig. 2.7 Google street view tools . . . . . . . . . . . . . . . . . . . . . . 34 Fig. 2.8 3D outdoor reconstruction . . . . . . . . . . . . . . . . . . . . . 36 Fig. 2.9 MSRC database . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Fig. 2.10 CEIeurope database . . . . . . . . . . . . . . . . . . . . . . . . 39 Fig. 2.11 17D filter bank . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Fig. 2.12 The process of image textonization . . . . . . . . . . . . . . . . 40 Fig. 2.13 Calculating feature response . . . . . . . . . . . . . . . . . . . . 42 Fig. 2.14 Multi-object recognition procedure. . . . . . . . . . . . . . . . . 44 Fig. 2.15 Comparison between k-mean and graphcut . . . . . . . . . . . . 45 Fig. 2.16 Example of sub-clustering based on connectivity property. . . . 47 List of Figures Fig. 2.17 The comparison between the proposed algorithm and that of Shotton et al. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Fig. 2.18 The principle of shadow detection . . . . . . . . . . . . . . . . . 50 Fig. 2.19 The result of shadow detection . . . . . . . . . . . . . . . . . . 51 Fig. 2.20 Road segmentation and orientation estimation . . . . . . . . . . 52 Fig. 2.21 Clustering of orientations . . . . . . . . . . . . . . . . . . . . . . 53 Fig. 2.22 Comparison of orientations . . . . . . . . . . . . . . . . . . . . . 54 Fig. 2.23 The result of road detection . . . . . . . . . . . . . . . . . . . . 55 Fig. 2.24 Frequently seen façade shapes of buildings . . . . . . . . . . . . 57 Fig. 2.25 Hierarchical shape matching . . . . . . . . . . . . . . . . . . . . 59 Fig. 2.26 Successful result of shape matching . . . . . . . . . . . . . . . . 61 Fig. 2.27 Shape registration . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Fig. 2.28 KML shape description shown on Google Earth . . . . . . . . . 64 Fig. 2.29 Image calibration and segmentation . . . . . . . . . . . . . . . . 65 Fig. 2.30 Thinning step and feature point extraction . . . . . . . . . . . . 66 Fig. 2.31 Smoothing and regularization. . . . . . . . . . . . . . . . . . . . 67 Fig. 2.32 Vectorization result. . . . . . . . . . . . . . . . . . . . . . . . . 68 Fig. 2.33 A wall is described by four vertexes. . . . . . . . . . . . . . . . 68 Fig. 2.34 Reconstructed 3D indoor map in different views. . . . . . . . . 69 Fig. 2.35 A toy example by using the downloaded map downloaded . . . 70 Fig. 3.1 Space division by kd-tree . . . . . . . . . . . . . . . . . . . . . . 76 Fig. 3.2 Conventional Polar sweep. . . . . . . . . . . . . . . . . . . . . . 77 Fig. 3.3 Polar sweep. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Fig. 3.4 Direct path. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Fig. 3.5 Reflection path searching. . . . . . . . . . . . . . . . . . . . . . 81 Fig. 3.6 Diffraction path searching. . . . . . . . . . . . . . . . . . . . . . 82 Fig. 3.7 Three different routes measured by COST231 group. . . . . . . 84 Fig. 3.8 Classification and radio propagation over Munich scenario . . . 85 Fig. 3.9 Simulation result for the first route METRO200. . . . . . . . 86 Fig. 3.10 Simulation result for the second route METRO201. . . . . . . 86 Fig. 3.11 Simulation result for the third route METRO202. . . . . . . . 87 Fig. 3.12 Four-layer architecture and physical view of the Cookie node . . 88 xvi List of Figures Fig. 3.13 ETRX2 ZigBee communication module on communication layer 89 Fig. 3.14 Radio pattern of antenna of ETRX2 module . . . . . . . . . . . 89 Fig. 3.15 Three scenarios for radio measurements . . . . . . . . . . . . . . 90 Fig. 3.16 Ray tracing demonstrations from different TXs and RXs . . . . 91 Fig. 3.17 Simulation result: example 1 . . . . . . . . . . . . . . . . . . . . 92 Fig. 3.18 Simulation result: example 2 . . . . . . . . . . . . . . . . . . . . 92 Fig. 3.19 Results and comparisons of Scenario A . . . . . . . . . . . . . . 93 Fig. 3.20 Results and comparisons of Scenario B . . . . . . . . . . . . . . 94 Fig. 3.21 Results and comparisons of Scenario C . . . . . . . . . . . . . . 95 Fig. 3.22 Demonstration of the toy example . . . . . . . . . . . . . . . . . 96 Fig. 3.23 Average time consumption of polar sweeping . . . . . . . . . . . 97 Fig. 3.24 Average time consumption of ray tracing . . . . . . . . . . . . . 98 Fig. 3.25 Polar sweeping with and without kd-tree traversing . . . . . . . 99 Fig. 4.1 Different topologies of WSN . . . . . . . . . . . . . . . . . . . . 103 Fig. 4.2 The searching of covered point . . . . . . . . . . . . . . . . . . . 107 Fig. 4.3 Products by Crossbow . . . . . . . . . . . . . . . . . . . . . . . 110 Fig. 4.4 BTnode rev3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Fig. 4.5 Waspmote,Wismote and SEED-EYE . . . . . . . . . . . . . . . 112 Fig. 4.6 Tyndall mote . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Fig. 4.7 Deployment cost configuration in vertical view . . . . . . . . . . 116 Fig. 4.8 Modelling strategy by using WSNet simulator . . . . . . . . . . 121 Fig. 4.9 Proposed planning algorithm strategy . . . . . . . . . . . . . . . 122 Fig. 4.10 Crossover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Fig. 4.11 Mutation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 Fig. 4.12 Configuration of scenario CEI-UPM . . . . . . . . . . . . . . . . 128 Fig. 4.13 Desirability values . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Fig. 4.14 Time consumption . . . . . . . . . . . . . . . . . . . . . . . . . 129 Fig. 4.15 Scenario East Lansing . . . . . . . . . . . . . . . . . . . . . . . 131 Fig. 4.16 Scenario Madrid . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Fig. 5.1 The mainframe of the planning tool . . . . . . . . . . . . . . . . 137 Fig. 5.2 User command on constructing new map . . . . . . . . . . . . . 137 Fig. 5.3 Environment property setting dialog . . . . . . . . . . . . . . . 138 xvii List of Figures Fig. 5.4 Node property configuration dialog . . . . . . . . . . . . . . . . 139 Fig. 5.5 Node configuration dialog . . . . . . . . . . . . . . . . . . . . . 140 Fig. 5.6 Generated topology demonstration . . . . . . . . . . . . . . . . 140 Fig. 5.7 3D navigation for outdoor scenario . . . . . . . . . . . . . . . . 141 Fig. 5.8 3D navigation for indoor scenario . . . . . . . . . . . . . . . . . 142 Fig. 6.1 BKR2400 antenna. . . . . . . . . . . . . . . . . . . . . . . . . . 144 Fig. 6.2 Application interface. . . . . . . . . . . . . . . . . . . . . . . . . 146 Fig. 6.3 Indoor modelling by using iMOST Fig. 6.4 User requirement over the indoor test. . . . . . . . . . . . . . . 149 Fig. 6.5 Topology comparison 1 . . . . . . . . . . . . . . . . . . . . . . . 149 Fig. 6.6 Topology comparison 2 . . . . . . . . . . . . . . . . . . . . . . . 150 Fig. 6.7 RSS comparison between real measurement and simulation result 153 Fig. 6.8 Remaining energy of N1 , N8 and N14 . . . . . . . . . . . . . . . 153 Fig. 6.9 Comparing the packet delivery status: Indoor . . . . . . . . . . 154 . . . . . . . . . . . . . . . . 148 Fig. 6.10 The sensed data of N4 . . . . . . . . . . . . . . . . . . . . . . . . 155 Fig. 6.11 Outdoor modelling by using iMOST . . . . . . . . . . . . . . . 156 Fig. 6.12 Topology comparison: Outdoor . . . . . . . . . . . . . . . . . . 159 Fig. 6.13 RSS comparison between real measurement and simulation result 160 Fig. 6.14 Remaining energy of N4 , N5 and N2 . . . . . . . . . . . . . . . 160 Fig. 6.15 Comparing the packet delivery status: Outdoor . . . . . . . . . 161 xviii List of Tables Table 1.1 Comparison of planning algorithms . . . . . . . . . . . . . . . 20 Table 2.1 Confusion matrix of classification result . . . . . . . . . . . . . 47 Table 2.2 Shape matching result . . . . . . . . . . . . . . . . . . . . . . 60 Table 3.1 Comparison of radio estimation result with other methods. . . 87 Table 3.2 Results comparison: Scenario A . . . . . . . . . . . . . . . . . 93 Table 3.3 Results comparison: Scenario B . . . . . . . . . . . . . . . . . 95 Table 3.4 Results comparison: Scenario C . . . . . . . . . . . . . . . . . 96 Table 3.5 Attenuation parameters of major objects indoors. . . . . . . . 98 Table 4.1 Important symbols . . . . . . . . . . . . . . . . . . . . . . . . 106 Table 4.2 Features of various platform. . . . . . . . . . . . . . . . . . . . 114 Table 4.3 Features of algorithms for comparison . . . . . . . . . . . . . . 130 Table 4.4 Results comparison for Scenario CEI-UPM. . . . . . . . . . . . 133 Table 4.5 Results comparison for Scenario East Lansing. . . . . . . . . . 133 Table 4.6 Results comparison for Scenario Madrid. . . . . . . . . . . . . 134 Table 6.1 Routing table format. . . . . . . . . . . . . . . . . . . . . . . . 145 Table 6.2 Packet format. . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Table 6.3 Evaluated performance of the two candidates. . . . . . . . . . 150 Table 6.4 Neighborhood table and RSS comparisons: Indoor . . . . . . . 152 Table 6.5 Evaluated performance of the selected candidate. . . . . . . . 157 Table 6.6 Neighborhood table and RSS comparisons: Outdoor . . . . . . 158 Chapter 1 Introduction Recent years have witnessed an increased interest in the use of Wireless Sensor Networks (WSNs) in various applications such as environmental monitoring, factory automation, habitat tracking, security surveillance, intelligent transportation and smart cities. This technology has brought a lot of benefits to the users from both research and industrial areas. Fig. 1.1 depicts a typical sensor network architecture. It is envisioned that tens to thousands of miniaturized sensor nodes, which operate on small batteries, will be deployed to operate autonomously to construct WSNs in different types of environments. Sensor networks may consist of various types of sensors such as seismic, low sampling rate magnetic, thermal, visual, infrared, acoustic and radar, which are able to monitor a wide range of ambient conditions including [1]: temperature, humidity, light condition, the presence or absence of objects, mechanical stress levels on attached objects, and the mobility characteristics such as speed and direction. The sensed data are collected and sent to a base station directly or via multiple hops depending on the network topology and routing protocols. In addition to the ability to probe its surroundings, each sensor node has one or more onboard radios to communicate with other nodes through wireless communication protocols such as ZigBeeT M [2], BluetoothT M [3] and Ultra-wideband (UWB)[4] among others. Therefore the combinations of micro-sensing and wireless communication offers a huge number of possibilities of WSN applications. The categorization of the applications of WSNs differs by different researches. Some works categorize the applications into military, environment, health, home and other commercial areas [5]; whereas some classify them into two categories [6]: monitoring and tracking. It is possible to expand this classification with other categories such as event detection and spatial process estimation [7]. In this chapter, we enumerate the WSN applications based on the monitoring and tracking frame with the items of different implemented areas as mentioned in [5], Chapter 1. Introduction Base station Applications Figure 1.1: A typical WSN architecture. the deployment experiences of some applications are reviewed as well. Afterwards, the challenges of deploying a WSN are identified to state out our motivations and targets. The main frame of the proposed planning methodology and planning tool are briefly described at the end of this chapter. 1.1 WSN applications and deployment experiences The origin of WSNs can be traced back to the 1950s during the cold war between Soviet Union and the United States. The SOund SUrveillance System (SOSUS) was developed by the United States Military to detect and track Soviet submarines. In order to echo the investments made in the 1960s and 1970s to develop the hardware for today’s Internet, the United States Defense Advanced Research Projects Agency (DARPA) launched the Distributed Sensor Network (DSN) program in 1980 to formally explore the challenges in implementing distributed wireless sensor networks. With the birth of DSN and its progression into academia, the governments and universities eventually show their interests in this topic and promote the research atmosphere to improve the performance of WSN and explore new areas of applications using WSNs, such as air quality monitoring, forest fire detection, natural disaster prevention and structural monitoring. Then there arises a strong 2 1.1. WSN applications and deployment experiences demand in the market as researchers made their steps into the corporation with technology giants of the day, such as IBM, CISCO and Bell Labs, they began to explore and expand the use of WSNs in our daily life, such as security surveillance, health monitoring and smart life. Lessons and extensive experiences are learnt during designing and constructing the wireless sensor networks, which continuously provide challenging issues and attracts more interests in this area. 1.1.1 Military applications The C4ISRT (Command, Control, Communications, Computers, Intelligence, Surveillance and Reconnaissance) systems integrate WSNs to realize military monitoring and tracking. The systems can be used to monitor friendly forces, equipment and ammunition. Every troop, vehicle, equipment and critical ammunition can be attached with small sensors to report the statuses. Those reports are gathered in sink nodes or be forwarded to the upper levels of the hierarchy and aggregated with the data from other units at each level, and sent to the troop leaders at the end. They are also implemented for battlefield surveillance where critical terrains, approach routes, paths and straits can be rapidly covered with sensor networks, to closely monitor the activities of the opposing forces. As a result, the deployment efficiency and reliability are very demanding for such applications to allow WSNs being constructed expeditiously and able to cover the area of interest. There are other military applications developed for targeting and tracking: Sensor networks can be incorporated into guidance systems of the intelligent ammunition, they can also be attached to soldiers to track their mobility and locations. In [8], the invasion of individual enemy soldiers are detected by using unattended acoustic and seismic sensors in the protecting military sites or buildings. The Early attack reaction sensor (EARS) [9] is a man-wearable gunshot system. It uses passive acoustic sensing system with small microphone array to detect gunshots (muzzle blast and/or shockwave) and provide relative azimuth and range information of the shot origin to the user. It has been tested in both open field and military operations in urban terrain (MOUT) environment and has provided useable bearing and range information against the firing positions. Some systems also perform localization such as [10, 11] to protect soldiers from potential menace by blasts 3 Chapter 1. Introduction and snipers. Such applications should pay sufficient attentions to the connections of nodes. In [12], hazardous chemicals are detected and identified by their unique infrared absorption signatures. The application is for deployment on expendable unmanned aerial vehicles in a nadir-viewing configuration from an altitude of 300 m while traveling at an air speed of 96 km/h. The applications of monitoring missile environment target at dramatically increasing missile active service life, saving millions of dollars and reducing the number of missiles needed. The challenge of such real-time monitoring systems is to collect and store data on environmental shock with high speed and observe vibration (up to 100 g) in missile canisters without electrical hazards. An optical sensor system capable to monitor shock and vibration in missile canisters in three dimensions at high speed (5 kHz) is proposed in [13]. 1.1.2 Environmental applications Environmental monitoring has been studied for a long time. It includes the detection and reaction towards natural disaster such as earthquake and avalanche, the detections on climate change and pollution, the tracking and observations of animal behaviors. The old mechanisms recorded data at specific intervals and required human intervention to download them. The employment of WSN technology in this field shortens the time and reduces efforts on data aggregation issue, and most importantly it is capable to monitor/track a huge quantity of the environmental data without disturbing the natural environment too much compared with the traditional approach. W-TREMORS [14] is a platform with high-frequency distributed data acquisition ability, and it is designed for earthquake engineering and structural monitoring. By adopting a novel communication protocol together with the developed software, it is able to test the shaking with inexpensive hardware resources. Wong et al. from UC Berkeley built a wireless sensor seismic response monitoring system based on MICA2 motes [15]. The system was tested by using a reinforced concrete bridge column (Fig. 1.2(a)). However, the accelerometer and the analog-to-digital converter implemented in the MICA2 motes do not possess the fidelity required for structural state evaluation. Both works mentioned about resolution problems, due to quantization significantly affects the low level acceleration readings. Besides, 4 1.1. WSN applications and deployment experiences (a) (b) (c) Figure 1.2: Environmental applications. (a) The test of WSN on specimen [15], (b) ZebraNet [16], (c) The volcanic monitoring system[17]. due to the interferences in radio transmission with multiple motes attempting to communicate simultaneously, the communication packet loss is quite pronounced at high sampling rates. ZebraNet system [16] shown in Fig. 1.2(b) is a mobile wireless sensor network used to track animal migrations. ZebraNet is composed of sensor nodes built into the zebra’s collar. The node consists of a 16-bit TI microcontroller, 4 Mbits off-chip flash memory, a 900 MHz radio, and a GPS unit. Positional readings are taken using the GPS and sent multi-hop across zebras to the base station. The goal is to accurately log each zebra’s position and use them for analysis. In the demonstration, a total of 6-10 zebra collars were deployed at the Sweetwaters game reserve in central Kenya to study the effects and reliability of the collars and to collect movement data. After deployment, the biologists observed that the collared zebras were affected by the collars. They observed additional head shakes from those zebra in the first week. After the first week, the collared zebra showed no difference than the uncollared zebra. A set of movement data were also collected during this study, from which the biologists can better understand the zebra movements. 5 Chapter 1. Introduction In volcanic monitoring, the challenges of a WSN application for data collection include reliable event detection, efficient data delivery, high data rates, and sparse deployment of nodes. Given these challenges, a network consisting of 16 sensor nodes was deployed on Volcn Reventador in northern Ecuador [17]. Each sensor node is a T-mote sky [18] device equipped with an external omni-directional antenna, a seismometer, a microphone, and a custom hardware interface board. Overall, the system performed well in this study. In the 19 days of deployment (Fig. 1.2(c)), the network observed 230 eruptions and other volcanic events. About 61% of the data was retrieved from the network due to short outages in the network from software component failure and power outage. Therefore, the node devices in this type of application should be smaller, lighter, and consume less power to facilitate distribution and prolong lifetime of the network. Macroscope of redwood [19] is an experimental application of a WSN that monitors and records the redwood trees in Sonoma, California. Sensor nodes are placed at different heights of the tree. Air temperature, relative humidity, and photo-synthetically-active solar radiation are measured by each sensor node. Plant biologists track changes of spatial gradients in the microclimate around a redwood tree and validate their biological theories. Sensor nodes may be deployed in a forest strategically to relay the exact origin of the fire to the end users before the fire is spread uncontrollable. Forest fire monitoring systems require a large amount of sensor nodes being deployed and integrated using radio frequency/optical systems. Also, they may be equipped with effective energy harvesting methods [20], such as solar cells, because the sensors may be left unattended for months and even years. The sensor nodes collaborate with each other to perform distributed sensing and overcome obstacles, such as trees and rocks, that block line of sight of sensors. DIMAP-FactorLink provides another example of forest fire detection. It has developed and integrated a forest fires detection system using the products of Libelium [21]. The covered area of the system was about 210 hectares in the north Spain region. Therefore in this type of application, we should focus not only on the coverage and communication issues, but also on the lifetime and cost. Biocomplexity mapping of the environment is done at the James Reserve in Southern California [22]. Three monitoring grids with each having 25-100 sensor 6 1.1. WSN applications and deployment experiences nodes are implemented for fixed view multimedia and environmental sensor data loggers. The ALERT system [23] is another example deployed in the US for flood detection. The types of sensors deployed in the system include rainfall, water level and weather sensors. The authors in [24] developed an underwater WSN application platform for long-term monitoring of coral reefs and fisheries. The deployed sensor network consists of static and mobile underwater sensor nodes. The nodes communicate with each other via point-to-point links using high speed optical communications with an acoustic protocol integrated in the TinyOS protocol stack. They have several types of sensors, including temperature and pressure sensing devices and cameras. Mobile nodes are needed to locate and move above the static nodes to collect data and perform network maintenance functions for deployment, re-location, and recovery. WSNs applications on precision agriculture have recently appeared. For instances, the experiences in the design, development and deployment of a WSN are described in [25] to improve water use efficiency for pasture production. Sensor pods should be designed carefully in this application to withstand seasonal weather changes and are resistant to damages that may be inflicted by cattle in the field. Temperature and humidity were measured using the Tmote Sky’s on-board sensors. 70 sensor pods were deployed in December of 2007, at the TIAR (Tasmanian Institute for Agricultural Research) Elliott Research Farm and managed to gather correct data from the field. The iCubes were designed with low-cost humidity sensors to monitor the soil wetness in the work of [26] and the authors in [27] developed a monitoring system by using TelosB wireless sensor nodes to acquire data such as temperature, humidity, illumination and voltage. In this application, a web-based platform integrated with Google Maps was developed to release the greenhouse environmental status and provide real-time voice and SMS alarm service. 1.1.3 Health applications Health applications can be categorized into activities of daily living monitoring, fall and movement detection, location tracking, medication intake monitoring, and medical status monitoring [28]. The Ultra Badge System [29] is one example of location tracking application that is used in a hospital setting. In Ultra Badge, a 3D tag system is designed to localize the patients. When a patient is in a specific area 7 Chapter 1. Introduction where a fall is most likely to occur, the system alerts the caregivers beforehand. The Ultra Badge System consists of ultrasonic receivers embedded in the environment and wireless ultrasonic emitters placed on objects as depicted in Fig. 1.3(a). The positions of the receivers are fixed and known beforehand, therefore the emitters can be positioned by using the multi-lateration technique. Two subsystems were developed for real implementations: the wheelchair locator and the ultrasonic radar. In the former subsystem, the nurse is notified when a patient uses a wheelchair approaching a ’detection area’, where a fall is most likely to occur. The latter subsystem aims to monitor the activities of the patients in their beds by using ultrasonic pulses. (a) (b) Figure 1.3: Health care applications. (a) The Ultra Badge System[29], (b) The iPackage of [30]. As patients may get allergic by taking some medications or risk of life with wrong dosage, if sensor nodes can be used in medication monitoring, the chance of getting and prescribing the wrong medication to patients can be minimized. An intelligent packaging prototype (iPackage) is developed by Pang et al.[30]. The system is capable of both remote medication intake monitoring and vital signs monitoring. It uses an array of Controlled Delamination Material (CDM) films along with the control circuits. The CDM film is a three-layer foil composed of aluminum bottom and top layers and an adhesive middle layer made of electrochemical epoxy. When a voltage higher than a particular threshold is applied on the bottom layer and top layer, an electrochemical reaction occurs in the middle layer. When the voltage is 8 1.1. WSN applications and deployment experiences applied for a certain amount of time, the epoxy layer is destroyed and delaminated. Therefore, the iPackage sealed with a CDM film can only be opened by the special control appliance which also enables the control of the dosage. The identification of the correct pill is accomplished by RFID. The prototype design of CDM and tagged capsule package is depicted in Fig. 1.3(b). 1.1.4 Other applications In the recent years, the concept of smart cities are proposed as the next stage in the process of urbanisation. As a city is a system of systems, the more we understand how those systems interact and share information, the better people can be helped to make decisions and to make the city better. The smart city can be identified along eight main axes or dimensions: smart environment, smart energy and water, smart transportation, smart education, smart healthcare, smart public safety, smart buildings and urban planning and smart government. Beyond the previously mentioned applications for smart environment, smart energy and water and smart healthcare, there exist smart transportation applications such as the PGS smart parking system [31], which is developed in ICU Korea based on a new T-Sensor hardware. And the California Partners for Advanced Transportation TecHnology (PATH) was established, with the mission to develop solutions to the problems of California’s surface transportation systems through cutting edge research, PATH research is divided into three program areas: Transportation Safety Research, Traffic Operations Research and Modal Applications Research. The traffic surveillance by WSNs was developed based on MICA2 DOTS. It was used to control traffic signal, on-ramp metering system to regulate the flow of traffic on freeway entrance ramps using traffic signals, the system was able to reduce delay by 102 million person-hours in 2003. The parking guidance and information system (PGIS), road condition sensing modality were also developed in PATH research. SmartSantander project [32] proposes a city-scale experimental research facility to support typical applications and services for a smart city. This unique experimental facility is sufficiently large, open and flexible to enable horizontal and vertical federation with other experimental facilities. The project envisions a 9 Chapter 1. Introduction Figure 1.4: Outdoor parking and environmental monitoring deployed architecture of Smartsantander project. deployment of 20,000 sensors in Belgrade, Guildford, Lbeck and Santander (12,000 nodes, Fig. 1.4), which exploits a large variety of technologies. One of the famous sensor nodes producers, Libelium, creates the inspirational and market research documents on 50 sensor applications. It has comprised and concluded necessary applications in a infographic (Fig.1.5) which combines Smart Cities, Internet of Things (IoT) and other sensing applications to construct a smarter world. 1.2 Challenges when deploying WSNs One of the major challenges in designing WSNs is the support of capturing and gathering data requirements while coping with the computation, energy, sensing ability and communication constraints. As a result, careful node placement can be a very effective optimization mean for achieving the desired design goals. In order to prolong the lifetime of WSN, energy conservation methods must be taken, scheduling and data aggregation are among the commonly used methods. Scheduling conserves energy by turning off the sensor whenever possible. While data aggregation tries to conserve energy by reducing the energy used in data transmission, efficient routing and topology construction can significantly reduce the energy consumption due to data aggregation. As the energy harvesting researches blossom nowadays, with the aim to tackle the energy constraint issues for different 10 1.2. Challenges when deploying WSNs Figure 1.5: Libelium smart world. areas, it will be able to solve the limited battery resource issue and allow the WSN applying to more areas. Connectivity and coverage problems are caused by the limited communication and sensing ability of sensor nodes. To solve both problems, the solution lays in how the sensors nodes are positioned with respect to each others. Coverage problem is regarding how to guarantee that each of the points to be monitored is covered by the sensors in the region. It is a trade-off problem, in maximizing coverage with low cost: the sensors need to be placed not too close to each other so that the sensing capability of the network is fully utilized while the cost is minimized; at the same time, they must not be located too far away from each other to avoid the formation of coverage holes (area outside sensing range of sensors). On the other hand, it is also a trade-off problem in terms of connectivity and cost: the sensor nodes need to be placed close enough so that they are within each other’s communication range and ensure the connectivity of WSN with robustness and reliability. 11 Chapter 1. Introduction 1.2.1 Necessity of simulation The emergence of wireless sensor networks brought many new challenging issues to WSN designers. Traditionally, the three main techniques for analyzing the performance of wired and wireless networks are analytical methods, computer simulation, and physical measurement. Due to many constraints imposed on sensor networks, as mentioned above, the energy limitation, sensing ability and communication constraints, the deployment of sensor networks tends to be quite complex and requires mastering of cross knowledge on networking, radio propagation and embedded system. Furthermore, although the aforementioned WSN applications have been implemented, there is still a lack of analysis and experiences to guide WSN designers to construct WSN topology successfully without many trials. Therefore, it appears that simulation is a feasible approach to the quantitative analysis of the performance of wireless sensor networks. The authors of [33] introduce their deployment experiences of various WSN applications and recommend that simulation should come first before real deployment. But for getting realistic results, one must have a realistic simulation environment, in which all parameters concur to an accurate description of the environment, platform and operation. In order to achieve realistic results, one can barely rely on the pre-defined parameters found in the literature. Currently, there are many conventional open source and freeware WSN simulation tools that are publicly available or in academic research use. Some examples are NS-2 [34], OMNeT++ [35], Worldsens [36] and TOSSIM [37], in which, the network protocols can be programmed and configured according to users’ desires and provide the convenience to simulate and evaluate performances at protocol levels. However, radio signal propagation is a very complex phenomenon since it is three-dimensional and influenced by many disturbances that are caused by the environment. Multi-path fading and attenuation directly contribute to the reliability and range of the wireless network. The current network simulators all use very simple physical channel and environment modeling. As default, the propagation models in these simulators are based on predefined empirical functions or assume as line-of-sight connection where no obstacles between the transmitter and receiver. Despite their powerful ability in validating WSN performance at protocol functionality level, they are not able to consider the impacts of realistic environments 12 1.3. Introduction to the planning algorithms and tools on topologies, protocols and deployments, and as a result, the performance of designed WSN can not be practically estimated, which might significantly influence the real implementation. Besides the network simulators for validating protocol levels, several other tools have been developed for radio propagation level or pre-deployment level to tackle different challenging issues for WSNs. In [38], the authors introduced several state-of-the-art radio propagation simulators, such as the EDX Signal pro [39], Winprop (AWE) [40] and CINDOOR [41], all of which are featured with 3D indoor modeling and ray-tracing propagation modeling in the simulation. The simulation and modeling of radio propagation for WSN can significantly reduce the work effort and costs to ensure the connectivity of WSNs. For obtaining more accurate results, the suitable radio simulator for WSNs is envisaged to support the importation of 3D indoor and outdoor environment model, the radio and antenna should be definable, the radio algorithm should be able to detect the multi-path effects including the direct path, reflections and diffractions. After investigating on the deployment issues and performance evaluation issues of WSNs, two questions arise: • Why can’t a good radio frequency simulator be used in collaboration with a network simulator to provide practical WSN performance analysis for any specific 3D environment and application? • Once provided with the evaluated performance of WSN topology, is there any smart tool to assist WSN designers improving the overall performance of WSN automatically and efficiently instead of manually adjust the designed topology and estimate repeatedly through network simulator? The answers to those questions are directedly pointed to smart planning tool for WSNs, which is desired and strongly demanded by the WSN designers. 1.3 Introduction to the planning algorithms and tools There are several WSN planning algorithms and tools developed in recent years. The 3D indoor planning heuristic (LowCost) [42], to the best of our knowledge, is the first indoor 3D WSN deployment heuristic that considers impacts of obstacles on sensing 13 Chapter 1. Introduction signal and radio communication. It consists of two steps: Provided a 3D indoor environment model with furniture and obstacles recorded, the first step calculates the coverage to deployment cost ratio for all the candidate points in the deployable area. Sensor nodes are iteratively put at the point with the maximum coverage to deployment cost ratio, so that the target region is covered with the minimum sensor node cost after this step. Then the connectivity of the deployed nodes is checked in the second step. The authors consider two options to satisfy the connectivity of WSN, the prior one is realized by moving the unconnected node towards the closest connected node without influencing the sensing coverage of the first step; otherwise, if the preferential option is not applicable, extra sensor nodes will be added along the line between the unconnected node and the closest connected node. Note that despite this approach manages to cover the sensing area with the ”minimum cost”, the connectivity of the WSN is ensured by simply moving or placing extra nodes without carefully selecting optimal positions to decrease the hardware cost, improve the link quality or prolong the network lifetime. Moreover, although the modelling of the sensing signal considers obstacles, the radio propagation model is too simple because the communication links are established only between line-of-sight (LOS) nodes, which is obviously not true in the real-world propagation. The MOGA algorithm [43] employs multi-objective genetic algorithm, which is proved to be efficient in solving NP-hard problem, to evolve the decision. Topology solution for the same network varies at different runs which provides more options than the deterministic approach of LowCost. However it focuses on maximizing the sensing coverage and prolonging the network lifetime with a predetermined number of nodes, as a result the hardware cost can not be optimized. Moreover the modeling of radio signal and sensing signal are based on ideal disc model thus it is not environmental sensitive. The previous two methods are developed for planning the homogeneous WSNs. However, in many prototypical systems available today, sensor networks normally consist of a variety of different devices. Nodes may differ in the type and number of attached sensors [32, 44]; some computationally more powerful computenodes may collect, process, and route sensory data from many more limited sensing nodes [45]. 14 1.3. Introduction to the planning algorithms and tools There are several works focused on planning heterogeneous network, such as in [46], the relay node placement problem for WSNs is concerned as placing a minimum number of relay nodes into a WSN to meet certain connectivity or survivability requirements. In that work, the authors assume that there may be some physical constraints on the placement of relay nodes and they study constrained versions of the relay node placement problem, where relay nodes can only be placed at a set of candidate locations. In the connected relay node placement problem, they want to place a minimum number of relay nodes to ensure that each sensor node is connected with a base station through a bidirectional path. In the survivable relay node placement problem, they want to place a minimum number of relay nodes to ensure that each sensor node is connected with two base stations (or the only base station in case there is only one base station) through two node-disjoint bidirectional paths. For each of the two problems, they discuss its computational complexity and present a framework of polynomial time O(1)-approximation algorithms with small approximation ratios. Numerical results show that their approximation algorithms can produce solutions very close to optimal solutions. The authors of [47] propose an approximation algorithm to find a feasible solution for relay node placement to deploy a minimum set of relay nodes in such a fashion that each sensor node must have at least one relay node within its one hop distance and all deployed relay nodes eventually form a connected network among themselves including one or more base-stations. The work reveals an approximation algorithm that runs in O(n2 ) time complexity, to find a feasible solution for above challenge. The authors also describe a framework to solve the above problem in non-convex shaped deployment region. [48] targets at providing hight efficiency and QoS and it presents a polynomial-time algorithms which is QoS-aware relay node placement using minimum Steiner tree on Convex hull. The work in [49] proposes Multiple-Objective Metric (MOM) for base station placement in wireless sensor networks to fairly increase various properties. It considers four different metrics for base station placement in WSNs. First, the ratio of sensor nodes which can communicate with a base station via either single-hop or multi-hop represents the coverage of sensor nodes. Second, the average ratio of connected sensor nodes after the failure of base stations represents the fault tolerance of a network. Third, the average distance between sensor nodes and their nearest 15 Chapter 1. Introduction base station represents the energy consumption of a network. However, as discussed before, not only the distance but also the obstacles lead to attenuation of received signal strength. Moreover, more energy is consumed at nodes with larger degree, as a result the energy consumption is not practically modelled by this work. Fourth, the standard deviation of the degree of base stations represents the average delay of a network due to congestion. The limitation of this algorithm is that sensor nodes should be pre-located by designers, which neither guarantees the sensing coverage without expert experience nor allows optimizing the hardware cost for WSN. The authors in [50] develop a tool that integrates a developed 3D indoor deployment heuristic together with NS-2 simulator to assist designers deploying and analyzing the performance of network. They propose a heuristic that minimizes hardware cost while satisfying requirements on coverage and connectivity. The network topology is constraint to the type of cluster tree and three different devices are provided: the coordinator, router and sensor. Sensors can only communicate with routers and coordinator. The heuristic considers radiation pattern of antenna as well as the effects of obstacles by using accurate ray-tracing algorithm. Once the topology is generated, the integrated NS-2 simulator is driven to simulate the packet drop rate and latency, and the results are demonstrated to users. The merit of this method is the integration of authorized network simulator to evaluate the performance of generated topology which provides a much more practical implication on packet delivery performance to designers. However, as the evaluation from NS-2 has no contribution to improve the generated topology, the proposed deployment heuristic should be run several times so that by a certain chance, designers can observe a satisfied solution with low cost, low drop rate and latency. The user interface of this work is shown in Fig. 1.6. It allows users prosecute many configurations including map, node properties, topology constraints and environment types. The generated topology can be shown and results evaluated by NS-2 are reported on the interface. Another state-of-the-art method for deploying relay node and sink node for indoor environment is proposed in [51, 52], the tool allows users defining the node demand zones, power source, sensing interval and transmission delay. By encapsulating those metrics into a complete requirements model, the tool optimizes the infrastructure of WSN and maximizes the utility function which provides 16 1.3. Introduction to the planning algorithms and tools Figure 1.6: The interface and demonstration of the work in [50]. a normalized equation that observes the coverage, link quality, lifetime and infrastructure cost. Fig. 1.7 and Fig. 1.8 are two examples of generated solutions for single-hop and multi-hop topologies respectively. The lifetime (L) of sensor node is considered in that work and is modelled by (1.2), The electric charge of a sensor node EC, expressed in mAh, is calculated according to (1.1) where Ia and Is are the power consumption in active state and sleep state respectively. ta and ts represents their time durations in a node interval. The current capacity of the battery CC is expressed in mAh. EC = 3600 × (ta × Ia + ts × Is ) ta + ts L= CC EC (1.1) (1.2) As it can be noticed from the formulations, the model of lifetime only considers the impacts of different states and their corresponding time durations, while the number of packets forwarded for other nodes are ignored which is not realistic especially for multi-hop topology. Besides, the authors did not consider the impacts of packet delivery ability (latency and drop rate) to ensure a more reliable WSN topology. 17 Chapter 1. Introduction Sensor nodes in this work should also be pre-determined by users and therefore node locations and cost are not optimized. Figure 1.7: Demonstration of the single-hop solution by the work in [51]. Figure 1.8: Demonstration of the multi-hop solution by the work in [51]. 18 1.3. Introduction to the planning algorithms and tools Some works are developed to tackle the modelling of sensing signal and radio signal to make the deployment algorithm more practical and accurate. The authors in [53] develop a probabilistic sensing model for sensors with line-of-sight-based coverage (e.g. cameras). The probabilistic sensing model takes into consideration sensing capacity probability as well as critical environmental factors such as terrain topography. Besides, they also implement several optimization schemes for sensor placement optimization. Sensor deployment in network-structured environments is studied in [54] and it aims to achieve k-coverage while minimizing the number of sensor nodes. The coverage problem of wireless sensor networks for the rolling terrains is studied in [55] to derive the general expression of the expected coverage ratio for regular terrains and irregular terrains. To enhance the WSN lifetime, the authors in [56] propose a deployment strategy with a non-uniform deployment method and an alternative duty mode to balance the energy consumption of sensor nodes in chain-type WSNs. To make the deployed network resilient to faults caused by communication errors, unstable network connectivity, and sensor faults, the authors in [57] present an approach, called FTSHM (fault tolerance in Structural health monitoring (SHM)), to repair the network with redundant backup nodes and guarantee a specified degree of fault tolerance. FTSHM searches the repairing points in clusters and places a set of backup sensors at those points by satisfying civil engineering requirements. FTSHM also includes a SHM algorithm suitable for decentralized computing in energy constrained WSNs, with the objective to guarantee that the WSN for SHM remains connected in the event of a sensor fault thus prolonging the WSN lifetime under connectivity and data delivery constraints. Table. 1.1 provides a comparative summary of the characteristics of the static node placement mechanisms discussed in this section. Optimal node placement is a very challenging problem that has been proven to be NP-Hard for most of the formulations of sensor deployment [59, 60, 61]. The deployment of WSNs should satisfy the requirements on the 3D sensing coverage, guaranteeing connectivity, prolonging the network longevity and reducing the hardware cost. However the existing planning algorithms and tools are to some extent have serious limitations to practically design reliable WSN topology. Only a few of 19 Chapter 1. Introduction [46] [43] Generic Generic Indoor Generic Generic Application 3D 1D 2D 2D 2D 2D 2D 3D Space Deterministic Deterministic Deterministic Deterministic Random Random Deterministic Random Deterministic Deployment Sensor Sensor,Sink Sensor Sensor Relay node Base station Relay node Relay node Sensor Sensor Node type Connectivity, Fault Tolerence Engergy consumption Congestion Lifetime,Cost QoS, Cost Coverage Cost Coverage Data fidelity Coverage Cost Lifetime – – Table 1.1: Comparison of planning algorithms [47] Generic Network Structure Random Sensor Fault tolerance [55] [56] Deployment cost Hardware cost Coverage, Life time Connectivity Lifetime Hardware cost - Fixed node count Place of relay node One sensor attached to one relay node – Linear align deployment Convex hull – – 3D Deterministic Deterministic Relay node, Sink Sensor Coordinator Router Link quality,Lifetime Infrastructure cost Cost Coverage, Connectivity Coverate, Connectivity Sphere sensing model Indoor 3D Connectivity, Hardware cost Constraints [49] Generic Generic 3D Deterministic Sensor Objectives [58] [48] Generic 1D Deterministic Paper [53] Generic 3D [42] [54] Outdoor Generic Generic Structure Monitoring [50] Indoor [57] [51, 52] 20 1.3. Introduction to the planning algorithms and tools them tackle the 3D deployment issue, and an overwhelming number of works are proposed to place devices in 2D scheme. Without considering the full dimension, the impacts of environment to the performance of WSN are not completely studied, thus the values of evaluated metrics such as connectivity and sensing coverage are not sufficiently accurate to make proper decision. Even fewer planning methods model the sensing coverage and radio propagation by considering the realistic scenario where obstacles exist. Radio signals propagate with multi-path phenomenon in the real world, in which direct paths, reflected paths and diffracted paths contribute to the received signal strength. Besides, obstacles between the path of sensor and objects might block the sensing signals, thus create coverage hole in the application. None of the planning algorithms model the network longevity properly and practically. They often employ unilateral and unrealistic formulations. The optimization targets are often one-sided in the current works. Without comprehensive evaluation on the important metrics, the performance of planned WSNs can not be reliable and entirely optimized. The diversity and variety of the deployed environment of WSNs have significant impacts on radio communication and sensing coverage. As a result, it is crucial for WSN designers having accurate environment model to prosecute realistic radio propagation and coverage estimation. However, in conventional approaches of constructing 3D environment model, people have to use a third party CAD tool or geometric scripts to reconstruct the target environment database. The prior knowledge of the detailed geographical information can only be obtained via manual measurements or from the city hall. Instead of doing so, people can buy such services from professional 3D modelling agencies. When the application scenario gets larger, the cost grows higher. Therefore, either time and efforts or money is needed in conventional approaches to construct 3D environment models. While none of the current network planning works or radio propagation researches figure out any method to model the 3D deployment environment efficiently and accurately. Many researchers are trapped by this issue and their algorithms/models can only be evaluated always in the same scenarios, without the possibility to test the robustness and feasibility for implementations in different environments. 21 Chapter 1. Introduction 1.4 Proposed methodology and work flow: Main contributions Motivated by all the reasons and challenging issues mentioned in the previous sections, this work dedicates to explore and develop state-of-the-art planning methodology and smart planning tool to assist WSN designers efficiently designing reliable WSN deployments. • A new method is proposed to efficiently and automatically model the 3D indoor and outdoor environments. The advantages of image understanding algorithm are applied to automatically recognize objects from the satellite images of the outdoor target regions and from the scanned floor plan of indoor area, thereafter 3D outdoor/indoor scenarios are reconstructed automatically and efficiently for signal propagation and network planning purpose. Its mechanism offers users a flexibility to reconstruct different types of environment without any human interaction. Thereby it significantly reduces human efforts, cost and time and allows WSN designers concentrating on the planning issues. • An efficient ray-tracing engine is developed to accurately and practically model the radio propagation and sensing signal on the constructed 3D map. By using the kd-tree space division algorithm and modified polar sweep algorithm, the signal rays are traced efficiently without detecting all the primitives in the scene. The proposed radio propagation model, which emphasizes not only the materials of obstacles but also their locations along the signal path, is applied to compute the received signal strengths for all candidate receivers. The sensing signal of sensor nodes is tracked by taking advantage of this obstacle sensitive approach. • WSN planning method is proposed to tackle the above mentioned challenges and efficiently deploy reliable WSNs. More metrics (connectivity, coverage, cost, lifetime, packet latency and packet drop rate) are modeled more practically compared with other works. Especially 3D ray tracing method is used to model the radio link and sensing signal which are sensitive to the obstruction of obstacles; network routing is constructed by using widely-used AODV protocol; the network longevity, packet delay and packet drop rate are 22 1.4. Proposed methodology and work flow: Main contributions obtained via simulating practical events in WSNet simulator, which to the best of our knowledge, is the first time that network simulator is involved in a planning algorithm. Moreover, a multi-objective optimization algorithm is developed to cater for the characteristics of WSNs. The capability of providing multiple optimized solutions simultaneously allows users making their own decisions accordingly, and the results are more comprehensively optimized compared with other state-of-the-art algorithms. • iMOST is developed based the novel planning methodology, by integrating the automatic 3D reconstruction method, the ray-tracing engine and the planning algorithm, to assist WSN designers efficiently planning reliable WSNs for different configurations. The abbreviated name iMOST stands for an Intelligent Multi-objective Optimization Sensor network planning Tool. iMOST features with: (1) Convenient operation with a user-friendly vision system; (2) Efficient and automatic 3D database reconstruction and fast 3D objects design for both indoor and outdoor environments; (3) It provides multiple multi-objective optimized 3D deployment solutions and allows users to configure the network properties, hence it can adapt to various WSN applications; (4) Deployment solutions in the 3D space and the corresponding evaluated performance are visually presented to users; and (5) The Node Placement Module of iMOST is available online as well as the source code of the other two rebuilt heuristics. Therefore WSN designers will be benefit from this tool on efficiently constructing environment database, practically and efficiently planning reliable WSNs for both outdoor and indoor applications. With the open source codes, they are also able to compare their developed algorithms with ours to contribute to this academic field. An overview of the planning tool is shown in Fig.1.9. As can be seen, it contains a user interface and three functionality modules: Image Processing Module, Ray-tracing Propagation Module and Node Placement Module. They are developed and embedded together to make the planning tool powerful and useful, thus contribute on the above mentioned aspects. The user friendly interface: provides user interaction with the tool and algorithms. It allows importing objects such as furniture for indoor 23 Chapter 1. Introduction WSN Planning Tool Brick wall Glass wall D 4 D 1 v1 w1 v2 w7 w6 TX ray ray 1 RX 2 D w2 2 w3 w5 D 3 w4 Image Processing Module Ray-tracing Propagation Module Node Placement Module Figure 1.9: The proposed methodology and tool. environment and vegetation and car for outdoor environment; it supports defining the sensing area AS in the space, pre-deploying nodes at some specific locations and configuring the parameters such as the transmission power and receiver sensitivity of the antenna and sensing ability of sensors; the deployment solution is visually provided to the user by indicating the locations of nodes, the constructed topology and the evaluation results. The Image Processing Module (IPM): is in charge of automatic 3D environment database reconstruction for indoor and outdoor environments. The algorithm is able to recognize different obstacles automatically with high accuracy. The birds’ view RGB images of the outdoor region (taken from satellite camera or sketched from websites like Google Maps) or the scanned map of indoor space should be prepared beforehand. Afterwards, the recognized result is vectorized and the 3D database is built accordingly without human supervision. The Ray-tracing Propagation Module (RPM): employs the developed ray-tracing method to trace the propagation path for both radio and 24 1.5. Conclusion sensing signal. Obstacles are considered along the ray path, therefore Non-line-of-sight (NLOS) obstruction is detected for the sensing signal and the multi-path effects and attenuations due to different obstacles are calculated in a proper way to enhance the accuracy of radio propagation estimation. The Node Placement Module (NPM): provides multi-objective optimized WSN deployment solutions according to the user configuration. This module employs the proposed multi-objective planning method and iteratively searches the best topologies to simultaneously trade-off among the sensing coverage, connectivity, cost, WSN longevity and packet delivery status. 1.5 Conclusion In this chapter, the widespread applications of WSNs are introduced in terms of military, environment, health monitoring and tracking, which nowadays are evolved to integrate different application categories to construct our world a smart world. Due to the lack of available experiences and guidance to assist WSN designers efficiently planning reliable and optimized WSN topology for various applications, practical and efficient planning methodology and tool are strongly demanded to facilitate WSN design step so as to reduce human efforts, cost and optimize network performance before real field deployment. Several state-of-the-art planning algorithms and tools are investigated and compared in this chapter, from the computation dimension, optimization objectives, modelling of objectives and application constraints. Besides of the advantages, the limitations of them are discussed thoroughly. A novel planning methodology is proposed in this work, to librate WSN designers from time consuming and costly 3D outdoor/indoor environment modelling, by using the proposed automatic image understanding and vectorization algorithm; to increase the accuracy and efficiency in radio propagation estimation and sensing signal tracking through the developed ray-tracing engine; to practically model important metrics; and to comprehensively optimize the performance of WSNs by 25 Chapter 1. Introduction properly selecting nodes and their properties based on multi-objective optimization method. An intelligent multi-objective optimization sensor network planning tool (iMOST) is developed in this work as well. It is a practical realization of the proposed planning methodology, the user-friendly interface facilitates the user operations, and visually demonstrates results. The three functional modules of iMOST work together to realize the proposed planning methodology. The planning tool, the planning methodology and the algorithms contribute to this work will be described in details throughout this dissertation. 26 Chapter 2 3D environment reconstruction method 2.1 Introduction WSN designers should be provided with the information of the region where the WSN will be deployed, so that the proper locations for placing sensor nodes can be decided to achieve application requirements. The important factors that affect the deployment are the obstacles and irregular terrains which can attenuate radio signal strength and block continuity of sensing region; there may exist particular areas that prohibit to place sensor nodes and also exist some other regions to be monitored. A traditional and standard approach to create a 3D model for a region is to build from scratch using tools such as CAD software or scripts that can be imported to a demonstrator. Roads, vegetation and building blocks can be described in the form of primitive 3D shapes based on which terrain data or maps should be provided in advance: either obtain from manual measurements, from city hall or purchase from professional company. Therefore, this geometry-based modeling technique is time-consuming and costly, especially for constructing large-scale scenarios where many obstacles exist. For many radio propagation experts, the major challenge is that rather than analyzing the radio performance, they are not skillful at preparing the environment model. Moreover, according to the survey in the academic conferences, we notice that some research institutes use the same scenario all the time to analyse their work without the possibility to test the robustness and accuracy for different environments. Chapter 2. 3D environment reconstruction method 2.1.1 3D environment reconstruction from Lidar systems The 3D environment reconstruction system may consists of automatic building extraction and reconstruction [62], road extraction and reconstruction [63], vegetation extraction and reconstruction [64]. Such applications require detailed and reference models that are still usually created manually. Several recent techniques [65] aim to increase the level of automation and realism by starting with actual images of the object or directly digitizing it with airborne laser scanning. Fig. 2.1 and Fig. 2.2 shows the general principle of the airborne laser scanning. The Figure 2.1: Airborne laser scanning. Figure 2.2: Example of the DSM and its original image. standard features of recent airborne lidar systems are their ability to discriminate between first and last pulse reflections. A laser pulse that is fired over an object usually has multiple reflections. Some of the laser pulses may be reflected by the 28 2.1. Introduction top of the object and therefore represents the first returning pulse. The remainder is likely to be reflected by the ground and hence generates the last returning pulse. Through computing the arrival time of the laser signal, lidar produces a fast and highly accurate three-dimensional Digital Surface Model (DSM) [66]. Thereafter the objects are extracted by using several image processing algorithms such as the edge detection and shape based classification [67],[68]. The limitation of this method is the high cost and efforts to obtain the scanned data, as helicopter or airplane is needed to carry the laser scanner and many human efforts are needed to realize the task. Therefore it is impractical to use such expensive technology for radio propagation simulation and network planning issues. 2.1.2 2.1.2.1 3D environment reconstruction from images Multi-view stereo reconstruction There are some works proposed to use multi-view stereo reconstructions that are efficient to provide dense and full 3D reconstructions from multiple views. These methods can be classified into: 1. Visual hull reconstruction. Obtain the 3D representation of an object through exploiting the silhouette information. The principle of this method is shown in Fig. 2.3. This technique assumes the texture of objects can be separated from the background. Under this assumption, the original image can be converted into a binary image, which we call a silhouette image. The foreground mask, known as a silhouette, is the 2D projection of the corresponding 3D foreground object from a specific view point. Along with the camera parameters, the silhouette defines a back-projected generalized cone that contains the actual object. This cone is called a silhouette cone and the intersection of the two cones is called a visual hull [69] which is a bounding geometry of the actual 3D object. 2. Space carving reconstruction. Generates an initial reconstruction that envelopes the object to be reconstructed. Then it iteratively removes unoccupied regions from an explicit volumetric representation. All voxels falling outside of the projected silhouette cone of a given view are considered inconsistent and are eliminated from the volume, see Fig. 2.4. 29 Chapter 2. 3D environment reconstruction method Figure 2.3: The intersection of silhouette cones defines an approximate geometric representation of an object called the visual hull. A visual hull has several desirable properties: It contains the actual object, and it has consistent silhouettes [70]. 3. Image-based rendering. Graph-cut algorithm is used in combination of carving approach, silhouette information to obtain higher precision. Fig. 2.5 shows the principles of the light field rendering method by [72]. This technique rotate the source of light along with the object platform, therefore different projections of the object with constant incident angle of light are taken by fixed camera. As can be seen, the multi-view stereo reconstruction requires multiple images of different views for the same scenario, where the objects are a few, in most cases there is only one object, with a texture quite different than the background. When the scenario becomes large and complex, or when there is no multiple stereo view available, this technique is not applicable any more. 2.1.2.2 Single image reconstruction Most recently, researchers begin to explore reconstruction from monocular view from a single image [73, 74]. The 3D reconstruction from single image is a challenging and attractive issue, as unlike human vision system and brain which have been trained for years and decades to predict the invisible part of an object by observing a single view, 30 2.1. Introduction Figure 2.4: Example of space carving reconstruction [71]. (a) The plane-sweep algorithm ensures that voxels are visited in order of visibility with respect to all active cameras. The current plane and active set of cameras is shown in orange. (b) The shape evolves and new cameras become active as the plane moves through the scene volume. Figure 2.5: Object and lighting support and the prototype camera gantry [72]. the computer is much simpler, and usually more than most other computer vision problems, single-view reconstruction is a highly ill-posed problem. To intimate the human vision, researchers integrate machine learning method to train the dense map database, to recognize objects or the shapes of specific images. Ashutosh Saxena et al.[75, 76] patch the images for both 3D location and 3D orientation and use a Markov Random Field (MRF) to infer a set of ”plane parameters” that belong to each small homogeneous path in the image. After training the MRF via supervised learning, the approach (see Fig. 2.6) is able to estimate the relationships between 31 Chapter 2. 3D environment reconstruction method different parts of the image thus generate the depth map for the test images. This method lacks the ability to model the invisible structure of the objects. In some works, human interaction is necessary to indicate the important feature markers. Zou et al. [77] uses a set of auxiliary reference grids to precisely reconstruct both polyhedral objects and curved-surfaced objects from a single image with unknown camera parameters. In that method, users should first define the edges and vertices of the objects, based on which, the camera is calibrated and reference grid is obtained accurately. Afterwards, the 3D wire frames of object is generated and surfaces are rendered, as a result it can be applied only when there are a few objects on the image, otherwise the human efforts spent on marking feature points will be unaffordable. Original image Ground truth of depth map (a)Training Database (b)Training procedure (c)Test result Figure 2.6: (a) The database of the depthmap is obtained by using the color map of laser scan; (b) The feature vector for a superpixel, which includes immediate and distant neighbors in multiple scales. The relationships are learnt through training with the groundtruth database; (c) An example of the test result which indicate the depth map of the test image with color scale. 2.2 3D outdoor environment reconstruction The outdoor environments, where the WSNs will be deployed in our work, are the European urban cities. The dominant types of obstacles for such environments are buildings, trees and cars of various colors, textures and shapes. 32 2.2. 3D outdoor environment reconstruction Although the airborne laser scanning (ALS) can accurately reconstruct large scale outdoor environment, it requires expensive hardware equipments (e.g. helicopter, laser scanner), time and a lot of manpower which violate the purpose of reducing the cost and efforts for deploying WSNs. Whereas the reconstruction from multi-stereo images are only suitable for simple scenarios with a few objects. It is also a very tough task to fetch city images with different stereos and views. For example, Google street view research group record street-level imagery by mounting the street view camera system on custom road vehicles, trike and modified snowmoble Fig. 2.7. A lot of human efforts are needed to register the information with the global scale and to construct the 3D models. Therefore this method is not suitable for our work either. In the early 21st century satellite imagery became widely available when affordable, easy to use software with access to satellite imagery databases was offered by several companies and organizations. Several other countries have satellite imaging programs, and a collaborative European effort launched the ERS and Envisat satellites carrying various sensors. All satellite images produced by NASA are published by Earth Observatory and are freely available to the public; Satellite images of different resolutions are available on Google and Yahoo Maps, which provide us the opportunity to access the satellite images freely. Since all the objects are visible on the images in pixels, it is convenient to indicate and locate the objects without measuring the geographic information in the real campaigns, which provides us the opportunity to reconstruct the cities and outdoor environments conveniently. However, as the scenario can be very large and complex, it is impractical and tedious for users manually indicating each primitive on the images. To avoid such a time and effort consuming procedure, we proposed a solution to automatically reconstruct the scenarios. This method should be able to simultaneously distinguish different objects on a image. Moreover, the recognitions must be realized pixel wisely so that it is suitable for further implementation for radio propagation and network planning. Thanks to the long historic development of image understanding researches, there are several state-of-the-art algorithms appeared in the most recent years which make it possible to recognize multiple objects pixel wisely without any human supervision. 33 Chapter 2. 3D environment reconstruction method Figure 2.7: Google street view tools. From top to bottom and from left to right: street view car, snowmobile, trekker, trike and trolley. He et al. [78] and Belongie et al. [79], are examples of those who propose to identify the objects by shape feature descriptor. Although the detailed shape-based matching algorithm can be quite different, the principles of them are similar, which basically consist of 3 steps: 1. Solve the correspondence between shapes; 2. Identify the model class to which the input object belongs; 3. Provide the point correspondences on the matched contours or further refine the matched points to provide accurate matched result between the actual shape and reference shape model. 34 2.2. 3D outdoor environment reconstruction The limitation of using shape-based algorithm to understand urban city environment lays in the variety of shapes for the same type of objects. Arivazhagan et al. [80] use color and texture features to extract fruit from simple background. He et al. [81] and Shotton et al. [82] recognize different obstacles based on machine learning incorporating texture, layout, and context information, which provide more comprehensive methods to distinguish multiple objects from complex scenarios. Shotton et al. [82] proposed a texton-boost algorithm that iteratively selects discriminative texture-layout filters v[r,t] (i) to compute weak learners, and M m combines them into a strong classifier of the form H(c, i) = m=1 hi (c). Each weak learner hi (c) is a decision stump based on the response hi (c) = ⎧ ⎨a[v [r,t] (i) > θ] + b ⎩k c if c < C (2.1) otherwise For those classes that share the feature c ∈ C, the weak learner gives hi (c) ∈ {a + b, b} depending on the comparison of feature response to a threshold θ. For classes not sharing the feature, the constant k c makes sure that unequal numbers of training examples of each class do not adversely affect the learning procedure. The machine learning algorithm, to some extent, intimates the learning procedure of human brain, turns out to be more suitable for our work. Ideally, once the method learns the features of different objects in different environments, it is able to indicate pixel wisely each objects on the images robustly no matter how the WSN deployment environment varies. 2.2.1 Proposed algorithm for outdoor environment reconstruction In this thesis, the algorithm proposed by Shotton et al. [82] is extended and sub-sampling and random feature selection techniques are used for iterative learning of the images. The estimated confidence value of each class for each pixel can be reinterpreted as a probability distribution using soft max transformation [83] to calculate the texture layout potentials. The work flow of the proposed method is indicated in Fig. 2.8. First of all, a set of images of the target region should be taken: either through super high resolution satellite camera or through standard hand-held camera. The 35 Chapter 2. 3D environment reconstruction method Data compression and Vectorization A large Scenario (a composition of small images of similar sizes) Recognition and Segmentation 3D reconstruction Figure 2.8: The work flow of the 3D outdoor environment reconstruction. properties of those images should be similar to that of the training images on the size, the angel of view, the resolution and so on. The construction of training image set will be discussed in Section 2.2.2. Thereafter the developed image understanding algorithm and segmentation algorithm are applied to automatically recognize objects in the provided images and extract them pixel-wisely. Image regularization and vectorization algorithms are employed to regularize the segmented objects into compact primitive shapes. Their position, rotation and scale information is registered with reference shapes, so that the feature points can be extracted and objects are vectorized with a 2D planar coordination. The GIS information is obtained and transform to the format of longitude and latitude. At the end, a ’KML’ log file is generated according to the vecotization result. The file can be demonstrated visually in software like Google Eearth and it provides an open access of GIS data for all kinds of purpose, especially for radio propagation simulation and network planning community that have urgent demands on GIS information of various environments. 2.2.2 Image database Our method starts from constructing the training of an image database, through which the features of images are learnt and discriminated for the concerned types of objects. In this work a bench mark image database and a user defined database are 36 2.2. 3D outdoor environment reconstruction used. The color code for different objects are indicated in Fig. 2.9(c) and example images of the two databases are shown in Fig. 2.9 and Fig. 2.10 respectively. The Microsoft Research Cambridge (MSRC) database is composed of 591 photographs of the following 21 object classes: building, grass, tree, cow, sheep, sky, airplane, water, face, car, bicycle, flower, sign, bird, book, chair, road, cat, dog, body, and boat. A subset of original images are shown in Fig. 2.9(a). The corresponding ground truths are shown in Fig. 2.9(b), where each color maps uniquely to an object class label according to the color codes. In this database all the images have similar size which is approximately 320 × 240 pixels. Note that, in reality there may exist some scenarios contain other objects rather than those in the 21-object database and different visual angles than that provided by the images. To tackle the problem, one can label new object classes manually by assigning different and unique color codes to create a new database that satisfy user’s requirements. However, the 21 objects are far many in this work for real world radio propagation and network planning in the European urban cities where only a few objects (e.g. buildings and cars) dominant the impacts on radio propagation and network topology. Besides we intend to use the satellite images, where bird’s view of the scenario is taken, the perspective of those 21-class images are not suitable for recognizing objects efficiently. As a result, CEIeurope database is constructed for recognizing the European style urban city and only four object classes are focused. The visual angle is the bird’s view from the high resolution satellite camera. A selection of images and their ground truths are shown in Fig. 2.10 all the images are 800 × 550 pixels with JPEG format and the resolution is the 18th level (among the 20 levels) on Google Map. The four-object classes are road, building, car and tree, they are labeled by using the same color code as that in MSRC database. 2.2.3 Image understanding and segmentation algorithm The meaningful object classes are recognized through a machine learning mechanism, which consists of a training phase followed by an evaluation phase. In the training procedure, the Joint-Boost algorithm [82] is employed to compute the weak learners, which are combined together at the end to compose a strong classifier that allows multi-object recognition from images. The evaluation procedure recognizes the objects based on a combination of possibility matrix, which indicates the 37 Chapter 2. 3D environment reconstruction method (a) (b) (c) Figure 2.9: The MSRC labeled image database. (a) A selection of images in the 21-class database. (b) The ground truth annotations corresponding to (a),(c) The color codes for the 21 classes. possibility of how each pixel belongs to each object class. The possibility matrix is provided by the trained strong classifier from training phase and a sub-cluster dominant class computation algorithm to refine the unprecise pixels without any human supervision. 38 2.2. 3D outdoor environment reconstruction (a) (b) Figure 2.10: The CEIeurope labeled image database. (a) A selection of images in the database. (b) The ground truth annotations corresponding to (a). 2.2.3.1 The training phase All the images, including the training images and the test images, are automatically converted from RGB color space to Lab color space at the very beginning. Lab color space with dimension L for lightness and a and b for the color-opponent dimensions, based on nonlinearly compressed CIE XYZ color space coordinates, is the only way to communicate different colors across different devices. The training images are convolved with a 17-dimensional filter bank (see Fig. 2.11) to extract the features of texture. The filterbank consists of: 3 Gaussians, 4 Laplacians of Gaussians (LoG) and 4 first order derivatives of Gaussians. The three Gaussian kernels (with σ = 1, 2, 4) are applied to each L, a, b channel, thus producing 9 filter responses. The four LoGs (with σ = 1, 2, 4, 8) are applied to the L channel only, thus producing 4 filter responses. The four derivatives of Gaussians are divided into the two x-and y-aligned sets, each with two different values of σ (σ = 2, 4). Derivatives of Gaussians are also applied to the L channel only, thus producing 4 final filter responses. Therefore, each pixel in each image has associated to a 17-dimensional feature vector. The choice of filter-bank is somewhat arbitrary, as long as it is sufficiently representative. This filter-bank was determined to have full rank in a singular-value decomposition (see [67]), and therefore there are no redundant elements. 39 Chapter 2. 3D environment reconstruction method (b) (a) (c) Figure 2.11: The 17D filter bank. (a) The 3 Gaussians are applied to L,a,b channels. (b) 4 LoGs are applied to L channel. (c) 4 derivatives of Gaussians are for L channel. 50 40 Kd-tree clustering 30 20 Filter bank 10 Input image Texton Map 0 Figure 2.12: The process of image textonization. The 17D responses for all training pixels are then whitened (to give zero mean and unit covariance), and an unsupervised clustering is performed. The Euclidean-distance K-means clustering algorithm is employed, which can be made dramatically faster by using the techniques of [84]. Finally, each pixel in each image is assigned to the nearest cluster center, producing the texton map. The procedure of image textonization is shown in Fig. 2.12. The texton map is denoted as T where pixel i has value Ti , i ∈ {1, ..., K}. 40 2.2. 3D outdoor environment reconstruction Afterwards, texture-layout filter is applied to learn the layout of texture and context information of T . Each texture-layout filter is a pair (r, t) of an image region r, and a texton index t. Region r is defined in coordinates relative to the pixel i being classified. For computational efficiency, only rectangular regions are investigated in our work, though an arbitrary region could be used. A set R of candidate rectangles are chosen randomly at the beginning of the training procedure to reduce the endless exploration of the shapes. As illustrated in Fig. 2.13, the feature response of texton index t at the location i is the proportion of the pixels under the offset region r + i that have texton index t and it is expresed by (2.2). Outside the image boundary there is zero contribution to the feature response. v[r,t] (i) = 1 [Tj = t] area(r) (2.2) j∈(r+i) We define that C is the set of object classes that is within the focus of application. c ∈ C represents a member of C. N ⊆ C represent a sub-set of the classes. Joint-Boost algorithm iteratively selects discriminative texture-layout filters v[r,t] (i) to compute weak learners. Each weak learner hi (c) is a decision stump based on the response hi (c) = ⎧ ⎨a[v [r,t] (i) > θ] + b ⎩k c if c ∈ N (2.3) otherwise At each iteration m, the subset N is randomly picked from C. For those classes c ∈ N that share the feature, the weak learner gives hi (c) ∈ {a + b, b} depending on the comparison of feature response to a threshold θ. For classes not sharing the feature, the constant k c makes sure that unequal numbers of training examples of each class do not adversely affect the learning procedure. At each round, a new weak learner is chosen by minimizing an error function 2 Jwse = Σc Σi wic (zic − hm i (c)) (2.4) Where each pixel i is attached with a target value zic {−1, +1} (+1 if i belongs to the class c, and −1 otherwise), wic is the weight function that specify the classification accuracy of i after m − 1 rounds of learning. 41 Chapter 2. 3D environment reconstruction method t1 r1 (a) Feature responses of v1 t2 r2 Texton Map (b) Feature responses of v2 Figure 2.13: Calculating the feature responses for textons. (a) the feature responses for v[r1 , t1 ]. (b) The feature responses for v[r2 , t2 ]. Theoretically, all the candidates r in R and t in T should be traversed at each iteration to find the best v[r,t] that produces the minimum Jwse for all the possible sharing class sets. However, the traversal is expensive and time consuming as there are |T | × |R| different combination candidates. To efficiently tackle this problem, each candidate pair of [r, t] is tested with a probability during an iteration and with thousands of iterations, all the possibilities are likely to be evaluated. Given the feature filter v and the threshold. The solutions of a + b, b and k c are expressed as follows: c c w z [v(i, r, t) ≤ θ] c∈N i i c i b= c∈N i wi [v(i, r, t) ≤ θ] c c w z [v(i, r, t) > θ] a + b = c∈N i i c i c∈N i wi [v(i, r, t) > θ] 42 (2.5) (2.6) 2.2. 3D outdoor environment reconstruction c c w z k = i i c i i wi c (2.7) We assume wic = 1 at the very beginning, and after a new weak learner is selected at round m, it is updated as wic := wic e−zi hi c m (c) (2.8) Thus at the end wic = e−zi Hi (c) and all the weak learners are combined into a strong c classifier of the form: H(c, i) = M hm i (c) (2.9) m=1 The texture-layout filters together with their thresholds and sharing classes compose the weak learners provided from the training phase, they are eventually stored in a training database for further evaluation. 2.2.3.2 The evaluation phase At the evaluation step, all the test images are converted into Lab color format and convolved with the 17D filter bank, and the responses are clustered to generate texton maps as the training procedure. The training database generated in the previous step are applied to the texton maps of all the test images. Hx (c, i) is computed on each pixel i in a test image x and the estimated confidence value can be reinterpreted as a probability distribution using soft max transformation [85] to give the texture layout potentials: Px (c, i) ∝ exp Hx (c, i) (2.10) Note that for each image a probability matrix Px is generated with three dimensions Hx × Wx × |C|, with H and W as the height and width of x, |C| is the number of classes in C. The temporary recognition result of each pixel i ∈ x, up to now, is the class with the maximum potential value. Ct (i, x) = arg max Px (c, i) c 43 (2.11) Chapter 2. 3D environment reconstruction method An example of the result is demonstrated in Fig. 2.14(b). Although most of the pixels are classified correctly and different objects are recognized, the precision of the edges between adjacent objects are not sufficient. Shotton et al. in [82] proposed a supervised method by manually indicating the misclassified parts, thereafter the algorithm adjusts those miss-classified parts into the right classes. Note that when there are a batch of images to be adjusted, the efforts of manually indication for each misclassified segments will be considerable and time consumption will be very high. Therefore, we proposed an automatic adjustment method to improve the results without any human interaction. (a) Input image (b)preliminary recognition and segmentation (c) Edge refined by Graph-Cut algorithm (d) Improved result by subdividing the clusters Figure 2.14: Multi-object recognition procedure. 2.2.4 Performance enhancement 2.2.4.1 Use of color information for refining edges Color cues are frequently used in image processing area for edge detection, as the color gradient variation at the border of adjacent objects are obvious, and that is also how human eyes distinguish the border of objects. In this work we propose to utilize the property of color variation information to improve the previous result Ct efficiently. First of all, the image is smoothly clustered into K clusters based on the RGB color distribution. Instead of using normal k-mean cluster method, a graph cut algorithm [86, 87] is employed to smoothly cut the edges of different clusters. Because our work requires the labels vary smoothly almost everywhere while preserving sharp discontinuities that may exist at object boundaries. Fig. 2.15 compares the results obtained from traditional k-mean cluster method to the Graph-cut algorithm, in both of the results, each pixel is assigned to a cluster and 44 2.2. 3D outdoor environment reconstruction (a) Input images (b) k-mean cluster (c) Graph cut algorithm Figure 2.15: Comparison clustering results between k-mean cluster and graphcut algorithm. (a) A selection of the input images from MSRC database. (b) The clustering result obtained by k-mean cluster.(c) The clustering result obtained by graphcut algorithm. labeled with different color code. The cluster boundaries of the graph-cut contain less noise and are more desired by this work. Basically the selection of the number of clusters K is arbitrary: the larger the value is, the more details there will be in the clustering results. As our purpose is to distinguish the non-trivial color information while maintain a certain tolerance of variation to avoid the over clustering issue, the proper range of K should be within 5 to 8 depending on the complexity of the target area. We denote Cdm (k) as 45 Chapter 2. 3D environment reconstruction method the dominant class of cluster k, and it is calculated by selecting the class with the maximum summation of likelihood for each cluster (see (2.12) and (2.13)), Px (cj |k) = 1 Px (cj , i) area(k) (2.12) i∈k Cdm (k) = arg max Px (cj |k) c (2.13) Px (cj |k) is the likelihood that the pixels in cluster k belong to class cj , and Px (cj , i) is the likelihood matrix of image x calculated by (2.10). Cdm (k) is the dominant class decision for cluster k. Therefore each object, constraint by color clusters and class likelyhood matrix, has much clearer boundary than the result when only Ct is used. The example results (Fig. 2.14(c)) visually indicate that the boundaries of adjacent objects are clearer than in Fig. 2.14(b) while the recognition accuracy is reduced. Due to the color similarity among different classes, for instances the color of sky and that of the windshields of cars are similar, parts of buildings and roads have similar colors, some pixels not in the same classes are labeled with same clusters. As a result, the dominant class method makes the adjustment inappropriate for the dominated members. To tackle this problem, we further divide each cluster into various numbers of subclusters based on their connectivity property (see Fig. 2.16). The likelihood matrix Px (cj |ks ) is calculated for each subcluster ks by using (2.12). Then Px (cj |ks ) is sort in descent order, and the class with the max potential value is the dominant class Cdm (ks ) of ks . An exception occurs when the class with the second max potential is also the dominant class of the parent cluster Cdm (k) with ks ∈ k, and if Px (Cdm (k)|ks ) ≥ μPx (Cdm (ks )|ks ), the class of ks is set to Cdm (k) rather than Cdm (ks ), otherwise C(ks ) = Cdm (ks ). The recognition result after this step is shown in Fig. 2.14(d) and more results are shown in Fig. 2.17. 46 2.2. 3D outdoor environment reconstruction k k1 (a) input image k k k2 (c) A selected cluster k (d) sub cluster map ks of k (b) graph-cut clustering Figure 2.16: Example of sub-clustering based on connectivity property. 2.2.4.2 Image understanding results and comparisons We perform experiments on a subset of the 21-object MSRC database, and only focus on 6 out of the 21 classes: building, grass, tree, sky, car and road, which might be the main objects that affect outdoor radio propagations. The experimental results Building Grass Tree Sky Car Road True class Building Grass Tree Sky Car Road Inferred class are given in Table. 2.1 in a format of confusion matrix. 75.7 2.4 16.8 10.0 12.8 7.1 0.1 44.3 0.6 0 0 0 8.1 35.0 71.6 5.7 6.0 1.5 4.9 0 8.9 84.3 0.4 0 9.5 1.1 1.8 0 76.0 2.6 1.7 17.2 0.3 0 4.9 88.8 Table 2.1: Confusion matrix. Number of cluster in texton booster is 400, the average accuracy is 76.1%. With 1000 rounds of boosting 45% of training images in the database, the number of texton clusters is set to be 400 to achieve the best average accuracy of 76.1% whereas the Joint-Boost algorithm gives an overall accuracy of 69.2%. Shotton et al. [82] then manually moved the misclassified parts to the right classes and the 47 Chapter 2. 3D environment reconstruction method improved result has an accuracy of 72.2%. We notice that the proposed method by our work outperforms the Joint-Boost algorithm and the edges of adjacent objects are much more accurate and clearer. Moreover, the proposed method has the capability to adjust the misclassified pixels to the correct classes: For instance in the second image of Fig. 2.17, some pixels belonging to the building are recognized as car by Joint-Boost algorithm. The proposed method is able to change the decision and move the pixels to the correct classes, and some pixels are moved to the building class and some are moved to tree class. Hence the proposed multi-object recognition method is fully automatic and more accurate compared with the Joint-Boost method, and it can be used to decide the location and the material of objects. It provides the opportunity to automatically assign the attenuate coefficient for each object, which will be useful in the following work to increase the efficiency of radio propagation simulation. 2.2.4.3 Eliminating shadow effect The available satellite images are taken for different places at different time, and the solar incident angles are abundantly different. If an object is partially in the shadow of another object, the discontinuity of texture variation occurs. The shadow effect has undesired impacts on the image understanding results. Therefore it is important to detect and eliminate shadow effect. We employ a scan scheme to scan for each row and column the line segments that fall inside the shadow region rs , the detection is based on color information. Although this scheme is arbitrary, it works well for urban city environment where hardly exists surface with large black color. Before detecting the shadows, a test image is converted from RGB space to Gray scale space. Each column and row are scanned separately, and Fig. 2.18 shows the principle of a column scan and row scan. The pixels of a column/row are filtered by a 1 pixel × 5 pixel window to realize zero-phase filtering on the high frequency variations, thus the filtered result (red) is more sensitive to the significant variations. Thereafter, the line segments, which begin with a decline trend below the threshold value and end with a climbing trend above the threshold, are considered belonging to rs . The scanning and detection procedure is repeated until the entire image is checked. As the histogram varies 48 2.2. 3D outdoor environment reconstruction Input Images Results of TextonBoost Improved Result Figure 2.17: The comparison of results between the proposed algorithm and [82] in a selection of images. 49 Chapter 2. 3D environment reconstruction method Column Scan Row Scan (a) original variation curve filtered curve start point end point 300 250 200 150 Threshold value 100 50 0 0 100 200 300 400 500 600 (b) Figure 2.18: The principle of scanning shadow. (a) Row scan and column scan of the shadowed line segments. (b) Filtered curves are calculated and compared with the threshold. The starting point and end point of each line segment are determined for each shadowed region. 50 2.2. 3D outdoor environment reconstruction (a) (b) Figure 2.19: The result of shadow detection. (a) There exist small holes after the column and row scanning. (b) The final result by filling the holes to reduce noises in the detection. with different images, there is no fix threshold value that can be pre-determined for all the satellite images, hence the value of ϕ should be determined for each image before the shadow detection. In our work, the mean value of the entire gray-scale image is considered as the value of threshold ϕ in the plot. The selected line segments compose grid-style shape with lots of small ”holes”, which can be seen from Fig. morphological operation [88]. 2.19(a). We eliminate those noises through Besides depending on the order of provided by the potential matrix in (2.12), the regions belong to shadow area have very high possibility to be road class. Assuming that rs is the connected region of the shadow area. If Cdm (rs ) = road, while the second most possible class is road, the pixels in rs are assigned to road class. 2.2.4.4 Road detection and regularization Road detection algorithm is used to enhance the performance of the recognition algorithm for outdoor environment. city (i.e. If the input image is recognized as urban through detecting the proportion of buildings, cars), road detection 51 Chapter 2. 3D environment reconstruction method (a) (b) Figure 2.20: Road segmentation and orientation estimation.(a) The original test image. (b) The classification result after applying the color information on segmenting and eliminate the shadow. The pixels of road class are extracted and highlighted with white color, the background is in black color, and the orientation vector is estimated for each pixel that belongs to road class. algorithm can be used to regulate the street network of the city and reduce unrealistic recognition result and improve the accuracy. Once provided image understanding result from the previous step, the pixels belong to road class are segmented from other objects with binary value (see Fig. 2.20(b) as an example). The orientation vector is evaluated through R. Wilson’s orientation estimation method [89]. We assume that all the pixels belong to the same road are likely to have similar direction vector with smooth change, based on which, the direction map is grouped into 8 clusters (Fig. 2.21) by using the Euclidean k-mean cluster algorithm, thus the 360 degree space is divided into 8 sections and each with a range of 45 degree. Each cluster is then segmented separately and regions are labeled based on the connectivity as the sub clustering method in the previous section. The orientation θe of each connected region is computed by the orientation of the smallest bounding ellipse (SBE), which is the angle between the x-axis and the major 52 2.2. 3D outdoor environment reconstruction axis of the ellipse, see Fig. 2.22(a) as an example. θe is then compared with the main direction of this region θm , as indicated in Fig. 2.22(b). If θd = |θe − θm | > θT , the centroid orientation Oc (ri ) of this region ri is defined as θm to maintain the continuity with its neighbor clusters, otherwise Oc (ri ) = θe . 8 50 7 100 6 150 200 5 250 4 300 3 350 2 400 1 450 500 100 200 300 400 0 500 Figure 2.21: The orientations are clustered into 8 groups Region expansion is realized to optimize the traffic network, and regularize the width, direction and connect the isolated road segment. The centroid location Cri is −−−−→ −−−→ , given the calculated for each region. Direction of Cri Crj is represented as O− C C ri rj −−−→ − Oc (ri )| and |O−−−−→ − Oc (rj )| centroid orientation Oc (ri ) and Oc (rj ), if |O− C C C C ri rj ri rj are less than an angle threshold ranging from 5 to 10 degree, and the connection between the two regions does not have important effect to the recognized buildings, the two regions are supposed to be connected. This assumption is useful when only a small proportion of pixels belong to the road class are misclassified, and they, by mistake, disconnect ri and rj that are assumed to be in the same road segment. 53 Chapter 2. 3D environment reconstruction method Tm TT Te (a) (b) Figure 2.22: A toy example of the comparison between θe of the SBE and the main orientation θm . (a) The SBE of a region where the orientations belong to the same direction cluster. (b) Computation of the difference between θe and θm . Two regions are expanded and connected by rectangles once the aforementioned constraints are fulfilled. The rectangle is expressed by r = [O, C, W, L] −−−→ is the orientation of the rectangle, C is the central of the where O = O− C C ri rj rectangle, W is the width that equals the median distance from the boundary to the skeleton and L is the length which equals the distance between furthest endpoints of the two regions. The pixels locate within the rectangle are assigned to road class. The pseudo code for the aforementioned road detection and expansion algorithm is: Listing 2.1: Road detection and region expansion Segment road class from image x; compute the orientation map Mo ; k-means cluster with 8 sections(45◦ ); for each connected region ri { compute Oc (ri ) } 54 2.2. 3D outdoor environment reconstruction for each region pair (ri , rj ) and i = j { if (orientation is similar) { WR = roadwidth(ri , rj ) HR = roadlength(ri , rj ) −−−− → OR = O− C C ri rj −−−−→ CR = median(Cri Crj ) rectangle: R(CR , OR , WR , HR ) if (Σi∈R [C(i) ∈ / road] < 0.5Area(R)) ∀i ∈ R, C(i) = road } } Finally, the result of the road detection algorithm is shown in Fig. 2.23. Compared with the result without any enhancement, the accuracy is significantly improved after applying the color information and road detection algorithm. (a) (b) (c) Figure 2.23: The result of road detection. (a) The test image. (b) The result without any enhancement. (c) The result of road detection. 2.2.5 Shape matching and vectorization Real world objects can be expressed by geometric primitives in the computer graphic area and CAD systems. In constructive solid geometry, primitives are simple geometric shapes such as a cube, cylinder, sphere, cone, pyramid, torus for 3D applications. Whereas a polygon is traditionally a plane figure that is bounded by a 55 Chapter 2. 3D environment reconstruction method closed path, composed of a finite sequence of straight line segments (i.e. by a closed polygonal chain), which is useful in 2D applications. The objects recognized by our image understanding algorithms, are to be described as 3D geometric primitives as well. By means of recording the closed vertex path for each object, the digital environment database is constructed. Even though the boundaries of recognized objects can be directly used to compose polygons, they contain far more number of pixels than that is actually needed, and the information stored in database will be highly redundant by doing so. A logical and natural way to tackle this problem, is to extract the feature points from the boundaries of segmented objects. As the proposed image understanding algorithm is not able to obtain 100% accuracy, there are still errors and coarse edges at the boundaries which make the detection of feature points a big challenge for some very simple primitives. Further more, in urban cities and indoor environments, an overwhelming number of objects have regular planar shapes such as quadrilateral, triangle, circle that can be described conveniently with only a few vertexes. Therefore we develop a shape matching algorithm to detect the shape for all the recognized objects and register them with the corresponding corners by rotation and deformation. In this section, the methodology for shape matching and registration to vectorize the recognized objects and compact database is introduced. At the end, the 3D map is written to KML format log file to be demonstrated visually on software (e.g. on Google Earth) and be accessed by other people. The façades of buildings in European urban cities are often seen with the shapes shown in Fig. 2.24, the set of such shapes is named as S in this work. Basically, shape-based image retrieval consists of measuring the similarity between shapes represented by their features. Some simple geometric features can be used to describe shapes. Usually, the simple geometric features can only discriminate shapes with large differences. Therefore they are usually used as filters to eliminate false hits or combined with other descriptors and they are not suitable to be used as stand-alone shape descriptors. A shape can be described from different aspects including: Rectangularity, Solidity, Circularity ratio and Zernike moments (ZM). 56 2.2. 3D outdoor environment reconstruction Figure 2.24: Frequently seen façade shapes of buildings. From top to bottom and from left to right are: circular, triangular, rectangular, H, L, Cross and G shape 2.2.5.1 Rectangularity Rectangularity represents how rectangular a shape is, i.e. How much it fills its minimum bounding rectangle: Rectangularity = AS /AR (2.14) where AS is the area of a shape; AR is the area of the minimum bounding rectangle box. 2.2.5.2 Solidity Solidity describes the extent to which the shape is convex or concave and it is defined by Solidity = AS /H (2.15) where H is the convex hull area of the shape. The solidity of a convex shape is always 1. 57 Chapter 2. 3D environment reconstruction method 2.2.5.3 Circularity ratio Circularity ratio represents how a shape is similar to a circle. It is the ratio of the area of a shape to the area of a circle having the same perimeter: (2.16) Circularity = AS /AC where AC is the area of the circle having the same perimeter as the shape. Assume the perimeter of the region is P, AC = P2 /4π. As 4π is a constant, the Circularity can be rewritten as Circularity ∝ AS /P2 2.2.5.4 Zernike moments Zernike Moments (ZM) are orthogonal moments [45]. The complex Zernike moments are derived from orthogonal Zernike polynomials: Vnm (x, y) = Vnm (r cos θ, sin θ) = Rnm (r)exp(jmθ) where Rnm (r) is the orthogonal radial polynomial: (n−|m|)/2 Rnm (r) = s=0 (−1)s (n − s)! s! × ( n−2s+|m| )!( n−2s−|m| )! 2 2 rn−2s Zernike polynomials are a complete set of complex valued functions orthogonal over the unit disk. The Zernike moment of order n with repetition m of shape region is given by: Znm = n + 1 rs (r cos θ, r sin θ) · Rnm (r) · exp (jmθ) r ≤ 1 π r (2.17) θ 2.2.5.5 Shape matching and registration As mentioned before, there is no a single shape descriptor that is able to tell so many shape types, as the tuning of shape parameters is a challenging problem. In this work, an hierarchical matching heuristic is employed by swapping the objects into the proper branch at each level to hierarchically detailing the shape description. 58 2.2. 3D outdoor environment reconstruction As a matter of course, different discrimination metrics are applied at different levels and the state flow is shown in Fig. 2.25. Dr Do Figure 2.25: Hierarchical shape matching. At the root of the hierarchical model, the discriminator Dr is in charge of distinguishing circle, triangle and the rest of shapes in S. Then at the second level, the 3rd branch for the rest shapes, discriminator Do is trained for the remaining 4 shapes. By using the Rectangularity, Solidity, Circularity ratio and Zernike moments (ZM) (Eq. (2.14) to Eq. (2.17)) of the regions, the discriminator is trained in Neural Network to learn the shape features. The training shape set is built by rotating, transforming and deforming the shapes shown in Fig. 2.24. Each training sample is black-white image of size 255 × 255, where the shape is in black color and with variation on orientations, scales. Dr is obtained by learning circles as type Sc , triangles as type St and the remaining types as So ; Do is obtained by learning rectangle as type Sr , H shape as type Sh , cross shape as Scro and the G shape as SG . Dr and Do are trained separately by using Bayesian regulation backpropagation function. The results of the shape matching are listed in table 2.2 and shown in Fig. 2.26. The accuracy is 100% without adding noise at the boundary of the test shapes, and 90% when adding the noise for applying to the recognized results where there are no serious errors. All the typical shapes in S are stored in data structure that 59 Chapter 2. 3D environment reconstruction method Table 2.2: Shape matching result XX XXX Input XXX Cicle XXX Output Circle Triangle Others Rectangles H L Cross G Triangle Rectangles H L Cross G contains the features like the centroid, orientation, scale of sides and so on, which also depend on how those shapes described in geometry. Once provided with the shape matched result. The objects are registered with the corresponding shape features to search the most suitable rotation and deformation by minimizing the evaluation metric. Assume S is the candidate shape, AS = i∈x [i ∈ S ] represents the area, and AIS = i∈x [i ∈ S && i ∈ S] represents the actual area of pixels that belong to S and S. The metric that evaluates the candidate shape is expressed by (2.18). The formula sums the ratio between candidate shape area and actual union region and the ratio between original shape area and the union region, which indicates how well the candidate shape matches with the original region. E(S ) = AS AS + AIS AIS (2.18) The object becomes finding the S b that provides minimum E S b = arg min E(S ) S (2.19) Thereafter, a iterative procedure is imposed on the shape matching and recognition through Eq. 2.18 and Eq. 2.19. 2.2.5.6 Delineating the buildings Recalling that the centroid of ri is Cri : {x, y} and the orientation of ri is Oc (ri ). If ri ∈ Sr , the polygon can be described by four vertexes (A, B, C and D) that are 60 2.2. 3D outdoor environment reconstruction Cross Rectangle L shape U shape H shape Figure 2.26: Successful result of shape matching. Each shape of Do is correctly matched. constraint by the following conditions: C= ⎧ yI −yA ⎪ ⎪ ⎪ xI −xA = tan θ, ⎪ ⎪ ⎪ −→ ⎪ −→ ⎨− AD = 2AI −−→ −−−→ ⎪ ⎪ DB = 2DCri ⎪ ⎪ ⎪ ⎪ → −−−→ ⎪ ⎩− AC = 2ACri yI −yCr i xI −xCr i = − tan1 θ (2.20) −−→ Where I is the median point of AD and θ = Oc (ri ) is the orientation of the shape. Note that the recognized region r is not perfect and we need to match the shape 61 Chapter 2. 3D environment reconstruction method with the condition in Eq. 2.20 by adapting θ and side lengths with coefficients: C = ⎧ yI −yA ⎪ 1 yI −yCri ⎪ ⎪ xI −xA = tan θ , xI −xCr = − tan θ ⎪ i ⎪ ⎪ −→ ⎪ −−→ ⎨− AD = αAD (2.21) −−→ −−→ ⎪ ⎪ AB = β AB ⎪ ⎪ ⎪ ⎪ −→ −−→ ⎪ ⎩− DC = β DC Where θ ∈ [θ − φ, θ + φ], normally φ ≤ 45o , and α(or β) ∈ [1 − τ, 1 − τ ] where τ ranges from 0 to 0.5. Fig. 2.27 shows the procedure of how a rectangle is registered by rotation and deformation. Rotation A B 100 I’ 200 300 Cr Deformation C Rotation D 400 500 600 100 200 300 400 500 Figure 2.27: An example of registering the rectangle primitives. 62 2.2. 3D outdoor environment reconstruction The similar heuristic can be applied to H shape, L shape and G shape by finding the vertexes that satisfy the conditions. Circular buildings are tested by a set of circles that centered at Cri and radius R ranging from the Bmin to Bmax which represent the boundary points that have the minimum and maximum distances with Cri . 2.2.5.7 Writing geometric primitives to KML file All the vectorized information are written into Keyhole Markup Language (KML) format, which is an XML notation for expressing geographic annotation and visualization within Internet-based, two-dimensional maps and three-dimensional earth browsers. Google Earth was the first program able to view and graphically edit KML files. Other projects such as Marble have also started to develop KML support. The advantage of using KML format is that the format is simple and descriptions are convenient to create for polygons, points and lines (see Listing 2.2 as an example of the script). It provides an open access possibility to the public and can be demonstrated in many popular free earth browsers as shown in Fig. 2.28. Listing 2.2: Common used Definitions in KML file <?xml version="1.0" encoding="UTF-8"?> <kml xmlns="http://www.opengis.net/kml/2.2"> <Style id="xxxx"><!--Define style of polygons, lines and points--> <LineStyle> </LineStyle> <PolyStyle> </PolyStyle> </Style> <Placemark> <Point><!--Draw point--> <coordinates>longitude, latitude, altitude</coordinates> </Point> <Polygon><!--Draw polygon--> <coordinates> longitude1, latitude1, altitude1 longitude2, latitude2, altitude2 ... </coordinates> 63 Chapter 2. 3D environment reconstruction method </Polygon> </Placemark> </kml> Figure 2.28: KML shape description of the reconstructed result demonstrated on Google Earth. 2.3 Indoor environment reconstruction Unlike the outdoor environment reconstruction, where the objects locate apart from each other and the scale is large, the indoor environment reconstruction is much easier. A method is developed to automatically model the 3D indoor environment from a scanned 2D map if the digital map is not available from the administrator. The reconstruction method consists of four steps: image calibration and classification step which recognizes walls from the scanned map; thinning and feature points extraction to compress the edge information; edge smoothing and vectorizing to build the 3D database. The reconstructed 3D scene is demonstrated to users at the end. 64 2.3. Indoor environment reconstruction Horizontal Distortion Vertical (a) (b) Figure 2.29: Image calibration and segmentation. (a) Scanned map. (b) Calibrated and segmented result. 2.3.1 Image calibration and classification Walls in the maps are normally labeled with black lines as shown in Fig. 2.29(a). Once the map is scanned, the image is converted from RGB color to gray color. Because the floor plan on paper has some distortion and transformation during scanning, the image should be calibrated with 2D horizontal-vertical direction to make sure that the axis of map matches well with real world axis. Thereafter, each pixel in the image is checked and pixels that belong to walls are recognized by using a single classifier h(i), ⎧ ⎨1 p(i) < v h(i) = ⎩0 p(i) ≥ v (2.22) where p(i) is the color of ith pixel in the image, v ∈ [0, 255] is the color threshold. If the pixel value is less than v, pixel i is marked by 1 which indicates the wall, otherwise it is marked with 0 which is the background. Hence, a segmentation result is obtained after this step, see Fig. 2.29(b). 65 Chapter 2. 3D environment reconstruction method 2.3.2 Thinning and feature points extraction In 2D vision of indoor scene, a wall can be expressed by an edge E < V1 , V2 > with V1 and V2 as two endpoints. As can be seen in Fig. 2.30, the extracted pixels from the previous step contain redundant information for constructing walls. Thus they are thinned to lines with 1-pixel width based on conventional thinning method and deliver the skeleton of wall efficiently. --Extracted pixels --Thinned line --Feature point --non-critical point Figure 2.30: Thinning step and feature point extraction However, as can be seen, the thinned lines (skeletons) still contain a large number of pixels which are not necessary to describe the critical variation of edges and some pixels are noises with high level fluctuations in a very short distance. Therefore, those non-critical points should be eliminated and only critical points along the thinned lines should be maintained. The critical points are also called feature points, which are either conjunctions shared by edges of different directions or the endpoints of edges. As a result, they can be used to represent the endpoints of edges. In this step, Harris-corner algorithm [90] is employed to search critical points throughout the segmented pixels and the results are demonstrated in Fig. 2.30 and Fig. 2.31. 2.3.3 Smoothing and vectorizing Critical points are clustered and map is vectorized in this step. Four windows with different directions are used to cluster feature points that belong to the same line. Fig. 2.31 shows the shapes of windows and an example of smoothing and regularization. As can be seen, the green dots are critical points obtained from previous step. After applying the windows, the locations of points are adjusted and marked by blue dots. Based on the smoothing algorithm, two points belong 66 2.3. Indoor environment reconstruction Windows with different directions Critical points Smoothed points Endpoints for regularizing edges Figure 2.31: Smoothing and regularization. to the same cluster and with largest distance are selected as endpoints of an edge. This step reduces the number of unnecessary planes and the database information is therefore compressed. 3D scene consists of obstacles which have shape description and material information that describe how signal strength is attenuated. Different materials have different attenuation values. The vectorization results are stored in ’.txt’ file, and the format is shown at the right side of Fig. 2.32. Vertices in 3D format are expressed as Vi = [xi , heighti , yi ]. In this work, all the walls are assumed to be quadrangle and have 4 vertices as indicated in Fig. 2.33. However, all the edges are discovered in 2D vision and only V1 = [x1 , 0, y1 ] and V2 = [x2 , 0, y2 ] are known. Therefore, once the height of wall is known, V3 = [x1 , height, y1 ] and V4 = [x2 , height, y2 ] are generated by assigning value to height. The structure of wall can be expressed by plane = V1 V3 V 2 V4 material where material is the index of material such as glass, wood, brick, concrete and so on. 67 Chapter 2. 3D environment reconstruction method Mismatched edges NUMBER OF PLANES 88 0 0 16.7 1 0 3 16.7 57.7 3 16.7 57.7 0 16.7 27.8 27.8 30.9 30.9 … 0 3 3 0 … 12.8 12.8 12.8 12.8 … 1 Figure 2.32: Vectorization result. Figure 2.33: A wall is described by four vertexes. 2.3.4 Demonstration and analysis The proposed 3D indoor reconstruction method is programmed in MATLAB, and run on a PC equipped with Intel i5-760 CPU of 2.8 GHz frequency. Fig. 2.32 compares the computed edges with the map. Only two edges have relative big errors (0.35 m) and one wall is missed classified, the rest edges match well with the map with an average error of 0.13 m. In this example, 86 edges are extracted. As the heights of walls are identically 3 m, they are included automatically to the result. Besides, after the endpoints are fetched, the horizontal range (X − range) and vertical range (Y − range) of target space are also obtained, therefore floor and ceiling are added automatically to the vectorization result and 88 planes are constructed at the end. The 3D views of the reconstructed indoor environment are demonstrated in Fig. 2.34. The size of the target space is 57.7 m×3 m×16.7 m. The time for constructing 68 2.4. Conclusion Figure 2.34: Reconstructed 3D indoor map in different views. this indoor map is 5.8 s, compared with conventional method which took almost 1 day per person by manually typing 88 × 4 endpoints. Fig. 2.35 is another example of 3D indoor reconstruction by randomly searching floor plan on Internet. page [91]. We select the East Lansing map provided on the web The size of the region is 77.4 m × 3 m × 36.6 m, the proposed indoor reconstruction method took place on the downloaded map, it took 23.5 s to reconstruct 156 planes on the floor plan. The recognized edges match well with the original image (see Fig. 2.35(b)) and the 3D reconstruction is viewed in Fig. 2.35(c). As a result the proposed method significantly reduces time and human efforts on modeling different indoor environments. 2.4 Conclusion 3D environment modelling is a difficult issue for radio engineers and network designers who do not have expert knowledge on this topic. In conventional approaches of constructing a 3D environment database, either a large amount of money should be spent to buy the digital vector map from professional companies, or time and human efforts are needed to manually measure and reconstruct the real campaigns. The expenses over this issue constraint the researchers and private users on validating their proposals or models, and they can be analyzed on a limited number of scenarios that are freely accessed by public such as the Munich scenario measured by COST 231 group [92]. To the best of our knowledge, this is the first time 69 Chapter 2. 3D environment reconstruction method (a) Original map of East Lansing (b) Vectorizing (c) 3D view of the reconstructed result Figure 2.35: (a) A toy example by using the map downloaded from [91]. (b) The vectorization result and the 3D view of reconstruction (c). 70 2.4. Conclusion that the advantages of image understanding algorithm is applied to automatically reconstruct 3D outdoor and indoor scenarios for signal propagation and network planning purpose. The principle of reconstruction method is introduced in this chapter and the proposed algorithms are detailed as well. The experimental results on outdoor reconstruction indicate that the algorithm is able to accurately recognize different objects from the satellite images of the target regions, it offers a flexibility to reconstruct different types of environment by training different environment database. The indoor reconstruction algorithm is proposed to reconstruct the indoor database from scanned floor plan, the efficiency and accuracy is satisfied without any human interaction. Thereby, the 3D environment reconstruction methodology proposed in this work significantly reduces human efforts, cost and time spent on reconstructing a 3D geographic database. 71 Chapter 3 Ray-tracing engine and radio propagation modelling There are numerous algorithms and methods that are developed to optimize the deployment, routing protocols, power consumption plans and etc. for WSNs. Before real implementation, those algorithms are evaluated through simulations which generally take place in network simulators such as NS-2, TOSSIM, EmStar or OMNeT++. Kamarudin et al. [93] review a variety of realistic propagation models for WSNs and discuss the modeling of vegetation propagation model in OMNeT++ simulation platform. They prove that propagation model has strong impact on the evaluation of network performance. However, the aforementioned simulators employ propagation models as simple as Free-space model and Log-normal model, which are too optimistic and independent of environment. The evaluations of those protocols and algorithms based on those empirical propagation models are not rigorous, which will lead to inappropriate algorithm design. Therefore accurate radio frequency (RF) propagation modelling and simulation are very important at the pre-deployment phase of WSN for predicting the design performance. There are many RF propagation simulation works focused on mobile communication. RF measurements have been made in several cities such as Munich scenario [92] and Ilmenau scenario [94]. Based on those experimental results, other works are done to model the radio propagation, tune critical parameters and evaluate coverage performance of base stations. Furthermore, signal attenuation parameters are also tested when signal penetrates through obstacles of different materials and sizes, such as different thickness of bricks, concrete, metal, etc. Most of the miniaturized wireless sensor devices implement ZigBee or 802.15.4 communication protocols, while there are only a few research works focused on modeling RF propagation for ZigBee WSNs, and even fewer work on measuring Chapter 3. Ray-tracing engine and radio propagation modelling ZigBee radio propagation in reality. For this reason, recently more researchers focus on modeling radio propagation for WSN and several experiments are made to characterize ZigBee propagation features in both indoor and outdoor environments [95], [96]. Korkalainen et al. [38] discuss advanced mobile signal tools that might be suitable for estimating 2.4 GHz ZigBee protocol. In order to ensure the results accurate and the simulation efficient, the tools should have features like 3D calculation, accurate environment modeling and radio modeling. In [97], the authors characterized wireless channels for indoor propagation at 2.4 GHz, but only direct path is considered. They conclude that free space propagation is unreliable, log-normal model is not completely matched with the trend of curvature of real measurements and the multi-wall-floor model is the most reliable and accurate among the discussed models. However, only considering the direct path is unilateral, as diffractions and reflections are also vital. Generally speaking, ray tracing algorithm has very high accuracy [98, 99, 100] compared with the empirical propagation models. It is obstacle sensitive which is able to trace multi-path effects in real world signal propagation. While the computation load is also very high as the method tests every intersection along the ray path. Especially when scenario becomes large, traditional method might take hours or days to finish simulation. Therefore many algorithms are developed to overcome the aforementioned drawback by slightly reducing accuracy as the compensation. Beam tracing algorithm [101] extends ray tracing algorithm to reduce intersection tests, as well as overcome sampling problems. Dominant path tracing algorithm [102] is developed to avoid redundant calculation, because the authors believe that 98% of received power is contributed by only a few radio rays. Ray tube tree method [103] increases the preprocessing speed in constructing trees for ray-tracing. A 2.5D outdoor ray launching tool is presented in [104]. The tool is very fast but the resolution is low (7 m) and the computation load is decreased by reducing 3D rays to 3D Line of sight (LOS), 2D Horizontal diffraction and reflection (HDV) and modified 2D Vertical diffraction (VD), therefore the accuracy of simulation result is constraint by the reduced resolution and the types of rays. A 3D ray-optical approach is presented in [105]. The calculation is in 3D, however in order to accelerate the calculation, the method preprocesses the environment by dividing 74 3.1. Space division the obstacles into tiles and the edges into segments, and ray paths are limited to only search over the tiles and segments. As a result the time consumptions of existing ray tracing methods are decreased by either reducing dimension of rays or reducing the details of the environment, the simulations are efficient while inaccurate. Therefore, in order to make the RF simulation method advanced, efficient efficient 3D ray tracing and accurate radio modelling should be developed and integrated. In this chapter, a 3D ray tracing engine is developed based on space division and polar sweeping is developed to efficiently search rays and the modelling of radio propagation is proposed to consider the obstacles and their order along the signal path. Our work is benefited from this part in terms of: 1. High accuracy in radio estimation for both indoor and outdoor environment. 2. High efficiency which allows the planning algorithm employing such technique to practically model the radio connectivity and sensing coverage with acceptable computational speed. 3. Generality. It can be used for the automatically reconstructed environment database from previous chapter as well as other GIS databases provided by designers. 3.1 Space division Before tracing rays, a 3D environment model, either from the aforementioned automatic reconstruction method or from other modelling methods, is loaded to the ray tracing engine, and the target space is split by using kd-tree algorithm. The target space is divided into small cubes with different volumes to balance the number of obstacles among the cubes. Vlastimil Havran did an extensive study of available spatial subdivision schemes (including regular grids, nested grids, octrees and kd-trees) and concluded in his thesis [106] that the kd-tree beats the other schemes in most cases. And Ingo Wald claimed that it is possible to limit the number of ray-triangle intersections to three or less using a well-constructed kd-tree [107]. 75 Chapter 3. Ray-tracing engine and radio propagation modelling A kd-tree is an axis-aligned Binary Space Partitioning (BSP) tree. Space is partitioned by splitting it in two halves, and the two halves are processed recursively until no half-space contains more than a preset number of primitives. While kd-trees may look like octrees at first, they actually are quite different: An octree always splits space in eight equal blocks, while the kd-tree splits space in two halves at a time. The most important difference though is that the position of the splitting plane is not fixed. In fact, positioning it well is what differentiates a good kd-tree from a bad one. Consider the images in Fig. 3.1: (a) (b) (c) Figure 3.1: Space division by kd-tree. (a) A scene with some primitives. (b) A bad example of kd-tree division. (c) A well constructed kd-tree. Fig. 3.1(a) shows a scene with some objects in it. In Fig. 3.1(b), this scene is subdivided; the first split is the vertical line, then both sides are split again by the horizontal lines. The split plane position is chosen in such a way that the number of primitives on both sides of the split plane should be roughly the same. Although this may sound like a good idea at the first glance, it actually isn’t: Imagine that a ray traverses through this subdivided scene but it never has chance to pass through an empty voxel. If we keep adding planes to this kd-tree following the same rule as Fig. 3.1(b), the result will end up with a tree that does not contain a single empty node, which is the worst possible situation, as the ray tracer has to check all primitives in each voxel it travels through. The subdivision in Fig. 3.1(c) is well constructed: A different heuristic by Jacco Bikker [108] is employed to determine the position of the split plane. The algorithm tries to isolate geometry from empty space, so that 76 3.2. Polar sweeping rays can travel freely without expensive intersection tests and therefore our space division mechanism employs the kd-tree algorithm proposed in [108]. 3.2 Polar sweeping At the beginning of ray tracing, the target environment is polar swept. Conventional polar sweep method is well known to solve geometric problems, by sweeping a line across the plane and halting at points where the line makes the intersection with any object within the target region. As depicted in Fig. 3.2, the solution is partially computed at those intersection points, so that at the end of sweep, a final intersection result is available. Polar sweep Figure 3.2: Conventional Polar sweep. In this work, conventional polar sweep algorithm is modified by bending the direction of the line whenever intersection occurs. A 3D line is rotated clock wisely and bottom up centering at the transmitter (TX) to discover for each direction the first intersected point and its corresponding plane. The rule of reflection is then applied to the line, which bends original direction of the line. Intersected planes are recorded whenever the direction changes at halting points. This procedure is repeated for each candidate direction, and each 3D line terminates shooting after 77 Chapter 3. Ray-tracing engine and radio propagation modelling Brick wall Glass wall D4 w7 w6 D1 ray1 w5 TX w1 ray2 D2 ray3 w2 w4 D3 w3 Figure 3.3: Polar sweep. a maximum number of intersections is reached or when the boundary is touched. Therefore, by sweeping the entire scene, all the possible orders of reflection planes and diffraction cones are discovered. The order of reflected plane is stored as a matrix with dimension of N ∗ depth, where N is number of possible reflection paths, depth is the maximum depth of reflections predefined in the ray tracing engine. For instance, the depth = 4 in Fig. 3.3 indicates no more than 4 reflections in a ray path, and the reflected plane is expressed as Ref plane. When visible plane exists and the length is less than depth, the ID of intersected primitive plane is recorded in a order along the signal path, such as w1 → w2 → w5 → w6. When the number of planes in a path is less than depth, N U LL is assigned to the remaining elements: ⎡ w7 N U LL N U LL N U LL ⎢ ⎢w1 ⎢ Ref plane = ⎢ ⎢w5 ⎣ .. . w2 w3 .. . ⎤ ⎥ w6 ⎥ ⎥ ⎥ N U LL N U LL⎥ ⎦ .. .. . . w5 (3.1) Diffraction happens at the convex edges, the diffraction points are extracted based on the resolution of z direction and Dif cone is a structure used to store all the information of a diffraction cone. structure Dif_cone{ point; 78 3.2. Polar sweeping visible_plane; invisible_plane; } Where Dif cone.point indicates the location of the diffraction Dif cone.visible plane is the ID of visible plane it incidents with, point, and Dif cone.invisible plane is the invisible plane connected with the visible plane, if there does not exist such a plane, Dif cone.invisible plane = N U LL. In ray tracing method, there exist many ways to search reflection paths. In this ray tracing engine, ray launching and ray tracing method are combined in order to be efficient. Ray launching is executed in the polar sweep step. Ray tracing happens when knowing the location of receiver (RX). After polar sweeping, the rays are traced for each RX, direct path, reflection paths and diffraction paths are selected and the received power strengths (RSS) are calculated separately according to models given in the following subsections. 3.2.1 Direct path A direct ray is launched from TX to RX, and each intersection is recorded when the ray penetrates through objects. The radio propagation model (3.2) which is developed by us in [109], is used to estimate power loss of direct path between TX and RX. Lp = 10 n log10 d + Lobstacle (3.2) Where n is path loss coefficient ranging from 2 to 5 and the value is dependent on the propagation environment, i.e. in free space n = 2, in others such as urban or rural environments 2 < n < 5. d is the direct 3D distance between TX and RX. Lobstacle is the power loss due to obstacles encountered along signal path. Lobstacle = N l(i)α(i)(i−1) (3.3) i=1 It is obtained by accumulating power reduction of each obstacle along the signal path. l(i) is attenuation parameter of the ith obstacle, α(i) ranges from 0 to 1, which is penetration rate of the material of ith obstacle. α(i)(i−1) decreases when i 79 Chapter 3. Ray-tracing engine and radio propagation modelling Brick wall Glass wall RX2 TX RX1 Figure 3.4: Direct path. increases, meaning that the first object, with which the signal intersects, produces the most significant power loss. Fig. 3.4 gives an example of calculating (3.2) for direct path. There are 2 RXs: RX1 and RX2 , and one transmitter T X. Signal penetrates through two glass walls before reaching RX1 , whereas another signal path penetrates through one brick wall to reach RX2 . For RX1 , l(1) = 2 dB, l(2) = 2 dB, assuming α = 0.9 as a constant value, we have Lobstacle = 2 + 2 × 0.9 = 3.8 dB. For RX2 , l(1) = 10 dB, we have Lobstacle = 10 dB. Eq. 3.2 indicates that not only distance, but also the materials of obstacles and their orders have impacts on received signal strength. In the ray tracing engine, the electric fields of radio wave are computed instead of calculating the path loss in dB, as the phase of propagation wave significantly affects the final result, thus resulting: Edir (d) = d √ n N Et i=1 Lobstacle(E) (i) e−j2πkd Where Et is the transmitted electric field at the sender. (3.4) Lobstacle(E) (i) is converted from Lobstacle (i), k is the wave length of radio, which is C/f , C is the speed of light and f is the frequency in Hz. 80 3.2. Polar sweeping Brick wall Glass wall D4 w7 w6 D1 TX v1 w5 ray1 w1 ray2 RX D2 w2 w4 D3 w3 v2 Figure 3.5: Reflection path searching. 3.2.2 Reflection path Starting from RX, each possible path discovered in Ref plane is traversed. The procedure is shown in Fig. 3.5, only two rays are shown as example. According to the second possible path in Fig. 3.3 and Eq. 3.1, Ref P lane(2) = [w1 w2 w5 w6], virtual source v1 of w1 is generated by mirroring T X along w1, v2 is obtained by mirroring v1 along w2. There are two reflection paths in this plane order, one is through T X → w1(v1) → RX, and the other one is T X → w1(v1) → w2(v2) → RX. w5 and w6 are ignored for current order. As a result, the format of reflection path matrix is expressed as: Ref path = w1 v1 w1 v1 w2 v2 (3.5) Note that although the example plot is by projecting 3D space into 2D, all the rays are traced in 3D approach. With this searching method, all candidate 3D paths from TX to RX are recorded at the end, and RSSs due to reflections are computed according to the model in Uniform Theory of Diffraction (UTD) [110]. For a RX at point P , its electric fiel Eref reflected from point Q is expressed by: Eref (dQP ) = Ei (Q)Re−j2πkdQP 81 (3.6) Chapter 3. Ray-tracing engine and radio propagation modelling Brick wall Glass wall D4 w7 w6 D1 w5 TX w1 w4 D2 w2 D3 w3 RX Figure 3.6: Diffraction path searching. Where Ei (Q) is the field of ray incident at Q. R is the reflection coefficient related to the material of encountered plane. 3.2.3 Diffraction path Diffractions happen at the diffraction points found in Dif point. When RX is shadowed by planes, diffraction will play a relative important role in RSS. Fig. 3.6 shows that a RX is in the diffraction region of diffraction point D2. The diffraction field is calculated by (3.7), Edif (dQP ) = Ei (Q)De−j2πkdQP (3.7) where Q represents D2 and P stands for RX in this case, D is diffraction coefficient, D= (1 + cos (θ)) √ , 2.0 2πf and θ = |(∠(T X, RX) − ∠(T X, Q))| is the angle differential between the rays from TX to Q and from TX to RX. At the end, according UTD theory, the Etot at P is the combination of Edir , Eref and Edif . Etot = Edir + Eref + Edif 82 (3.8) 3.3. Measurements and experimental results The RSS is calculated as RSS = 10log((Etot · 3.3 3.3.1 λ 2 ) ) 4π (3.9) Measurements and experimental results Outdoor RF propagation verification The proposed method was verified by comparing the results to that of Munich scenario [92], which was comprehensively studied by the European COST 231 working group. Fig. 3.7 illustrates the top view of the target region in Munich city. The size of the region is 2400 m × 3400 m and there are 3 different routes measured by COST 231 group (e.g. the red color is the first route rout0 named as METRO200, the black one rout1 is the second route METRO201, and the green one rout2 is the third rout METRO202). A new training database is created through similar methods described in Chapter 2. The training images are fetched randomly from Google Maps within the region of Munich city. Buildings, roads and trees are labeled in order to guarantee the learning of reliable and transferable knowledge. Fig. 3.8 shows the evaluation procedure: At the beginning, the same area in which the COST 231 group did measurements is captured on Google Maps. The images are downloaded and divided into small sub-images, each of which is of the similar size as the training images. Afterwards, they are passed through the image classifier to recognize and segment the objects belong to three classes (building, tree and road). The recognized objects are then vectorized and the 3D database is automatically generated. Then the propagation simulator is run to simulate the signal transmission in the target area. Finally, the signal coverage map is visually presented to the user. The proposed method is programmed by combining C#, Matlab and C++ code. It was run on a PC equipped with Intel i5-760 CPU, 2.8 GHz. It took less than 1 hour to construct the 3D database and approximately 15 minutes to finish radio estimation with a resolution of 1 m. Hence it took around 0.3 s to process 2356 points for all the three routes. The simulation results are compared with the practical measurements. As shown in Fig. 3.9 ∼ Fig. 3.11, the results of the 3 different routes correlate well with the 83 Chapter 3. Ray-tracing engine and radio propagation modelling Figure 3.7: Three different routes measured by COST231 group. real measurements. The mean error (ME) and the standard deviation (STD) of the results are the two metrics used to evaluate the accuracy of the simulation method. The ME of METRO200 is -0.3 dB and the STD is 5.7 dB. The ME is -0.4 dB and STD is 5.5 dB in METRO201. The ME and STD of METRO202 are 2.2 dB and 6.7 dB respectively. The performances on the first two routes are similar and are 84 3.3. Measurements and experimental results A small region of original image Classification result 3D reconstruction Radio simulation on the constructed 3D scene Figure 3.8: Classification result and radio propagation procedure over Munich scenario. 85 Chapter 3. Ray-tracing engine and radio propagation modelling comparison on METRO200 180 170 real measurement simulated result 160 pathloss(dB) 150 140 130 120 110 100 90 80 70 0 200 400 600 800 1000 number of location Figure 3.9: Simulation result for the first route METRO200. comparison on METRO201 180 170 real measurement simulated result 160 pathloss(dB) 150 140 130 120 110 100 90 80 70 0 50 100 150 200 250 300 350 400 number of location Figure 3.10: Simulation result for the second route METRO201. 86 3.3. Measurements and experimental results comparison on route METRO202 180 170 real measurement simulated result 160 pathloss(dB) 150 140 130 120 110 100 90 80 70 0 200 400 600 800 1000 1200 number of location Figure 3.11: Simulation result for the third route METRO202. Table 3.1: Comparison of radio estimation result with other methods. Method This thesis Ericsson CNET COST-WI RAY-TRI [111] METRO200 STD Mean 5.7 -0.3 6.7 0.3 6.9 -2.1 7.7 10.8 7.1 -2.6 METRO201 STD Mean 5.5 -0.4 7.1 2.3 9.5 -3.6 5.9 15.4 6.2 -0.7 METRO202 STD Mean 6.7 2.2 7.5 1.4 5.6 -0.2 7.3 16.3 8.3 -1.4 better than the third route which has more measured points and some are located in the streets with much more buildings. Table. 3.1 compares the proposed method with other state-of-the-art technologies. The STD and ME of the proposed method are the smallest on METRO200 and METRO201, while they are slightly greater than CNET in METRO202. The experimental results indicate that this novel approach has good efficiency and accuracy. A quantitative explanation for this improvement can be directed towards three reasons: the construction accuracy of the environment database is 82%, the miss-classified pixels are 18%. The resolution of simulation 87 Chapter 3. Ray-tracing engine and radio propagation modelling Figure 3.12: Four-layer architecture and physical view of the Cookie node. is 1 m which is higher than in other methods. Furthermore, the propagation model is adequate in describing reduced dimensionality environments, and most of the calculations used in the radio propagation model are based on efficient image manipulation rather than slow geometrical computation. 3.3.2 Indoor RF propagation verification In this section, indoor experiments are made to verify the simulation method. Sensor nodes named Cookies[112], developed by CEI-UPM, are used in this experiment. Cookieshave modular architecture of four layers as shown in Fig. 3.12. The layers are bonded through vertical connectors which contain all the signals within the node. As all the layers use the same connectors in the same position, reusability and interchangeability are much easier for catering different application requirements. Since the experiments are focused on signal strength measurement, the communication layer with ZigBee communication protocol is to be concerned. The communication layer includes a ZigBee module ETRX2 from Telegesis as shown in Fig. 3.13, which is a low power 2.4 GHz band transceiver based on the Ember EM250 SoC ZigBee/IEEE802.15.4 solution. The antenna pattern is near Omni-directional (see Fig. 3.14) with linear polarization. The transmitter is connected and power supplied by laptop through USB cable, the transmit power is set to be 3 dBm, the sensitivity of antenna is -97 dBm, the receiver is powered by two 1.5 V batteries. TX sends packet to RX to request link 88 3.3. Measurements and experimental results Figure 3.13: ETRX2 ZigBee communication module on communication layer. Figure 3.14: Radio pattern of antenna of ETRX2 module. information. Then RX fetches RSSI value of the received packet and replies to the transmitter. By placing the receivers at different locations, different RSSIs are obtained. The measurement is carried out in the lab of CEI-UPM. Three different scenarios are measured, as shown in Fig. 3.15. The Scenario A is realized in a room equipped with several desks and computers. The transmitter is fixed at 1.04 m high above the ground. The receiver is kept at the same height as the transmitter, and RSSIs are measured at five locations as indicated in Fig. 3.15(a). The purpose of this scenario is to verify the propagation model with trivial obstacles, exam the value of path loss coefficient n and RSS(d0 ). In scenario B (Fig. 3.15(b)), the position of transmitter is unchanged, while the locations of receiver are selected further away from transmitter, and are in different rooms around the lab. This is to verify the attenuation model 89 Chapter 3. Ray-tracing engine and radio propagation modelling BS (a) Scenario A BS (b) Scenario B ZigBee Coordinator ZigBee Sensor Device 12 1 BS (c) Scenario C Figure 3.15: Three scenarios for radio measurements. of non-trivial obstacles, and obtain empirical penetration parameters. Scenario C (Fig. 3.15(c)) is realized in the corridor where the communication between TX and RX are LOS, and because the width is much smaller compared with the length, the tunneling effect occurs. Not only both transmitter and receiver are maintained at same height, but also antennas are pointed toward directions with maximum RSSI to minimize the radiation pattern degradation. The ray tracing engine is programmed in C++ for Windows operating system. Fig. 3.16 shows the visible planes (with blue color) and the related information 90 3.3. Measurements and experimental results rx tx tx rx tx rx Figure 3.16: Ray tracing demonstrations from different TXs and RXs. discovered at the polar sweep step. Actually, the floor and ceiling are also visible planes to TX, but they are not highlighted for providing a friend vision. Those three figures show all the ray paths from TX (magenta point) to a RX (yellow point). As can be seen, the number of rays that could arrive at a RX is varied. Directed, reflected, diffracted rays are computed through full 3D approach without simplification during ray tracing. Fig. 3.17 and Fig. 3.18 show the simulated RSS maps for the campaigns. Both ray tracing algorithm and radio propagation model are developed for 3D computation. As can be seen, the walls significantly reduce signal strength when encountered by the signal path. During simulation, some parameters are constant, the resolution is set to be 0.2 m. Different groups of simulation results are obtained by varying depth from 0 to 6, which are then compared with the real measurements, as well as the Free-space model (3.10) and Log-normal model (3.11), where P Lref is the path loss at reference 91 Chapter 3. Ray-tracing engine and radio propagation modelling Figure 3.17: Simulation result: example 1. Figure 3.18: Simulation result: example 2. distance and Xσ is gaussian random value with 0 mean and σ as the standard deviation which is equal to 3 in this study, P Lf s (dB) = 20 log10 (d) + 20 log10 (f ) − 147.55 (3.10) P LLN (dB) = P Lref + 10 nln log10 (d/d0 ) + Xσ (3.11) Scenario A is a typical indoor environment without important obstacles between transmitter and receiver. Fig. 3.19 compares the proposed ray tracing method with Free-space path loss model, Log-normal model and real measurements. Table. 3.2 shows their mean error M E and standard deviation error ST D in dB, compared with the real measurements. Free-space model is too optimistic in estimating indoor propagation with the highest M E and ST D. While Log-normal model is much better than the Free-space model. The ray tracing is better than the Log-normal model. Moreover when depth = 3, the ray tracing method is the most accurate result. If all the RXs are in the same room of TX, the dominant contributions of 92 3.3. Measurements and experimental results Result Comparison for Scenario A -30 -40 -50 RSSI (dBm) -60 -70 -80 -90 Direct path Depth 3 Free Space Log-normal Real Measurment -100 -110 -120 1 2 3 Sensor ID 4 5 Figure 3.19: Results and comparisons of Scenario A. RSS are direct, reflected and diffracted paths without considering penetration losses. Some parameters of the proposed radio propagation model are determined through this case study to calibrate with the real world impacts, thus n = 2, Lobstacle(watts) = 3.6 and α = 0.91. The reference distance loss of the Log-normal model P Lref is calibrated to 60 dB and nln = 3 which will be maintained unchanged for the remaining two scenarios. One should also notice that the gaussian random variable Table 3.2: Results comparison: Scenario A. XXX Proposed XXXMethod XXX depth = 0 depth = 3 Result XX ME (dB) STD (dB) 3.18 3.22 93 1.99 2.00 FS LN 30.87 6.28 6.42 6.67 Chapter 3. Ray-tracing engine and radio propagation modelling Result Comparison for Scenario B -30 -40 -50 RSSI (dBm) -60 -70 -80 -90 Direct path Depth 3 Free Space Log-normal Real Measurment -100 -110 -120 1 2 3 Sensor ID 4 5 Figure 3.20: Results and comparisons of Scenario B. in the Log-normal model may slightly change the performance for the same scenario at different simulations. In Scenario B, RXs are in different rooms than TX, all possible paths may make contributions to RSS. In this scenario, the penetration attenuations have strong impacts on the RSS, see Fig. 3.20. Table. 3.3 shows the M E and ST D for each method. As expected for the ray tracing method, the results that only consider direct path are slightly worse than the ones considering multi-path effects. Since Log-normal model does not consider obstacles between TX and RX, it can not accurately model the indoor environment when obstacles exist and the ray tracing method turns out to be the best in this case. In Scenario C, there is no any obstacle between the TX and RXs. However due to the narrow width of corridor, reflections and diffractions play more important 94 3.3. Measurements and experimental results Table 3.3: Results comparison: Scenario B. XX XXX Method Proposed XXX XXX depth = 0 depth = 3 Result ME (dB) STD (dB) 2.71 2.91 2.42 1.65 FS LN 37.29 4.02 7.49 4.61 role than the previous two scenarios. As can be seen from Fig. 3.21, the ray tracing results match quite well with the real measurement even with small fluctuations. The slope of Free-space model is almost the same as ray tracing, which is reasonable, as a corridor can be considered as a free space environment in a long distance. However, the entire indoor environment is definitely not a free space. Table. 3.4 indicates that the proposed ray-tracing method outperforms the other two algorithms. Result Comparison for Scenario C -30 -40 -50 RSSI (dBm) -60 -70 -80 -90 Direct path Depth 3 Free Space Log-normal Real Measurment -100 -110 -120 1 2 3 4 5 6 7 Sensor ID 8 9 10 Figure 3.21: Results and comparisons of Scenario C. 95 11 12 Chapter 3. Ray-tracing engine and radio propagation modelling (a) Bird’s view of the deployment on the toy scenario (b) Zoom in of the deployment Figure 3.22: Demonstration of the toy example which has similar deployment as scenario C. As it can be noticed, the accuracy rises as the value of depth grows. Through the comparisons of the three scenarios, it is concluded that the proposed ray tracing method performs more accurately and robustly than other methods. Therefore it is suitable to be applied for different scenarios in indoor environment. The ray tracing engine is run in a PC with Intel Core i5-760 CPU of 2.8GHz frequency. Besides the three deployments in CEI-UPM environment, a virtual WSN deployment is realized in East Lansing scenario with the topology similar as scenario C, see Fig. 3.22. Table 3.4: Results comparison: Scenario C. XXX Proposed XXXMethod XXX depth = 0 depth = 3 Result XX ME (dB) STD (dB) 1.19 0.80 96 1.15 0.80 FS LN 30.23 1.41 10.65 1.41 3.3. Measurements and experimental results 20 Scenario A Scenario B Scenario C East Lansing Time (s) 15 10 5 0 direct 1 2 3 4 Reflection depth 5 6 Figure 3.23: Average time consumption of polar sweeping, by varying the depth of reflection rays. The average time consumptions of polar sweeping and ray tracing, by varying the reflection depth for each tested scenario, are shown in Fig. 3.23 and Fig. 3.24 respectively. Both figures indicate that the computation time increases as depth increases. The time consumptions of polar sweeping for scenarios A∼C in CEI-UPM are quite close to each other, while it took 2.8 times longer in average to finish the polar sweeping in East Lansing environment. The time cost of ray tracing of East Lansing goes much higher than that of Scenario C when depth exceeds 3, as much more new reflection planes are involved in the former environment. Fig. 3.25 shows that the polar sweeping without kd-tree traversing takes 3.35 times longer than the opposite choice, therefore the kd-tree traversing can significantly improve the efficiency of the ray tracing engine. The engine takes 37 ms in average to construct a kd-tree for CEI environment and 78 ms by doing so for the East Lansing environment. Note that the kd-tree construction is realized based on splitting properly all the primitives in the target region, the time spent on kd-tree construction is dependent on the number of planes in the environment. While the time spent on tracing rays and polar sweeping 97 Chapter 3. Ray-tracing engine and radio propagation modelling 40 Scenario A Scenario B Scenario C East Lansing 35 Time (ms) 30 25 20 15 10 5 0 direct 1 2 3 4 Reflection depth 5 6 Figure 3.24: Average time consumption of ray tracing, by varying the depth of reflection rays. depends on the value of depth and the complexity of surrounding environment of the transmitter. The more reflections and planes exist around a TX, the longer the simulation time will be. Table 3.5: Attenuation parameters of major objects indoors. Materials Antenuation Parameter (dB) Brick Glass 8 2 Metallic board Human Body of PC 5 2∼10 During the experiment, the attenuation parameters of different obstacles are also measured along the propagation path, as indicated in Table 3.5. The main obstacles are walls, glass of windows, metallic boards and human bodies. The attenuation caused by stilled human body even varies widely. Human movements are dynamic and there are some researchers focusing on the interference of human body on the radio propagation. The significance of the effect of human body towards signal strength depends on how high the sensor nodes are installed, and whether human 98 3.4. Conclusion 20 polar sweep with kd-tree polar sweep without kd-tree Time (s) 15 10 5 0 direct 1 2 3 4 Reflection depth 5 6 Figure 3.25: Average time consumption of polar sweeping with and without kd-tree traversing. movement is important for sensor applications, such as in localization systems or health care applications. 3.4 Conclusion The developed ray tracing engine propagation simulation method contributes on efficiency and accuracy to the radio estimation. By using image processing concepts including the kd-tree space division algorithm and modified polar sweep algorithm the rays are traced efficiently without detecting all the primitives in the scenario. The ray-tracing algorithm ensures accurate and practical results. The radio propagation model emphasizes not only the materials of obstacles but also their locations. Hence, the performance of simulation is robust and accurate compared with conventional propagation models. The experimental results imply that this methodology is suitable for both outdoor urban scenes and indoor environments, moreover it can be applied to GSM communication and ZigBee protocol by varying 99 Chapter 3. Ray-tracing engine and radio propagation modelling frequency parameter in the model. Moreover, the ray tracing engine is featured with generality that can be used for any environment database with the similar description format on primitives. The experimental data are available in this chapter, which can be used for comparisons by other researchers. The time consumptions are recorded for all the steps of ray tracing, the measurements on RSSs for different deployments and the attenuation parameters of different materials are also available in this work. Beyond the RF propagation estimation, the sensing signal of sensor nodes, which are sensitive to the obstacles, can also be benefit from the ray-tracing engine. The details for modeling the sensing coverage is introduced in the next chapter together with the planning algorithm. The indoor furniture also have influences on RF propagation and sensing signal, although it is not introduced in this chapter, with the ray tracing method, they can be detected. Therefore once provided with their models together with the environment database, the calculated results will automatically consider their impacts without any simplification. 100 Chapter 4 Planning the WSN One of the major challenges in designing wireless sensor networks is the support of various application requirements while coping with the computation, energy, communication, sensing and cost constraints. Careful node placement can be a very effective optimization means for achieving the desired design goals. However, optimal node placement is a very challenging problem that has been proven to be NP-Hard for most of the formulations of sensor deployment [59, 60, 61], and the modelling of important metrics turns out to have significant impacts on the deployment decision. In this chapter, design parameters including the network topology, heterogeneity of nodes and the modelling of important metrics are discussed. The current researches on optimized node placement have been reported in Chapter 1. According to the reviews, those algorithms and heuristics somehow have serious limitations. From the aspect of metric modelling, only a few of them tackle the 3D deployment issue and they are developed only for 3D indoor applications [42, 51, 52]. Even fewer works model the sensing coverage and radio propagation by considering the realistic scenario where obstacles exist. None of the aforementioned algorithms modeled the network longevity properly and practically, they often employ unilateral and unrealistic formulations. Moreover, the optimization targets are one-sided. Without comprehensive evaluation on the important metrics, the performance of WSNs can not be entirely optimized and reliable. An efficient WSN planning algorithm is proposed in this work to tackle the above mentioned challenges and efficiently assist designers on deploying reliable WSNs. This algorithm contributes on the following aspects: • Comprehensive metrics are considered. This work considers connectivity, sensing coverage, cost, lifetime, packet delay and packet loss rate, Chapter 4. Planning the WSN which to our best knowledge, is the most comprehensive evaluation scheme for analyzing the performance of WSN. • Practical metrics modelling by integrating network simulator. The connectivity and sensing coverage are modelled in assistance of 3D ray-tracing method which is sensitive to the existence of obstacles; hardware cost refers to the number of devices as well as their types. Routes of network are constructed by using AODV protocol based on the computed connectivity information. Network longevity, packet delay and packet drop rate are obtained through triggering events in WSNet simulator according to a user defined sensing task and the provided topology. It is the first time that network simulator is involved in a planning algorithm to tackle the difficulty on modelling those vital metrics and provide practical evaluations. • Efficient and multi-objective optimization. A multi-objective optimization algorithm is developed for WSN to optimize the cost, coverage, lifetime, packet delay and packet drop rate. The individual length is changeable so that the cost can be optimized, meanwhile crossovers and mutations are designed to eliminate invalid modifications to improve the computation efficiency. NSGA-II ranking method, which is proved with high efficiency, is employed by this work. 4.1 Topology One important property of a sensor network is its diameter, that is, the maximum number of hops between any two nodes in the network. In its simplest form, a sensor network forms a single-hop network, with every sensor node being able to directly communicate with every other node. An infrastructure-based network with a single base station forms a star network with a diameter of two. A multi-hop network may form an arbitrary graph, but often an overlay network with a simpler structure is constructed such as a tree or a set of connected stars. The topology affects many network characteristics such as latency, robustness, and capacity. The complexity of data routing and processing also depends on the topology. Fig. 4.1 shows four different topologies that have been applied to WSN applications. 102 4.1. Topology SN BS BS SN (a) Star Network (b) Tree Network SN CH SN CH CH BS BS CH (c) Mesh Network CH (d) Cluster Network Figure 4.1: Different topologies of WSN. 4.1.1 Star network In star networks, nodes are connected to a centralized communications hub. Each node cannot communicate directly with one another; all communications must be routed through the centralized hub. Each node is then a clientwhile the central hub is the serveritself or a gateway node that is in direct communication with the base station. An example of a star network is shown in Fig. 4.1(a). The failure of a link does not affect the entire network and leaves the rest of structure unchanged. However, it is not fault tolerant as there is no alternative path from unconnected node to the base station once the link is failed or obstructed. Due to the constraint on radio communication ability, sensor nodes in star topology can not be placed far away from the sink, thus its applications are limited to small networks. As there is no delay due to buffering at routers along the path, the data latency in star topology 103 Chapter 4. Planning the WSN is quite low, nevertheless there is likely to be more loss due to collision when network density increases. 4.1.2 Tree network A natural and logical extension of the star topology is the tree structure where sink node is the root and a collection of star networks are arranged at different levels in hierarchy, see Fig. 4.1 (b). All the communications between children (lower level) must be routed through their parent (upper level). The parent node has higher traffic load thus the energy is depleted faster than its children. Once die out of battery, the children attached to it will be affected and become isolated. Accordingly, as the level of node becomes higher, more nodes will be influenced once the critical link is broken. 4.1.3 Mesh network Mesh Networks are multi-hop local area networks in which each sensor node not only sends and receives its own message but also functions as a router to relay messages for its neighbors through the network. Mesh topology facilitates multiple communication paths from the sensor nodes to the base station. A special case of mesh is the grid topology where each grid point represents a sensor node, and the links are the edges of the grid. An example of mesh topology is shown in Fig. 4.1 (c). Multi-hop routing methods effectively overcome shadowing and path loss effects, thus mesh WSNs are self-configuring networks that dynamically optimize routes through the network based on the best link quality between neighbor nodes. If the node density is high enough, multi-hop routing uses a large amount of nodes to balance the relay traffic all over the WSN. However data latency increases as the number of hops increases, WSN designers should carefully plan the transmission and duty-cycle scheduling to overcome the collision and interference. Generally speaking, the amount of data, which are relayed for each node, increases as a node getting closer to the base station, therefore some redundant nodes are necessary to backup in the bottleneck area of WSN. Many applications employ mesh topology, such as 104 4.2. Heterogeneity the one for monitoring active volcano [17] and the GreenOrbs deployment [113] for ecological surveillance in the forest. 4.1.4 Cluster network In clustered hierarchical topology, all nodes in a WSN are joined at the lowest level. The sensors use their local neighborhood information to form a set of clusters and elect a cluster head (CH) for each cluster. The CH election process can be based on various parameters such as available energy resources, proximity to the base station, and number of neighbors. The CH in the lowest level are arranged into clusters in a higher level. The process is repeated for each level in the hierarchy. The number of hierarchical levels depends on several criteria including coverage requirement, deployment region, node density, and transceiver and sensing range. The clustered hierarchical architecture maintains a tree routed at the sink node, with a hierarchy of CH as the internal nodes and sensor nodes as leaf nodes of the tree. Nevertheless, different from the tree topology, it still maintains the multi-hop mesh within the same cluster. Compared with mesh mode, data aggregation is more efficient in cluster topology as only the CHs are in charge of aggregating and forwarding data to the sink, hence it can be extremely effective in one-to-many, many-to-one, or one-to-all (broadcast) communications. On the other hand, all data directed to the sink results in the CH near the sink come to have high relay traffic. As a result, CH around sink uses much more energy compared with other CHs, and this is one of the main problems that shorten the network life time in clustered WSN. Due to its complexity in CH election, each node should be aware of status of its neighbors and to our best knowledge, cluster topology has not been practically employed by any WSN application. 4.2 Heterogeneity When considering the heterogeneity, WSN can be classified into two categories: homogeneous WSN and heterogeneous WSN. Early sensor network visions anticipated that sensor networks would typically consist of homogeneous devices that were mostly identical from a hardware and software point of view. Some 105 Chapter 4. Planning the WSN projects, such as Amorphous Computing [114], even assumed that sensor nodes were indistinguishable, that is, they did not even possess unique addresses or IDs within their hardware. This view was based on the observation that otherwise it would not be feasible to cheaply produce vast quantities of sensor nodes. However, in many prototypical systems available today, sensor networks consist of a variety of different devices. Nodes may differ in the type and number of attached sensors [32, 44]; some computationally more powerful nodes may collect, process, and route sensory data from many more limited sensing nodes [45]; some sensor nodes may be equipped with special hardware such as a GPS receiver to act as beacons for other nodes to infer their locations such as the projects in [115, 116], which use GPS and satellite to locate the animals; some nodes may act as gateways to long-range data communication networks (e.g. 3G gateways are used in [117], satellite networks, or the Internet [118]). The degree of heterogeneity in a sensor network is an important factor since it affects the complexity of the software executed on the sensor nodes and also the management of the whole system. 4.3 4.3.1 Introduction and modelling of important metrics Preliminaries and assumptions We summarize the important symbols that are used in this section in Table 4.1. Our Table 4.1: Important symbols Symbol Si Ni As Φ Si C cost L Pl Pd DC,cost,L,Pl ,Pd Meaning sensor node with ID = i node device with ID = i and S ∈ N sensing/monitoring area a set of points that are the covered points of Si coverage cost lifetime packet latency packet drop rate desirability formulation of C, cost, L, Pl , Pd 106 4.3. Introduction and modelling of important metrics sphere ( S i , rS I ) m ISi rS i Si O(mSi ) Figure 4.2: The searching of covered point method employs a ray-tracing method described in our previous work [109] as well as in Chapter 3 to emulate the propagation paths of both radio and sensing signals. In the real world propagation, multi-path phenomena occurs on the radio signal transmission: when a radio signal encounters obstacles, reflections and diffractions happen. Thus received signal strength at the receiver is computed by accumulating the arrived waves from all directions. Unlike the radio propagation, the sensing signal is usually only considered by using direct path, which halts at the intersected points with the surfaces of obstacles. Thus a covered point of a sensor node Si is defined as following: Definition 1 Covered point: A point m is said to be covered by Si , if and only if it is within the sensing range of Si and is not obstructed by any obstacle. ΦSi represents a set of all the points that are the covered points of Si . −−→ ΦSi = {m|m ∈ sphere(Si , rSi )∧!O(mSi )} As indicated in Fig. 4.2, sphere(Si , rSi ) is the sphere with radius rSi centered at −−→ node Si and indicates the ideal sensing area of Si . O(mSi ) indicates whether the sensing path from m to Si is obstructed. 107 Chapter 4. Planning the WSN Routing scheme: There have been many algorithms proposed for routing data in sensor networks, which consider the characteristics of sensor nodes together with the application requirements. Nowadays, an overwhelming number of commercial sensor devices support the distance vector based routing protocol such as the Ad hoc On-Demand Distance Vector (AODV) routing [119] and link-state based routing protocol Dynamic Source Routing (DSR) [120]. However, the routing results of minimum-weight/minimum-hop based routing protocols are very similar and can be computed based on the shortest-path searching by Dijkstra’s algorithm, in which data are collected and forwarded to BS via the path with the best distance metric. In this work, the distance metric is modelled by the qualities of established links according to the ray-tracing results, which not only considers the distance between transmitter and receiver but also indicates the impact of surrounding environment. According to the comparison and traversing procedure over the WSN, the pseudo code of Dijkstra algorithm is shown in Listing. 4.1. Listing 4.1: Pseudo code of Dijkstra algorithm S={all nodes except the base station V}; For each node u{/*search the path from u to V*/ int temp = maxint; int u = v; For each node j in S { /*search a un-traversed node with shortest path to V*/ if ((!S[j])&&(dist[j]<temp)){ u = j; temp = dist[j]; } } S[u] = true; /*Remove u from the list*/ For each node j { if ((!S[j])&& j is directely connected wih u){ int newdist = dist[u][v] + max(c[u][j], c[j][u]); /*if the path through u is smaller than the older distance from j to v the shottest route is updated for node j*/ if (newdist<dist[j][v]) { dist[j][v] = newdist; 108 4.3. Introduction and modelling of important metrics next[j] = u; /*the next hop of j is u */ } } } } 4.3.2 The cost of WSN One of the design goal, from economic point of view, is to reduce the cost while fulfilling requirements of an application. Many companies and research organizations arise in the recent decade to design and manufacture sensor nodes, which provide various options on the budget. Besides, the installation of sensor nodes requires extra human efforts. For instance, the cost of placing sensors on the ceiling and walls is different from placing them within a human-active space; even attaching sensor nodes to different heights can vary the costs. As a result, the Cost of WSN is categorized into hardware cost and deployment cost in this work. 4.3.2.1 Hardware cost MICA and MICA2 were once the most successful families of Berkeley motes. The MICA2 platform, whose layout is shown in Fig. 4.3, is equipped with an Atmel ATmega128L and has a CC1000 transceiver. Berkeley motes up to the MICA2 generation cannot interface with other wireless-enabled devices [121]. However, the newer generations MICAz and Telos support IEEE 802.15.4, which is part of the 802.15 Wireless Personal Area Network (WPAN) standard being developed by IEEE. At this point, these devices represent a very good solution for generic sensing nodes, even though their unit cost is still relatively high (about $100 ∼ $200). Various platforms have been developed for the use of Berkeley motes in mobile sensor networks to enable investigations into controlled mobility, which facilitates deployment and network repair and provides possibilities for the implementation of energy-harvesting. UCLAs RoboMote [122], Notre Dames MicaBot [123] and UC Berkeleys CotsBots [124] are examples of efforts in this direction. UCLAs Medusa MK-2 sensor nodes [125], developed for the Smart Kindergarten project, expand Berkeley motes with a second microcontroller. An on-board power 109 Chapter 4. Planning the WSN management and tracking unit monitors power consumption within the different subsystems and selectively powers down unused parts of the node. Intel has designed its own iMote [126] (see Fig. 4.3) to implement various improvements over available mote designs, such as increased CPU processing power, increased main memory size for on-board computing and improved radio reliability. In iMote, a powerful ARM7TDMI core is complemented by a large main memory and non-volatile storage area; on the radio side, Bluetooth has been chosen, thus the price of an iMote is about $299 which is very high. MICA2 TelosB iMote2 Figure 4.3: Products by Crossbow. The BTnode rev3 hardware designed in 2007 by ETHZ [121] (see Fig. 4.4), is based on an Atmel ATmega128L microcontroller, a Bluetooth module and a low-power radio which is the same as that used on Berkeley MICA2 Motes. BTnode costs around $200 which is slightly higher than MICA2 due to an extra cost on Bluetooth module. Figure 4.4: BTnode rev3. 110 4.3. Introduction and modelling of important metrics Waspmote [127] in Fig. 4.5 is a sensor device developers oriented product. It works with different communication protocols (ZigBee, Bluetooth and GPRS) and frequencies (2.4GHz, 868MHz, 900MHz) and creates links with distance up to 12 km. Waspmote is compatible with more than 50 sensors. The flexible property makes Waspmote suitable for different types of applications and the price varies according to the configuration. For example, sensors with ZigBee module cost the least compared with that attached with Wi-Fi and Bluetooth module, while the one with 3G+GPS costs around $300 and is the most expensive configuration. WiSMote [128] in Fig. 4.5 appears since 2011. It is a sensor/actuator module well adapted to WSN applications. The wireless link operates over the 2.4 GHz ISM band. With its wide range of embedded sensors and its variety of extension connectors, it is able to monitor any kind of physical measurements in fields like environment, healthcare, domotics, smart building, logistics or industrial applications. WiSMote embeds an small footprint operating system (Contiki) plus an IEEE 802.15.4 protocol stack compatbile with Zigbee and 6LoWPAN (IPv6). The availability of inexpensive hardware such as CMOS cameras and microphones has fostered the development of Wireless Multimedia Sensor Networks (WMSNs), i.e., networks of wirelessly interconnected devices that are able to ubiquitously retrieve multimedia content such as video and audio streams, still images, and scalar sensor data from the environment. One of the recent multimedia wireless sensor is the SEED-EYE [129] in Fig. 4.5, which provides an advanced board for implementing low-cost (about $150 per unit)WMSN. It hosts a powerful Microchip PIC32, and has a full set of communication interfaces such as Ethernet, IEEE802.15.4 / Zigbee, and USB. Moreover, it is integrated with a CMOS Camera, making it an ideal board for implementing next generation imaging WSNs. Beyond the on-shelf sensor products, there are enormous research organizations dedicating to develop and prototype their own sensor platforms. Typically, those prototypes are based on add-on/modular hardware design. Although the designs are not highly optimized in terms power consumption or size and price, one can still foresee the bright future once they are launched in the market. The Tyndall’s mote family [130] developing a range of ISM band wireless sensing systems for deployment in the environment focusing on the key areas of the environment and fitness and health, the structure is shown in Fig. 4.6; Cookies[112] developed by the 111 Chapter 4. Planning the WSN Waspmote Wismote SEED-EYE Figure 4.5: Waspmote(left),Wismote(middle), SEED-EYE(right). researchers at Centro de Electronica Industrial of Universidad Politecnica de Madrid (CEI-UPM), have a modular architecture of four layers, as depicted in Fig. 3.12. Each layer fulfills a specific functionality in the node, and the layers are changeable for different applications. Moreover, it is possible to have a heterogeneous network with nodes composed of different layers. In the current motes, 2.4 GHz ZigBee communication protocol is used and AODV protocol is embedded. Tyndall Mote Figure 4.6: Tyndall mote. During the investigation of different sensor node manufacturers, we find out that some of them also produce other types of node to cater the heterogeneity and various applications of WSN. WiSGate of Arago Systems and Meshlium produced by Libelium act as a gateway between a wireless sensors/actuators network and other type of network. In [131], the authors analyzed hybrid sensor networks consisting of transceiving (cluster-heads) and transmit-only sensors. By using the developed 112 4.3. Introduction and modelling of important metrics mathematical model of physical and MAC layer, they demonstrated how much the dollar-cost and the power consumption of a sensor network can be decreased while maintaining the same network coverage. Table 4.2 summarize the hardware features of above mentioned sensor productions and compare their prices. Due to the heterogeneity property and various topologies of WSNs, motes with different functions should be considered to obtain a optimum cost solution rather than simply using a uniform type of all-function sensor mote for the whole network. Based on the above survey, motes are classified into three types in this work: • Sensor Node (SN): equipped with sensors to monitor the surrounding environment. In this work, each sensor node is static and has wireless communication and routing ability. • Relay Node (RN): has the ability of communication and routing. RN is usually needed to fill radio communication hole or to balance traffic load. • Base Station (BS): is in charge of aggregating data and is directly connected with the central server. A WSN has one BS and its location is predetermined by users. Since the number and location of BS is assumed to be fixed in this work, the cost of BS is also fixed and is not included in the model of hardware cost. SN contains extra sensor module besides the communication and routing module, therefore a SN costs more than a RN. The hardware cost model costhw is expressed by accumulating the price of each node deployed in the area A: costhw = M (P (Ni .type)) (4.1) i=1 where P (Ni .type) indicates the relative price that is dependent on the type of node Ni , for instance P (RN ) = 1, P (SN ) = 3. The available budget is considered as the maximum cost costmaxhw . The normalization of costhw provides a desirability component of the design goal to minimize the hardware cost, and Dcost is used in this work to represent the 113 Chapter 4. Planning the WSN BTnode rev3 MICA2 Telos Tmote Sky Sun Spot Imote2 Platform 2009 2011 2012 2007 2002 2004 2005 2006 2007 Year ATmega1281 MSP430 5x PIC32MX795F512L ATMega128L ATMega128L MSP430F149 TI MSP430 AT91RM920T Intel PXA271 CPU CC1000 CC2420 CC2420 CC2420 CC2420 Bluetooth CC1000 8 radio modules CC2520 MRF24J40MB Communication 128 KB Flash External Memory 512 kB Flash 512 kB Flash 1 M Flash 4 M Flash 32 M Flash 2×AA 2×AA N/A 2×AA Power Supply 2×AA 2×AA 2×AA Lithium ion 2×AA $130˜$300 N/A $200 $215 $150 $110 $99 $750 $299 Price 128 KB Flash 2 M Flash N/A Table 4.2: Features of various platform. Waspmote WiSMote SEED-EYE 114 4.3. Introduction and modelling of important metrics desirability on hardware cost. Dcost = 4.3.2.2 costmaxhw − costhw costmaxhw (4.2) Deployment cost During the deployment of nodes and equipments, human efforts and extra tools are needed in order to mount the nodes in the decided locations. The investigation on the lessons learnt by other researchers indicates that mounting problem occurs in all the works. The authors of [132] deployed WSN for precision agriculture, and they observed that multi-path fading which was exacerbated by the movement of leaves of the maize plants played a very crucial role on RSSI. From 2004-2005, Langendoen et al. [133] tried to deploy more than 100 sensor nodes to monitor the potato crops for precision agriculture. To avoid obstruction of when the potato crop is flowering and leaves cover the (ground-based) antennas, the nodes were installed on poles at a height of 75 cm. Besides, they included a safety margin to ensure that the nodes could not be hit by farming equipment attached to a tractor. The authors also learned very interesting lessons in [134], they developed a WSN to monitor the indoor environment. However, they found in two occasions that the sensor nodes are taken from where they were. As a result, they decided to place the sensor nodes not for the best coverage but for the best security. Hence those sensor nodes are eventually either hidden from field of vision or placed high up on the wall. There might exist non-deployable area where nodes are not allowed to be placed, those area should be pre-defined and the corresponding deployment cost can be assigned with the maximum value. In this work, deployment cost costd is modeled as boolean value and the forbidden area is assigned by users through GUI interface as shown in Fig. 4.7, costd = 1 if the area is not accessible, otherwise costd = 0. 4.3.3 Coverage Sensing coverage is one of the key issues that should be considered when deploying a WSN, as it corresponds to the quality of service that can be provided by a WSN. The coverage concept can be defined and categorized based on the node density level. If only parts of the area are covered by the sensor nodes, the coverage is sparse; if the area is completely (or almost completely) covered by sensors, the coverage 115 Chapter 4. Planning the WSN cost=0 h=1 m Width=3 m cost=0 Forbidden Area h=2 m cost=1 cost=1 Figure 4.7: Deployment cost configuration in vertical view. is dense; otherwise, if the same detected location is covered by multiple sensors, the coverage is called redundant. The density of coverage is normally determined by user requirements which may vary across different applications. An adequate coverage is a key to robust WSN application, and it may also be exploited to extend the network lifetime by switching redundant nodes to sleep modes to reduce power consumption. In this work, K-coverage problem is investigated and is concerned as the potential problem to be tackled by planning algorithm. Definition 2 Target K-coverage Requirement: At any given moment, any target point m ∈ As is the covered point of at least k different SNs (k = 1 · · · M ). Therefore the desirability of K-coverage requirement C is expressed as: DC = 4.3.4 | M i=1 {m|m∈As ∧m∈ΦSi }| |{m|m∈As }|×k (4.3) Connectivity Wireless sensor networks are represented by a graph G = (V, E) where V is the set of nodes and E ⊆ V 2 is the set of edges: (Nu , Nv ) ∈ E means that Nu and Nv are neighbors. The neighborhood set N(u) of Nu is expressed by N(u) = {Nv |(Nu , Nv ) ∈ E ∨ (Nv , Nu ) ∈ E} 116 4.3. Introduction and modelling of important metrics wireless links are determined according to received signal strength calculated by accurate ray-tracing method, thus edges are defined as : E = {(Nu , Nv ) ∈ V 2 |u = v ∧ RSS(uv) ≥ RXs } Where RSS(uv) is the received signal strength from Nu to Nv , and RXs is the sensitivity of antenna at the receiver. As can be seen from the edge definition, communication links can be established if the RSS is above the sensitivity of antenna. A WSN is said to be connected, if any two nodes belong to a WSN are linked together by edge(s) via single hop or multiple hops. Connectivity is intermittent if the network is occasionally partitioned. If nodes are isolated most of the time and enter the communication range of other nodes occasionally, the communication is said sporadic ([135]). Note that despite the existence of partitions, messages may be transported across partitions by mobile nodes, which is not the case in this work where only static network is considered. Connectivity mainly influences the design of communication protocols and methods of data gathering. Generally speaking, the this concept can be categorized into two directions: Definition 3 Connected K-Coverage Problem: Given a sensor network consisting of n sensors and an interest region, the network should satisfy the following two conditions at any moment: 1. Satisfy the conditions of the K-Coverage requirement 2. The communication graph G is connected Definition 4 K-connected Problem: A graph G is said to be k-connected if for each pair of vertices there exist at least k mutually independent paths of edges connecting them. In other words, the graph G is still connected even after removal of any k − 1 vertices from G. In [136], three localized K-CDS construction protocols were proposed. The first one is a probabilistic approach which is based on K-Gossip. The second is a deterministic approach which is an extension from the K-coverage condition. The last one is Color-Based K-CDS Construction. The authors in [137] proposed two 117 Chapter 4. Planning the WSN algorithms: one centralized algorithm CGA and one distributed algorithm DDA, to tackle the limits of k = m in [136]. Although the protocols and theorems have been focused for redundant connection for a robust WSN, the practical heuristic to construct a k-connected WSN is rarely studied. The work in [138] proposes deployment patterns to achieve full coverage and three-connectivity, and full coverage and five-connectivity under different ratios of sensor communication range (denoted by Rc ) over sensing range (denoted by Rs ) for WSNs. The authors also discover that there exists a hexagon-based universally elemental pattern which can generate all known optimal patterns. However, as stated in the paper, when considering the non-disc sensing model or geographical constraints or heterogeneity of sensor nodes, the proposed deployment pattern is not optimal. In this work, connectivity of WSN is constructed to tackle the Connected K-Coverage Problem with K-connected network topology by using the previous mentioned sensing model, practical multi-path radio propagation model on heterogeneous WSNs. 4.3.5 4.3.5.1 Lifetime, Packet latency and Packet drop rate Lifetime As most of the sensor nodes are powered by batteries, they will exhaust energy after a certain time once deployed in the environment. Therefore the WSN will be disconnected and no longer satisfy the sensing requirement. The lifetime of sensor network is a very important metric and WSN designers have done many efforts to prolong it. The methods include developing proper MAC periods, optimizing the topology to reduce bottleneck nodes, developing back up plans and routing algorithms. The network lifetime in our work is defined as the time that the first node exhausts its energy. The desirability of this metric is expressed by a ratio between the actual lifetime (L) and the maximum expected lifetime (Lmax ) by the WSN designer. The expression of DL is: DL = L Lmax 118 (4.4) 4.3. Introduction and modelling of important metrics 4.3.5.2 Packet latency Packet latency is defined as a average end-to-end delay from the source to the destination (BS). There are many factors that affect packet latency and the most important factors are: the usage of channel, the hops between source and destination and the scheduling of nodes along the routing path. The desirability of packet latency is expressed as: D Pl = 1 − Mp i Pl (i) Mp (4.5) where Mp is the total number of data packets generated by all the sensors and Pl (i) is the latency of packet i. 4.3.5.3 Packet drop rate Packet drop can be caused by signal degradation over the network medium due to multi-path fading, channel congestion, corrupted packets rejected in-transit, faulty networking hardware, faulty network drivers or invalid routes. The packet drop rate is a ratio between the number of dropped packet (Pd ) and the number of generated data packets, and the desirability over this metric is: D Pd = 1 − 4.3.5.4 Pd Mp (4.6) Proposed strategy by using WSNet simulator to model L, Pl and Pd Because L, Pl and Pd can be affected by network topology, real-time communication and packet load, it is difficult to precisely model them through simple formulas. That is the reason why protocol designers usually estimate such performance through network simulator, where WSN can be simulated with a determined or random topology with nodes being scheduled and data load being assigned based on user specifications. Inspired by those works, we integrate WSNet1 simulator in this work to observe the three complex metrics through practical network simulation. WSNet simulator allows researchers to analyze WSN performance based on network configuration 1 http://wsnet.gforge.inria.fr/ 119 Chapter 4. Planning the WSN which can be read from the files that indicate topology, routing and connectivity information. Users can either use the embedded provided layer modules or extend the operation by developing customized modules that will be used in the real application. The network protocol can be the IEEE 802.15.4 standard for physical layer and MAC layer, or other desired protocols. The energy model should be programmed previously to imply how energy is consumed for transmitting/receiving data packets and for different status such as wake, sleep and idle. The key functions of energy model are in Listing 4.2. Listing 4.2: Key functions of energy models in WSNet simulator /* Battery model*/ /* The editable energy comsumption function for *transmitting and receiving packets, as well as the idle state */ void consume_tx(id,duration); void consume_rx(id,duration); void consume_idle(id,duration); /*802.15.4 MAC layer key functions*/ int check_channel_busy(id); /*check the status of channel for a node*/ int state_machine(id); /* the state assignment for a node*/ As a result the strategy of the modelling method can be described by the flowchart in Fig. 4.8. When a candidate topology is generated, the node location, routing and connectivity files are created in a shared folder between the planning algorithm engine and WSNet simulator. A ’xml’ script is created to configure the property of WSN for the simulation in WSNet, including the network size, region scale, network protocols, energy consumption models and directory of the generated output files. After WSNet finishing the simulation, it returns the values of network lifetime, packet latency and packet drop rate to the planning algorithm engine, for analyzing desirability values by the aforementioned formulas (4.4, 4.5 and 4.6). 120 4.4. The proposed multi-objective optimization methodology User Requirements Xml configuration file Lifetime Packet latency Log files: connection, routing table, location Packet drop rate Figure 4.8: Modelling of L, Pl and Pd by using WSNet simulator. 4.4 The proposed multi-objective optimization methodology Once provided application requirements and the deployment environment model, the problem of planning a WSN is formulated as: Determine the topology of the network to maximize the 5 desirability values calculated by (4.2 ∼ 4.6). This is a multi-objective optimization problem, which is proven to be NP-hard. Multi-objective optimization genetic algorithms are proposed to effectively and efficiently solve the NP-hard problems, among which the Nondominated Sorting Genetic Algorithm II (NSGA-II) by [139] is an ideal approach that features with elitism selection, high computation efficiency O(M N 2 ) (where M is the number of objectives and N is the population size) and does not need to specify the sharing parameter. As a result, the multi-objective optimization method in the present work is based on NSGA-II. It concerns the constraints and rules on formulating the genes as well as the mutations and crossovers, to cater to the features of WSN, and therefore the results are achieved efficiently and effectively. Fig. 4.9 shows an overview of the proposed planning method. Network deployment is generated based on the deployment constraints and user requirements on the detected regions and forbidden regions, the value of cost is obtained at this step. There are three ways to create the deployment: 1. User can determine the locations and properties of nodes manually via the GUI interface; 121 Chapter 4. Planning the WSN User Requirements Detected region Deployment constraints Detection load Cost Network deployment generation Locations Coverage Radio and sensing coverage analysis Topology generation Multi-Objective Evaluation Function L Pl Pd Xml file Log files Feedback Loop Figure 4.9: The strategy of the proposed planning method. 2. Nodes can be randomly generated based on the constraints; 3. Crossover and mutation modify the node properties during the evolutionary strategy. Thereafter, radio propagation and sensing signals are computed for each node by using the ray-tracing method so that the connectivity and sensing coverage are obtained, and the topology of the WSN is constructed according to the routing protocol pre-defined by WSN designer. As discussed in section 4.3, the ’xml’ file and log files are generated to trigger WSNet simulator so that lifetime, packet latency and packet drop rate are analysed after WSNet finishing simulation. With all those metrics provided, the multi-objective evaluation method computes the objectives (desirability) and selects those candidates with best performance base on NSGA-II, and the selected population are fed back to the network deployment generation function to create new populations, thus the objectives are gradually progressed. At the end, the algorithm can provide multiple elitist solutions to WSN designers. In genetic algorithm, a candidate solution can also be called individual, creature, or phenotype. Each candidate solution has a set of properties (its chromosomes 122 4.4. The proposed multi-objective optimization methodology or genotypes) which can be mutated and altered. Traditionally, individuals are represented in a vector of binary value, but other representations are also possible. The evolution usually starts from a population of randomly generated individuals and in each generation, the fitness value of every individual in the population is evaluated. The individuals with better performance are selected from the current population, and each individual’s genome is modified (recombined and possibly randomly mutated) to form a new generation to be evaluated in the next iteration. Commonly, the algorithm terminates when either a maximum number of generations has been produced, or a satisfactory fitness level has been reached. In this work, an individual is expressed as DV represented by (4.7), where Ni is the node with ID = i and it is a ”chromosome” of DV . M is the number of nodes, which also indicates the size of WSN. Each ”chromosome” has properties including 3D location of the node, type of node, transmission power Ptx , radio sensitivity RXs and sensing range Rsense . We assume that M , Ni .location, Ni .type can be modified, as a result this method is based on changeable length which will bring difficulties to crossover and mutation. Besides the locations and type must be modified according to certain rules to construct a valid WSN. The details of the strategy are introduced and problems are tackled later in this section. ⎧ ⎪ ⎪ location : x, y, z ⎪ ⎪ ⎪ ⎨ type : BS, SN, RN DV = [N1 , N2 , · · · NM ] , Ni = ⎪ Ptx , RXs ⎪ ⎪ ⎪ ⎪ ⎩ Rsense ⎫ ⎪ ⎪ ⎪ ⎪ ⎪ ⎬ ⎪ ⎪ ⎪ ⎪ ⎪ ⎭ (4.7) The algorithm defines two types of population: P arent and Children, for storing the parents and children respectively. The format of population is defined by Np individuals attached with their corresponding desirability values: ⎡ 1 Dcost,c,L,P l ,Pd 2 Dcost,c,L,Pl ,Pd DV1 , ⎢ ⎢ DV , 2 ⎢ population = ⎢ ⎢ ··· ⎣ Np DVNp , Dcost,c,L,P l ,Pd 123 ⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦ (4.8) Chapter 4. Planning the WSN 4.4.1 Initialization of individuals Initial population of candidates are traditionally generated in a random way, and some of them may occasionally satisfy the constraints and requirements, not to mention optimizing the performance at the same time. When the scale of region or the size of WSN becomes large, there will be less chance that an initial candidate has a valid WSN topology. To efficiently tackle this issue, a high ”quality” initial seed is generated to guarantee the basic requirements on connectivity and coverage. The LowCost heuristic proposed in [42] is employed to add a valid individual at the initial phase: At the beginning, the Coverage is computed for each deployable point m ∈ At and the heuristic selects mi that with the maximum Coverage as the best location and a sensor node Su is then placed on mi . The coverage level of the monitoring points newly covered by Su is updated, and those points with a sufficient coverage level are removed from the set of sensing area As . This procedure is repeated until all the monitoring points are k-covered. Afterwards, LowCost focuses on the connectivity problem. Let Nu be the node of unconnected nodes U. The algorithm selects a node Nc in the connected sensor nodes C that is the closest to Nu and computes the new virtualposition m of Nu by moving it towards Nc as long as the set of monitoring points initially covered by Nu remains unchanged. If Nu is still unconnected after changing its position, extra relay nodes are put on the line between Nu and Nc so that Nu and Nc are connected. The resulted initial seed is expected to be better than a randomly generated seed which do not guarantee the coverage and connectivity. However, as discussed in Chapter 1, the result does not solve the optimization between connectivity and cost. Besides the individual generated by LowCost heuristic, the rest individuals are generated based on the constraints on location with various length (size of WSN), thus the initial population of parents (P arent) are obtained and evaluated. 124 4.4. The proposed multi-objective optimization methodology I offset Lcrop SN RN A I 'offset Lcrop I 'offset Lcrop B I offset Lcrop A’ B’ Figure 4.10: Crossover with different lengths. M'tM M' M A A A’ A’ A’’ A’’ SN RN Figure 4.11: Mutation with changeable length. 4.4.2 Crossover and mutation At each generation, the parents are recombined (crossover) and mutated with different probability. The demonstrations of crossover and mutation with variable individual lengths are shown in Fig. 4.10 and Fig. 4.11 respectively. 125 Chapter 4. Planning the WSN 4.4.2.1 Crossover Two candidates (P arent(i) and P arent(j)) are randomly selected from the population of parents and crossover occurs between them with a chance of Pco . Note that the length of both parents might be different, and the crop length Lcrop is limited by the shorter length: Lcrop < min(Mi , Mj ). And the offsets can be different in both parents with constraints: Iof f + Lcrop ≤ M . Afterwards, the two generated children (A and B ) are stored into the population of Children. If crossover does not happen on the current couple, the two parents are stored into Children directly for further modification. Similar procedure repeats on other possible pairs of P arent until Children is filled by individuals. 4.4.2.2 Mutation Mutation occurs on the individuals of Children. The size of Children(i) is Mi which mutates with a probability of Pmu . Note that the change on Mi will to some extent increase the risk of constructing an un-connected topology. The new length Mi is obtained by randomly selecting a value within the range limited by [Tmin , Tmax ], where Tmin = max(Mi − T, 1), and Tmax = min(Mi + T, Mmax ). T is a small integer value and is equal to 2 in our work. Mmax is the maximum number of nodes that allowed to be used and is determined by user. If Mi is less than Mi , Mi nodes are randomly picked from Children(i); otherwise, if Mi > Mi , Mi − Mi random nodes are added to Children(i). Thereafter mutation happens on each node of Children(i) with probability of Pmu , on the 3D location and the type of node. The movement of Nj ∈ Children(i) is limited within the sphere of radius dmax centered at Nj .location and the type can be selected randomly between SN and RN. 4.4.3 Evaluation based on desirability models and constraints Routes are searched from each node to the BS by using the Dijkstra’s method. Several further steps are prosecuted to make the algorithm converge faster: A SN is changed to RN when it does not cover any point m ∈ As ; A RN is deleted if it does not act as a router for other nodes; If two nodes of the same type are located too 126 4.5. Experimental results and analysis close to each other, one of them are moving apart in a similar way as the mutation on location. The desirability values of all the 5 metrics are computed and then attached to each corresponding individual. P arent and Children are mixed so that all the individuals from both populations are ranked based on nondominated sorting by NSGA-II. As a WSN must focus on fulfilling the sensing tasks, the desirability of coverage Dc is considered as the only constraint among the five objectives. By doing so, if Dci > Dcj , the rank of individual i is always higher than individual j no matter how the other metrics are; otherwise, the ranking is based on all the desirability values equivalently, the greater a desirability is, the better the corresponding metric will be. At the end of each generation, Np best individuals are selected and formulate new P arent for the next generation. The evolutionary procedure repeats until the maximum generation is reached. The proposed method is able to provide multiple WSN deployment solutions with optimized performance from different aspects. As a result, it gives designers flexibility to observe different optimized deployments and assist them making deployment decision accordingly. 4.5 Experimental results and analysis The performance of planning algorithm is evaluated through observing the fitness value and time efficiency compared with other heuristics, as well as checking the feasibility in real applications. The performance comparisons are realized with three comparable state-of-the-art algorithms which have 3D computation ability. All the algorithms including the proposed one are programmed in C++ and they are run on a PC with Intel Core i5-760 2.8 GHz CPU so that the results are fairly compared. 4.5.1 The impact of maximum number of generation As the proposed algorithm is based on evolutionary strategy, the larger the number of generations, the more outstanding the population will evolve. Therefore, we first evaluate how the maximum number of generations N U MmaxGen impacts on WSN performance. The application requirements are shown in Fig. 4.12: The 3D map is the floor plan of CEI-UPM with a scale of 57 m × 16 m × 3 m and a resolution of 127 Chapter 4. Planning the WSN 1 m. The red point indicates the location of BS (17.52 m, 10.02 m, 1.5 m) and blue rectangles are the sensing areas As which contain 70 points to be covered. Target region BS Target region Figure 4.12: Configuration of Scenario CEI-UPM. In this study, the size of population is Np = 8 for both P arent and Children. Crossover and mutation possibilities are Pco = 0.1 and Pmu = 0.2 respectively. The data period for each sensor node is 1 s and the simulation lasts 2400 s in WSNet simulator. The maximum value of generation N U MmaxGen increases from 10 to 150 with a step of 10, hence there are 15 different N U MmaxGen s. The algorithm runs 5 times for each N U MmaxGen , and as a result 8 × 5 × 15 optimized solutions are obtained after the simulation. The results are grouped for each N U MmaxGen and the pareto front is shown in Fig. 4.13, the mean value of each group data are calculated for each desirability metric, the five metrics construct a plot with five axes. The area constructed from each group indicates that as N U MmaxGen increases and the area grows larger, and the overall performance becomes more stable when N U MmaxGen ≥ 100. Fig. 4.14 indicates that although time consumption fluctuates at the same N U MmaxGen , consumed time increases approximately linearly with N U MmaxGen . Therefore, a proper trade-off decision should be made between the performance and efficiency of computation, and N U MmaxGen = 100 can be selected for this configuration. 128 4.5. Experimental results and analysis Cost Lifetime Coverage maxGen maxGen maxGen Packet latency Packet drop rate Figure 4.13: Desirability values vary with N U MmaxGen . 350 Time Consumption (s) 300 250 200 150 100 50 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 Max number of generation Figure 4.14: Time consumption varies with N U MmaxGen . 129 Chapter 4. Planning the WSN Table 4.3: Features of algorithms for comparison Algorithm WSN type Solutions MOGA SN, RN, BS SN, RN, BS SN, BS Single LowCost SN, BS Single proposed WMOGA 4.5.2 Multiple Single Radio Raytracing(RT) Raytracing(RT) distance Line-ofSight(LoS) Objectives C Cost L Pl Pd Y(RT) Y Y Y Y Y(RT) Y Y Y Y Y(dist.) N Y N N Y(LoS) Y N N N Performance comparison with other heuristics Two comparable heuristics (LowCost [42] and MOGA [43]) are selected and programmed in the same platform as the proposed algorithm, so that their performance are fairly evaluated in exactly the same configuration. Besides, a weighted multi-objective fitness function (WMOGA) developed in our previous work by [140] is modified and implemented to evaluate the impacts of NSGA-II on the final solutions, the formula of the weighted function is: f = w1 DC + w2 Dcost + w3 DL + w4 DPl + w5 DPd (4.9) Table 4.3 compares the features of the algorithms, and the proposed method considers more objectives and more practical modelling for heterogeneous WSN. We do not analyze the impact of modelling the radio and sensing signal in this thesis, however it has proved in the previous work, the practical ray-tracing algorithm outperforms other distance based empirical models. By setting reflection and diffraction depth as depth = 0 for the ray-tracing engine, only direct paths are traced in this evaluation. Therefore the algorithms are compare in time efficiency and optimization performance. The population of Children is 8 for the first 3 algorithms, and the parent is 8 for the proposed method, 1 for the other two. N U MmaxGen = 150 for the first 3 algorithms. Three scenarios are tested for all the algorithms and the configurations are: the first scenario is in the CEI-UPM floor plan, As = 70 and BS.location = (17.52 m, 10.02 m, 1.5 m) (Fig. 4.12); the second scenario is based on the floor plan of East Lansing which has a scale of 77.4 m × 36.6 m × 3 m and the resolution is 130 4.5. Experimental results and analysis 1 m. As = 232 and BS.location = (23.43 m, 13.60 m, 1.5 m) (Fig. 4.15); outdoor region of Madrid city is the third scenario. Its scale is 233.36 m × 297 m × 73.67 m and resolution is 3 m. As = 310 and BS.location = (107.8 m, 60.1 m, 3 m) (Fig. 4.16). Target region BS Target region Target region Figure 4.15: Scenario East Lansing. The five desirability metrics used by the proposed method are also computed based on the topologies generated by the other three algorithms, the solutions are compared according to those values as indicated in Table 4.4, 4.5 and 4.6 respectively. Red color marks the best value and blue color marks the worst. The performance of MOGA is limited by using a constant WSN size and unilateral objectives. WMOGA considers all the objectives without the ability of providing multiple solutions simultaneously, which explains why this method obtains good optimization performance. As expected, most of the best results are obtained by the proposed algorithm, allowing designers making decisions from different aspects to construct a reliable topology. The computation time of all the algorithms increases as the scale of scenario grows. LowCost is the most efficient with deterministic heuristic while it obtains worst performance on some objectives at each scenario. Especially when there are significant obstacles between the unconnected and connected part of 131 Chapter 4. Planning the WSN Target region BS Target region Figure 4.16: Scenario Madrid. a WSN, more nodes are placed across the obstacles instead of placing around them to optimize the cost. The proposed method provides 8 different multi-objective optimized solutions simultaneously and leads to the highest time consumption at each scenario. However if divided by 8, the average time consumption per solution is at least 45% better than MOGA and at least 60% better than WMOGA. As a result, even by integrating external WSNet simulation in the loop of evolutionary strategy and with changeable size of WSN, the proposed method is still much more efficient than MOGA and WMOGA. 132 4.6. Conclusion Table 4.4: Results comparison for Scenario CEI-UPM. Algorithm Time (s) Proposed 255.09 WMOGA MOGA LowCost 456.64 67.66 1.3 Objectives DC DCost 1 0.84 1 0.835 1 0.83 1 0.84 1 0.845 1 0.84 1 0.85 1 0.85 1 0.835 1 0.82 1 0.82 DL 0.24 0.29 0.25 0.22 0.24 0.22 0.26 0.25 0.26 0.28 0.21 D Pl 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.98 0.99 D Pd 0.96 0.86 1 0.92 0.88 0.99 0.98 1 0.98 0.9 0.96 Table 4.5: Results comparison for Scenario East Lansing. 4.6 Algorithm Time (s) Proposed 799.32 WMOGA MOGA LowCost 581.03 177.14 3.64 Objectives DC DCost 1 0.805 1 0.81 1 0.83 1 0.81 1 0.815 1 0.815 1 0.82 1 0.815 1 0.815 1 0.82 1 0.82 DL 0.26 0.24 0.21 0.26 0.30 0.30 0.21 0.32 0.36 0.28 0.19 D Pl 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 D Pd 0.91 0.97 0.94 0.92 0.96 0.87 0.89 1 1 0.94 0.96 Conclusion A planning algorithm is proposed in this thesis, by taking the advantage of proposed ray-tracing scheme for both radio and sensing signal propagation, the modeling on coverage and connectivity turns out to be accurate. Moreover, we also consider the important impact of lifetime and link quality on the WSN, the optimization is more complete compared with other works. This algorithm is suitable for both outdoor and indoor environment with the ability to consider deployable area and 133 Chapter 4. Planning the WSN Table 4.6: Results comparison for Scenario Madrid. Algorithm Time (s) Proposed 4048.57 WMOGA MOGA LowCost 1236.32 1581.14 35.29 Objectives DC DCost 1 0.83 1 0.83 1 0.83 1 0.86 1 0.85 1 0.86 1 0.83 1 0.865 0.99 0.755 1 0.58 1 0.58 DL 0.19 0.18 0.20 0.22 0.21 0.23 0.20 0.20 0.18 0.17 0.05 D Pl 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 D Pd 0.95 0.97 0.95 0.91 1 0.99 0.94 1 1 0.99 0.83 forbidden area. The scalability and probabilistic model of sensing signal are also very challenging topic in this area, and should be tackled in the future. 134 Chapter 5 iMOST: an Intelligent Multi-objective Optimization Sensor network planning Tool iMOST is developed by integrating the introduced algorithms, to assist WSN designers efficiently planning reliable WSNs for different configurations. The abbreviated name iMOST stands for an Intelligent Multi-objective Optimization Sensor network planning Tool. As mentioned in Section 1.4 of Chapter 1, the structure of iMOST is illustrated in Fig. 1.9, which is composed of an user friendly interface and three core functional modules: Image Processing Module (IPM), Ray-tracing Propagation Module (RPM) and Node Placement Module (NPM). iMOST contributes on: (1) Efficient and automatic 3D database reconstruction and fast 3D objects design for both indoor and outdoor environments; (2) It provides multiple multi-objective optimized 3D deployment solutions and allows users to configure the network properties, hence it can adapt to various WSN applications; (3) Deployment solutions in the 3D space and the corresponding evaluated performance are visually presented to users; and (4) The Node Placement Module of iMOST is available online as well as the source code of the other two rebuilt heuristics. Therefore WSN designers will benefit from this tool on efficiently constructing environment database, practically and efficiently planning reliable WSNs for both outdoor and indoor applications. With the open source codes, they are also able to compare their algorithms with ours to contribute this academic field. The current version of iMOST works on Win7 operating system and its mainframe is programmed by using MFC of Visual Studio. The details of important features of iMOST are introduced in this chapter. Chapter 5. iMOST: an Intelligent Multi-objective Optimization Sensor network planning Tool The mainframe of the user interface is shown in Fig. 5.1. It is consist of Menu bar, Tool bar where functions can be realized based on user demands and Demonstration area where 3D view of scenario and WSN evaluation result can be demonstrated. The meaning of each icon and the corresponding function is indicated in the figure as well. The IPM is called by the NewMap menu in File option (ref. to sec. 5.1.1), map will be loaded to the Demonstration area by Open. The tool bar provides 7 options including: manually place nodes on the loaded map, view the performance of WSN, define sensing region, construct WSN topology based on the planning algorithm, reset the scenario and navigate the scenario and deployment in 3D. 5.1 5.1.1 Menu bar Image Processing Module By selecting NewMap menu under File, the IPM is triggered and automatic environment reconstruction algorithm is run for either indoor or outdoor scenario based on the procedure introduced in Chapter 2. As shown in Fig. 5.2, user should first define the directory of tested images, the folder of trained database and the folder of output results. The automatic 3D environment reconstruction method is launched after user finishing configuration, images are loaded and objects are recognized automatically. This procedure allows users freely preparing their own environment and reconstructing the environment via the trained database, so that the 3D database is constructed efficiently and accurately without spending many human efforts or high cost compared with conventional methods. 5.1.2 Environment property configuration Once the 3D environment database is constructed or there is any 3D map available in ’txt’, ’kml’ or ’3DS’ file with the polygonal description format, the map can be imported to the tool and demonstrated in the visual interface by clicking Open menu. During the importing of map, the Environment Property dialog is pop up, which is shown in Fig. 5.3. It provides opportunity for users setting the environment properties including the resolution and scale ratio compared with the real world size. 136 5.1. Menu bar View candidate result Manual node deployment mode Finish configuration Sensing region define Clean the deployment Planning algorithm Navigation of scenario Scenario and result demonstration IPM: Automatic environment reconstruction Import available 3D map: ‘txt’, ‘kml’, ‘3DS’ format Figure 5.1: The mainframe of the planning tool. Figure 5.2: User command on constructing new map. 137 Chapter 5. iMOST: an Intelligent Multi-objective Optimization Sensor network planning Tool Figure 5.3: Environment property setting dialog. The resolution also determines how many points the ray-tracing engine will calculate for, and the number of possible locations of nodes as well. Therefore, resolution will impact on the computation time of the algorithms, whereas the scale ratio only affects the real scale of the scenario and topology of nodes. 5.2 5.2.1 Toolbar Node deployment Beyond the automatic topology generation by the proposed planning algorithm, nodes can be placed by users for other purpose such as to evaluate the performance of a designed topology. The node deployment button allows users placing nodes on determined location, and a Node Property dialog ( see Fig. 5.4) will pop out each time the left mouse button clicks on the valid region, to allow configuring the property of the placed node on type: SN, RN and BS, Tx power in dBm, radio sensitivity Rxs in dBm and if it is a sensor node the sense range can be set in meters. As expected each node may have different communication ability and sensing ability defined by users. Different types of nodes are distinguished by different colors and point sizes in this work, for example red color point with the largest size represents a BS, the point with purple color and middle size RN and yellow color point with the smallest size 5.2.2 indicates a SN. Network Planning Module The NPM requires users make 3 steps to configure the network: 138 is a 5.2. Toolbar Figure 5.4: Node property configuration dialog. • Define sensing area As . By pressing , the function of definning sensing area is triggered. User click the left mouse button on the scenario to mark the border corners of sensing region as shown in Fig. 4.12. In the current version, each region is represented by four vertexes to construct a quadrangle. However, the shape of region is not limited in the real world, this function can be extended by allowing more vertexes for a region. • Node pre-deployment. Several nodes can be pre-deployed by users by pressing . This tool supports semi-auto and auto planning of WSN, where semi-auto means that user pre-define parts of WSN on specified locations with determined properties, whereas only the BS is determined manually in auto planning mode. • Terminate user configuration and launch the planning algorithm. Once the button is pressed, the configuration is done and user can launch the proposed planning algorithm by pressing . At the end, multiple multi-objective optimized solutions can be viewed visually by clicking on the icon and then selecting the individual ID from the dialog shown in Fig. 5.5. See Fig. 5.6 the results of individual ID = 4 as an example, with the topology and the value of each objective demonstrated. 139 Chapter 5. iMOST: an Intelligent Multi-objective Optimization Sensor network planning Tool Figure 5.5: Node configuration dialog. Coverage 1.00 Cost 0.855 Lifetime 0.22 Packet latency 0.98 Packet drop rate 1 Figure 5.6: Generated topology of individual with ID=4. 5.2.3 Ray-tracing Propagation Module The RPM can not be visually seen from the mainframe, however it is embedded in the function behind the icons of view results and the planning algorithm . Whenever the radio propagation or sensing coverage is needed, they are all computed via RPM so that the accuracy and reliability of the estimated result is guaranteed. 5.2.4 3D navigation The tool offers the capability to navigate the loaded 3D scenario by pressing . The scenario or deployment of WSN can be viewed with different angles of view and zooming scales. The zooming function is realized by rolling the scroll wheel of the mouse forward or backward. The view point is adjusted through pressing the right mouse button and moving the mouse at the same time. The angle of views 140 5.2. Toolbar Figure 5.7: 3D navigation for outdoor scenario. are changed by pressing the left button and moving the mouse simultaneously. Fig. 5.7 and Fig. 5.8 demonstrate the 3D navigation at outdoor and indoor environment respectively. The generated topology, the radio rays computed from RPM and the evaluated network performance can all be shown in 3D view. Moreover, additional objects in the scenario can be eliminated according to the user requirement. By doing so, they don’t have any impacts on radio propagation and sensing signal modelling. 141 Chapter 5. iMOST: an Intelligent Multi-objective Optimization Sensor network planning Tool Figure 5.8: 3D navigation for indoor scenario. 5.3 Conclusion iMOST is a tool developed to assist designers planning WSN topology with optimized performance. It features with high efficiency, flexibility and reliability with low cost by integrating the proposed IPM, RPM and NPM by this work. With the ability of automatic 3D outdoor and indoor environment reconstruction, it significantly reduces economic cost, human efforts and time that spent on this crucial but non-planning issue. This tool also explores the feasibility of a new direction of image processing application. User-friendly interface allows designers to configure the scenario and WSN properties conviniently. It visually demonstrates the deployment solutions which not only satisfiy the design requirements but also overall optimize the WSN performance by NPM on sensing coverage, connectivity, hardware cost, network longevity, packet latency and packet drop rate. The node placement module and the rebuilt heuristics are accessible through internet so as to benefit other WSN designers from different aspects. As part of the work, the WSN deployment solutions will be evaluated under real measurements. 142 Chapter 6 Real measurements and results analysis Two environment monitoring demonstrations were set up to validate the performance of the proposed planning algorithm and the developed planning tool. The overall methodology, from 3D environment reconstruction and user configuration to radio propagation, topology generation and performance estimation, is employed for both demonstrations. In both tests, ’Cookie’ is equipped with ZigBee communication protocol layer, environment sensors that are able to sense temperature, light and humidity, and external antennas made by EAD, see Fig. 6.1. The BKR2400 antenna is 1/2 wave dipole with 2 dBi peak gain, it has linear polarisation with omni-directional radiation pattern at horizontal plane. Lithium-based batteries have been used to supply the energy to the modular nodes during the WSN deployments and experimental tests, providing up to 500 mAh which covers the power consumption requirements of the devices. Moreover, they can be charged by using the power supply layer of the Cookie architecture, so that the autonomy of the nodes is enhanced. 6.1 Aggregation mechanism of measured data Besides gathering the ambient data, the most important parameters that should be observed are RSS value of data packets, neighborhood table of each node, routing table with the BS as destination, battery level and packet delivery states. The request of those observations is implemented by means of using the HW-SW co-design platform proposed by [141], which is a framework based on libraries and controllers that allows designers realizing applications by programming in C code Chapter 6. Real measurements and results analysis Figure 6.1: BKR2400 antenna. and compiling to generate the bitstreams for the microcontroller of sensor node. The mechanism of aggregating the aforementioned data is described as follows: 1. RSS. Whenever a node Ni receives packets from other nodes, it records the RSS in dBm, and noted as RSS(Nj , tk ) Where Nj is the source of a packet, tk ∈ [0, tp ] is the arrival time stamp of the packet, tp is the overall testing time of WSN. 2. Neighborhood table of each node. The neighborhood table Ti of Ni contains the IDs of neighbors and corresponding RSS records. The format of Ti is expressed as following and for simplicity in real application, 10 samples of RSSs are cached for each neighbor of Ni . Nj1 Ti = Nj2 .. . RSSI(Nj1 , t0 ), RSSI(Nj1 , t1 ), · · · , RSSI(Nj1 , tp ) RSSI(Nj2 , t0 ), RSSI(Nj2 , t1 ), · · · , RSSI(Nj2 , tp ) .. . 3. Routing table of the network. It records for the whole WSN the routes from each node to BS. Each node maintain the next hop information, so that at the base station, routing table R of the WSN is constructed: 144 6.2. Application interface Table 6.1: Routing table format. Source Destination Next hop Ni BS Nk Nj .. . BS Nv .. . BS 4. Power consumption (Battery level). This observation is to estimate the lifetime of the network. By observing battery level, the battery status of each node can be evaluated. Each node records its battery status BL at every period tB and reports to the BS every n · tB . By doing so, the battery level table B of the whole network is expressed as following: Nj1 B= Nj2 .. . BL(Nj1 , tB ), BL(Nj1 , 2tB ), · · · , BL(Nj1 , ntB ) BL(Nj2 , tB ), BL(Nj2 , 2tB ), · · · , BL(Nj2 , ntB ) .. . 5. Format of data packet. The BS gathers sensed data from sensors. Each SN periodically (Td ) sends data packet to BS, the data packet contains the following information: Table 6.2: Packet format. Source ID Arrival time stamp Sequence Sensed data TX time stamp Once the test is terminated, the packet loss rate can be analyzed based on the continuity of packet sequence for each node, and packet delay of the WSN is computed from the differences between TX time stamp and arrival time stamp. Moreover, environmental monitoring data are obtained as well to prove that those solutions can satisfy the application requirements. 6.2 Application interface There is usually a user application of observing data and maintain the functions of nodes in any type of WSN applications. We also developed a user interface for our 145 Chapter 6. Real measurements and results analysis demonstrations to allow simple operations from users. The application interface is programmed by using JAVA with all the aforementioned function to fetch all the required information. It works in cooperation with the BS and Fig. 6.2 shows the structure of the interface. Figure 6.2: Application interface. • Data packet is shown once received by BS. As can be seen, the source ID, TX time, temperature(T), humidity (H), light (L), battery level and sequence number are included. Arrival time stamp is added at the end. • Routing table can be generated according to user’s command, which is realized by clicking ’Generate Routing Table’ button. • User can select node ID and click the ’Neighbor Table’ to observe the neighborhood table of a node. 146 6.3. Indoor measurements • Time is synchronized by clicking ’Configure Time’. • All the information is saved to ’.txt’ file once ’Save Current Information’ button is pressed. After preparing all the aforementioned hardware devices, aggregation mechanism and user interface software are used to monitor the WSN. Two real deployments are launched for an indoor and an outdoor environment to monitor temperature, humidity and light level and validate the planned topology and performance estimated by iMOST. 6.3 Indoor measurements The first test is realized in the indoor environment of CEI-UPM. The 3D indoor environment database is constructed based on the Chapter. 2, where the reconstructed results are analyzed in details. The scanned map is shown in Fig. 6.3(a). The reconstructed result (Fig. 6.3(b)) has 88 planes accurately constructed and 2 edges misclassified. Beyond the automatically provided environmental database, we manually import other 3DS models (Fig. 6.3(c)) to show the presence of office desks and consider their impacts on the topology planning. Environment database is loaded to iMOST, resolution is equal to 1.0 m and scale ratio is 1.0. User requirements on sensing regions are demonstrated in Fig. 6.4, As = 90 and BS.location = (18.73 m, 12.00 m, 1.5 m). Nodes are set with TX power as −12 dBm, RX sensitivity as −98 dBm, Rsense = 3 m. A population of planed topologies are generated by iMOST based on the user configuration with optimized coverage, cost, lifetime, packet latency and packet drop rate. In this work, we select two candidates for real deployments, which have topologies indicated in Fig. 6.5(a) and Fig. 6.6(a). All the nodes are placed approximately at the locations indicated by the planned solutions and the constructed topologies are shown in Fig. 6.5(b) and Fig. 6.6(b) respectively. The routing from N5 to BS in the planned solution is via N4 whereas it routes though N8 in the real deployments. Except N5 , the remaining nodes have the same next hop as indicated by the iMOST solutions. 147 Chapter 6. Real measurements and results analysis (a) Original scanned floor plan of CEI-UPM (b) Automatic reconstructed result (c) 3DS imported models (c) 3D view of the scenario with furniture Figure 6.3: Indoor modelling by using iMOST: automatic 3D reconstruction+3DS models. The evaluated performance of the two candidate solutions are shown in Table .6.3, topology 1 performs better than topology 2 in terms of cost, lifetime and data drop rate. Table 6.4 shows the measured data of topology 1 in details. The detected neighbors and corresponding RSS values are shown (the first sub-row) for each node (N1 -N17 ) in the scenario. All the detected RSS values are computed by averaging the fetched samples along different time, and they are compared with 148 6.3. Indoor measurements Figure 6.4: User requirement over the indoor test. (a) iMOST solution 1 BS 2 8 11 12 13 14 15 16 3 9 4 5 6 7 10 17 (b) Real topology Figure 6.5: Topology comparison 1: (a) One of the eight solutions generated by iMOST for CEI-UPM. (b) The topology of real deployment. 149 Chapter 6. Real measurements and results analysis (a) iMOST solution 1 BS 2 3 4 5 6 7 18 8 11 12 13 14 15 16 9 10 17 (b) Real topology Figure 6.6: Topology comparison 2: (a) Another solution generated by iMOST for CEI-UPM. (b) The topology of the corresponding real deployment. Table 6.3: Evaluated performance of the two candidates. Topology solution 1 2 Objectives DC DCost DL 1 0.83 0.432 1 0.825 0.428 D Pl 0.99 0.99 D Pd 0.99 0.95 the simulation results, by setting the ray-tracing engine with maximum reflection depth depth = 0(the second sub-row) and depth = 3 (the third sub-row). The number of traced rays grows as depth increases, however the results will be more accurate as more multi-path effects are considered. As can be seen, there are some errors in discovered neighbors of some nodes (e. g.N2 and N14 ), and the errors are marked by red color. Assuming that N (u) is the set of actual neighbors of a node u, and N (u) the set of neighbors known to u (i.e. whose identifier is present in its neighborhood table). Neighborhood accuracy Accnb is the average value of the accumulation for all the nodes in the WSN the proportion 150 6.3. Indoor measurements of actual neighbors of node that have been indeed detected. It is formulated as (6.1): |N (u) N (u)| u=1 |N (u)| M Accnb = M (6.1) × 100%. Accordingly, we have the accuracy of neighborhood Accnb = 88% in this case study. The main reasons of this error are from the embedded HELLO protocol of the ZigBee layer. As N2 cannot detect N7 and N8 while N7 and N8 is able to discover N2 as their neighbor, the similar reason can be applied for the neighborhood of N5,6,10,12,13,14,16 . Fig. 6.7 demonstrates graphically RSS values of the measured data and simulation results when depth = 0 and depth = 3, by aligning the actual neighbors for each node (from N1 to N17 ). Table 6.4 shows the mean error (M E) and standard deviation error (STD) for those two simulations. When depth = 0, M E0 = 4.29 dB and ST D0 = 5.06 dB; If depth = 3, M E3 = 3.80 dB and ST D3 = 3.61 dB, which as expected, performs better (11% reduction in M E and 28.6% reduction in ST D) than the former case. Therefore WSN designer should make a trade-off decision between the time consumption and accuracy, if the planning strategy can be realized without time constraint, this work suggests users configure ray tracing engine with depth > 0 to improve the accuracy of topology estimation. This scenario is tested on 22nd and 23rd November, 2013. Sensors detect environment and send data packet every one minute, the battery status is reported from each node to BS every one minute as well. This traffic load mechanism aims to speedup the battery consumption with a ratio of 120 times faster than the simulation period (100 days). Therefore network longevity is predicted as 8.64 hours and we should change battery since that moment. The curves of battery consumption shown in Fig. 6.8 are the measured remaining energy of N1,8,14 varied along working time of WSN. According to the test N1 has the lowest lifetime around 8.3 hours, which obtains 96.1% of match with the predicted value the estimated result and indicates a good performance of the proposed work on lifetime modelling in iMOST. Fig. 6.9 compares packet delivery status in a period of 30 minutes for topology 1. N7 has the highest packet loss rate (20%) because it needs the maximum number of hops to reach BS compared with other nodes. The average packet latency is around 3.5 s therefore Dpl ≈ 0.97 which is slightly less (3%) than the estimated result. 151 Chapter 6. Real measurements and results analysis BS(-73.14),1(-69.40),2(-67.37),3(-63.72),4(-66.81),5(-72.12),6(-75.28),7(-78.03),8(-62.21),10(-60.28) BS(-83.58),1(-68.95),2(-66.33),3(-64.00),4(-66.76),5(-72.06),6(-75.27),7(-78.01),8(-62.74),10(-60.25) BS(-56.00),1(-72.00),2(-64.00),3(-66.00),5(-64.00),6(-72.50),7(-69.00),8(-69.00),9(-56.00) BS(-75.80),1(-72.00),2(-54.25),3(-67.37),4(-63.82),5(-69.43),6(-72.25),7(-74.75),8(-68.07),9(-60.28) BS(-68.89),1(-72.03),2(-69.95),3(-66.96),4(-63.85),5(-68.99),6(-72.15),7(-74.77),8(-68.72),9(-59.58) 12(-59.00),13(-67.00),14(-64.00),15(-71.50),16(-73.00),17(-79.00) BS(-75.34),12(-54.25),13(-65.99),14(-68.13),15(-73.25),16(-75.59),17(-77.67) BS(-74.90),12(-54.27),13(-67.00),14(-64.76),15(-74.42),16(-75.60),17(-78.90) 11(-59.00),13(-60.00),15(-81.00),16(-75.00),17(-81.00) BS(-73.75),11(-54.25),13(-62.41),14(-65.24),15(-71.26),16(-73.81),17(-76.05) BS(-73.30),11(-54.27),13(-62.32),14(-62.95),15(-71.71),16(-74.72),17(-77.28) 11(-64.00),12(-62.00),14(-56.00),15(-69.00),16(-65.00),17(-69.00) BS(-72.62),11(-65.99),12(-62.41),14(-56.87),15(-66.37),16(-69.85),17(-72.68) BS(-73.12),11(-66.24),12(-62.61).14(-55.71),15(-66.10),16(-69.49),17(-73.04) 11(-64.00),13(-56.00),15(-54.00),17(-72.00) BS(-69.82),11(-68.13),12(-65.24),13(-56.87),15(-63.19),16(-67.38),17(-70.60) BS(-69.54),11(-66.30),12(-65.30),13(-56.61),15(-59.55),16(-67.81),17(-69.74) BS(-76.00),11(-75.00),12(-81.00),13(-69.00),14(-54.00),16(-58.00),17(-67.00) BS(-67.94),11(-74.00),12(-72.01),13(-67.12),14(-63.94),16(-57.78),17(-64.55) BS(-67.83),11(-77.01),12(-71.71),13(-69.27),14(-61.73),16(-57.67),17(-64.48) 11(-73.00),12(-75.00),13(-64.00),15(-56.00),17(-55.00) BS(-67.03),11(-76.33),12(-74.56),13(-70.60),14(-68.13),15(-57.78),17(-57.78) BS(-66.94),11(-76.14),12(-74.72),13(-69.99),14(-63.97),15(-57.10),17(-57.52) BS(-70.00),11(-77.00),12(-80.00),13(-71.00),14(-72.00),15(-66.00),16(-55.00) BS(-69.31),11(-78.42),12(-76.80),13(-73.43),14(-71.35),15(-64.55),16(-57.78) BS(-70.32),11(-78.89),12(-77.29),13(-74.32),14(-72.24),15(-64.46),16(-58.17) BS(-69.00),2(-55.00),3(-65.00),4(-70.50),5(-78.00),6(-79.00),7(-84.00),8(-63.00),9(-65.00),10(-83.00) BS(-68.98),2(-54.25),3(-64.55),4(-68.82),5(-76.35),6(-78.43),7(-80.68),8(-65.32),9(-69.40),10(-72.00) BS(-68.85),2(-54.27),3(-64.51),4(-70.05),5(-86.31),6(-88.52),7(-90.52),8(-65.68),9(-69.31),10(-82.08) BS(-74.00),1(-55.00),3(-48.00),4(-65.67),5(-79.00),6(-73.00),9(-65.00),10(-70.00) BS(-71.10),1(-54.25),3(-60.28),4(-65.88),5(-74.58),6(-76.82),7(-79.21),8(-64.57),9(-67.37),10(-69.99) BS(-72.85),1(-55.67),3(-59.62),4(-66.11),5(-85.83),6(-86.68),7(-89.21),8(-64.74),9(-67.42),10(-69.95) BS(-72.00),1(-63.50),2(-62.00),4(-61.75),5(-75.00),6(-69.50),7(-78.25),8(-71.00),9(-63.50),10(-69.00) BS(-74.12),1(-64.55),2(-60.28),4(-57.78),5(-71.35),6(-74.02),7(-76.80),8(-67.99),9(-63.72),10(-67.37) BS(-74.94),1(-64.01),2(-60.23),4(-57.84),5(-81.80),6(-84.00),7(-76.8),8(-88.05),9(-53.00),10(-68.40) BS(-70.00),1(-74.00),2(-69.00),3(-61.00),5(-66.00),6(-74.00),7(-77.00),8(-73.00),9(-63.00),10(-64.50) BS(-76.27),1(-68.82),2(-65.88),3(-57.78),5(-68.13),6(-71.35),7(-74.56),8(-69.24),9(-66.81),10(-63.82) BS(-74.89),1(-70.05),2(-66.23),3(-57.84),5(-68.22),6(-71.15),7(-75.07),8(-69.15),9(-65.56),10(-63.96) BS(-78.50),1(-86.00),3(-76.00),4(-62.25),7(-55.00),8(-81.00),9(-72.33),10(-73.00) BS(-80.98),1(-76.35),2(-74.58),3(-71.35),4(-68.13),6(-57.78),7(-65.99),8(-76.26),9(-72.12),10(-69.43) BS(-80.32),1(-86.63),2(-84.33),3(-72.17),4(-68.22),6(-57.71),7(-60.45),8(-76.52),9(-72.06),10(-69.66) BS(-72.00),1(-83.00),3(-72.00),4(-74.30),5(-55.00),7(-45.00),9(-74.00),10(-70.00) BS(-81.85),1(-78.43),2(-76.82),3(-74.02),4(-71.35),5(-57.78),7(-60.58),8(-77.55),9(-75.28),10(-72.25) BS(-79.33),1(-78.52),2(-77.71),3(-74.49),4(-71.15),5(-57.71),7(-56.86),8(-77.31),9(-75.27),10(-72.14) BS(-43.00),1(-91.00),2(-87.60),3(-80.33),4(-75.00),5(-54.25),6(-44.00),8(-79.50),9(-76.00),10(-71.00) BS(-84.42),1(-80.68),2(-79.21),3(-76.80),4(-74.56),5(-65.99),6(-60.58),8(-79.89),9(-78.03),10(-74.75) BS(-54.30),1(-80.33),2(-80.87),3(-76.80),4(-75.05),5(-60.45),6(-56.85),8(-79.69),9(-76.39),10(-74.77) BS(-68.00),1(-60.00),2(-65.00),3(-73.00),4(-77.00),5(-76.00),6(-71.00),7(-76.50),9(-52.00),10(-76.00) BS(-69.23),1(-65.32),2(-64.57),3(-67.99),4(-69.24),5(-76.26),6(-77.55),7(-79.89),9(-62.21),10(-68.07) BS(-69.04),1(-65.68),2(-63.70),3(-68.37),4(-69.10),5(-76.22),6(-77.84),7(-79.69),9(-62.37),10(-68.09) BS(-83.00),1(-67.50),2(-56.00),3(-58.00),4(-66.00),5(-75.00),6(-75.00),7(-76.00),8(-65.50),10(-64.50) Neighbor ID and Average RSSs (dBm) 88% Accnb ME3 = 3.80 STD3 = 3.61 ME0 = 4.29 STD0 = 5.06 RSS ME, STD Table 6.4: Neighborhood table and RSS comparisons between real measurement and simulation results: indoor scenario. Node ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 152 6.3. Indoor measurements -30 Real measurement Simulation result depth=0 Simulation result depth=3 -40 RSS (dBm) -50 -60 -70 -80 -90 -100 0 20 40 60 80 Measured value index aligned in the table 100 120 140 Figure 6.7: RSS comparison between real measurement and simulation result with depth = 0 and depth = 3. 100 node 1 node 8 node 14 90 80 Remaining energy (%) 70 60 50 40 30 20 10 0 0 1 2 3 4 5 Working time (hours) 6 7 8 9 Figure 6.8: Remaining energy of N1 , N8 and N14 along the working time of WSN. The packet drop rate is calculated as the proportion between lost data packets and total number of packets that have been sent in the WSN. According to the observed 153 Chapter 6. Real measurements and results analysis packet sequence number for each node, there is 10% of packet loss in this case, thus the desirability metric DPd = 90%. Compared with the estimated result (99%), the error is approximately 9%. 25 Number of packets Arrived packets Sent packets 20 15 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Node ID Figure 6.9: Comparing the number of arrived packet and the number of sent packet for each node in the deployment. The three nodes: N1,8,14 are the bottlenecks of this deployment. Although N1 has less children than N8 and same as N14 , the accumulated dropped packets hoping through N1 are less than that via N8 and N14 . As a result, N1 should forward more data packets to BS and a faster exhaustion on the battery energy occurs, which also reveals why (see Fig. 6.8) the lifetime of N1 is shorter than other nodes. The three nodes N1,8,14 are the bottleneck of the deployment, although N1 has less children than N8 and same as N14 , the accumulated packet loss rate hoping through N1 is less than that via N8 and N14 . As a result, N1 should forward more 154 6.4. Outdoor measurements data packets to BS and a faster exhaustion on the battery energy occurs, which also reveals why (see Fig. 6.8) the lifetime of N1 is shorter than other nodes. Sensed data are gathered at the coordinator and the environment variation is recorded for each sensor. Fig. 6.10 shows the variation of temperature, humidity and light in the room where N4 is placed, and the observation time is from 16 : 00 to 21 : 00 on 22nd Nov, 2013. The average temperature is 29.25◦ C, humidity is 47.13 without significant changes along time. While the light level suddenly reduces from 90.35% (3700) to 0% (0) at 20:30, it indicates that the light is turned off and people may have left room after this moment. 50 Measured level 40 30 20 10 0 16 Temperature Humidity Light (x0.01) 17 18 19 20 Time of the day (from 16hr~21hr) 21 Figure 6.10: The sensed data of N4 . 6.4 Outdoor measurements The second test is realized to monitor the outdoor environment at the parking lot of UPM. The bird’s view of the environment is taken from Google Maps, as shown in Fig. 6.11(a), the automatic image understanding algorithm, using Madrid training database, provides the recognition result shown in Fig. 6.11(b) which achieves 93.3% 155 Chapter 6. Real measurements and results analysis of accuracy compared with the ground truth. It is expected to assigned uniform (a) Original image (b) Image recognition result (c) 3D view of the reconstructed result by adding terrain information from Google Earth Figure 6.11: Outdoor modelling reconstruction+terrain information. by using iMOST: automatic 3D height to the building, while in this case we register the terrain information of this area is available, and therefore it was used to enhance the performance. The 3D view of the reconstructed model is shown in Fig. 6.11(c). This 3D environment modelling took 6.83 minutes to reconstruct a region of 233.36 m × 297 m × 73.67 m. 156 6.4. Outdoor measurements Table 6.5: Evaluated performance of the selected candidate. DC 1 Objectives DCost DL D Pl 0.91 0.155 0.99 D Pd 0.98 User configuration on sensing area for this test is As = 99 and TX power for all the nodes are set as 4 dBm which is the maximum communication ability of the antenna. Nodes are set with RX sensitivity as −98 dBm, Rsense = 8 m. iMOST generates topology (Fig. 6.12(a)) accordingly, and the candidate, which with the lowest cost and competitive desirability values of other metrics, is selected in this case. The predicted performance is shown in Table. 6.5. By placing 9 nodes in target locations, the real topology is shown in Fig. 6.12(b). N1 ∼ N4 are, as estimated, directly connected with BS, N7 routes via N4 rather than through N6 in the solution. Table 6.6 shows the measured data of the deployed topology in details. All the detected RSS values are computed by averaging the fetched samples along different time, and they are compared with the simulation results with depth = 3 (the second sub-row). N5 and N8 can not discover each other in the real deployment while the simulated result shows connections between them, and therefore Accnb = 98%. Fig. 6.13 demonstrates graphically RSS values of the measured data and simulation results when depth = 3, by aligning the actual neighbors for each node (from N1 to N9 ). Table 6.6 indicates that in this case M E3 = 2.45 dB and ST D3 = 2.45 dB. As the outdoor environment is an open area without significant obstacles between nodes, this performance is slightly better than indoor performance. Both indoor and outdoor neighborhood results indicate that the proposed modelling methods on radio propagation and link establishment are practical and reliable to be applied in real deployments. This scenario is tested on 23rd November, 2013. Sensing period and battery period are one minute. The estimated network longevity is predicted as 1.86 hours with the maximum transmission power. The curves of battery consumptions shown in Fig. 6.14 are the measured remaining energy of N4,5,2 varying along working time of WSN. According to the test, N4 has the lowest lifetime (1.95 hours), which is 4.8% higher than the estimated result. N5 is in charge of forwarding packets only 157 Chapter 6. Real measurements and results analysis BS(-61.00),2(-61.50),3(-67.00),4(-75.00),5(-70.00),6(-79.00),7(-78.00),8(-80.00) BS(-65.65),2(-62.26),3(-68.25),4(-71.40),5(-74.47),6(-76.41),7(-77.99),8(-81.13) BS(-77.00),1(-62.30),3(-58.00),4(-68.00),5(-76.00),6(-74.00),7(-76.00),8(-75.00),9(-86.00) BS(-72.45),1(-62.25),3(-62.19),4(-67.69),5(-75.17),6(-74.53),7(-76.29),8(-78.85),9(-88.02) BS(-57.00),1(-68.50),2(-59.00),4(-62.65),5(-66.00),6(-71.00),7(-75.00),8(-73.00),9(-86.00) BS(-68.11),1(-68.24),2(-62.20),4(-61.15),5(-68.86),6(-72.11),7(-74.51),8(-72.31),9(-80.15) BS(-79.00),1(-74.00),2(-67.00),3(-63.00),5(-70.00),6(-71.00),7(-70.00),8(-77.00),9(-85.00) BS(-71.18),1(-71.44),2(-67.69),3(-61.17),5(-64.31),6(-69.24),7(-72.45),8(-84.86),9(-85.83) BS(-73.00),1(-75.50),2(-73.00),3(-67.00),4(-68.00),6(-63.00),7(-69.00),9(-83.00) BS(-74.42),1(-74.57),2(-72.19),3(-68.87),4(-64.31),6(-61.96),7(-68.21),8(-69.44),9(-83.56) BS(-77.00),1(-81.00),2(-74.00),3(-69.00),4(-70.33),5(-65.00),7(-60.00),8(-64.00),9(-78.00) BS(-76.27),1(-76.39),2(-74.51),3(-72.11),4(-69.24),5(-61.99),7(-62.44),8(-66.05),9(-79.89) BS(-79.00),1(-76.00),2(-72.00),3(-76.00),4(-67.00),5(-69.00),6(-59.00),8(-63.00),9(-63.00) BS(-77.81),1(-77.96),2(-76.42),3(-74.55),4(-72.46),5(-68.21),6(-62.45),8(-62.08),9(-68.10) 1(-80.00),2(-76.00),3(-78.00),4(-86.00),6(-65.00),7(-61.00),9(-61.00) 1(-80.95),2(-88.60),3(-73.21),4(-84.86),5(-69.44),6(-65.95),7(-62.09),9(-62.07) 2(-85.00),3(-79.00),4(-85.00),5(-82.00),6(-77.00),7(-67.00),8(-61.00) 2(-88.21),3(-80.23),4(-85.83),5(-83.56),6(-69.72),7(-68.09),8(-62.08) Neighbor ID and Average RSSs (dBm) 99% Accnb STD3 = 2.45 ME3 = 2.45 RSS ME, STD Table 6.6: Neighborhood table and RSS comparisons between real measurement and simulation results: outdoor scenario. Node ID 1 2 3 4 5 6 7 8 9 158 6.4. Outdoor measurements for N6 , therefore within the same time it consumes less energy than N4 . While N2 9 8 7 6 4 5 3 2 1 (a) iMOST solution with the lowest cost 9 8 7 6 5 4 3 2 1 (b) Real topology Figure 6.12: Topology comparison: (a) One of the eight solutions generated by iMOST for outdoor environment. (b) The topology of real deployment. 159 Chapter 6. Real measurements and results analysis -30 Real measurement Simulation result depth=3 -40 RSS (dBm) -50 -60 -70 -80 -90 -100 0 10 20 30 40 50 Measured value index aligned in the table 60 70 80 Figure 6.13: RSS comparison between real measurement and simulation result with depth = 3. 100 90 80 Remaining energy (%) 70 60 50 40 30 20 10 0 0 node 4 node 5 node 2 0.1 0.2 0.3 0.4 0.5 Working time (hours) 0.6 0.7 0.8 0.9 1 Figure 6.14: Remaining energy of N4 , N5 and N2 along the working time of WSN. sends packets only for itself, compared with N4 and N5 , it has the longest battery life. 160 6.5. Conclusion 38 Arrived packets Sent packets Number of packets 36 34 32 30 28 26 1 2 3 4 5 6 Node ID 7 8 9 Figure 6.15: Comparing the number of arrived packet and the number of sent packet for each node in the deployment. Fig. 6.15 compares packet delivery status in a period of 0.5 hour for the topology. DPd = 99% which is better than the estimated result of 98%. N9 and N8 , which perform similarly as N7 in the indoor measurement, have the highest packet loss rate (10%), because they also need more number of hops to reach BS compared with other nodes. The average packet latency is around 2.6 s, therefore Dpl ≈ 0.97 which is slightly less (2%) than the estimated result. 6.5 Conclusion The whole methodology introduced through this thesis is comprehensively validated in this chapter, by launching real WSN deployments. 161 Chapter 6. Real measurements and results analysis 3D environment models are constructed automatically for both indoor and outdoor scenarios. Real deployments for ambient monitoring are realized for target regions. Experiments are carefully designed by programming the sensor nodes coping with the application requirements, and the placements of nodes strictly follow the planned topologies generated by iMOST. The experimental data are categorized and compared in terms of environment reconstruction accuracy, routing table accuracy, neighborhood table accuracy, RSS accuracy, lifetime and packet delivery status. Both applications show the potentials of the proposed algorithm and the developed planning tool to fulfill user requirements with optimized and practical performance. 162 Chapter 7 Conclusions and future works 7.1 Conclusions In this thesis, a series of novel methodologies have been proposed, in terms of environment modelling, metric modelling, network evaluation and multi-objective optimized planning, for efficiently planning reliable WSNs. A planning tool iMOST is developed by integrating those proposed methodologies to assist WSN designers conveniently constructing and evaluating network topology for outdoor and indoor environments. The proposed automatic 3D outdoor and indoor environment modelling methodology liberates WSN designers and wireless communication engineers from the traditional time consuming and costly approaches, in which environment is usually purchased from professional GIS companies or manually reconstructing from field measurements. With the best average accuracy of 76.1% for outdoor image understanding within one hour and 97% accuracy for indoor recognition in less than one minute, this method is capable to recognize and segment objects from images pixel wisely with sufficient accuracy. The 3D vectorization procedure eliminates redundant information thus objects are stored succinctly to reduce memory occupations. Moreover we also prove its flexibility to be applied for different outdoor environments by training customized environment databases and the robustness for indoor reconstruction. To our best knowledge, this is the first time that image understanding algorithm being applied to automatically reconstruct environment database for signal propagation and network planning purpose. This novel approach allows reconstructing large scale 3D map in a very short time, accurately and for free. Based on the practical and accurate 3D modelling, the ray tracing engine is developed to tracing the radio and sensing signal path. The experimental results Chapter 7. Conclusions and future works proved the improvement on computation efficiency (in average 335%) by using the kd-tree space division algorithm and modified polar sweep algorithm. The radio propagation model is proposed for ray tracing engine, which emphasizes not only the materials of obstacles but also their locations along the signal path and the performance is evaluated through comparison with both indoor and outdoor measurement data, and the mean error is less than 2.2 dB in outdoor test and less than 2.42 dB for indoor scenarios, which outperforms than the compared state-of-the-art works. The sensing signal of sensor nodes, which are sensitive to the obstacles, is benefit from the ray-tracing algorithm via obstacles detection. The performance of this modelling method is robust and accurate compared with conventional methods and experimental results imply that this methodology is suitable for both outdoor urban scenes and indoor environments, moreover it can be applied to GSM communication and ZigBee protocol by varying frequency parameter in the radio propagation model. An automatic 3D multi-objective optimization WSN planning algorithm is proposed in this work. More comprehensive metrics (connectivity, coverage, cost, lifetime, packet latency and packet drop rate) are modeled practically compared with other works, especially 3D ray tracing method are used to model the radio link and sensing signal which are sensitive to the obstruction of obstacles; routing of network is constructed by using AODV protocol; the network longevity, packet delay and packet drop rate are obtained via simulating practical events in WSNet simulator, which to the best of our knowledge, is the first time that network simulator is involved in a planning algorithm. Moreover the multi-objective optimization methodology is developed to cater the characteristics of WSNs. The individual length is changeable so that the cost can be optimized, meanwhile crossovers and mutations are designed to eliminate invalid modifications to improve the computation efficiency. The capability of providing multiple optimized solutions simultaneously allows users making their own decisions, and the results are more comprehensive optimized compared with other state-of-the-art algorithms. iMOST is developed by integrating the introduced novel algorithms, to assist WSN designers efficiently planning reliable WSNs for different configurations. iMOST features with convenient operation with user-friend vision system allow users configuring the network properties freely; It supports the efficient and automatic 164 7.2. Future works 3D database reconstruction algorithm and fast 3D objects design for both indoor and outdoor environments; The multiple multi-objective optimized 3D deployment solutions in the 3D space and the corresponding evaluated performance are visually presented to users; and the NPM of iMOST is available online as well as the source codes of the other two rebuilt advanced heuristics. Therefore WSN designers will be benefit from this tool on efficiently constructing environment database, practically and efficiently planning reliable WSNs for both outdoor and indoor applications, efficiently and accurately estimate the performance of a WSN. With the open source codes, they are also able to compare their algorithms with ours to make contributions to this academic field. The whole methodology introduced through this thesis is comprehensively validated, by launching real WSN deployments for both indoor and outdoor environment. Experiments are carefully designed by programming the sensor nodes coping with the application requirements, and the placements of nodes strictly follow the planned topologies generated by iMOST. The environment reconstruction accuracy, routing table accuracy, neighborhood table accuracy, RSS accuracy, lifetime and packet delivery status are computed and analyzed through comparisons. The results indicate that the proposed methodologies and the developed planning tool iMOST are able to assist WSN designers efficiently planning reliable and optimized WSN topology for both indoor and outdoor scenarios. 7.2 Future works In the future, this work can be continued from different aspects: First of all, the image understanding result for indoor environment can be improved. By observing different types of line segments with different thickness, the materials of internal walls can be distinguished and different dielectric parameters can automatically assigned instead of using uniform parameter by the current work. More training database should be constructed for different kinds of environments. Besides the choices of some parameters in outdoor image understanding are arbitrary, which leads to uncertainty in the performance and hence deep insight study over those parameters will contribute to the work. 165 Chapter 7. Conclusions and future works Secondly, computer science knowledge shall be used in the future to optimize the computation method for large scale network planning and large scenario demonstration. When scenario becomes larger, more points are involved in the computation which will increase time spent on searching proper placements. One proposal to this problem is the partition computation: each detected region is extracted and only the areas with high deployment possibility are concerned. By doing so, the planning algorithm does not have to traverse the entire environment database and the speed is improved. There are several issues that we have not yet explored in this work, including planning mobile sensor networks, the study of human movements on the network performance and the interference from other wireless signals over the deployed environment and the antenna radiation pattern towards the radio propagation modelling. Therefore more efforts should be done to make contribution to this academic field and bring more benefits to WSN designers. 7.3 Publications based on this work This section shows the complete list of publications resulting from this thesis: Refereed journal papers 1. Danping He, Guixuan Liang, Jorge Portilla, Teresa Riesgo, A Novel Method for Radio Propagation Simulation Based on Automatic 3D Environment Reconstruction. Radioengineering. 21 - 1, pp. 985 - 992. 12/2012. ISSN 1210-2512. (Invited paper) 2. Danping He, Gabriel Mujica, Guixuan Liang, Jorge Portilla, Teresa Riesgo, Radio Propagation Modeling and Real Test of ZigBee Based Indoor Wireless Sensor Networks. Submitted to JSA. (Invited paper by Journal of Systems Architecture: Embedded Software Design (JSA)) 3. Danping He, Gabriel Mujica, Jorge Portilla, Teresa Riesgo, Modelling and planning reliable wireless sensor networks based on multi-objective optimization genetic algorithm with changeable length. Submitted to Journal of Heuristics. 166 7.3. Publications based on this work Refereed conference papers 1. Danping He, Nathalie Mitton, David Simplot-Ryl, An Energy Efficient Adaptive HELLO Algorithm for Mobile Ad Hoc Networks. In Proceedings of the 16th ACM international conference on Modeling, analysis & simulation of wireless and mobile systems (MSWiM ’13), pp. 65 - 72, 2013. 2. Danping He, Gabriel Mujica, Guixuan Liang, Jorge Portilla, Teresa Riesgo, Radio Propagation Modeling and Measurements for ZigBee Based Indoor Wireless Sensor Networks. In Proceedings of the Jornadas de Computacin empotrada, pp. 98 - 103, 2013. Best paper award, invited to publish on JSA 3. Danping He, Jorge Portilla, Teresa Riesgo, A 3D Multi-objective Optimization Planning Algorithm for Wireless Sensor Networks. Published in IECON, 2013. 4. Danping He, Guixuan Liang, Jorge Portilla, Teresa Riesgo, A Novel Method for Radio Propagation Simulation Based on Automatic 3D Environment Reconstruction. In Proc. 6th European Conf. Antennas and Propagation (EUCAP), pp. 1445 - 1449, 2012. Invited to publish at special issue of Radioengineering Journal 5. Danping He, Gabriel Mujica, Jorge Portilla, Teresa Riesgo, Simulation Tool and Case Study for Planning Wireless Sensor Network. In Proceedings of Annual Conference of the IEEE Industrial Electronics Society (IECON), pp. 6028 - 6032, 2012. 6. Guixuan Liang, Danping He, Jorge Portilla, Teresa Riesgo, A Hardware In The Loop Design Methodology For FPGA System and Its Application To Complex Functions. In Proceedings of VLSI, Design, Automation and Test (VLSI-DAT ), pp. 1 - 6, 2012. 7. Guixuan Liang, Danping He, Jorge Portilla, Teresa Riesgo, Functional Validation of MB-OFDM System Using HW-in the loop. In Proceedings of Conference on Design of Circuits and Integrated Systems (DCIS), pp. 131 136, 2012 167 Chapter 7. Conclusions and future works 8. Guixuan Liang, Danping He, Eduardo de la Torre, Teresa Riesgo, Low-power, High-speed FFT Processor for MB-OFDMUWB Application. Microtechnologies for the New Millennium 2011 (SPIE), 2011. 7.4 Implementation of this work This work is implemented in the Working Package 5 of European project WSN-DPCM, which is funded by the ARTEMIS Joint Undertaking (the European technology platform representing the field of advanced research and technology for embedded intelligence and systems), national authorities and European partner companies with a total amount of 3.4 million euros. It is officially launched in October 2011 and lasts for 36 months [142]. With the cooperation of several technical universities and companies from Spain, Italy, Lithuania and Greece, the project targets to address the WSN deployment, testing, and maintenance challenging issues by developing an integrated platform for smart environments that will comprise a middleware for heterogeneous wireless technologies as well as an integrated engineering tool for quick system development, a planning tool and a commissioning & maintenance tool for expert and non-expert users. The Working Package 5 (WP5) of the WSN-DPCM project aims at developing the WSN visual modeling, simulation and deployment tool for the optimal target WSN deployment scheme definition. In the proposal, the planning tool should have rich graphical end-user interface with high level of automation, supporting various proposals such as models of WSN motes, network topology, communication models, environmental conditions and deployment environments. With the assistance of the planning tool, WSN developers are expected to be facilitated during the WSN distributed application design for investigating alternative deployment schemes in a transparent and guided way, thus saving efforts and time. 168 Bibliography [1] D. Estrin, R. Govindan, J. Heidemann, and S. Kumar. Next century challenges: scalable coordination in sensor networks. In Proceedings of the 5th annual ACM/IEEE international conference on Mobile computing and networking, MobiCom ’99, pages 263–270, New York, NY, USA, 1999. ACM. (Cited on page 1.) [2] Zigbee Alliance. Zigbee protocol. http://www.zigbee.org/. (Cited on page 1.) [3] Bluetooth. https://www.bluetooth.org/. (Cited on page 1.) [4] UWB. Iso/iec 26907:2009 information technology – telecommunications and information exchange between systems – high-rate ultra-wideband phy and mac standard, 2009. (Cited on page 1.) [5] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci. Wireless sensor networks: a survey. Computer Networks, 38(4):393–422, 2002. (Cited on page 1.) [6] J. Yick, B. Mukherjee, and D. Ghosal. Wireless sensor network survey. Computer Networks, 52(12):2292 – 2330, 2008. (Cited on page 1.) [7] C. Buratti, A. Conti, D. Dardari, and R. Verdone. An overview on wireless sensor networks technology and evolution. Sensors, 9(9):6869–6896, 2009. (Cited on page 1.) [8] P. Naz, S. Hengy, and P. Hamery. Soldier detection using unattended acoustic and seismic sensors. In Proceedings of SPIE, Baltimore, Maryland, USA, 2012. (Cited on page 3.) [9] J. Chang, W. Mendyk, L. Thier, P. Yun, A. LaRow, S. Shaw, and W. Schoenborn. Early attack reaction sensor (EARS), a man-wearable gunshot detection system. In Proceedings of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, volume 6201 of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Orlando (Kissimmee), FL, USA, June 2006. (Cited on page 3.) [10] H. E. de Bree and J. W. Wind. The acoustic vector sensor: a versatile battlefield acoustics sensor. In Proceedings of the Society of PhotoOptical Instrumentation Engineers (SPIE) Conference Series, volume 8047 of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Orlando, Florida, USA, May 2011. (Cited on page 3.) Bibliography [11] G. de Mel, T. Pham, P. Sullivan, K. Grueneberg, W. Vasconcelos, and T. Norman. Intent-based resource deployment in wireless sensor networks. In Proceedings of the Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, volume 8389 of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Baltimore, Maryland, USA, May 2012. (Cited on page 3.) [12] R. T. Zehr, S. K. Holland, and G. Laufer. A low-cost remote chemical sensor for e-uav platforms. In Proceedings of SPIE Defense & Security Symposium, Orlando, Florida USA, 2007. (Cited on page 4.) [13] L. Bukshpun, R. D. Pradhan, V. Tun, N.and Esterkin, and G. Tomczyk. Novel optical sensor system for missile canisters continuous monitoring. In Proceedings of SPIE, 2007. (Cited on page 4.) [14] E. Goldoni and P. Gamba. W-tremors, a wireless monitoring system for earthquake engineering. In Proceedings of IEEE Workshop on Environmental Energy and Structural Monitoring Systems (EESMS 2010), pages 26–31, Taranto, Italy, 2010. (Cited on page 4.) [15] J. Wong, J. Goethals, and B. Stojadinovic. Wireless sensor seismic response monitoring system implemented on top of neesgrid. In Proceedings of SPIE 5768, Health Monitoring and Smart Nondestructive Evaluation of Structural and Biological Systems IV, 74, volume 5768, pages 74–84, San Diego, CA, USA, 2005. (Cited on pages 4 and 5.) [16] P. Zhang, C. M. Sadler, S. A. Lyon, and M. Martonosi. Hardware design experiences in zebranet. In Proceedings of the 2nd international conference on Embedded networked sensor systems, SenSys ’04, pages 227–238, New York, NY, USA, 2004. ACM. (Cited on page 5.) [17] G. Werner-Allen, K. Lorincz, M. Welsh, O. Marcillo, J. Johnson, M. Ruiz, and J. Lees. Deploying a wireless sensor network on an active volcano. IEEE Internet Computing, 10(2):18–25, 2006. (Cited on pages 5, 6 and 105.) [18] Tmote Sky, 2009. (Cited on page 6.) [19] G. Tolle, J. Polastre, R. Szewczyk, D. Culler, N. Turner, K. Tu, S. Burgess, T. Dawson, P. Buonadonna, D. Gay, and W. Hong. A macroscope in the redwoods. In Proceedings of the 3rd international conference on Embedded networked sensor systems, SenSys ’05, pages 51–63, New York, NY, USA, 2005. ACM. (Cited on page 6.) [20] A. Chandrakasan, R. Amirtharajah, S. Cho, J. Goodman, G. Konduri, J. Kulik, W. Rabiner, and A. Wang. Design considerations for distributed 170 Bibliography microsensor systems. In Proceedings of the IEEE Custom Integrated Circuits, pages 279–286, San Diego, CA, USA, 1999. (Cited on page 6.) [21] Forest fire detection system. http://www.libelium.com/wireless_ sensor_networks_to_detec_forest_fires/. (Cited on page 6.) [22] A. Cerpa, J. Elson, D. Estrin, L. Girod, M. Hamilton, and J. Zhao. Habitat monitoring: application driver for wireless communications technology. SIGCOMM Comput. Commun. Rev., 31(2 supplement):20–41, April 2001. (Cited on page 6.) [23] Alert system website. page 7.) http://www.alertsystems.org/. (Cited on [24] I. Vasilescu, K. Kotay, D. Rus, M. Dunbabin, and P. Corke. Data collection, storage, and retrieval with an underwater sensor network. In Proceedings of the 3rd international conference on Embedded networked sensor systems, SenSys ’05, pages 154–165, New York, NY, USA, 2005. ACM. (Cited on page 7.) [25] J. McCulloch, P. McCarthy, S. M. Guru, W. Peng, D. Hugo, and A. Terhorst. Wireless sensor network deployment for water use efficiency in irrigation. In Proceedings of the workshop on Real-world wireless sensor networks, REALWSN ’08, pages 46–50, New York, NY, USA, 2008. ACM. (Cited on page 7.) [26] G. Sklivanitis, J. Kimionis, and E. Kampianakis. Towards precision agriculture: Building a soil wetness multi-hop wsn from first principles. In Proceedings of the Second International Workshop in Sensing Technologies in Architecture, Forestry and Environment (ECOSENSE), Belgrade, Serbia, 2011. (Cited on page 7.) [27] J. Xia, Z. Tang, X. Shi, L. Fan, and H. Li. An environment monitoring system for precise agriculture based on wireless sensor networks. In Proceedings of the Seventh International Conference on Mobile Ad-hoc and Sensor Networks (MSN 2011), pages 28–35, Beijing, China, 2011. (Cited on page 7.) [28] H. Alemdar and C. Ersoy. Wireless sensor networks for healthcare: A survey. Computer Networks, 54(15):2688 – 2710, 2010. (Cited on page 7.) [29] T. Hori and Y. Nishida. Ultrasonic sensors for the elderly and caregivers in a nursing home. In Proceedings of the 7th International Conference on Enterprise Information Systems (ICEIS 2005), pages 110–115, Miami, USA, 2005. (Cited on pages 7 and 8.) 171 Bibliography [30] Z. Pang, Q. Chen, and L. Zheng. A pervasive and preventive healthcare solution for medication noncompliance and daily monitoring. In Proceedings of the 2nd International Symposium on Applied Sciences in Biomedical and Communication Technologies (ISABEL 2009), pages 1–6, Bratislava, Slovakia, 2009. (Cited on page 8.) [31] S. Yoo, P. K. Chong, T. Kim, J. Kang, D. Kim, C. Shin, K. Sung, and B. Jang. Pgs: Parking guidance system based on wireless sensor network. In Proceedings of the 3rd International Symposium on Wireless Pervasive Computing (ISWPC 2008), pages 218–222, Santorini, Greece, 2008. (Cited on page 9.) [32] Smartsantander. http://www.smartsantander.eu/. (Cited on pages 9, 14 and 106.) [33] J. Rousselot, Ph. Dallemagne, and J.-D. Decotignie. Deployments of wireless sensor networks performed by csem. In Proceedings of COGnitive systems with Interactive Sensors (COGIS 2009), Paris, France, 2009. (Cited on page 12.) [34] Ns-2. http://www.isi.edu/nsnam/ns/. (Cited on page 12.) [35] Omnet++. http://www.omnetpp.org/. (Cited on page 12.) [36] A. Fraboulet, G. Chelius, and E. Fleury. Worldsens: development and prototyping tools for application specific wireless sensors networks. In Proceedings of the 6th international conference on Information processing in sensor networks, IPSN ’07, pages 176–185, New York, NY, USA, 2007. ACM. (Cited on page 12.) [37] P. Levis, N. Lee, M. Welsh, and D. E. Culler. Tossim: accurate and scalable simulation of entire tinyos applications. In Proceedings of SenSys, pages 126–137, Los Angeles, California, USA, 2003. (Cited on page 12.) [38] M. Korkalainen and M. Sallinen. A survey of rf-propagation simulation tools for wireless sensor networks. In Proceedings of the Fourth Int Sensor Technologies and Applications (SENSORCOMM) Conf, pages 342–347, Venice, Italy, 2010. (Cited on pages 13 and 74.) [39] Edxpro. http://www.edx.com/products/signalpro.html. (Cited on page 13.) [40] Winprop: Software tool for the planning of radio communication networks (terrain, urban, indoor & tunnel). http://www.awe-communications. com/. (Cited on page 13.) 172 Bibliography [41] Cindoor. http://www.gsr.unican.es/cindoor/. (Cited on page 13.) [42] M. T. Kouakou, S. Yamamoto, K. Yasumoto, and M. Ito. Cost-efficient deployment for full-coverage and connectivity in indoor 3d wsns. In Proceedings of IPSJ Dicomo 2010, 2010. (Cited on pages 13, 20, 101, 124 and 130.) [43] D. Jourdan and O.L. de Weck. Layout optimization for a wireless sensor network using a multi-objective genetic algorithm. In Proceedings of IEEE 59th Vehicular Technology Conference (VTC 2004), volume 5, pages 2466–2470 Vol.5, Los Angeles, CA, USA, 2004. (Cited on pages 14, 20 and 130.) [44] Citysense website. https://www.sensenetworks.com/products/ macrosense-technology-platform/citysense/. (Cited on pages 14 and 106.) [45] M. Ahlberg, V. Vlassov, and T. Yasui. Router placement in wireless sensor networks. In Proceedings of the IEEE International Conference on Mobile Adhoc and Sensor Systems (MASS 2006), pages 538–541, Vancouver, BC, Canada, 2006. (Cited on pages 14 and 106.) [46] S. Misra, S. D. Hong, G. Xue, and J. Tang. Constrained relay node placement in wireless sensor networks: Formulation and approximations. Networking, IEEE/ACM Transactions on, 18(2):434–447, 2010. (Cited on pages 15 and 20.) [47] S.M.S. Shams, A.H. Chowdhury, Ki-Hyung Kim, and Noh Bok Lee. A fast approximation algorithm for relay node placement in double-tiered wireless sensor network. In Proceedings of IEEE Military Communications Conference (Milcom 2008), pages 1–6, San Diego, CA, USA, 2008. (Cited on pages 15 and 20.) [48] S. Lee and M. Lee. Qrmsc: Efficient qos-aware relay node placement in wireless sensor networks using minimum steiner tree on the convex hull. In Proceedings of The International Conference on Information Networking (ICOIN 2013), pages 36–41, Bangkok, Thailand, 2013. (Cited on pages 15 and 20.) [49] S. Kim, J. Ko, J. Yoon, and H. Lee. Multiple-objective metric for placing multiple base stations in wireless sensor networks. In Proceedings of the 2nd International Symposium on Wireless Pervasive Computing (ISWPC ’07), pages –, San Juan, Puerto Rico, 2007. (Cited on pages 15 and 20.) [50] Y. Huang, P. Hsiu, W. Chu, K. Hung, A. Pang, T. Kuo, M. Di, and H. Fang. An integrated deployment tool for zigbee-based wireless sensor networks. In Proceedings of IEEE/IFIP International Conference on Embedded and Ubiquitous Computing (EUC ’08), volume 1, pages 309–315, Shanghai, China, 2008. (Cited on pages 16, 17 and 20.) 173 Bibliography [51] A. Guinard, M.S. Aslam, D. Pusceddu, S. Rea, A. McGibney, and D. Pesch. Design and deployment tool for in-building wireless sensor networks: A performance discussion. In Proceedings of IEEE 36th Conference on Local Computer Networks (LCN 2011), pages 649–656, Bonn, Germany, 2011. (Cited on pages 16, 18, 20 and 101.) [52] A. McGibney, A. Guinard, and D. Pesch. Wi-design: A modelling and optimization tool for wireless embedded systems in buildings. In Proceedings of IEEE 36th Conference on Local Computer Networks (LCN 2011), pages 640–648, Bonn, Germany, 2011. (Cited on pages 16, 20 and 101.) [53] V. Akbarzadeh, C. Gagne, M. Parizeau, M. Argany, and M.A. Mostafavi. Probabilistic sensing model for sensor placement optimization based on line-of-sight coverage. Instrumentation and Measurement, IEEE Transactions on, 62(2):293–303, 2013. (Cited on pages 19 and 20.) [54] S. Xiong, L. Yu, H. Shen, C. Wang, and W. Lu. Efficient algorithms for sensor deployment and routing in sensor networks for network-structured environment monitoring. In Proceedings of the IEEE INFOCOM 2012, pages 1008–1016, Orlando, FL, USA, 2012. (Cited on pages 19 and 20.) [55] L. Liu and H. Ma. On coverage of wireless sensor networks for rolling terrains. Parallel and Distributed Systems, IEEE Transactions on, 23(1):118–125, 2012. (Cited on pages 19 and 20.) [56] S. Qian, P. Guo, and T. Jiang. A novel lifetime-enhanced deployment strategy for chain-type wireless sensor networks. In Proceedings of IEEE International Conference on Communications (ICC 2012), pages 513–517, Ottawa, ON, Canada, 2012. (Cited on pages 19 and 20.) [57] M.Z.A. Bhuiyan, J. Cao, and G. Wang. Deploying wireless sensor networks with fault tolerance for structural health monitoring. In Proceedings of IEEE 8th International Conference on Distributed Computing in Sensor Systems (DCOSS 2012), pages 194–202, Hangzhou, China, 2012. (Cited on pages 19 and 20.) [58] Y. Chen, C. Chuah, and Q. Zhao. Sensor placement for maximizing lifetime per unit cost in wireless sensor networks. In Proceedings of IEEE Military Communications Conference (MILCOM 2005), pages 1097–1102 Vol. 2, Atlantic city, New jersey, USA, 2005. (Cited on page 20.) [59] A. Efrat, Sariel H., and J. S. B. Mitchell. Approximation algorithms for two optimal location problems in sensor networks. In Proceedings of 2nd International Conference on Broadband Networks (BROADNETS 2005), Boston, Massachusetts, USA, 2004. (Cited on pages 19 and 101.) 174 Bibliography [60] X. Cheng, D. Du, L. Wang, and B. Xu. Relay sensor placement in wireless sensor networks. Wirel. Netw., 14(3):347–355, June 2008. (Cited on pages 19 and 101.) [61] S. Poduri, S. Pattem, B. Krishnamachari, and G. S. Sukhatme. Sensor network configuration and the curse of dimensionality. In Proceedings of the Third IEEE Workshop on Embedded Networked Sensors, 2006. (Cited on pages 19 and 101.) [62] H. Mayer. Automatic object extraction from aerial imagery - a survey focusing on buildings. Computer Vision and Image Understanding, 74(2):138 – 149, 1999. (Cited on page 28.) [63] C. Heipke, H. Mayer, C. Wiedemann, and O. Jamet. Evaluation of automatic road extraction. In Proceedings of International Archives of Photogrammetry and Remote Sensing, pages 47–56, 1997. (Cited on page 28.) [64] M. Chikr El-Mezouar, N. Taleb, K. Kpalma, and J. Ronsin. A high-resolution index for vegetation extraction in ikonos images. In Proceedings of the SPIE, volume 7824, pages 78242A–78242A–9, 2010. (Cited on page 28.) [65] N. Haala, C. Brenner, and A. Karl-heinrich. 3d urban gis from laser altimeter and 2d map data. In Proceedings of International Archives of Photogrammetry and Remote Sensing, pages 339–346, 1998. (Cited on page 28.) [66] F. Rottensteiner and Ch. Briese. Automatic generation of building models from lidar data and the integration of aerial images. In Proceedings of ISPRS working group III/3 workshop on 3-D reconstruction from airborne laserscanner and InSAR data, pages 174–180, Dresden, Germany, 2003. (Cited on page 29.) [67] D. G. Jones and J. Malik. A computational framework for determining stereo correspondence from a set of linear spatial filters. In G. Sandini, editor, Computer Vision?ECCV’92, volume 588 of Lecture Notes in Computer Science, pages 395–410. Springer Berlin Heidelberg, 1992. (Cited on pages 29 and 39.) [68] H. Lin, J. Gao, Y. Zhou, G. Lu, M. Ye, C. Zhang, L. Liu, and R. Yang. Semantic decomposition and reconstruction of residential scenes from lidar data. ACM Trans. Graph., 32(4):66:1–66:10, July 2013. (Cited on page 29.) [69] A. Laurentini. The visual hull concept for silhouette-based image understanding. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 16(2):150–162, 1994. (Cited on page 29.) 175 Bibliography [70] W. Matusik, C. Buehler, R. Raskar, S. J. Gortler, and L. Image-based visual hulls. In Proceedings of the 27th annual on Computer graphics and interactive techniques, SIGGRAPH 369–374, New York, NY, USA, 2000. ACM Press/Addison-Wesley Co. (Cited on page 30.) McMillan. conference ’00, pages Publishing [71] K.N. Kutulakos and S.M. Seitz. A theory of shape by space carving. In Proceedings of the Seventh IEEE International Conference on Computer Vision (ICCV’99), volume 1, pages 307–314 vol.1, Kerkyra, Greece, 1999. (Cited on page 31.) [72] M. Levoy and P. Hanrahan. Light field rendering. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, SIGGRAPH ’96, pages 31–42, New York, NY, USA, 1996. ACM. (Cited on pages 30 and 31.) [73] M. R. Oswald, E. Töppe, and D. Cremers. Fast and globally optimal single view reconstruction of curved objects. In Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012), pages 534–541, Providence, Rhode Island, USA, 2012. (Cited on page 30.) [74] Mukta Prasad and Andrew W. Fitzgibbon. Single view reconstruction of curved surfaces. In CVPR (2), pages 1345–1354, 2006. (Cited on page 30.) [75] A. Saxena, M. Sun, and A.Y. Ng. 3-d reconstruction from sparse views using monocular vision. In Proceedings of IEEE 11th International Conference on Computer Vision (ICCV 2007), pages 1–8, Rio de Janeiro, Brazil, 2007. (Cited on page 31.) [76] A. Saxena, M. Sun, and A.Y. Ng. Learning 3-d scene structure from a single still image. In Proceedings of IEEE 11th International Conference on Computer Vision (ICCV 2007), pages 1–8, Rio de Janeiro, Brazil, 2007. (Cited on page 31.) [77] C. Zou, J. Liu, and J. Liu. Precise 3d reconstruction from a single image. In Proceedings of the Asian Conference on Computer Vision (ACCV 2012), pages 271–282, Daejeon, Korea, 2012. (Cited on page 32.) [78] L. He and H. Liu. Shape context for image understanding. In Proceedings of the 5th WSEAS international conference on Signal, speech and image processing, SSIP’05, pages 276–281, Stevens Point, Wisconsin, USA, 2005. World Scientific and Engineering Academy and Society (WSEAS). (Cited on page 34.) 176 Bibliography [79] S. Belongie, J. Malik, and J. Puzicha. Shape matching and object recognition using shape contexts. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(4):509–522, 2002. (Cited on page 34.) [80] S. Arivazhagan, R. N. Shebiah, S.S. Nidhyanandhan, and L. Ganesan. Fruit recognition using color and texture features. Journal of Emerging Trends in Computing and Information Sciences, 1(2):90–94, January 2010. Published by Foundation of Computer Science. (Cited on page 35.) [81] X. He, R.S. Zemel, and M.A. Carreira-Perpindn. Multiscale conditional random fields for image labeling. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), volume 2, pages II–695–II–702 Vol.2, Washington, DC, USA, 2004. (Cited on page 35.) [82] J. Shotton, J. Winn, C. Rother, and A. Criminisi. Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int. J. Comput. Vision, 81(1):2–23, January 2009. (Cited on pages 35, 37, 44, 47 and 49.) [83] L. Breiman. Special invited paper. additive logistic regression: A statistical view of boosting: Discussion. The Annals of Statistics, 28(2):pp. 374–377, 2000. (Cited on page 35.) [84] C. Elkan. Using the Triangle Inequality to Accelerate k-Means. In Proceedings of the 30th International Conference on Machine Learning (ICML 2013), pages 147–153, Atlanta, GA, USA, 2003. (Cited on page 40.) [85] J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28:2000, 1998. (Cited on page 43.) [86] Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell., 23(11):1222–1239, November 2001. (Cited on page 44.) [87] Y. Boykov and V. Kolmogorov. An experimental comparison of min-cut/maxflow algorithms for energy minimization in vision. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 26(9):1124–1137, 2004. (Cited on page 44.) [88] P. Soille. Morphological Image Analysis: Principles and Applications. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2 edition, 2003. (Cited on page 51.) 177 Bibliography [89] R. Wilson, S. C. Clippingdale, and A. H. Bhalerao. Robust estimation of local orientations in images using a multiresolution approach. In Proceedings of SPIE 1360, Visual Communications and Image Processing ’90: Fifth in a Series, 1393, volume 1360, pages 1393–1403, Lausanne, Switzerland, 1990. (Cited on page 52.) [90] C. Harris and M. J. Stephens. A combined corner and edge detector. In Proceedings of the fourth Alvey Vision Conference, pages 147–152, University of Manchester, 1988. (Cited on page 66.) [91] East Lansing. Floor plan of east lansing, mi 48823. (Cited on pages 69 and 70.) [92] E. DAMOSSO. Final report of COST 231. Digital Mobile Radio towards future Generation Systems. European Comission,Bruxelles, 1999. (Cited on pages 69, 73 and 83.) [93] L. M. Kamarudin, R. B. Ahmad, B. L. Ong, F. Malek, A. Zakaria, and M. A. M. Arif. Review and modeling of vegetation propagation model for wireless sensor networks using omnet++. In Proceedings of the Second Int Network Applications Protocols and Services (NETAPPS) Conf, pages 78–83, Kedah, 2010. (Cited on page 73.) [94] J. Kenyeres, S. Sajban, P. Farkas, and M. Rakus. Indoor experiment with wsn application. In Proceedings of the 33rd Int MIPRO Convention, pages 863–866, Opatija, Croatia, 2010. (Cited on page 73.) [95] R. M. Pellegrini, S. Persia, D. Volponi, and G. Marcone. Rf propagation analysis for zigbee sensor network using rssi measurements. In Proceedings of the 2nd Int Wireless Communication, Vehicular Technology, Information Theory and Aerospace & Electronic Systems Technology (Wireless VITAE) Conf, pages 1–5, Chennai, India, 2011. (Cited on page 74.) [96] V. Kolar, S. Razak, P. Mahonen, and N. B. Abu-Ghazaleh. Measurement and analysis of link quality in wireless networks: An application perspective. In Proceedings of IEEE Conf. Computer Communications Workshops (INFOCOM 2010), pages 1–6, San Diego, CA, USA, 2010. (Cited on page 74.) [97] T. Chrysikos, G. Georgopoulos, and S. Kotsopoulos. Wireless channel characterization for a home indoor propagation topology at 2.4 ghz. In Proceedings of Wireless Telecommunications Symp (WTS 2011), pages 1–10, New York City, NY, USA, 2011. (Cited on page 74.) [98] K. R. Schaubach, N. J. Davis, and T. S. Rappaport. A ray tracing method for predicting path loss and delay spread in microcellular environments. In 178 Bibliography Proceedings of IEEE 42nd Vehicular Technology Conference, pages 932–935, Denver, CO, USA, 1992. (Cited on page 74.) [99] L. Schmitz, A.and Kobbelt. Wave propagation using the photon path map. In Proceedings of International Symposium on Performance Evaluation of Wireless Ad Hoc, Sensor, and Ubiquitous Networks (PE-WASUN 06), pages 158–161, Torremolinos, Malaga, Spain, 2006. (Cited on page 74.) [100] S. Kim, Jr. Guarino, B. J., III Willis, T. M., V. Erceg, S. J. Fortune, R. A. Valenzuela, L. W. Thomas, J. Ling, and J. D. Moore. Radio propagation measurements and prediction using three-dimensional ray tracing in urban environments at 908 mhz and 1.9 ghz. 48(3):931–946, 1999. (Cited on page 74.) [101] A. Schmitz, T. Rick, T. Karolski, L. Kobbelt, and T. Kuhlen. Beam tracing for multipath propagation in urban environments. In Proceedings of the 3rd European Conf. Antennas and Propagation (EuCAP 2009), pages 2631–2635, Berlin, Germany, 2009. (Cited on page 74.) [102] R. Wahl and G. Wolfle. Combined urban and indoor network planning using the dominant path propagation model. In Proceedings of the First European Conf. Antennas and Propagation (EuCAP 2006), pages 1–6, Nice, France, 2006. (Cited on page 74.) [103] H. Son and N. Myung. A deterministic ray tube method for microcellular wave propagation prediction model. 47(8):1344–1350, 1999. (Cited on page 74.) [104] Z. Lai, H. Song, P. Wang, H. Mu, L. Wu, and J. Zhang. Implementation and validation of a 2.5d intelligent ray launching algorithm for large urban scenarios. In Proceedings of the 6th European Conf. Antennas and Propagation (EUCAP 2012), pages 2396–2400, Prague, Czech Republic, 2012. (Cited on page 74.) [105] T. Rautiainen, R. Hoppe, and G. Wolfle. Measurements and 3d ray tracing propagation predictions of channel characteristics in indoor environments. In Proceedings of the IEEE 18th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC 2007), pages 1–5, Athens, Greece, 2007. (Cited on page 74.) [106] V. Havran. Heuristic Ray Shooting Algorithms. Ph.d. thesis, Department of Computer Science and Engineering, Faculty of Electrical Engineering, Czech Technical University in Prague, November 2000. (Cited on page 75.) [107] I. Wald. Realtime Ray Tracing and Interactive Global Illumination. PhD thesis, Computer Graphics Group, Saarland University, 2004. (Cited on page 75.) 179 Bibliography [108] J. Bikker. Ray tracing in real -Time games. PhD thesis, NHTV University, 2012. (Cited on pages 76 and 77.) [109] D. He, G. Liang, J. Portilla, and T. Riesgo. A novel method for radio propagation simulation based on automatic 3d environment reconstruction. In Proceedings of the 6th European Conf. Antennas and Propagation (EUCAP 2012), pages 1445–1449, Prague, Czech Republic, 2012. (Cited on pages 79 and 107.) [110] D. A. McNamara, C. W. I. Pistorius, and J. A. G. Malherbe. Introduction to the Uniform Geometrical Theory of Diffraction. Boston, MA: Artech House, 1990. (Cited on page 81.) [111] Z. Yun, Z. Zhang, and M.F. Iskander. A ray-tracing method based on the triangular grid approach and application to propagation prediction in urban environments. Antennas and Propagation, IEEE Transactions on, 50(5):750–758, 2002. (Cited on page 87.) [112] J. Portilla, A. de Castro, E. de la Torre, and T. Riesgo. A modular architecture for nodes in wireless sensor networks. J. UCS, 12(3):328–339, 2006. (Cited on pages 88 and 111.) [113] Greenorbs. http://www.greenorbs.org/. (Cited on page 105.) [114] H. Abelson, D. Allen, D. Coore, C. Hanson, G. Homsy, T. F. Knight Jr., R. Nagpal, E. Rauch, G. J. Sussman, and R. Weiss. Amorphous computing. Commun. ACM, 43(5):74–82, May 2000. (Cited on page 106.) [115] R Handcock, D. Swain, G. Bishop-Hurley, K. Patison, Wark T., P. Valencia, P. Corke, and C. O’Neill. Monitoring animal behaviour and environmental interactions using wireless sensor networks, gps collars and satellite remote sensing. Sensors, 9(5):3586–3603, 2009. (Cited on page 106.) [116] R. Zviedris, A. Elsts, G. Strazdins, A. Mednis, and L. Selavo. Lynxnet: Wild animal monitoring using sensor networks. In Proceedings of the 4th International Workshop on Real-World Wireless Sensor Networks (REALWSN 2010), volume 6511 of Lecture Notes in Computer Science, pages 170–173, Colombo, Sri Lanka, 2010. Springer. (Cited on page 106.) [117] Xu Xu, Weifa Liang, Tim Wark, and Jaein Jeong. Maximizing network lifetime via 3g gateway assignment in dual-radio sensor networks. In Proceedings of the 37th Annual IEEE Conference on Local Computer Networks (LCN 2012), pages 479–486, Clearwater Beach, FL, USA, 2012. (Cited on page 106.) 180 Bibliography [118] D.l. Bimschas, S. P. Fekete, S. Fischer, H. Hellbrück, A. Kröller, R. Mietz, M. Pagel, D. Pfisterer, K. Römer, and T. Teubler. Real-world g-lab: Integrating wireless sensor networks with the future internet. In Proceedings of The 6th International Conference on Testbeds and Research Infrastructures for the Development of Networks & Communities (TridentCom 2010), volume 46, pages 577–579, Berlin, Germany, 2010. (Cited on page 106.) [119] C. Perkins, E. Belding-Royer, and S. Das. Ad hoc on-demand distance vector (aodv) routing, 2003. (Cited on page 108.) [120] D. Johnson, Y. Hu, and D. Maltz. The dynamic source routing protocol (dsr) for mobile ad hoc networks for ipv4, 2007. (Cited on page 108.) [121] J. Beutel, O. Kasten, and M. Ringwald. Poster abstract: Btnodes – a distributed platform for sensor nodes. In Proceedings of the 1st International Conference on Embedded Networked Sensor Systems, pages 292 – 293, Los Angeles, California, USA, January 2003. ACM Press. (Cited on pages 109 and 110.) [122] K. Dantu, M. H. Rahimi, H. Shah, S. Babel, A. Dhariwal, and G. S. Sukhatme. Robomote: enabling mobility in sensor networks. In Proceedings of the Fourth International Symposium on Information Processing in Sensor Networks (IPSN 2005), pages 404–409, UCLA, Los Angeles, California, USA, 2005. (Cited on page 109.) [123] M. B. McMickell, B. Goodwine, and L. A. Montestruque. Micabot: a robotic platform for large-scale distributed robotics. In Proceedings of the 2003 IEEE International Conference on Robotics and Automation (ICRA 2003), pages 1600–1605, Taipei, Taiwan, 2003. (Cited on page 109.) [124] S. Bergbreiter and K. S. J. Pister. Cotsbots: an off-the-shelf platform for distributed robotics. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003)., pages 1632–1637, Nevada, USA, 2003. (Cited on page 109.) [125] A. Savvides and M. B. Srivastava. A distributed computation platform for wireless embedded sensing. In Proceedings of the 20th International Conference on Computer Design (ICCD 2002), pages 220–225, Freiburg, Germany, 2002. (Cited on page 109.) [126] R. M. Kling. Intel mote: An enhanced sensor network node. In Proceedings of the International Workshop on Advanced Sensors, Structural Health Monitoring, and Smart Structures, pages 1–6, Raiosha, Japan, November 2003. (Cited on page 110.) 181 Bibliography [127] Waspmote - wireless sensor networks 802.15.4 zigbee mote. http://www. libelium.com/products/waspmote. (Cited on page 111.) [128] Wismote. http://www.aragosystems.com/en/wisnet-item/ wisnet-wismote-item.html. (Cited on page 111.) [129] Seed-eye. http://www.evidence.eu.com/products/seed-eye. html. (Cited on page 111.) [130] Tyndall mote. http://www.tyndall.ie/content/ wireless-sensor-networks. (Cited on page 111.) [131] B. Blaszczyszyn and B. Radunovic. Using transmit-only sensors to reduce deployment cost of wireless sensor networks. In Proceedings of the 27th IEEE International Conference on Computer Communications, Joint Conference of the IEEE Computer and Communications Societies (INFOCOM 2008), pages 1202–1210, Phoenix, AZ, USA, 2008. (Cited on page 112.) [132] M. Mafuta, M. Zennaro, A. Bagula, G. Ault, H. Gombachika, and T. Chadza. Successful deployment of a wireless sensor network for precision agriculture in malawi. In Proceedings of the 3rd IEEE International Conference on Networked Embedded Systems for Every Application ( NESEA 2012 ), pages 1–7, Liverpool, UK, 2012. (Cited on page 115.) [133] K. Langendoen, A. Baggio, and O. Visser. Murphy loves potatoes: experiences from a pilot sensor network deployment in precision agriculture. In Proceedings of the 20th international conference on Parallel and distributed processing, IPDPS’06, pages 174–174, Washington, DC, USA, 2006. IEEE Computer Society. (Cited on page 115.) [134] S. Lau, T. Chang, S. Hu, H. Huang, L. Shyu, C. Chiu, and P. Huang. Sensor networks for everyday use: the bl-live experience. In Proceedings of IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC 2006), volume 1, pages 336–343, Taichung, Taiwan, 2006. (Cited on page 115.) [135] K. Romer and F. Mattern. The design space of wireless sensor networks. Wireless Communications, IEEE, 11(6):54–61, 2004. (Cited on page 117.) [136] F. Dai and J. Wu. On constructing k-connected k-dominating set in wireless networks (ipdps 2005). In Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium, pages 81a–81a, Denver, Colorado, USA, 2005. (Cited on pages 117 and 118.) 182 Bibliography [137] Y. Wu and Y. Li. Construction algorithms for k-connected m-dominating sets in wireless sensor networks. In Proceedings of the 9th ACM International Symposium on Mobile Ad Hoc Networking and Computing, MobiHoc ’08, pages 83–90, New York, NY, USA, 2008. ACM. (Cited on page 117.) [138] X. Bai, D. Xuan, Z. Yun, T. H. Lai, and W. Jia. Complete optimal deployment patterns for full-coverage and k-connectivity (k≤6) wireless sensor networks. In Proceedings of the 9th ACM International Symposium on Mobile Ad Hoc Networking and Computing, MobiHoc ’08, pages 401–410, New York, NY, USA, 2008. ACM. (Cited on page 118.) [139] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: Nsga-ii. Evolutionary Computation, IEEE Transactions on, 6(2):182–197, 2002. (Cited on page 121.) [140] Danping He, Jorge Portilla, and Teresa Riesgo. A 3d multi-objective optimization planning algorithm for wireless sensor networks. In Proceedings of the 39th Annual Conference of the IEEE Industrial Electronics Society (IECON 2013), Vienna, Austria, 2013. (Cited on page 130.) [141] G. Mujica, V. Rosello, J. Portilla, and T. Riesgo. Hardware-software integration platform for a wsn testbed based on cookies nodes. In Proceedings of the 38th Annual Conference on IEEE Industrial Electronics Society (IECON 2012), pages 6013–6018, Montreal, Quebec, Canada, 2012. (Cited on page 143.) [142] Wsn dpcm project. http://www.wsn-dpcm.eu/. (Cited on page 168.) 183