Team 19: Kegan Leberman Nathan Leduc Chad Malinowski Tom Van Noord Final Report May 11th, 2016 © 2016, Calvin College and Nathan Leduc, Chad Malinowski, Kegan Leberman, and Tom Van Noord. 1 Executive Summary Project Brösel is an interactive whiteboard system which doubles as a means of controlling the computer’s cursor and mouse buttons from a distance. The goal of the project is to make the smart classroom available to all students and teachers by producing a high quality interactive whiteboard at a price which is affordable to both high­budget and low­budget schools. While the device that the design team produced is useful primarily to teachers and students, it can also be used in a business context. The target market for this device is focused primarily on schools and businesses as those are the markets that will find the system most useful and applicable. In order to be competitive in this market, the team is focusing on several key areas. The goal is to use as much of the existing technology in the classroom as possible to reduce waste and to limit how much the consumer needs to purchase. The system that the team is producing is to be useable in any classroom environment that has a projector and a screen or similar surface on which to project. Since the technology is integratable into an existing classroom, the project does not involve redesigning expensive technologies such as projectors and integrated whiteboards which would inflate the cost of the system. Thus, the products can be sold at lower price. Along with a lower selling price, the designed technology combines important features together so that a classroom is better able to use the technology effectively. Additionally, by combining several features into one system, the amount of hardware necessary to accomplish these functions is reduced which, in turn, results in a more sustainable solution. Project Brösel is comprised of four members: Kegan Leberman, Nathan Leduc, Chad Malinowski, and Tom Van Noord. All four of them are senior engineering students in the Electrical and Computer Concentration at Calvin College. This document describes the design process and decisions that the team has made over the past year and describes in detail the system prototype they have produced and plans for future revisions. The team determined that the best method of implementing the design was to have two camera sensors on the plane of the board that communicate with the computer as well as a stylus which the user would hold. The cameras send image data to the computer to be processed and the stylus outputs location data and button press data. The stylus has a gyroscope which enables the gyro mouse portion of the system. The stylus is also the device that the cameras detect in order to enable the interactive whiteboard function. All of these peripheral devices are connected to the computer which has software to process the data and translate the stylus motion into cursor movement on the computer. 2 Table of Contents Table of Figures
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 08 Table of Tables
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Abbreviations
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1 Introduction
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.1 Calvin College
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.1.1 Calvin College Engineering Department
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.1.1.1 Senior Design Course . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.2 Team Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.2.1 Team Name and Logo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.2.2 Team Members
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.2.2.1 Kegan Leberman
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.2.2 Nathan Leduc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.2.3 Chad Malinowski
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.2.2.4 Tom Van Noord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.3 Report Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.4 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.5 Project Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2 Project Requirements
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1 System Requirements
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.1 Functional Requirements
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.2 Electrical Requirements
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.3 Software Requirements
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.4 Communication Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.1.5 Physical Requirements
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2 Design Norm Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.1 Stewardship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.2 Trust . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.3 Justice
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3 Project Management
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.1 Team Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.1.1 Team Advisor and Course Instructors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.1.2 Industrial Consultant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.2 Team Schedule Management and Division of Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2.1 Team Meetings
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2.2 Task Organization​
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2.2.1 Team Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.2.3 Schedule
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.2.3.1 Milestones
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.2.3.2 Schedule Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3 3.2.3.3 Time Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.3 Budget Management
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.3.1 Sources of Funding and Budget Details
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.4 Method of Approach
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.4.1 Research Method
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4.2 Team Communication Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4.3 Design Method
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4.4 Testing Method
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4. System Components and Architecture
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.1 Conceptual Diagram
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 4.2 Component Summaries
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.2.1 Stylus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.2.2 Sensors
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.2.3 Motion Tracking Software
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.2.4 Implementation Software
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5. System Design Alternatives
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 5.1 Motion Tracking
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 5.1.1 Possible Solutions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 5.1.1.1 Determine Position from Acceleration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 5.1.1.2 Determine Position by Infrared Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 5.1.1.3 Determine Position by Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 5.1.1.4 Determine Position by Radar
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 5.1.1.5 Determine Position by GPS
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.1.2 Decision Criteria
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.1.3 Decision Matrix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.1.4 Final Decision
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.2 Stylus
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5.2.1 Possible Solutions and Final Component Selections
. . . . . . . . . . . . . . . . . . . . . . 34 5.2.1.1 Microprocessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5.2.1.2 Inertial Measurement Unit
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 5.2.1.3 Bluetooth Transceiver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 5.2.1.4 Battery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.2.1.5 Physical Buttons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 5.3 Peripheral Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 5.3.1 Possible Solutions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 5.3.1.1 Sensors on Side of Board
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 5.3.1.2 Sensors in Corner of Board
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.3.1.3 Sensors on Projector
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.3.2 Decision Criteria
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.3.3 Decision Matrix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.3.4 Final Decision
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.4 System Communication
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.4.1 Possible Solutions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4 5.4.1.1 Bluetooth
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 5.4.1.2 WiFi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.4.1.3 ZigBee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.4.2 Decision Criteria
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.4.3 Decision Matrix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 5.4.4 Final Decision
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.5 Implementation Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.5.1 Possible Solutions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.5.1.1 No Graphical Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.5.1.2 Create a Program that Enables Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.5.1.3 Create a Program that Overlays and Enables Interaction . . . . . . . . . . . . . . . . 49 5.5.2 Decision Criteria
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.5.3 Decision Matrix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.5.4 Final Decision
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 5.6 Design Decision Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6. System Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 6.1 Stylus
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 6.1.1 Stylus Hardware
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 6.1.1.1 Arduino Nano . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 6.1.1.2 Inertial Measurement Unit
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 6.1.1.3 Bluetooth transceiver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 6.1.1.4 Battery and Battery Charger
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 6.1.1.5 Printed Circuit Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 6.1.2 Stylus Physical Enclosure
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 6.1.3 Stylus Software
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 6.2 Sensors
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 6.3 Motion Tracking Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 6.3.1 Object Detection
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 6.3.2 Scaling
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 6.4 Implementation Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 7. Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 7.1 Stylus Testing
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 7.1.1 Durability Test
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 7.1.2 Communication Range Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.1.3 Battery Life Test
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.1.4 Bluetooth Transmission Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.2 Sensors Testing
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 7.3 Motion Tracking Software Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 7.4 Implementation Software Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 8. Prototype and Deliverables
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 8.1 Final Prototype
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 8.2 Deliverables
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 8.2.1 Fall Semester Deliverables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5 8.2.2 Spring Semester Deliverables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 8.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 9 Business Plan
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 9.1 Mission and Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 9.2 Industry Background and Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 9.2.1 Major Customer Groups
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 9.2.2 Regulatory Restrictions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 9.2.3 Significant Trends
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 9.2.4 Growth Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 9.2.5 Barriers to Entry and Exit
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 9.2.6 Key Success Factors in the Industry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 9.2.7 Outlook for the Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 9.3 Business Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 9.3.1 Desired Image and Position in Market . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 9.3.2 Company Goals and Objectives
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 9.3.2.1 Operational Goals and Objectives
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 9.3.2.2 Financial Goals and Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 9.3.3 SWOT Analysis
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 9.3.3.1 Internal Strengths
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 9.3.3.2 Internal Weaknesses
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 9.3.3.3 External Opportunities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 9.3.3.4 External Threats. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 9.3.4 Competitive Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 9.3.4.1 Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 9.3.4.2 Response
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 9.4 Company Products and Services
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 9.4.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 9.4.2 Patent and Trademark Protection
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 9.4.3 Future Products and Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 9.5 Marketing Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 9.5.1 Target Market. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 9.5.1.1 Problem To Be Solved . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 9.5.1.2 Demographic Profile
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 9.5.2 Customer’s Motivation to Buy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 9.5.3 Market Size and Trends
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 9.5.4 Advertising and Promotion
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 9.5.4.1 Message
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 9.5.4.2 Medium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 9.5.4.3 Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 9.5.5 Pricing
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 9.5.5.1 Discount Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 9.5.6 Distribution Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 9.6 Competitive Analysis
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 6 9.6.1 Existing Competitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 9.6.1.1 SMARTBoard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 9.6.1.2 Promethean Board
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 9.6.1.3 Epson BrightLink
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 9.6.2 Potential Competitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 9.6.2.1 Apple TV
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 9.6.2.1 ChromeCast
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 9.7 Management Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 9.7.1 Key Managers and Employees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 9.7.2 Experience, Skills, and Know­How They Bring to the Company
. . . . . . . . . . 94 9.7.3 Future Additions to Management Team
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 9.7.4 Board of Directors
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 9.8 Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 9.8.1 Legal Form of Ownership
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 9.8.2 Decision Making Authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 9.8.3 Compensation and Benefits Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 9.8.4 Production Methodology
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 9.8.4.1 Raw Materials and Costs
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 9.8.4.2 Key Supply Chain Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 9.8.5 Facilities
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 9.9 Financial Forecasts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 9.9.1 Key Assumptions
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 9.9.2 Financial Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 9.10 Loan and Investment Proposal
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 10 Conclusion
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 10.1 Project Results
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 10.2 Lessons Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 11 Acknowledgements
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 12 Appendices
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Appendix A: Gantt Chart
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Appendix B: Financial Statements and Product Price Estimations
. . . . . . . . . . . . . . . 104 Appendix C: Senior Design Budget Breakdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Appendix D: Technical Drawings
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 7 Table of Figures Figure 1: Image that led to the name Project Brosel
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Figure 2: Project Brösel Logo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Figure 3. Project Brösel Team Picture. From left to right: Kegan Leberman, Chad Malinowski, Nathan Leduc, and Tom Van Noord
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Figure 4: Weekly time allocation over the two semesters
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Figure 5: Time spent on each subject per week . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Figure 6: Percentage of time allocated to each subject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Figure 7: Basic system block diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Figure 8: IR LED and IR sensor prototype boards
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Figure 9: Test results from IR sensor and IR LED
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Figure 10: Arduino Nano
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Figure 11. Raspberry Pi Zero . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Figure 12: Adafruit 9DOF IMU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Figure 13: Sparkfun IMU
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Figure 14: Bluetooth module
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Figure 15: USB Bluetooth Dongle
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Figure 16: 9V Battery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Figure 17: AA Batteries
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Figure 18: Lithium Coin Batteries
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Figure 19: LiPo Battery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Figure 20: A push button option that was found to be unusable . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Figure 21: A push button that would not be able to extrude through the shell
. . . . . . . . . . . . . . . . 43 Figure 22: A surface mounted push button that was too difficult to solder onto a wire
. . . . . . . . . . 44 Figure 23: The final push button that was implemented into the design . . . . . . . . . . . . . . . . . . . . . . 44 Figure 24: Circuit Schematic for the Stylus Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Figure 25: Schematic of the final PCB design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Figure 26: Final PCB design
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Figure 27: First board design in physical form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Figure 28: View of the topside of the PCB
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 Figure 29: View of the underside of the PCB
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Figure 30: The first three­dimensional print of the physical enclosure
. . . . . . . . . . . . . . . . . . . . . . 57 Figure 31: The first prototype of the lower half of the enclosure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Figure 32: View of the inside of the top of the first enclosure
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Figure 33: Results of the second revision of the bottom half enclosure . . . . . . . . . . . . . . . . . . . . . . 59 Figure 34: Inside view of the top half of the final enclosure
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Figure 35: Bottom half stylus assembly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Figure 36: Completed stylus enclosure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Figure 37: Flow Chart of stylus software
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Figure 38: Haar­like features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Figure 39: Train image for cascade classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 8 Figure 40: Frame used to ensure object recognition
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Figure 41: Ping­pong ball being detected by the cascade classifier
. . . . . . . . . . . . . . . . . . . . . . 65 Figure 42: Image processing program loop
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Figure 43: Illustrates the effect of moving an object away from the camera in 3D space . . . . . . . . . . 66 Figure 44: Graphical representation of relationship between angle Beta and camera pixels
. . . . 68 Figure 45: Geometry used to determine the location of the object in the first quadrant
. . . . . . . . . . 69 Figure 46: Block diagram of the implementation software
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Figure 47: A broken top half enclosure after an 18’ drop
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Figure 48: Mobile Screenshot of bluetooth data transmission
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Figure 49: A Computer Aided Design (CAD) assembly of a possible sensor solution
. . . . . . . . . . 81 9 Table of Tables Table 1: Basic breakdown of spending . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Table 2: Current drawn by Arduino circuit under test conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Table 3: Battery Decision Matrix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Table 4: Sensor Position Decision Matrix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 Table 5: Decision matrix for wireless communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Table 6: Decision matrix for software design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Table 7: Trigonometric equations for each quadrant of the test surface . . . . . . . . . . . . . . . . . . . . . . 69 Table 8: Bill of Materials (BOM) of the final prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Table 9: Summary of Brösel Competition
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Table 10​
: ​
Raw materials and Costs for single and bulk purchases
. . . . . . . . . . . . . . . . . . . . . . 96 10 Abbreviations ABET ­ Accreditation Board of Engineering and Technology API ­ Application Programming Interface BSE ­ Bachelors of Science in Engineering CAD ­ Computer­Aided Design CEAC ­ Calvin Engineering Advisory Council DMM ­ Digital Multimeter DOF ­ Degrees of Freedom BOM ­ Bill of Materials DRC ­ Design Rule Check EB ­ Engineering Building ERC ­ Electrical Rule Check GPS ­ Global Positioning System GUI ­ Graphical User Interface IDE ­ Integrated Development Environment IEEE ­ Institute of Electrical and Electronics Engineers IMU ­ Inertial Measurement Unit I/O ­ Input/Output IR ­ Infrared LED ­ Light Emitting Diode LiPo ­ Lithium Polymer LLC ­ Limited Liability Company mA ­ Milliamp mAh ­ Milliamp­hour PC ­ Personal Computer PCB ­ Printed Circuit Board PPFS ­ Project Proposal Feasibility Study SBA ­ Small Business Administration SCL ­ Serial Clock SDA ­ Serial Data USB ­ Universal Serial Bus V ­ Volts 11 1 Introduction Prior to describing the analysis of the project system design and implementation as well as the business proposal for a company based on the sale of the system, background information is provided that relates to the purpose of the project, introduces the team, and discusses the Christian values­ the design norms­ upon which the project was designed. 1.1 Calvin College Calvin College, founded in 1876 by the Christian Reformed Church in North America, is a private liberal­arts college located in Grand Rapids, Michigan that “equips students to think deeply, to act justly, and to live wholeheartedly as Christ’s agents of renewal in the world.1” Consisting of a student body of 3,894 students, the college provides programs in over 100 majors and minors although the top majors, according to student enrollment, are business, nursing, biology, engineering, and education.2 1.1.1 Calvin College Engineering Department Calvin College’s Engineering Department is accredited by the Accreditation Board of Engineering and Technology (ABET) and offers a Bachelors of Science in Engineering in four distinct concentrations: Chemical, Civil and Environmental, Mechanical, and Electrical and Computer. During the 2015­16 academic year, the engineering department consisted of 467 students, 83 of which are graduating in May 2016, according to assistant registrar, Melisa Hubka. 1.1.1.1 Senior Design Course ENGR 339 and 340, Senior Design, are the capstone courses of the engineering department and coalesce the various skills and knowledge students have learned during their four years at Calvin. The senior design course focuses on applying the skills and knowledge students have learned through the design of a project relevant to their concentration. Teams are typically composed of three to five members and may include members from different concentrations if it would be beneficial to the scope of the project. 1.2 Team Information In order to understand the project better, it is important to first understand who it is that is designing the project as well as to understand why the team chose the name Project Brösel which, upon first glance, seems to have no obvious connections to the nature of the project. 1
http://www.calvin.edu/about/who­we­are/ ​
http://colleges.usnews.rankingsandreviews.com/best­colleges/calvin­college­2241 2
12 1.2.1 Team Name and Logo Project Brösel is the codename for the project and it is important to note that it is not the product name that would be used in marketing the product. In an attempt to secure the intellectual property of the design, the actual marketed name would not be chosen until the design is ready to mass produce. Still, the story as to how Team 19 chose the codename Project Brösel is rather humorous. Team 19 wants the design to be a low­cost alternative to the SMART Board but also double as a gyro mouse. Knowing this, various names were thrown around such as MotionMouse, SmartMouse, and WonderMouse. However, none of those names had the particular ring to them that made Team 19 happy. In a moment of desperation, Chad Malinowski, one of the team members on the team, typed “smartmouse” into Google Images and stumbled upon the image seen in Figure 13. The team was so amused by the intricate combination of cuteness and ridiculousness seen in the image that, from that day on, the codename for the project was Project Brösel. Figure 1: Image that led to the name Project Brösel A meaning behind the name Project Brösel was later derived from the definition of “Brösel”, which.means breadcrumbs in German. Naming the project Brösel is fitting because the purpose of the device is to enable the presenter to better illustrate his or her ideas to the audience. It is as if the presenter is leaving a trail of breadcrumbs to their ideas for the audience to follow. Additionally, breadcrumbs tend to be rather small and one of the main goals of the project is to reduce the amount of hardware needed. The logo for the project, seen in Figure 2, was designed by Jacob Meyer, a graphic design major at Calvin College graduating in 2016. 3
​
https://i.ytimg.com/vi/Pb7Woa4­qDQ/maxresdefault.jpg 13 Figure 2: Project Brösel Logo The equal sign conveys that the movement as recorded on the computer screen is identical to the user’s movements of the stylus. For a more complete understanding of this desired parallelism, please read about the project goal in section 1.5. It is composed of two identical calligraphy strokes which are notoriously difficult to imitate convincingly. This all conveys a sense of accurate replication which is the end goal of the system designed by Project Brösel. Additionally, the equal sign conveys one of the main values to which the team holds: equality in education. Project Brösel wants to design a low­cost alternative to the SMART Board ​
so that low­income schools can have the same access to educational technology as more privileged higher­income schools. 1.2.2 Team Members As previously mentioned, senior design teams are composed of three to five senior engineering students. This next section provides a short introductory biography of each of the four team members. Figure 3 shows a picture of the team members. Figure 3. Project Brösel Team Picture. From left to right: Kegan Leberman, Chad Malinowski, Nathan Leduc, and Tom Van Noord. 14 1.2.2.1 Kegan Leberman Kegan Leberman is a senior at Calvin College. He is graduating with a Bachelors of Science in Engineering (BSE) with a concentration in Electrical and Computer Engineering as well as a minor in Mathematics. Kegan is from Manchester, New Hampshire. After graduation, Kegan will begin a career as an electrical engineer for ADAC Automotive, located in Grand Rapids, Michigan. 1.2.2.2 Nathan Leduc Nathan is a senior at Calvin College and is graduating with a BSE with a concentration in Electrical and Computer Engineering as well as a minor in Mathematics. He grew up as a missionary kid and is an American and Canadian dual­citizen who was born in England and spent nine years growing up in France. Outside of engineering, Nathan enjoys rock climbing, spending time with his friends and girlfriend, and never refuses an opportunity to travel. After graduation, he will begin a career as a controls engineer at JR Automation located in Holland, Michigan. However, he hopes that, someday, his career takes him back overseas. 1.2.2.3 Chad Malinowski Along with Kegan and Nathan, Chad is graduating with an honors BSE with a concentration in Electrical and Computer Engineering as well as minors in Computer Science, Business, and Mathematics. He grew up in Ada, Michigan. In the summer of 2015, Chad had the opportunity to work as a research intern at Carnegie Mellon University. Chad’s research focus was hardware security. His interests are in controls engineering or embedded systems engineering. After graduation, Chad will work at JR Automation as a control engineer. 1.2.2.4 Tom Van Noord Tom Van Noord is also graduating with a BSE with a concentration in Electrical and Computer Engineering and a minor in Computer Science. His hometown is Ada, Michigan Through working in an engineering internship, Tom has gained experience in working with microprocessing and precision motion. After graduating, he will begin working as an electronics engineer at ZF TRW in the Ann Arbor, Michigan, area. 1.3 Report Overview This report presents a comprehensive look at Team 19’s senior design project. The report documents every stage of the project from brainstorming and research to the design process to, finally, the implementation of the design process. Additionally, the report provides a glance at the business plan which would be used to start a venture capital firm based upon the sale of the product the team has designed. 15 Specifically, the report is divided into ten primary sections. Section 1 acts as the introduction to the project in which the context of the project is highlighted as well as the problem which the team sought to solve. Section 2 details the various requirements of both the design process as well as the design itself. In Section 3, the methodology behind the design is highlighted. In this section, the team dynamics and logistics, design schedule, and project budget are discussed. Section 4 provides a broad overview of the system design while Section 5 discusses the various design alternatives and choices that were made. Section 6 acts as a follow­up to Section 4 and presents a much more in­depth look at the implementation of the system design. Section 7 explains the testing which the system underwent. In Section 8, the final products and deliverables of both semesters of the Senior Design course are presented. Section 9 deviates from the technical details of the project and focuses on a potential business plan which could be used to sell the system. Finally, Section 10 acts as the conclusion to the entire report. Two minor sections are presented after the conclusion: the team’s acknowledgements and the appendices. 1.4 Problem Statement The team noticed a gap between high­income schools and low­income schools in terms of the ease of access to educational technology. While schools located in areas with many high­income families are able to afford state of the art technology and curriculum materials, schools located in low­income neighborhoods can struggle to afford the same level of educational technology and, as such, the quality of the education that students receive may be compromised. Additionally, the majority of the current interactive whiteboard solutions require an extensive use of hardware. Early versions of the SMART Board, for example, use a large board which has resistive sensors behind it. These sensors use the pressure applied by the marker onto the board to track the location of the pen.4 Since these systems require a large amount of hardware, they are not particularly environmentally friendly. 1.5 Project Statement To rectify this situation, Project Brösel decided to design a low­cost alternative to the commonly used SMART Board. The final system that would be designed would be able to achieve all of the same basic functionality as high­end interactive whiteboards at a lower cost by reducing the amount of hardware necessary to achieve full functionality of the system. The prototype which was designed for the senior design project, however, was simply a proof of concept that the core functionality of interactive whiteboards can be achieved without the use of resistive boards. Project Brösel’s goal was to create a new interactive whiteboard that can be used in business presentations and classrooms. The device provides a method of interfacing with what is shown on the screen by manipulating the cursor position on the screen. This, in turn, would allow for software to be 4
http://www.teq.com/blog/2014/03/magic­smart­boards­touch­technology­works/#.VzERrt9VhBc 16 created which would enable a writing function. However, there would be no need for markers in order to accomplish this writing function; the user would be able to simply draw or write onto the interactive whiteboard with the stylus and the output would be displayed on the screen in the form of digital ink. Additionally, the system designed for senior design also works as a gyro mouse where the user can simply point the stylus at a location on the computer or projector screen and the computer cursor will move to that location. The hardware for the first prototype consists of two sensors and a stylus. In other words, the prototype does not involve the resistive board which other systems often use. The stylus uses a gyroscope to determine the relative motion of the device so as to enable the gyro mouse function of the project. Two buttons on the stylus act as left click and right click buttons while a third button enables and disables the gyro mouse function of the stylus. When the sensors detect the presence of the stylus, the system is able to enter a mode where it behaves like an interactive whiteboard. Conversely, when the sensors do not detect the presence of the stylus, the system will simply treat the stylus like a gyro mouse. The third button on the stylus switches the stylus’ function between the two modes. 17 2 Project Requirements Before the design process could begin, it was essential to first know what it is that is being designed. The first step in answering this question was to formulate project requirements which establish the core structure of how the design process would proceed. These requirements cover a wide range of design areas related to the project. The requirements relate to the functional design of the project, as well as the electrical, software, communication, and physical design. In addition to technical requirements, some of the project requirements also included design norms which defined the manner in which the system design would be approached in order to meet the team’s Christian beliefs. 2.1 System Requirements The requirements for the new computer interface had to meet the following requirements discussed in this section. These requirements are essential in order to successfully create a working and fully functional system which could be marketed to the target demographic ­ low­income schools. 2.1.1 Functional Requirements The basic function of the device shall be similar to both a gyro mouse and an interactive whiteboard. This means that the stylus shall operate as a standard mouse except that the stylus can perform standard mouse functions in the three dimensional space instead of the two dimensional space. In addition to these functions, when the user moves the stylus on to the projection surface, the cursor of the host computer shall match the location of the physical device. This shall enable the device to draw or write on the screen when a particular button is triggered. 2.1.2 Electrical Requirements The device shall be powered by a rechargeable battery to provide the device enough life to last for at least four hours before a recharge is required. This way, it could be used for most of a school day without needing a recharge. In order for the battery to last the entire day, the teacher could simply recharge it during an hour in which he or she does not have a class or during the school’s lunch period. 2.1.3 Software Requirements The software is the backbone of the system. It shall use all of the information gathered from the stylus and the sensors to determine the proper mouse location on the board. The software shall run in the background of the computer without interfering with other computer programs and processes. 2.1.4 Communication Requirements The stylus shall be wireless so as to provide the user enough mobility to move freely around the classroom or conference room. A 40 to 50­foot range from the computer shall provide the user plenty of room for freedom to move around during their lecture. However, to allow multiple devices to be used in 18 adjacent rooms simultaneously, the means for communication between the sensors, stylus, and host computer shall be unique. In so doing, two or more systems in close proximity to each other shall not interfere with each other or interfere with other devices or systems using a similar communication protocol. 2.1.5 Physical Requirements The stylus which the user holds shall be comfortable to hold and easy to control. A user­friendly encasing means that the user will be able to interface with the computer in an intuitive and unobtrusive manner for as long as is needed. The entire system shall be portable and easy to set­up. This would allow for the user to take the system wherever is most convenient to them at any given time. The setup time shall be less than five minutes while the teardown time shall be less than one minute. 2.2 Design Norm Requirements All four of the design team members are Christians and, as such, their Christian faith plays a crucial role as to how they want to approach the project. As a way to provide guidelines for how to honor God through their design process as well as their final product, they have chosen three design norms­ stewardship, justice, and trust­ which capture the essence of what they want to accomplish through the project. 2.2.1 Stewardship Stewardship is a key component of the design. By designing an alternative to the SMART Board that utilizes motion tracking technology instead of the standard resistive touch boards, less hardware can be used to accomplish the same core functionality. Using less materials in the system helps to preserve the earth’s resources and, thus, results in wise stewardship. 2.2.2 Trust A second key component of the manner in which the team approached the project is trust. This means that the customers who use the design must trust that the system is reliable and will work when it is needed. If the system is unreliable, it will not be used consistently and the entire project will have been for naught. 2.2.3 Justice Many interactive whiteboards can cost upwards of $5,0005. By designing a system that costs approximately $200 to manufacture and, thus, has a final price point far under $5,000, schools with a smaller budget, such as those found in inner­city neighborhoods, can benefit from the same technology as schools with a higher budget. This helps to level the playing field between schools and helps to promote a fair and just access to a high­quality education. 5
https://modernchalkboard.com/article­cost.html 19 3 Project Management This section describes the process by which the team was organized into different roles to most efficiently design the system. In addition, the section will explain the team’s methodology in terms of the manner in which the team approached the design process. 3.1 Team Organization As previously mentioned, members of the team included Kegan Leberman, Nathan Leduc, Chad Malinowski, and Tom Van Noord. As a means of both providing an adequate amount of work to each team member and ensuring that each portion of the project was completed, the project was broken into four sections; each team member worked on one of those sections. Kegan initially worked on determining the feasibility of the sensor hardware which the team had planned on using. Later, he spent time designing the plastic case for the stylus. Nathan designed the hardware for the stylus, the code to send the gyroscope data to the computer over Bluetooth, and was the primary author of the final report. Meanwhile, Chad wrote all of the software needed to receive the data from the sensors and the stylus and convert that data into the smart board and gyro mouse functions. Finally, Tom worked on the image processing software as this was the means by which the team decided to tackle the motion tracking via sensors portion of the project. Additionally, Kegan and Nathan worked together to create, maintain, and update the team’s website. While not technically part of the team, the course advisor, instructors, and industrial consultant worked very closely with the team and, as such, are presented in this section. 3.1.1 Team Advisor and Course Instructors The team faculty advisor was Mr. Mark Michmerhuizen, Assistant Professor of Electrical and Computer Engineering at Calvin College. In addition to Mr. Michmerhuizen, instructors for the Senior Design course included Dr. Jeremy Van Antwerp, Mr. Ned Nielsen, and Mr. Robert Masselink who are respectively, a professor of chemical engineering, a professor of mechanical engineering, and the teaching staff who worked with the civil engineering students. 3.1.2 Industrial Consultant The industrial consultant for the team was Mr. Eric Walstra of Gentex Corporation. The team met with him once each semester to go over the design process as well as to receive feedback on obstacles the team was facing. Mr. Eric Walstra is also a member of the Calvin Engineering Advisory Council (CEAC) with whom the team met in April for a design review. During the CEAC review, however, the team met with two other members of the CEAC: Mr. J. Voogt and Mr. P. Rozeboom. 20 3.2 Team Schedule Management and Division of Work In order to achieve the project goals in an efficient and timely manner, the team members maintained constant communication with each other and attempted to stick to a schedule throughout both semesters so as to not be pressed for time within the last final weeks of the project. This section highlights the methodology with which the team approached the scheduling of the project as well as the various steps of the project. 3.2.1 Team Meetings During the fall semester, the team had structured meetings twice a week to go over the work which had to be done as well as to brainstorm together about the project. However, during the spring semester, team meetings became more informal as each team member was working on his own individual portion of the project. Still, the team was able to remain updated on other parts of the project during designated work time during the senior design class time and at other times as needed. 3.2.2 Task Organization As previously mentioned, the team worked very closely together during the fall semester to research the various components of the project and do some preliminary design work. During the spring semester, however, the team split up and each team member worked independently on his own part of the project. The beginning of the spring semester involved a lot of preliminary design work. Specifically, for the stylus, this involved designing the circuitry needed between all the components. During this stage of the stylus development, the work done on the software involved creating the basic framework for the code by determining a means to manually alter the cursor location through the use of code. This transitioned into a phase in which the stylus’ printed circuit board (PCB) could be designed. Meanwhile, work was done on the integration software and code was written that enabled control of the cursor’s location using the data transmitted by the stylus. Finally, the stylus hardware was placed in the plastic enclosure which had been designed and the motion tracking software­ which had been developed independently throughout this whole process­ was integrated with the integration software. 3.2.2.1 Team Documentation In order to keep track of all of the various steps and components of the project, all of the reports, briefs, tables, and presentations for the project were kept in a shared folder on Google Drive. In addition, many of the design and software files were kept on the C drive of the computer in the team’s senior design room, Engineering Building (EB) 128. As a safety measure, multiple copies of important documents were kept on both the team members’ personal H drives, the shared S drive, and the local computer’s C drive. Some of the work, primarily the software which Tom and Chad worked on, were kept on the team members’ personal laptops. 21 3.2.3 Schedule Chad was the primary schedule manager for the project. He oversaw the timeline of the project and worked on figuring out in what order the various steps of the project needed to be completed. To see the full timeline for the project, please refer to Appendix A. 3.2.3.1 Milestones The project’s schedule included many hard deadlines as well as many soft deadlines. However, there were several milestones throughout the year which were put in place by Prof. Michmerhuizen and the other course instructors. The full list of course deadlines were: ­ September 11th: Team formed ­ September 16th: Project Proposed ­ September 23rd: Project objectives and requirements defined ­ October 7th: Project Proposal and Feasibility Study (PPFS) outline submitted ­ October 14th: Work breakdown structure and Gantt chart submitted ­ October 19th: Project brief for Mr. Eric Walstra submitted ­ October 22nd: First presentation to Senior Design class ­ October 28th: Project website posted ­ November 6th: Updated project poster posted by Senior Design room, EB128 ­ November 10th: Meeting with Mr. Eric Walstra ­ November 16th: PPFS draft submitted ­ November 30th: Second presentation to Senior Design class ­ December 11th: PPFS submitted ­ February 29th: Updated project brief for Mr. Eric Walstra submitted ­ March 4th: Team blurb submitted and third presentation to Senior Design class ­ March 10th: Meeting with Mr. Eric Walstra ­ March 23rd: 1­page team description for Senior Design banquet program submitted ­ April 11th: Project Brief for CEAC review submitted ­ April 15th: Presentation to CEAC committee ­ April 25th: Draft final report submitted to Prof. Michmerhuizen and fourth presentation to Senior Design class ­ April 29th: Senior Design night project poster submitted ­ May 7th: Senior Design open house and final presentation ­ May 11th: Final report submitted Many of these deadlines were deliverables which were presented to various parties who were involved with the project. The aforementioned milestones do not include soft milestones which were set in place by the team so as to help the team remain on schedule. 22 3.2.3.2 Schedule Management The schedule was managed by evaluating the team's progress and adjusted accordingly. During the fall semester, Chad­ the schedule manager­ reviewed the schedule at each meeting in order to inform the team of any upcoming deadlines or to make any needed adjustments. However, during the slightly less structured spring semester, the team simply kept each other accountable of the work that was being done on each portion of the project and informed each other of how work that was being done in one area affected work on another part of the project. 3.2.3.3 Time Allocation Figure 4 below shows an estimated timeline of the amount of work that was put into the project by week. Figure 5 displays the time allocation per area of the project by week while Figure 6 displays the percentage of time spent in each area over the course of the year. It is worth noting that these values are simply estimates of the time spent and that the margin of error for this data is +/­ 20%. In Figures 5 and 6, the project is subdivided into various parts of the project: stylus and sensors; research; presentations, posters, and briefs; image processing; integration software; and PPFS and final report. Figure 4: Weekly time allocation over the two semesters 23 Figure 5: Time spent on each subject per week Figure 6: Percentage of time allocated to each subject 24 3.3 Budget Management Tom was the budget manager and kept track of how much of the budget had been spent and what the money had been used to purchase. The budget was managed through a spreadsheet on the Google Drive that was accessible to every other team member. Every time a purchase was made, the spreadsheet was updated so that the team would know how much they could still afford to spend on the project. This next section provides an analysis of the team budget and the various components which were purchased throughout the course of the project. 3.3.1 Sources of Funding and Budget Details The team was given a budget of $500 by the Calvin College Engineering Department. Of the $500, the team spent $385.73. Table 1 below displays a broad overview of the budget expenditures. Each one of the categories in Table 1 is broken down into further detail in Appendix C. Table 1: Basic breakdown of spending Category Cost Infrared Testing $26.17 Stylus $282.66 Sensors $59.90 Shipping $17 ​
TOTAL $385.73 As will be seen further in the report, a lot of the items purchased were not actually components that were used in the final prototype. Some items were purchased but were then were unable to be implemented due to design changes while other items were simply duplicates of components that were implemented in the design. By purchasing duplicate components and yet remaining within budget, the team was able to mitigate the risk of time lost due to needing to order a replacement part should the original component be damaged. 3.4 Method of Approach As previously mentioned, the team met several times a week during the fall semester in order to better define what it wanted the project to resemble. After fully defining the project, the team moved into the research phase. This phase took up the majority of the fall semester. At the beginning of the spring semester, the team moved into the design phase during which time the team split up with each person tackling a different component of the project. The design phase was followed by the implementation phase and, finally, the testing phase. The methodology to each one of these phases is detailed below. 25 3.4.1 Research Method The research phase involved performing two separate steps: using databases and performing experiments. The first step was to acquire as much information as possible from online databases about products related to the project or systems whose design could be beneficial in helping to design the project. The Engineering Research Databases, Patents, and IEEE (Institute of Electrical and Electronics Engineers) were the primary databases used to uncover useful information related to Project Brösel as they are the databases in the Hekman Library (Calvin College’s library) most pertinent to electrical engineering. No patents were found that could be particularly useful in designing the system; however, the research led the team to explore three design alternatives for the motion tracking aspect of the project. These three alternatives considered were positioning via accelerometer and gyroscope, positioning via infrared (IR) sensors and, finally, positioning via image processing. All three of these design alternatives are described in further detail in section 5.1. After acquiring as much information as possible from the online databases, experiments were conducted in order to test out the design alternatives. These experiments enabled the team to conduct research specifically related to Project Brösel with the end goal being to determine the project's feasibility. These experiments are presented in section 5.1 along with the description of the design alternatives. 3.4.2 Team Communication Method During the fall semester, the primary method through which the team members communicated was through biweekly meetings and group text messages. During the spring semester, however, a lot of the communication was a lot more fluid in that the team members communicated whenever they saw each other whether that be in the team’s senior design room, during senior design class, or during ENGR 302, the engineering electromagnetics course which the team members all took during the spring semester. Additionally, communication also occurred through emails and text messages at times when a team member needed a quick response to a question. 3.4.3 Design Method There were several different steps and processes which the team used in order to determine the design for the system. After the research phase, the team transitioned into the preliminary design phase in which they determined the feasibility of the various alternative designs by systematically eliminating those that were infeasible and further researching those that were deemed feasible. If further research proved that the idea would not work, the idea was eliminated and the next one was considered. Once the team determined an idea would work, it was adopted into the design. This idea was then researched even further to gain a thorough understanding of the process or technology and how to effectively integrate it into the final design. This stage of the design process involved looking at documentation for the various hardware and software components that were chosen and using them to implement the design. 26 3.4.4 Testing Method Small tests were performed incrementally throughout the design, implementation, and assembly processes. For the software, this involved testing small sections of code for proper functionality and, once those sections were working well, adding more code onto the software. For the hardware, this involved using design software to check for a functional circuit design and then testing the connections between the various components throughout the assembly process. Once assembled, the system has a whole was tested for proper functionality and system integration. Each component of the system was tested multiple times and, if possible, changes were implemented to further improve the system functionality. 27 4. System Components and Architecture As previously mentioned, the system involves two sensors and a stylus. The sensors are cameras which rely on image processing to track the location of the stylus on the board. The stylus is the device which the user holds in order to write on the board as well as to remotely control the computer’s cursor. This next section of the report highlights the various components of the project and presents a bird’s eye view at their architecture and functionality. Section 5, presented further in this report, describes the manner in which the various components were chosen so as to result in the system architecture presented in this section. Section 6, meanwhile, provides a much more in­depth look at each element that was chosen to be implemented in the final design. 4.1 Conceptual Diagram The broad overview of the system is seen in Figure 7 below. As previously mentioned, the system acts as both an interactive whiteboard by using image processing to track the location of the stylus as well as a means of controlling the computer’s cursor by moving the stylus in three dimensional space. Figure 7: Basic system block diagram 28 4.2 Component Summaries The system uses three distinct pieces of hardware and two software systems. The hardware systems are the two cameras as well as the stylus while the two software systems are the motion tracking software and the implementation software which translates the data from the camera and the stylus to a cursor position on the computer. This next section provides a summary of each one of these components. 4.2.1 Stylus The stylus is the physical device which the user holds and it acts as both the “pen” which the cameras detect as well as the gyro mouse. It connects to the computer via Bluetooth and has three buttons. Two of the buttons act as left click and right buttons for the gyro mouse while a third button switches the mode by which the cursor is controlled: gyro mouse or motion tracking via image processing. It also includes a rechargeable lithium polymer (LiPo) battery. 4.2.2 Sensors The sensors are the components in the system which are attached to the screen and which are used by the motion tracking software to determine the location of the stylus on the board. For the final senior design prototype, these sensors are simple USB webcams; one webcam is attached to the top of the board while the other is fixed onto the side of the board. While USB webcams successfully accomplish the needed functionality for these sensors, further work could be done to make the cameras wireless. 4.2.3 Motion Tracking Software The motion tracking software is the program on the computer which takes the pictures received from the sensors and calculates an X and Y coordinate for the stylus based on how the stylus appears in each frame of the live webcam video feed. Once the X and Y position is determined, the coordinates are sent to the implementation software. 4.2.4 Implementation Software The implementation software is the code that utilizes the data gathered from the stylus and motion tracking software and uses the data to interact with the computer through cursor movement as well as left and right clicks. It receives raw gyroscope and button status data from the stylus and X and Y coordinates from the motion tracking software. 29 5. System Design Alternatives As with any engineering design, many different alternatives were considered for the various aspects of this project. This next section highlights the design alternatives for the motion tracking software, the stylus, the sensors, and the implementation software. While some alternatives were ruled out because they were deemed to be unrealistic within the scope of the project, other alternatives were ruled out because another option was found which performed the same functionality more efficiently or more reliably. 5.1 Motion Tracking In the process of choosing the means by which the motion tracking technology would be implemented for the senior design prototype, several criteria were kept in mind. It was essential that whichever method of motion tracking would be chosen had the ability to provide precise and consistent location data so that an accurate position could be calculated. 5.1.1 Possible Solutions Possible design alternatives to the motion tracking aspect of the project are presented below. 5.1.1.1 Determine Position from Acceleration The initial design idea was to track the device location throughout the room using accelerometers and gyroscopes. The accelerometers and gyroscopes would determine the acceleration of the device and angle of acceleration. From the gathered data, the computer would perform the double integral on the data gathered from the accelerometer in order to determine the position of the writing device.6 However, further research indicated that, while this calculation is possible, the error grows exponentially over time.7 The major source of error in this method is the inability to record initial conditions of the device without error. When taking the integral of acceleration, which yields the velocity of the device, the initial velocity of the device needs to be known. In this application, the initial velocity term would be equal to the final velocity from the previous point. However, this value will have a slight amount of error and will be continually folded into the next calculation, compounding the error as long as the device does not calibrate itself to an external point of reference to correct for any error introduced into the calculations. This error is then compounded again in taking the integral of the velocity to get the position. This situation is much like walking with a blindfold on and not removing it or not getting position data from another person nearby. For a brief period of time, the user will be able to determine where they are in three dimensional space. However, the longer the user walks with the blindfold on and does not take it 6
​
http://math2.uncc.edu/~bjwichno/spring2010/math1121/Lecture_Notes/unit_3/Lectures/lec_pos_vel_acc.htm https://arxiv.org/ftp/arxiv/papers/1311/1311.4572.pdf 7
30 off to see where they are in space, the further they get from where they think they are. Thus, the three dimensional position tracking method would be extremely inaccurate without an external point of reference. 5​
.1.1.2 Determine Position by Infrared Sensors Four IR sensors could be attached to the edge of the projection space. The IR sensors would then detect the illumination of an infrared (IR) light­emitting diode (LED) attached to the stylus. Ideally, when the user would move the stylus, the IR sensors would detect the movement of the IR LED. The distance could then be triangulated between all four sensors based on the intensity of the IR light detected by the sensors. To make the IR sensor system work, the IR LED on the stylus would need to emit a uniform light across the plane perpendicular to the board since the intensity of the light read by the photodiode is not only affected by the distance between the LED and the sensor but also the angle of the LED to the sensor. Without the LED emitting a uniform intensity of light, it would be impossible to calculate the position of the LED on the stylus. This could be a cause for concern since each user could hold the stylus differently which means that both the angle at which the device is held could change making it difficult to obtain a uniform light intensity. 5.1.1.3 Determine Position by Cameras One or two cameras could be used to determine when the user places the stylus on the board’s surface and where the device is with respect to the board at some predefined points in the projection area. The camera would record an image and the computer would process the image to determine the device location. The most difficult part about implementing this method would be placing the cameras in such a way that the cameras have a clear view of the board since an object could be placed between the camera and the stylus and, thus, block the camera’s ability to process the data. 5.1.1.4 Determine Position by Radar The use of a radar system could also determine the position of the stylus by measuring the time that light or an ultrasound wave takes to travel between the sensor and the stylus. The time it takes to travel between the handheld device and the sensors could then be used to triangulate the stylus’ position on the board. While this could prove to be a good and reliable method to fulfill the motion tracking requirement, it would require a very accurate synchronized clock between the sensors and the handheld device to track the time the wave travels and, by extension, the distance that the wave travels. This could be a rather difficult feat to accomplish. Another downside to using radar technology is that this technology is limited to tracking only one object. In order for this technology to detect multiple objects, the sensor would have to be programmed to time multiple echoes and software would have to be written to prevent the stylus from no longer listening for echoes after the first one. Another fact to consider, which could be both a downside and an upside, is that the sensor will detect a wide range of materials. Should the system be created so that the user could use his or her finger to control the computer, the fact that the sensor would detect a multitude of materials would be beneficial as the sensor would be able to detect 31 the presence of the finger. The downside of this would present itself in any situation where some other object gets between the sensor and the stylus; under this scenario, the stylus would not be detected. 5.1.1.5 Determine Position by GPS A final possible way to determine the stylus’ position on the board would be to do so via global positioning system (GPS). To determine position using GPS, the system would need access to at least three of the GPS satellites in orbit. Using the signals from these satellites, it is possible to determine the position of an object on the surface of the earth. With access to a fourth satellite, it is possible to determine the elevation as well. The problem with this method is that there is no guarantee that the system would be able to establish a stable connection with four satellites when the system is inside a building as it can sometimes be difficult to do so even the system is in the open. Another issue with the use of GPS is that there would be a delay between the data transmission and the position calculation; this delay is most likely too high to be effective for the system to be considered real­time. The final, and perhaps the most obvious, obstacle to this technology is that the stylus would be moving extremely small distances from the point of view of the GPS; moving a fraction of an inch on the board may not even register as movement on the GPS. 5.1.2 Decision Criteria The chosen sensor array must be able to provide a reasonable amount of resolution across a projected screen in a classroom. This system will take the position of the stylus that is placed on the board and send the positional data back to the computer. This data must be sufficiently accurate for the computer to accurately display the motion of the stylus on the board as cursor movement on the computer. Additionally, for the solution to be feasible within the scope of a senior design project, the motion tracking technology must be able to be implemented without having to achieve too many significant hurdles. 5.1.3 Decision Matrix While a decision matrix could have been used to analyze the various motion tracking methods, it was deemed that such a matrix was not necessary. A decision matrix is only useful in situations where there are more than one viable alternatives. However, in this case, there was really only one viable alternative. Most of the alternatives would not have been accurate enough or reliable enough and, as such, could be rejected fairly easily. 5.1.4 Final Decision Choosing which method to use for the motion tracking was fairly easy after some initial research and testing. Using the second integral of acceleration to track the position would not work due to the exponential growth in the error. The IR sensors, seen in Figure 8, initially looked promising but were determined to be infeasible due to the difficulties in achieving a wide enough range of distance where the output voltage would be beneficial to use. 32 Figure 8: IR LED and IR sensor prototype boards To confirm the infeasibility of using IR sensors to track the stylus’ motion, a test was performed to determine the relationship between LED position and sensor output. The IR LED was shone into the sensor at incrementally further distances and the voltage output by the sensor was recorded. The result of this test is seen in Figure 9. Figure 9: Test results from IR sensor and IR LED As can be seen in the figure, the relationship between the distance of the LED and the output voltage is not linear. This would make it difficult to accurately compute the location of the stylus; this would be a much easier obstacle to overcome if the relationship were linear. Additionally, the sensor simply doesn’t have enough range for it to be a feasible solution. The sensor did not record any voltage for distances closer than 11 inches from the board and, at 35 inches, the output voltage was so low that any noise in the signal would make it difficult to record the output properly. Another challenge to using IR sensors is that an IR sensor might be affected by ambient light in the room and would thus not be able to reliably detect the IR LED. Using radar and GPS as the motion tracking technology was quickly eliminated as well because of the complexities that would need to be overcome in order to make the technology work properly. The 33 synchronization of the clocks in the radar method and the connectivity problems in the GPS method render both of these methods infeasible. While the IR LED was the most promising solution aside from the image processing, the method that has been chosen is using motion tracking software via cameras to track the position of the stylus. Cameras provided multiple advantages that the other methods could not such as the ability to track multiple objects, the ability to change the exposure time, and the ability to filter out different colors of light if needed. Two downsides to the camera solution are that the use of cameras consumes a large amount of power and it requires a significant amount of data processing compared to the other alternatives.. 5.2 Stylus The stylus is the device that the user holds in order to simulate the act of “writing” on the board as well as to control the mouse remotely. In order to design this device, it was first important to determine what the functionality of the system would be. At one point, the team was considering using IR LEDs in order to perform the motion tracking feature of the system. As such, the stylus was originally going to have an IR LED on the tip of it in order for the sensors to track the stylus. However, this method of motion tracking was quickly disregarded after some initial tests. For an explanation as to why it was discarded, please refer to section 5.1.4. This section describes the component alternatives that were considered for the stylus once the motion tracking technology was chosen to be implemented via image processing. One solution that was considered to implement the functionality of the stylus was to simply retrofit an existing gyro mouse or traditional computer mouse with the technology needed to fulfill the system requirements. However, this idea was quickly discarded since the system had to be able to switch between using the gyro mouse as a stylus and using it in conjunction with the camera sensors. Doing so would require altering the gyro mouse or traditional mouse software and, since the software was not open­source, it was deemed too difficult to approach the stylus design in this manner even though doing so would have much reduced the task of designing the stylus hardware. 5.2.1 Possible Solutions and Final Component Selections The design of the stylus and the consideration of components for the stylus assumed that the motion tracking ability of the system would be performed using image processing. However, most of the hardware in the stylus pertained to the stylus’ function as a gyro mouse. As such, the components described below all pertain to enabling the gyro mouse function of the stylus. 5.2.1.1 Microprocessor The brain of the stylus consists of an Arduino Nano as seen in Figure 10. Since all of the hardware had to fit into a fairly confined area in order to make the stylus easy to hold, the Arduino Nano was a logical design choice due to its small size as well as the Arduino’s ability to easily interact with input/output (I/O) devices. 34 8
Figure 10: Arduino Nano The Raspberry Pi Zero, seen in Figure 11, was also considered as a possible option but, due to the fact that the Zero had not been on the market very long, it was deemed too difficult to acquire one because of the high demand and low supply of Raspberry Pi Zeros. Regardless of the device’s availability on the market, the Raspberry Pi Zero would not have been a good design decision since Raspberry Pis are known primarily for their computing power and less for their ability to interact with I/O devices. Figure 11. Raspberry Pi Zero9 5.2.1.2 Inertial Measurement Unit An Inertial Measurement Unit (IMU) was needed in order to track the relative motion of the stylus so that the gyro mouse functionality of the stylus would work. Several IMUs were considered but, ultimately, the 9 degree of freedom (DOF) IMU from Adafruit was chosen. This component is seen in Figure 12.. ​
http://www.etechpk.net/wp­content/uploads/2016/02/ARDUINO_NANO_03.png ​
http://betanews.com/wp­content/uploads/2015/12/Pi­Zero.jpg 8
9
35 10
Figure 12: Adafruit 9DOF IMU This IMU was chosen due to its relatively low cost of $19.95 compared to some other IMUs such as the 6 DOF IMU from Sparkfun seen in Figure 13 which was available for $39.95. While the Sparkfun IMU would have been smaller and thus could potentially result in a smaller stylus, one of the other goals of the project was to reduce the cost of the project. After considering the two options, it was decided that the reduction in size would not outweigh the reduction in cost so it was on that basis that the Adafruit IMU was chosen. 11
Figure 13: Sparkfun IMU Additionally, Adafruit is known to be highly compatible with Arduino devices while the compatibility between Sparkfun devices and Arduino is not as well documented so that was yet another reason why the Adafruit 9 DOF IMU was a good choice. ​
https://cdn­shop.adafruit.com/970x728/1714­05.jpg ​
https://cdn.sparkfun.com//assets/parts/4/3/8/2/10121­01d.jpg 10
11
36 5.2.1.3 Bluetooth Transceiver It was decided that the communication between the stylus and the computer would be done via Bluetooth. For an explanation as to why Bluetooth was chosen instead of other wireless protocols, please refer to section 5.4. Once Bluetooth was chosen to be the means by which the stylus would communicate with the computer, a Bluetooth transceiver had to be chosen. This component selection process took into account several different factors. First and foremost, the team wanted the transceiver to be able to fit onto a PCB. As such, transceivers with through­hole mounts were prioritized in the selection process. Next, the transceiver needed to be able to operate at a voltage level between 3V and 5V since that was the range of voltage at which the Arduino would most likely be operating. Taking into account these two requirements, the number of possible alternatives for the Bluetooth transceiver was limited to a surprisingly low number of options. While various Bluetooth transceivers were taken into consideration, none of them were considered to a significant enough degree that it is worth mentioning them in this report. The team decided to proceed with the HC­06 Bluetooth transceiver shown in Figure 14 which could operate at a voltage level between 2.7V and 4.2V. A benefit of this transceiver is that there was a large amount of documentation available online as to how to get the Bluetooth to successfully transmit data. Another benefit of this transceiver is that it was fairly thin which meant that it would be able to be installed into the stylus fairly easily and compactly. Figure 14: Bluetooth module The team also purchased a standard USB Bluetooth dongle, as shown in Figure 15, which should have provided the system with the ability to enable Bluetooth connectivity between the stylus and any computer into which the dongle was inserted. 37 Figure 15: USB Bluetooth Dongle 12 However, testing the dongle on the computers in the electrical engineering lab resulted in the discovery that there were certain libraries which would need to be installed onto the computers in order to make the dongle work successfully. As such, the team decided that, for the purposes of the senior design prototype, a better alternative would be to simply use computers which had a Bluetooth transceiver pre­installed into the computer. For a final product, however, the system might include a wireless Bluetooth dongle that would enable Bluetooth technology on any computer. 5.2.1.4 Battery One of the main design criteria for the stylus was that it needed to be portable. If it were not portable, its functionality as both a method of writing on the board as well as acting as a gyro mouse would be severely limited. However, while it needed to be portable, it also needed to receive power in order for the PCB inside of the stylus to work. Thus, a battery was necessary. Before figuring out which type of battery should be used, it was first important to determine the amperage needed to operate the circuit. This was the first step in determining a battery type because the amount of power in the stylus would determine what power capacity the battery would need to provide. While some battery types work well in low­power situations, other battery types are most effective in high­power scenarios. So, with the circuit constructed on a breadboard, data was taken of the circuit’s power consumption under various modes and with the power being supplied to different pins on the Arduino. (The Arduino has two pins for supply voltage; one of the pins works well with voltages over 5V while the other works better with a supply voltage under 5V.) In order to perform this test, the circuit was powered through a power supply. The voltage was set and the current was read from the power supply. Table 2 below presents the data that was taken from this test. 12
https://developer.mbed.org/media/uploads/simon/_scaled_bluetooth­dongle.jpg 38 Table 2: Current drawn by Arduino circuit under test conditions Bluetooth Connection Established No Bluetooth Connection With LED Without LED With LED Without LED Connected to +5V 47mA ­ 50mA pin with 3.7V supply 48 mA 38mA ­ 72mA* 37mA ­ 71mA* Connected to VIN 60mA ­ 66mA Pin with 7.4V supply 60mA 48mA ­ 75mA* 47mA ­ 81mA* Connected to VIN 61mA ­ 65mA pin with 9V supply 59mA 49mA ­ 81mA* 46mA ­ 81mA* Note: The asterisk (*) indicates that the current fluctuated wildly in a range between the two values. At the time at which this data was collected, it was thought that a blinking LED might be placed onto the stylus so that the motion tracking software would have something more distinguishable to track. It was thought that the LED could be set to blink in a synchronized fashion with the camera’s frame rate so as to ensure that the camera always perceived the LED but that, by not having the LED turned on continuously, the stylus could have a longer battery life. When the current was measured with the LED as part of the circuit, the current draw transitioned in a regular fashion between the two numbers seen in Table 2. However, throughout the course of the semester, the motion tracking software had to be continuously simplified and, as such, it was deemed unnecessary to have the LED as part of the stylus. Additionally, the current draw fluctuated very inconsistently during times when the Bluetooth was powered up yet didn’t have a connection established to the computer. While it is unlikely that there would ever be a situation in which the stylus was on, and by extension, the Bluetooth transceiver was on without being connected to the computer, it is still important that the current draw spiked under these conditions meaning that, should the user stop using the software but forget to turn off the device, the battery could be drained very quickly. Table 2 was useful in that it provided two key results. First, it made a lot more sense to power the stylus through the +5V pin instead of the VIN pin because, while the stylus provided the same functionality regardless of which pin was used, using the +5V pin meant that significantly less power was necessary. Secondly, once it was determined that it was preferable to power the stylus through the +5V pin, it could be seen that the average power draw for the stylus was 50mA. Multiplying this number by the desired battery life of four hours meant that a 200mAh or more battery should be used. (A 400mAh battery would be able to provide 400mA of current for one hour. However, we only need approximately 50mA of current output so a 200mAh battery would be able to provide 50mA of current for four hours.) However, to be rather conservative in the estimation of power needed, it was decided that a 500mAh 39 battery should be used. This provides a safety factor of 2.5. Finally, at this point, it was possible to begin comparing battery types since it was known how much power the battery needed to provide. The first battery that was considered was a simple 9V battery as shown in Figure 16. The benefit of the 9V battery would have been that it provides more than enough voltage to power the Arduino and the other components. While a 9V battery would have required powering the circuit through the VIN pin rather than the +5V pin, This would have been an easy change to make to the design. Figure 16: 9V Battery13 However, it would not have been preferable to power the circuit through the VIN pin since the chosen Bluetooth transceiver, the HC­06, could operate only under a voltage maximum of 4.2V. Thus, using a 9V battery would have required an additional resistor be placed in the circuit to reduce the voltage to a point where the Bluetooth transceiver could operate properly. Alternatively, the 9V battery could have been connected to the +5V pin through a linear regulator which would reduce the voltage output. However, this would have resulted in a less efficient system since there would essentially be wasted voltage in the system. Additionally, the 9V battery is large and offers a rather small mAh (approximately 500mAh for most 9V batteries14) compared to other battery types. Finally, the 9V battery could be replaced by the user but, since it is not rechargeable, it could potentially be a hassle for the user to replace the stylus’ batteries every four or so hours. The next battery that was considered to power the stylus was standard AA or AAA batteries as seen in Figure 17. The benefit of this type of battery is that it provides more than enough milliamp­hours; the typical AA/AAA battery offers 1500­2000 mAh. Providing this much power to the stylus would mean that the stylus would have a lengthy battery life. Figure 17: AA Batteries 15
13
​
https://upload.wikimedia.org/wikipedia/commons/8/8d/Duracell_9_Volt_0849.jpg ​
http://www.techlib.com/reference/batteries.html 15
http://rightbattery.com/wp­content/uploads/2013/08/1­5v­aa­duracell­alkaline­battery.jpg 14
40 However, the voltage output of this battery type is only 1.5V per battery16. Tests of the circuit demonstrated that the system stops working once the voltage reaches 2.75V. This means that two batteries would have been necessary to operate the stylus. Additionally, a downside of this battery type is that, just like the 9V battery, it would require external batteries which the user would have to provide. While the batteries would not need to be changed as often as they would for the 9V battery, they would still need to be changed decently often which, after a while, could result in a rather expensive “upkeep” cost for the user. Next, the possibility of using lithium coin batteries, as seen in Figure 18, to power the stylus was considered. The benefit of these batteries is that they are small and lightweight; they would fit perfectly into the encasing of the stylus. Figure 18: Lithium Coin Batteries17 However, this type of battery came with several considerable drawbacks. First, they are typically more expensive than other battery types as they can cost anywhere between $1­$3 per battery18. While this would not normally be a huge problem as $1­$3 is still rather inexpensive, the problem presents itself when you consider that these batteries are not rechargeable and that they typically only have 25­1000mAh and that, in order to increase the mAh rating, the size would have to decrease which could present a problem as to how it fits into the case. The relatively low mAh rating on the batteries means that $1­$3 would be invested in the system fairly regularly which, over a period of time, could result in a high maintenance cost. The final battery type that was considered was the use of a lithium polymer (LiPo) battery as seen in Figure 19. LiPo batteries are easily rechargeable via a USB cable provided that a separate charger module be added to the stylus. Additionally, LiPo batteries can provide the necessary amount of voltage to the current as most of them provide 3.7V19. Additionally, LiPo batteries are available in a variety of mAh ratings and one could be easily obtained to provide 500mAh. While they do have a higher initial cost than most other battery types, they do not need to be replaced as often since they are rechargeable. As a result, the total cost over the lifetime of the product actually makes this battery type rather affordable compared to other batteries. 16
http://www.greenbatteries.com/aa­and­aaa­battery­faq/ ​
http://www.vetco.net/catalog/images/VET­CR2032­1.jpg 18
​
https://learn.adafruit.com/all­about­batteries/lithium­batteries­and­coin­cells 19
​
http://www.rogershobbycenter.com/lipoguide/ 17
41 20
Figure 19: LiPo Battery The main factors considered in terms of which battery should be used in the stylus were the size of the stylus since a smaller battery could result in a smaller package, the cost of the battery since a lower cost would result in a lower cost to manufacture the stylus and, in turn, a lower end­cost for the consumer, the power provided since the battery had to be able to power the circuit adequately and, finally, how easy it is to recharge the battery since a rechargeable stylus would be a great perk for the consumer. The decision matrix seen in Table 3 below helped to narrow down the design alternatives to one final solution. As can be seen in Table 3, the clear winner as to which battery would be implemented into the stylus was the LiPo battery. Table 3: Battery Decision Matrix Weight 9V Battery AA/AAA Battery Lithium Coin Lithium Polymer Size 8 1 5 9 7 Cost 3 5 5 2 7 Power Provided 5 4 6 4 8 Ease of Recharge 7 3 4 2 10 TOTAL 64 113 112 187 5.2.1.5 Physical Buttons Physical buttons were used to accomplish three functions: the left click, right click, and mode switch functions. The requirements of the buttons were quite simple. The lower half of the buttons needed to be 20
​
https://cdn­shop.adafruit.com/970x728/1578­01.jpg 42 thin enough in order to be mounted to the inside wall of the top half of the stylus case without interfering with the electrical components that would be placed underneath them. Another criterion that was considered in the process of selecting a button was the ease of mounting. If a button did not have room to fit in the case or a way of mounting to the side of the case, the button would not be usable. For example, one of the buttons considered, pictured below in Figure 20, did not have any plastic contour that was not a part of the physical button itself. This made it impossible to secure the button to the side of the stylus case. Figure 20: A push button option that was found to be unusable.21 The last criterion for the button was that the moving component of the switch had to be long enough to extrude through the shell of the enclosure. The button below in Figure 21 was originally considered but was not deemed as a viable option due to the button not being long enough for the user to press it. Figure 21: A push button that would not be able to extrude through the shell.22 There were some buttons that were both able to be mounted to the side of the stylus case as well as long enough to extrude through the case but still were unable to be used due to the surface mount pins on the button being too small to reasonably solder a wire onto the pin. Since the location in the stylus case to which the buttons would be mounted was elevated from the printed circuit board, the switches had to have wires soldered to them. Initial tests on the button in Figure 22 revealed that the pins were so small that the wires simply snapped off when any force was applied to the button. 21
http://media.digikey.com/Photos/E­Switch%20Photos/KS­01Q­01_sml.jpg http://media.digikey.com/photos/CW%20Ind%20Photos/GPTS203211B_sml.jpg 22
43 Figure 22: A surface mounted push button that was too difficult to solder onto a wire.23 Finally, a push button was selected that fulfilled all of the requirements for the button. The final button implemented into the design is seen in Figure 23. This button uses through­hole pins and, in a future prototype, a move to a surface mounted button would be preferable but would require some specialized printed circuit boards for it to work properly in the current design. Figure 23: The final push button that was implemented into the design24 5.3 Peripheral Sensors The peripheral sensors are the components which track the motion of the stylus on the board. Using the data which the sensors produce, the position of the cursor on the board can be altered. As described in section 5.1, it was decided that the sensors would be cameras and that the motion tracking would be done via image processing. Even though the type of sensor had already been chosen, the location of the sensors still had to be determined. This section describes the three possible locations where the sensors could have been placed. 5.3.1 Possible Solutions The three locations where the sensors could have been placed were: on the sides of the board, on the corner of the board, or on the projector itself. Each one of these solutions would alter the manner in which the software would work. These alternatives are detailed below. 5.3.1.1 Sensors on Side of Board A side mounted sensor would require the use of two sensors. The first sensor could track the stylus on the X axis while the second sensor would track the stylus on the Y axis. Without both sensors, the 23
​
https://cdn.sparkfun.com//assets/parts/4/3/3/7/10063­01.jpg ​
https://cdn.sparkfun.com//assets/parts/2/6/2/9/09190­03­L.jpg 24
44 software would only be able to detect movement on one axis. This would be the most simple solution to implement in software but, unfortunately, would require the use of two sensors. Additionally, placing the cameras on the side of the board would require either a lens that is able to see a 180° field of view or for the cameras to be placed further away from the board so that the entire board could fit within a 90° field of view. 5.3.1.2 Sensors in Corner of Board Corner mounted sensors present the advantage of being able to detect both the X and Y axes on the board with a single sensor type. Though the algorithms to calculate the position of the stylus would be a little more complicated than if the sensors were mounted on the sides of the board, the motion tracking software could theoretically work through the use of only one camera. In contrast to placing the cameras on the side of the board, this solution would only require the camera to have a 90° field of view and would still be able to be placed very close to the surface of the board. 5.3.1.3 Sensors on Projector Sensors mounted on or near the projector have the advantage of being able to easily view the entire board and do not require a wide angle camera lens which minimizes any distortion in the image. One problem with this solution is caused because of the greater distance between the projector and the stylus compared to the other solutions. In order to detect small movements in the stylus from a distance, a high resolution camera would be needed and there would need to be extensive training done in the image recognition software so that the software knows to disregard other objects within its frame. Both of these obstacles would require more network bandwidth and processing power than the other solutions. The most significant downfall of this solution is the camera’s line of sight. The system needs the cameras to be able to see the stylus in order to calculate the position of the object. If either the user, another person, or another object obstructs the view of the camera, the system would be unable to determine a position for the object. This could result in an inconsistent data stream which could lead to the movement on the computer experiencing a delay or not being responsive to the user’s movements. This solution would also require the user to be conscious of their positioning and to hold the stylus in such a way that the camera would still be able to detect the stylus. 5.3.2 Decision Criteria The design of the sensors was based on several different factors. First, it was necessary that the sensors not get in the way of the user’s movements and that the sensors would be easy to mount on the board and move between locations if the user wishes to move the system to a different classroom. It was also necessary that the sensors have a clear view of the board. The three possible solutions listed above fit this criteria with varying degrees of success. The final decision and the means by which the decision was reached can be seen in the next section. 5.3.3 Decision Matrix In order to determine which of the three positions would be the best place to put the sensor, a decision matrix was created and can be seen in Table 4. 45 Table 4: Sensor Position Decision Matrix Weight Side of Board Corners of Board Projector Mount Positioning 4 7 9 5 View 6 7 8 9 Ease of Use 3 8 7 5 Total 13 94 105 89 5.3.4 Final Decision Looking at the decision matrix, it is clear that the corners of the board would be the most ideal location to place the sensors. Sensors mounted on the corners of the board would be out of the way of the user, would have a clear view of the board, and would be easy to move and set up. However, for the senior design prototype, it was decided that the sensors would be placed on the side of the board which is contrary to the result determined by the decision matrix. There are two primary reasons for this design choice. First, the team was unable to locate a camera which had a 90° field of view as would be necessary if the camera was placed on the corner of the board. Secondly, the team decided that there was not enough time in the semester to write the more complicated software to determine both an X and a Y position from one camera. The team knew that it would be easier to write the image recognition software if the camera was placed on the side of the board. Hence, for those two aforementioned reasons, the senior design prototype had the cameras placed on the side of the board. In future prototypes, however, work would be done to place the cameras at the corner of the board. 5.4 System Communication Another crucial component of the design is the wireless communication between the stylus and the computer. In order to most efficiently and effectively achieve this wireless communication, three communication methods were considered: Bluetooth, WiFi, and ZigBee. 5.4.1 Possible Solutions The three system communication design alternatives, their benefits, and their downsides are presented below. 5.4.1.1 Bluetooth Bluetooth is a wireless communication standard that enables data communication over short distances. Bluetooth is incorporated into a large percentage of today’s technology ranging from wireless 46 headphones and speakers to wireless keyboard and mice to hands­free calling in cars. Bluetooth would be easy to implement since it is so widely used and, as such, documentation for helping to design the system is readily available. The downside of using Bluetooth is that all computers, especially older computers, might not support the use of Bluetooth. To alleviate this problem, a specific USB dongle would be necessary to provide Bluetooth capabilities to the host computer. 5.4.1.2 WiFi WiFi is a local area wireless computer networking technology that allows enabled devices to communicate across a distance. Often, the communication frequency is on the 2.4GHz frequency, but recent developments have allowed for the 5GHz band to be used as well. The advantage to using WiFi for the device’s communication is that WiFi is a widely used technology and is thus well documented and fairly easy to implement. The disadvantage is that a WiFi network is required to make the devices work and there may not be a wireless network easily available. In contrast, a Bluetooth connection simply uses a point­to­point connection which does not require an external network. 5.4.1.3 ZigBee ZigBee is a low cost, low power communication standard. It uses a low duty cycle in order to provide a communication standard at low power usage. The communication range can reach upwards of 300 feet. ZigBee is mostly used for building network topologies as it is great at creating wireless network meshes. One downside to this technology is that, due to its relatively low bandwidth, there is a limit to how much data can be sent over the network. 5.4.2 Decision Criteria The wireless communication must be able to communicate over a distance of approximately forty to fifty feet. It must be easy for the user to set up and connect all of the devices. The communication must also be fast enough to allow for near real­time translation of the user’s movement into movement on the board. 5.4.3 Decision Matrix To assist with making the decision of which type of wireless communication to use, a decision matrix was created as shown in Table 5. Table 5: Decision matrix for wireless communication Weight Bluetooth WiFi ZigBee Speed 6 5 7 5 Bandwidth 6 5 6 2 Reliability 3 9 2 7 Total 15 87 84 63 47 5.4.4 Final Decision After considering all of the options, Bluetooth seems to fit best in accordance to the specifications outlined above. The downside to WiFi is that it relies on the customer’s network and, thus, it is more difficult to control the ability to transmit data wirelessly which could potentially lead to an unreliable system and violate the design norm of trust. By using Bluetooth, all of the components required to achieve wireless communication are contained within the system. While ZigBee could be a viable wireless communication method, its low data bandwidth might potentially cause some problems when sending image processing data over the network. Since Bluetooth has a higher bandwidth, it was deemed most reasonable to implement Bluetooth technology as the system’s means of wireless connectivity. 5.5 Implementation Software The system operates essentially as two separate means to interact with the computer’s cursor. All the data gathered from the gyro mouse hardware as well as the image processing software is collected and transmitted into a pair of X and Y coordinates. The implementation software is the software that translates the data into cursor movement and is required so that the user can interact with the presentation by either highlighting, drawing, or writing on the screen. This section will describe the type of software that was considered and the final decision that was made as to how to incorporate the implementation software. Three alternatives were considered as to how to implement the software of the system. These three solutions are slightly different in nature to the design alternatives in the previously mentioned sections. These software solutions all build on to each other so that, by selecting the third alternative, the basic premise of the first two would also be selected. 5.5.1 Possible Solutions The three software design alternatives are detailed below. 5.5.1.1 No Graphical Interface This alternative is not ideal but the software design could simply act as a mouse and interface with the different programs in the same way that a mouse would. This limits the effectiveness of the device to only being able to ‘write’ in visual editing programs such as Microsoft Paint or the pen tools in Microsoft Powerpoint. 5.5.1.2 Create a Program that Enables Writing A program that enables writing would essentially be an overlay to the existing screen. The overlay would be a clear window so that the user could see everything that is being displayed. While in this program, buttons would not be able to be clicked because the clear backdrop would be running in the foreground of the computer. 48 5.5.1.3 Create a Program that Overlays and Enables Interaction Increasing the complexity of the program further would enable the writing on the board as well as interaction with the programs in the background. This final iteration of the software would be absolutely ideal as it would allow the user the freedom to use the full functionality of the device. 5.5.2 Decision Criteria In order to emphasize the various features of the software, the user must have some sort of software installed on their computer. If nothing else, the software would implement the basic characteristics of the program. Specialized software would be especially useful in the calibration of the system and could be used to emphasize the various features of the system. The software must provide some kind of interface for calibration and would ideally provide additional features such as an ability to write with digital ink on the computer screen. 5.5.3 Decision Matrix As can be seen in the decision matrix in Table 6, each alternative is better than the previous one. Table 6: Decision matrix for software design Weight No GUI Digital ink display Overlay Program on screen Calibration 7 6 7 9 Gesture Integration 5 7 7 8 Additional Functionality 3 2 5 7 Total 15 83 99 124 5.5.4 Final Decision In a final product, creating an overlay program would emphasize the strengths of what the technology can do and provide the best experience for the user. The end product would be able to use its features much more efficiently and to a greater capacity than using a third party software or a software that does not put an emphasis on the strengths of the technology. As such, a team written software would be the best implementation of a design for this implementation. However, for the sake of the scope of a senior design project, the decision was made to not create a graphical user interface. As such, the first prototype of the system could only control the cursor. Further prototypes would add the ability to write on the computer’s monitor using digital ink. 49 5.6 Design Decision Summary Final design decisions of the project were made in five key areas: the motion tracking software, the stylus, the peripheral sensors, the system communication, and the implementation of the software. According to the decision matrices in the previous sections, the final product can be described as follows. The design consists of a stylus having three push buttons, an Arduino Nano, an IMU, and a Bluetooth transceiver for wireless transmission. The software consists of a motion tracking method using cameras that will be placed on the corner of the board although, for the senior design prototype, the cameras were placed on the side of the board. A final product would include a custom­made fully functional GUI although the senior design prototype only included a means of controlling the computer’s cursor. 50 6. System Implementation Once all of the components for the system were selected and the preliminary design was finalized, work could finally begin on implementing the design into a prototype. This next section describes the design implementation process and provides an analysis of the various components in the system. 6.1 Stylus The stylus is the component in the project which acts both as the pen that the user holds to write on the board as well as the gyro mouse. It differentiates between the two based on the status of a button press on the stylus. If the button is pressed once, it acts as a pen while if it is pressed a second time, it acts as a gyro mouse. It sends positional data to the computer via Bluetooth. The following sections present the design and operation of the stylus in more detail. 6.1.1 Stylus Hardware The primary components in the stylus are an Arduino Nano, an IMU containing a gyroscope and an accelerometer, a Bluetooth transceiver, and a battery. In addition to these components, the stylus includes three buttons, a slide switch, a battery charger, and four resistors. These were all soldered onto a PCB then attached to the physical enclosure. Figure 24 displays the overall hardware schematic for the stylus. Green wires represent a data signal while red wires represent power signals and black wires represent ground signals. Figure 24: Circuit Schematic for the Stylus Hardware 51 Each one of these hardware components as well as the final PCB and the physical enclosure for the stylus is presented in further detail below. 6.1.1.1 Arduino Nano The “brain” of the stylus consists of an Arduino Nano as seen previously in Figure 24. The Arduino can operate from either a 5.5V source or a 3.3V source which made it an ideal development board to use for interacting with the various components. The Arduino also includes both digital and analog input and output (I/O) pins and, due to this versatility, it was able to interact with both the IMU through the analog pins and the buttons and Bluetooth module via the digital pins. Power was supplied to the Arduino through the 5.5V power pin. Power was supplied to the Bluetooth transceiver and the three buttons through the 3.3V pin. Software was loaded onto the Arduino through the USB port mounted on the top of the breakout board. The specific software that was written onto the Arduino is presented in Section 6.1.3. 6.1.1.2 Inertial Measurement Unit An IMU was needed in order to track the relative motion of the stylus so that the gyro mouse functionality of the stylus would work. Several IMUs were considered but, ultimately, the 9 DOF IMU from Adafruit was chosen as seen previously in Section 5.2.1.2. For the final design, the IMU recorded the angular acceleration of the stylus and sent that data to the Arduino via the serial data (SDA) and serial clock (SCL) pins. Initially, the software on the Arduino sent data from both the accelerometer and gyroscope to the computer. However, it was quickly discovered that the movement of the cursor on the screen was much smoother if only the gyroscopic data was used. As such, the Arduino software was modified to send only the gyroscope data to the computer since the implementation software no longer needed the accelerometer data. 6.1.1.3 Bluetooth transceiver The Bluetooth transceiver, the HC­06, was set to transmit data at a rate of 9600 baud, or 9600 bits/s and it transmits the data from the IMU as well as the button presses from the Arduino to the computer. As can be seen in Figure 24, the Bluetooth transceiver pins include a pin named RXD which receives data, a pin named TXD which transmits data, a power pin, and a ground pin. The Bluetooth is powered directly from the battery as opposed to receiving power from the Arduino. As a side note, the user will need to look for the device named HC­06 and then enter the pin 1234 into the computer should the user wish to pair the stylus to a computer. This is the default name and password which came with the Bluetooth transceiver. 52 6.1.1.4 Battery and Battery Charger The battery is a LiPO type battery with 500 mAh of power at a voltage level of 4.2V when fully charged and 3.7V when discharged. It has a 1C specification meaning that the battery can handle a maximum continuous current of 500mA. However, since the system only draws approximately 50mA of current, this is not a problem. The battery is connected to the power switch which switches between either connecting the battery to the battery charger or connecting the battery to the Arduino. The battery includes a protection current which prevents the charger from over­charging the battery and prevents the battery from draining to a level under 3V; both of these scenarios can cause permanent damage to the battery. The standard life of a LiPo battery is 300­500 recharge/discharge cycles.25 The battery charger is a standard LiPo charger. It has been set to charge at a current of 500mA which is the highest charge current that the battery can safely handle since it is a 1C battery. A standard MicroUSB cable can be plugged into the charger which then, in turn, charges the battery. 6.1.1.5 Printed Circuit Board The printed circuit board for the stylus was designed using Eagle CAD and was manufactured by OSHPark, a PCB manufacturer located in Oregon. Designing the PCB first required having a fully functional schematic for the stylus hardware. The finalized schematic can be seen in Figure 25. Next, a library was developed which consisted of the various components in the stylus; specifically, this involved creating the symbol as well as the package of each part. The symbol is used for creating the schematic, also known as the circuit diagram, for the PCB while the package is the drawing for the actual physical component that will be put onto the PCB. While the symbol can be simply a rough approximation of the dimensions of the component, the package needs to be extremely accurate to the dimensions of the component since any flaws in the package will result in a faulty PCB. After the symbol and package of each component were created, a device was created for each component which assigns each pin in the symbol to a pin on the package. Next, the schematic of the board could be put together. For all intents and purposes, this schematic is simply a replica of the schematic in Figure 24. This schematic can be seen in Figure 26. Once the schematic was drawn, the electrical rule check (ERC) was conducted on the schematic. The ERC checked that there were no major electrical design flaws or errors in the schematic. 25
http://batteryuniversity.com/learn/article/how_to_prolong_lithium_based_batteries 53 Figure 25: Schematic of the final PCB design 54 Once the schematic was developed, the board was ready to be designed. After exporting the schematic into a board, a task that is very easily achieved thanks to features implemented in Eagle CAD, the various components of the schematic could be laid onto the board. The final board layout for the PCB is seen in Figure 26. The board was able to be designed as a two­layer board; the red lines represent the top traces ­ the “wires” on the PCB which connect the various components ­ while the bottom traces are represented by the blue lines. Once designed, it was verified to pass the design rule check (DRC) function on Eagle CAD. This ensured that all of the components were connected properly and that no components or traces overlapped. The width for all of the traces was 0.024” while the majority of the pins were drilled so that the holes had a diameter of 0.0354”. Figure 26: Final PCB design OSHPark was chosen to be the third­party source which would manufacture the boards because of its relatively low cost compared to other PCB manufacturers. OSHPark charged the team $5 per square inch to manufacture the PCB so, since the dimensions of the PCB came out to 4.31” by 1.09”, the team was charged $23.45 to manufacture the boards. OSHPark required a minimum of three boards per order so the team ordered three copies of their PCB. This meant that while the three boards together cost $23.45, each PCB actually cost only $7.82. OSHPark also provided its own DRC which could be imported into Eagle CAD. By passing OSHPark’s DRC, it was confirmed that OSHPark would properly be able to manufacture the PCB. Unfortunately, the team had to design two separate PCBs for the stylus because the first PCB was found to have two major design flaws and one minor design flaw. Figure 27 displays the top of the first version of the PCB with the three design flaws highlighted. Figure 27: First board design in physical form 55 The red oval in Figure 27 displays the first major design flaw. In order to reduce the size of the stylus case, the Bluetooth transceiver needed to be moved back closer to the middle of the board. Since the Bluetooth transceiver was to be mounted directly on top of the PCB but facing toward the back of the stylus as can be seen in Figure 27, moving the Bluetooth transceiver forward toward the middle of the PCB would allow for a smaller total length of the stylus. The next major design flaw ­ and arguably the biggest design flaw ­ in the first iteration of the PCB is seen in the pink circle. There were no pins on the PCB for the battery and the battery charger. The pins in the pink circle display the pins for the power switch. The power switch was to be connected to the battery charger, the battery, and the Arduino so that, by switching the position of the on­off switch, the user could cut off power to the Arduino and, once current was no longer flowing to the PCB, the user could charge the battery. The team attempted soldering both the battery and battery charger as well as the power switch to the six holes seen in the pink circle which meant placing two components into the same pin but, ultimately, it was determined that it would simply be better to design a new board. Finally, the minor design flaw, as seen in the green oval, was the addition of pins for the I2C connections on the Arduino. These pins were not needed as the I2C pins faced upward on the Arduino rather than downward. By eliminating these unnecessary pins, there would be more room on the PCB for the traces. To rectify the design flaws, a second iteration of the PCB was created. This second design is the one shown in Figure 28. Figures 28 and 29, seen below, display the top and bottom of the second PCB design. Figure 28: View of the topside of the PCB 56 Figure 29: View of the underside of the PCB Before soldering components onto the PCB, a Digital Multimeter (DMM) was used to ensure that all of the components and pins were connected properly. During the assembly process, each connection between the various pins was verified for proper connectivity every time a new component was soldered onto the PCB. By performing these two tests, the proper functionality of the PCB was checked multiple times throughout the process and, in so doing, the risk for error could be mitigated. 6.1.2 Stylus Physical Enclosure The process of designing the physical enclosure for the stylus began once all of the electrical components had been selected and a rough size of the printed circuit board was known. Rough representations of the various components on the PCB were created in Autodesk Inventor in order to visualize the size of the assembled PCB and to determine what size of object the enclosure would need to contain. The drawing of all of the PCB components along with the drawings of the physical enclosure and related assemblies can be found in Appendix D. Having determined a rough estimate of the size that the enclosure would need to be, a rough sketch of the shape and size of the enclosure was 3D printed in order to determine if the enclosure felt comfortable to hold. The results of this print can be seen in Figure 30. Due to the manner in which the components were arranged on the PCB, the enclosure took on the rough shape of a common carpenter’s pencil. Figure 30: The first three­dimensional print of the physical enclosure 57 Once the team felt comfortable with the shape and size of the stylus, the enclosure was divided into two pieces and the interior was designed so that it would fit all of the PCB components in a compact manner. The first stage of this design process was to determine where objects would attach to the lower half of the enclosure as seen in Figure 31. The four outermost holes near the edges of the stylus were used to connect the top half of the enclosure to the lower half. The innermost six holes and two pins were used to mount the PCB and the battery charger component. The center compartment in the bottom half of the enclosure holds the battery. The remaining space houses the power switch and allows enough room for wires to slide between the PCB and the enclosure’s outer shell. Figure 31: The first prototype of the lower half of the enclosure The top half of the enclosure contains less features than the bottom half. The top simply has holes to mount the push buttons, a small cutout out for the power switch and screw holes to connect to the bottom half. In the first version of this part, pictured in Figure 32, there are two holes for the push buttons and a small recession in the shell in order to mount the button to the shell. Figure 32: View of the inside of the top of the first enclosure After the first prototype was 3D printed, the circuit board assembly and other components were then mounted to the enclosure. Several flaws, errors, and possible improvements were identified during the 58 assembly of the first 3D enclosure. Changes that were made in the second revision included the removal of two mounting points in order to make more space for wiring, the addition of an extruded feature to help hold the battery in place, several minor revisions to the area around the back of the bottom piece, as well as changes in the holes for push buttons on the top half to accommodate the button seen in Figure 33 as opposed to the button seen in Figure 32 and, finally, the addition of a hole for the mode button. The results of this revision can be seen in Figures 33 and 34. Figure 33: Results of the second revision of the bottom half enclosure Figure 34: Inside view of the top half of the final enclosure This second revision is the design that has been implemented in the senior design prototype. All of the components that are required for the functionality of the gyro mouse fit within the enclosure as seen in Figure 35. The two halves are then able to be screwed together. 59 Figure 35: Bottom half stylus assembly Finally, the stylus was painted white for aesthetic appeal as well as to differentiate the stylus from the black sphere which the motion tracking software was trained to detect. For more information on the motion tracking software, please see section 6.3. The final stylus assembly is seen in Figure 36. Figure 36: Completed stylus enclosure 6.1.3 Stylus Software The software in the stylus was written in C++ using the Arduino Processing integrated development environment (IDE). The software is rather simple since all the Arduino needs to do is record data from the IMU, monitor the buttons for anytime they’re pressed, and transmit those two pieces of data over Bluetooth. Figure 36 below displays a flowchart of the stylus’ software. 60 Figure 37: Flow chart of stylus software Recording a button press for the left click or right click buttons was simply a matter of checking whether the status of the pin was high or low ­ that is, whether or not there was current flowing to the pin. Additionally, to properly make the signal high or low, a pull­up resistor was connected to each button. Detecting whether the mode button was in camera mode or gyro mouse mode required a little bit more software. This was done by keeping a count of the number of times that the button had been pressed. If the count was divisible by two, the Arduino output that the gyro mouse was enabled. However, if the count was not divisible by two, the output was set to enable the camera mode. As can be seen in Figure 37, the software was a loop in which the values of the Y and Z axes of the gyroscope was outputted followed by the status of the buttons. However, there was a 2.5ms delay intentionally inserted into the code before the loop iterated through once more. This was necessary because, without the delay, the Bluetooth transceiver was unable to keep up with the data that the Arduino output. 61 6.2 Sensors The sensors were implemented using the OpenCV library which is an image processing library which contains many different functions that can be useful in image processing applications. The library provides a function called VideoCapture that allows the user to open the camera stream and take individual frames from that stream and process them separately. Each camera attached to the computer was given an individual index so that the user could specify from which camera to take the image frame. The sensors are simply positioned in the correct place and then the images are read. They provide no other function in the prototype. 6.3 Motion Tracking Software The motion tracking software was created in several stages. The first stage involved implementing the cascade classifier while the second stage used parsing software and, finally, the third stage integrated the mouse movement into the integration software. Each portion of the software was written using Microsoft Visual Studio and the C++ language. In order to perform the various functions, the software incorporated several external libraries including some of the basic Windows libraries in order to increase the efficiency of the mouse movement. 6.3.1 Object Detection One of the most difficult portions of this portion of the project involved creating the object detection software. The actual code for the object detection function is quite simple to understand, but the method by which the code work is difficult and tedious to implement. In order to detect an object in an image, the team chose to use a method called cascade classification which uses a file called a cascade classifier to determine if an object is in a frame.26 The cascade classifier uses multiple stages where each stage is applied to specific regions in an image that might contain the desired object and depending on whether or not the object is present, the region is either accepted or rejected. A region is accepted or rejected based on whether or not it contains Haar­like features. These Haar­like features consist of three major categories: edge features, line features, and center features. If the region of interest passes all of the stages in the classifier, the object is detected. Examples of each type of Haar­like features can be seen in Figure 38. 26
http://docs.opencv.org/2.4/modules/objdetect/doc/cascade_classification.html 62 Figure 38: Haar­like features Creating a cascade classifier involved gathering many positive images ­ images that contain the object you wish to detect ­ as well as negative images that do not contain the images for which you are looking and running a process called a “train”. The positive images can be gathered one of two ways. The first way is to manually gather the images into a single source file. It is essential that each of the images have the same dimensions in order for the train to work. The source images should also be scaled down in size. Otherwise, the train will take up too much of the computer’s memory and be unable to finish. The second way to gather pictures for the train is to create positive images from a single source image. This latter method was the one the team decided to use. During the initial design phase, the team wanted the software to be able to detect hands as well as the stylus. This proved to be infeasible as a hand is not a rigid object. The shape of one’s hand and one’s finger varies from person to person and it is therefore difficult to make a single file recognize an arbitrary hand. It is possible to create a cascade classifier that can recognize non­rigid objects like hands but creating this type of file is incredibly complex and was deemed to be out of the scope of the project. Generating numerous positive images from a single positive image was done through an OpenCV utility called the createsamples utility. This utility takes a single image and applies it to numerous negative images with various distortions. Distorting the positive image allows for a more robust train file that will recognize the image even if it is slightly distorted. This utility then outputs a vec file which can be used in the final training process. Training the cascade classifier is done using another OpenCV utility. The utility takes the positive images, negative images, and the number of desired stages and creates a cascade classifier. The train itself takes numerous hours and goes through many stages. After each stage successfully completes the train process, the next stage of the cascade classifier is trained. After the desired number of stages are completed, the utility combines each of them into a single cascade classifier which is then applied to the images. The cascade classifier that the team used was trained to look for a single image: a black dot or ball on a white background. This means that for image recognition technology used in the senior design prototype to be reliable, it was necessary to have a fixed white background behind the stylus so that the software 63 could successfully recognize the ball in the image and, from that image, translate the stylus’ movement into cursor movement. The train itself was created using approximately 2000 positive images generated from the createsamples utility. These images were then used in a twenty stage train applied to each image passed to it. The source image used for the train can be seen in Figure 39. Figure 39: Train image for cascade classifier The source image was a black dot on a white background. Thus, in order for the software to recognize the object correctly, the background of each image frame must be white. It was possible to recognize the object without using a white background but the recognition was not nearly as reliable. As a result, the team constructed a moveable white background to ensure that the ping­pong ball which was used as a real­world representation of the source image would be recognized effectively. The white frame that was constructed can be seen in Figure 40. Figure 40: Frame used to ensure object recognition 64 Applying the cascade classifier to each image returns an array detailing the status of the object in each frame. An example of the image returned by the classifier can be seen in Figure 41 below. From this array, the software can return the X and Y coordinates of the object in the picture. The values returned by this portion of the code are used in the overall calibration of the board. Calibrating the software involves taking values at each of the four corners and using these values as well as the known locations of the board to compute the result of the scaling function. This scaling function, described next, allows the software to transform the pixels data as seen by the cameras to the pixel data on the screen. Figure 41: Ping­pong ball being detected by the cascade classifier After completing the cascade classifier stage of the software, the pixel coordinates obtained by the cameras are converted to pixel locations on the screen. Then, the implementation software is used to move the mouse. Overall, the object detection software repeats in a continuous loop when it is enabled via the stylus’ mode button. The overall object detection software loop can be seen in Figure 42 below. Figure 42: Image processing program loop 65 6.3.2 Scaling Due to the nature of cameras, a scaling procedure needs to occur to convert the data obtained from the camera into data that can be used to control the cursor. When the camera takes a picture, the picture consists of an array of pixel data. Each pixel in the image can be thought of as having a specific set of X and Y coordinates that specify the pixel’s location in the image. However, this array is unable of detecting depth in an image. As such, two images are needed to determine the specific location of the object on the board since one camera could determine the position on the X axis and the other camera could determine the position on the Y axis. This would be the case if the field of view of the cameras was 180°; however, as will be seen, it is not so simple to determine the X and Y coordinates of the stylus. A second difficulty with image processing is that, as an object moves away from the source, it moves to the center of the image. In other words, as the object moves away from the camera (movement in the Z axis), the image taken by the camera records the movement as occurring on the Y axis. See Figure 43 below to illustrate this effect. Figure 43: Illustrates the effect of moving an object away from the camera in 3D space Motion of this type means that as the object moves away from the camera, its X and Y position might change. Therefore, a simple movement of the stylus will not result in the same motion of the cursor on 66 the computer screen. Without fixing this error, a vertical movement of the stylus results in a diagonal movement of the cursor and, as such, the whole monitor cannot be accessed while still maintaining within the bounds of the physical screen. Therefore, the device needs to be calibrated and the motion of the stylus needs to be translated in order to be able to use the whole computer screen using a fixed rectangular test space. After the pixel location has been determined by the cascade classifier, the pixel location is passed to the conversion software. This method takes the pixel location from the frame read by the camera and translates it into a pixel location on the screen of the computer. The algorithm for translating the camera pixel location to the pixel location on the screen of the computer consists of three steps. First, the camera pixels are converted to angles, then a system of equations is applied to the angles and, finally, the physical location is converted to a pixel location on the computer screen. During the first step of the conversion algorithm, the camera pixels are translated into α and β angles. The α angle refers to the angle between the line connecting the top camera to the midpoint of the board and the line connecting the camera to a given location on the board, while the β angle refers to the same angle except for the camera to the left of the board rather than on top of the board. The equations for both angles were derived by dividing the X and Y coordinates of the pixels in the camera’s image by the pixels per degree. Prior to the division, the zero offset and middle pixel value, mx and my , must be subtracted in order to determine the angle direction. The resulting angle calculations are shown below. α = X reading −mx− zero of f set
pixels per degree
β =
Y
−my − zero of f set
reading
pixels per degree
These equations hold true as long as the relationship between the pixels and angles is linear. Through experimentation, this assumption does hold true. The experiment consisted of recording the camera pixel locations for various points on the physical board. The X reading in the above equation is taken from the camera above the board and the Y reading is taken from the camera to the left of the board. The angles from the data points collected on the physical board were calculated by measuring the horizontal and vertical distances from the points to the cameras. The result of the experiment confirmed the assumption of the linear relationship between camera pixels and angles as shown by Figure 44. 67 Figure 44: Graphical representation of relationship between angle Beta and camera pixels The pixel locations in Figure 43 have been subtracted from the value of the pixel at the middle of the screen to show the relationship more easily. The graph shows that an angle in the positive direction has the same change in pixels as a negative angle. Thus, this confirms the assumption that the relationship between camera pixels and angles is linear. The second step of the algorithm is to use eight different trigonometric functions to calculate the physical board location. All eight equations are not used simultaneously. Two equations are used for one of the four quadrants in a standard cartesian coordinate system. One of the equations calculates the X position of the stylus while the other equation calculates the Y position. These equations assume that the length between the left camera and the center of the screen (L) and the length from the top camera to the center of the screen (H) is known. As a result, these lengths need to be measured during the prototype’s setup process. Measuring these lengths will not be necessary for the final product because the cameras will be located at the corners of the screen. This will allow for the length form the cameras to the center of the screen to be calculated during the calibration stage. The eight derived trigonometric equations are shown in Table 7. 68 Table 7: Trigonometric equations for each quadrant of the test surface Quadrant X equations Y equations I (H − Y )tan(α) [ L + Htan(α) ]tan(β)
1 + tan(α)tan(β)
II (H − Y )tan(α) [ L − Htan(α) ]tan(β)
1 − tan(α)tan(β)
III (H + Y )tan(α) [ L − Htan(α) ]tan(β)
1 + tan(α)tan(β)
IV (H + Y )tan(α) [ L + Htan(α) ]tan(β)
1 − tan(α)tan(β)
The Y equations are solved for first and then the result from the Y equations is used to solve for X. The Y equations were derived from the ratio of two tangent equations. Figure 45 illustrates the geometry used to derived both X and Y for the first quadrant of the board. Figure 45: Geometry used to determine the location of the object in the first quadrant Two right triangles can be formed where both triangles have a common angle of β . These triangles have lengths L and L + x and heights of y ′ and Y respectively. These two triangles are used as ratios to solve for y as shown below. Y = y ′( L + X
L ) 69 y ′ can be determined from rearranging the tangent of β to equate to y ′ = L * tan(β) . To continue to solve for y, x must be calculated. This is done using the strategy of similar ratio of tangents, but only using α and x. The x location is found in terms of x ′ . x = x ′( HH−Y ) x ′ can be determined from rearranging the tangent of α which equates to x ′ = H * tan(α) . The resulting equations can be substituted and simplified to the final x equation. x = (H − Y )tan(α) The x equation substitutes into the equation for y and can be rearranged into an equation that solves for y using only known values. [ L + Htan(α) ]tan(β)
Y = 1 + tan(α)tan(β) The resulting equation for Y is the final form of the Y equation that is used to determine the physical Y coordinates. The other six equations in Table 7 are derived in similar manners for the physical X and Y locations. However, some of the signs are flipped from positive to negative. The differences come from the addition or subtraction between the L and x or H and y, depending on if the location is greater or less than L or H. The last step of the algorithm for translating the camera pixel location to the pixel location on the screen is to scale the physical X and Y locations to the screen pixel locations. The scaling factor for scaling the X physical coordinate is the number of pixels on the width of the monitor divided by the physical screen height. The scaling factor for the Y physical coordinate is found in a similar manner. For the prototype, the physical screen height and width is measured after the cameras are placed into position. In the final product, the cameras would be stationed at the corner of the board which may eliminate the need for knowing the width and height of the physical board because the distances can be calculated in camera pixels instead of physical distances. The prototype the team designed uses two USB webcams to take the image streams used for image processing. These webcams work reasonably well for prototyping purposes. A final design would have cameras with wider viewing angles and a higher frame rate. The higher the framerate of the camera, the faster the object can be moved across the screen. Additionally, a future prototype of the system would include a more robust train process for the motion tracking software. Creating a train that could recognize the object without a white background would be ideal. Doing so would mean that the user would not need to have a background set up behind them in order to use the product. 70 6.4 Implementation Software The implementation software is the code that pulls all the components of the system together. It manages the Bluetooth connection between the stylus and the computer while interpreting the stylus and camera inputs. Moreover, the implementation software moves the cursor to a particular location based on the inputs. The cursor movement then allows the user to interact with computer applications such as interactive whiteboard software where the user can write on the computer screen using digital ink. The implementation software also enables the gesture control functionality from the stylus. Currently, the only gestures that are supported in the software are the “swipe right” and “swipe left” gestures which move a Powerpoint presentation to the next slide or the previous slide, respectively. The entire software is written in C++ and uses windows application programing interface (API) functions to accomplish its tasks. Figure 46 illustrates the block diagram depicting the general flow of the code. The actual code can be found on Google Drive. Figure 46: Block diagram of the implementation software 71 The implementation software starts by initializing the software's global variables and setting up the communication link between the computer’s Bluetooth and the Bluetooth transceiver in the sylus. The communication link is initialized by connecting to the correct Bluetooth communication port which is specific to each computer. This leads into the next block of the flow chart: parsing the data. The buffer was setup to be the size of a one byte character and it stores the data after the function which reads the data is executed. This choice was made because the parse algorithm is setup to interpret the data one character at a time. Parsing one character at a time allows for system reliability in case an invalid character is received. Since the product is a real­time system, any invalid data is simply ignored and the program moves onto the next set of data. Once all the data for one iteration of the stylus is collected, the program then implements the data. The program must know if the user is in the sensor mode or in the gyro mouse mode prior to implementing the data. The program does this by outputting either a 0 or a 1 depending on which control method is currently enabled. Every time the mode button on the stylus is pressed, the variable toggles between the two states. If the system is in the sensor mode, the program starts the image detection software, as described previously, to find the X and Y computer screen position to which the cursor should move. If the system is in the gyro mouse mode, the yaw and pitch gyroscope values parsed from the Bluetooth data stream are integrated over the sampling frequency to determine the amount of rotation. The distance is then scaled up to the correct sensitivity level. The X and Y pixel position of the computer screen is added to the scaled distance of rotation to result in the new location to which the cursor should move. To improve the gyro mouse function’s stability, the sensitivity around the zero values for the gyroscopic data was decreased. Any gyroscopic data ranging from ­0.08 to 0.08 was forced to zero. This allows the user to keep the pointer steady in one particular position during the presentation. Otherwise, the gyroscope will detect the small amounts of jitter in the user’s hand and move the mouse pointer slightly. This improvement made it easier for the the user to click buttons and icons on the screen. Another improvement made to allow the user to click or double click on the computer was to temporarily disable the cursor movement for one loop iteration every time the left or right click button was pressed. The improvement was necessary otherwise there was a chance that the small amount of force it takes to push the button on the stylus would cause the user’s hand to move slightly and, as a result, the cursor position would move slightly. This slight movement would cause the computer to interpret the user’s input as a click and drag instead of a single or double click. Disabling any mouse movement for one loop iteration fixed this problem. The physical movement of the mouse was conducted through an API function provided by Microsoft Windows called sendInput. When paired with the proper size buffer, this function enabled the user to move the cursor and simulate a mouse click. A separate mouse movement class was created to handle the initialization of the buffer. All the main loop has to do to move the mouse cursor is call the move mouse function and pass it the X and Y coordinates. Separate functions are provided in the class to simulate mouse clicks. The mouse movement function works by inserting the mouse commands into the 72 cursor input stream. After the new cursor position is calculated from either the gyroscopic values or object detection, the cursor is updated to the new position. The implementation software then uses the data received from the stylus to determine if any gestures were performed or if the right or left mouse buttons were pressed. The stylus implemented two gestures: the left swipe and the right swipe. Left swipe would move a presentation slide show back one slide while right swipe would move a presentation slide show forward one slide. The gestures were also implemented using the sendinput function. The function would simulate an up arrow keystroke to go to the previous slide and a down arrow keystroke for moving forward a slide. After the status of the mouse clicks and the gestures have been implemented, the program loops back to read the next data set from the stylus. The implementation software also includes the GUI to allow the user to use the full functionality of the device and to provide easy calibration. The software provides a command prompt for calibrating the prototype. In the final design, the calibration would be a more aesthetically pleasing GUI. The command prompt was chosen as the GUI for the prototype to provide the engineer who is testing the product with more “behind­the­scenes” information to ensure that the system is functioning properly. This additional “behind­the­scenes” information includes the camera pixel locations where the object was detected. The implementation software did not include the software to allow the user to write on the screen. Software for allowing the user to write and interact on the board began undergoing development in December 2015 but, due to the scope of the project, it was determined that it would be preferable to limit the device interaction with the computer to only being able to control the cursor. The actual writing on the screen would be left to open­source interactive whiteboard software such as ​
Open­Sankoré. Open­Sankoré is a program that allows the user to write on a transparent screen and allows the user to interact with the background. This software contains all the functionality that a fully working interactive whiteboard software would need to demonstrate the full potential of the Brösel system. 73 7. Testing Once the system was designed and assembled, each component underwent testing to ensure proper functionality and reliability. While some of the tests could allow the team to tweak their design depending on how well it underwent the tests, other tests simply gave the team a sense of relief that their system worked properly. Each test that the system underwent is described further in detail in these next couple pages. 7.1 Stylus Testing Several tests were performed to test the functionality of the stylus; this included testing the durability of the 3D printed case, the length of the battery life as well as the battery’s ability to recharge, the transmission of data over Bluetooth, the functionality of the PCB, and the communication range of the stylus. Additional tests were performed in order to test the functionality of the stylus but these are described in section 7.4 as any further tests dealt primarily with the stylus’ interaction with the implementation software rather than the stylus itself. 7.1.1 Durability Test The first test performed was to test the strength of the 3D printed case. To do so, the case was dropped multiple times from various heights. When the case fell onto concrete from a height ranging from 4 feet to 6 feet, the probable height at which a user would hold the stylus, the case held up perfectly and suffered no damage. However, when the stylus was dropped from the mezzanine of the engineering building onto the floor of the engineering building, a distance of 18’1”, the top of the stylus broke as seen in Figure 47. Figure 47: A broken top half enclosure after an 18’ drop The results of this test indicate that the stylus can hold up a minor fall perfectly well but it would break if subjected to a greater fall. This is perfectly acceptable though because, in most scenarios where the user would be using the device, they would not be in a position to drop it 18’. 74 7.1.2 Communication Range Test The next test that was performed on the stylus was a test to determine the communication range of the Bluetooth. To do so, a connection was first established between the computer and the stylus. Then, the stylus was continuously taken further away from the computer. The connection was still established at a distance of approximately 125ft ­ the distance, according to Google Earth, between the team’s senior design room in the engineering building and the far corner of the mezzanine in the engineering building. It is possible that the connection would have continued beyond this distance but 125ft is more than enough range for the system’s intended use in a classroom. 7.1.3 Battery Life Test The next test that the stylus underwent was a test to determine the battery length of the device. This test involved establishing a Bluetooth connection between the stylus and the computer and simply waiting to see how long it took before the stylus’ battery died. The stylus’ battery started out fully charged and died after five hours and three minutes of data transmission. Per the design requirements, the stylus was designed to last four hours. As such, five hours and three minutes exceeds the design requirement. 7.1.4 Bluetooth Transmission Test This test was performed on the stylus primarily during assembly. It involved using a terminal emulator program ­ primarily TeraTerm on the personal computer (PC) or Bluetooth Terminal on an Android phone ­ to check that the stylus hardware was correctly sending transmitting data. By performing this test multiple times during the assembly process, it could be verified that the system was working correctly during each step. The output of this test can be seen in Figure 48. Figure 48: Mobile Screenshot of bluetooth data transmission 75 7.2 Sensors Testing Sensor testing occurred in two phases. The first phase was using built in webcams to determine if the software was working correctly. This phase took place over several weeks and was used to determine if a cascade classifier would work. After determining if a classifier was acceptable the sensor testing moved on to the second phase of testing. This stage involved testing the classifier with two cameras and determining if the method would work with two cameras. 7.3 Motion Tracking Software Testing Testing the motion tracking software was interwoven with the sensor testing scheme. After determining that the cameras would work as theorized, the team began testing different classifiers using the cameras. After an acceptable classifier was trained, the tracking software was implemented inside the wider software. When implemented, the team thoroughly tested the software determining if the movements on the test screen mirrored those of the mouse on the screen. A phase of testing showed that the motion tracking software did not map the pixels returned from the camera to those of the computer screen. In order to properly map each of the pixels of the test screen onto those of the computer screen the software required an algorithm to make the correct adjustment. After thoroughly testing this portion of the software the team deemed this portion of the software acceptable. 7.4 Implementation Software Testing Testing the implementation software composed of a short duration test and a long duration test. The short duration test consisted of executing the software in gyro mouse mode and keeping the stylus still on a table. The purpose of this test was to determine how well the gyro mouse stayed in a single location and it did not drift across the screen. The results of this test led to adjustments to the X and Y coordinate calculations and disabling the mouse movement briefly during button presses as discussed in Section 6.4. The long duration test consisted of letting the software run for multiple two hour periods. The test ensured that the program is reliable over a long duration. When the test was first conducted, the software crashed after approximately seven minutes. The cause of the error was that the parsing portion of the software occasionally passed empty strings which were processed as data. This error was corrected and the software successfully executed for the multiple two hour periods. During the duration of the longer tests, the amount of memory the program took up was analyzed in order to identify any memory leaks. The memory size of the software was measured by the windows task manager and compared with Microsoft Visual Studio’s memory analyzer. The tests resulted in the conclusion that there were no memory leaks in the program. 76 8. Prototype and Deliverables A prototype was constructed of the entire Brösel system to demonstrate the proof of concept. This section describes the final prototype delivered at the end of the year and additional deliverables throughout the semester. 8.1 Final Prototype The final prototype demonstrates the feasibility of an interactive whiteboard that uses two directly adjacent cameras to determine the position of the stylus on the board. In addition, the prototype consists of the stylus that allows the user to control the board from a distance. Table 8 displays the various components of the stylus, the number of each component that is in the stylus, the price per component, the total cost of each component, and the vendor from which the component was purchased. 77 Table 8: BOM of the final prototype Description Price Quantity Cost Vendor Arduino Nano $20.19 1 $20.19 Arduino IMU $19.95 1 $19.95 Adafruit Bluetooth Transceiver $9.99 1 $9.99 Amazon LiPo Battery $7.95 1 $7.95 Adafruit LiPo Charger $6.95 1 $6.95 Adafruit On­Off Power Switch $0.95 1 $0.95 Digikey PCB $7.82 1 $7.82 OSH Park 3D Printed Case $26.39 1 $26.39 Calvin College 3­48x1/4 Screws $0.06 4 $0.24 McMaster­Carr 5­40x9/16 Screws $0.88 2 $1.76 McMaster­Carr 5­40x3/8 Screws $0.09 2 $0.18 McMaster­Carr Stylus Buttons $0.50 3 $1.50 Sparkfun Ping Pong Ball $0.05 1 $0.05 Tom Van Noord White paint $0.59 1 $0.59 Lowe’s Total Cost for Stylus $104.51 Webcams $39.99 2 $79.98 Calvin College Total Cost for Sensors $79.98 TOTAL COST $184.49 The cameras used in the prototype are two wired web cameras instead of the wireless camera sensors mentioned previously. Web cameras were used in this prototype due to problems connecting the Bluetooth module with a Raspberry Pi. The original design for the camera was to construct a sensor consisting of a camera connected to the Raspberry Pi that transmits the feed to the computer through a Bluetooth transceiver. The documentation for connecting Bluetooth to a Raspberry Pi was scarce which made the sensor difficult to construct. The team decided to substitute the camera sensors with web cameras to ensure that the prototype would be completed on time. A prototype using web cameras would 78 still provide a proof of concept of the design and, as such, the design change was not considered to be detrimental to the project’s original goals. 8.2 Deliverables At the end of both ENGR 339 and ENGR 340, the team submitted their work for the semester in the form of several reports and other deliverables. These specific deliverables are detailed below. 8.2.1 Fall Semester Deliverables The PPFS was the final report for the fall semester that outlined the proposed project as well as the team’s mission statement, objectives, and goals. This report also detailed the team’s preliminary research and tests. The report outlined what the team planned to accomplish in the following spring semester. In conjunction with a Business 357 class, a business plan and market analysis for the project and product were also conducted and included in the PPFS. Over the course of the fall semester, the team also made a few presentations to engineers, classmates, and college faculty. These presentations outlined the team’s current progress periodically throughout the semester. These presentations also provided the platform for the audience to ask questions and make suggestions regarding the project. 8.2.2 Spring Semester Deliverables Deliverables for the spring semester included a prototype, a final report, two presentations, several posters, and a project brief. Similar to the fall semester, progress presentations were made periodically to the senior class and college faculty. The final report included material from the PPFS as well as details of the work that was done during the spring semester. This included elements of the prototype design delivered in the spring semester and future work that would be done if there was more time. The prototype and posters were created to be put on display at the Engineering Senior Design Night on May 7th. 8.3 Future work Future work for the stylus would be implemented in several various ways if there were more time to work on the project. Once the hardware was tested, there was no longer any need to access the internal hardware of the stylus. Therefore, it is unnecessary to have the upper and lower pieces of the stylus screw together. In future revisions, the two pieces would snap together which would allow for a much simpler, cheaper, and cleaner enclosure. Another improvement which could be made to the stylus would be making the enclosure more circular which would provide the stylus with a more shape and feel similar to traditional markers. Improvements in reporting the status of the stylus’ battery could also be added. A flaw of the current prototype is that it is difficult to determine the remaining charge of the battery and its charging status. LEDs would be 79 integrated into future revisions to report the status of components in the stylus. The last modification for future revisions of the enclosure would be to use an injection mold to create the enclosure; this would result in savings in both manufacturing cost and time. Improvements could also be made to the PCBs in the stylus. In the current solution, four breakout boards are mounted to the main circuit board. Space could be saved by designing all of those components on a single PCB with surface mounted components. This would also provide the luxury of designing a custom board size which would make it possible to further reduce the size of the stylus enclosure. Creating a PCB that uses surface mount components rather than through­hole components increases the diversity of components that could be used. One component that would benefit from this improvement would be the push buttons. In one revision of the stylus, the surface mounted five­directional push buttons seen in Figure 22 were used but were ultimately substituted due to complications in the assembly process. However, if the PCBs used surface mount technology, these buttons could be used. The software of the system could also be improved. With more time and a larger team, several improvements could be made to the features of the software as well as its overall performance. One of the improvements would be adding to the amount of motion gestures that the system would be capable of handling and adding the functionality of having user­defined gestures in which the user would customize the gesture functionality of the stylus to meet their own needs and preferences. Improvements could be made to the overall system regarding the time it takes to process a command. Improving the response time of both the software and the hardware would mean that future versions could process more commands per minute which, in turn, would make the mouse movement seem more fluent and responsive. The most resource­intensive component of the software is the object recognition software which recognizes the stylus. Improvements could be made to the system in terms of its capability to recognize other writing systems and its ability to recognize multiple styluses simultaneously. Future work in this category would include the recognition of fingers, markers, and other writing objects. In addition, another improvement that would greatly affect the performance of the software would be to increase the efficiency of the object recognition portion of the software so that the system is able to recognize objects quicker and with more reliability. The largest focus of future work to the system would involve improvements to the system’s camera sensors. Currently, two USB webcams are hard wired to the host computer but to make the system truly portable and to achieve several of the original objectives of the project, these sensors would have to be wireless. One of the biggest challenges to accomplishing this feat is the large amount of data that has to be transmitted over a wireless network. In order to increase the number of frames the cameras process per second, the microprocessors onboard the sensors would need to pre­process the picture information. Two­way communication with the host computer would be required to retrieve commands about what information to send based on the previous location of the writing device. Similar to the stylus, this would require custom hardware on a single printed circuit board in order to keep the sensors small and light enough to mount to the projection surface. 80 The final peripheral sensor would be a single enclosure that would include an optical sensor, wireless communication, a microprocessor, a battery, and hardware that would allow the sensor to be mounted on the corners of a typical whiteboard or projector screen. A possible final sensor would look similar to Figure 49 below. Figure 49: A CAD assembly of a possible sensor solution. 81 9 Business Plan The following section provides a broad overview of the way in which a company selling the system designed by Project Brösel might conduct their business. The business plan begins by analyzing the current industry of educational technology in terms of both its demographics and the current competitors on the market. Then, the business and marketing strategies that would be implemented are described. Finally, the operations and financial forecasts of the company are disclosed. This business plan was written for a Business 357 class which Chad did not take due to having already fulfilled the requirement through classes for his business minor. Ryan Beezhold, a junior in the class who was not part of a senior design project, was assigned to this team for the class and contributed to this business report. 9.1 Mission and Vision Project Brösel’s mission is to provide teachers and presenters with a low­cost, portable, easy to use and set up interactive whiteboard or projection solution. This mission statement along with the design norms previously mentioned leads to the company treating both their employees and the end consumer kindly. Project Brösel’s vision is to help to create an educational system in which schools with a small budget are able to provide the same quality of education to students as schools with a higher budget by improving accessibility to educational technology. 9.2 Industry Background and Overview The designed system fits primarily into the educational technology industry. This industry can be divided into hardware and software sub­sectors. On the hardware side, the industry includes equipment and technology for teachers and students that improves the experience inside the classroom and seeks to facilitate learning. Examples of this type of technology include tablets, smart projectors, and interactive whiteboards. The software sub­sector includes digital content providers, online textbooks and curriculum materials, instructional support systems which help teachers organize and plan their lessons, and administrative software which facilitates the operations of the school board and administration27. 9.2.1 Major Customer Groups The majority of major customers of the educational technology industry are involved with education in some form. This includes teachers, school administrators, homeschool parents, and intermediate school districts. An intermediate school district is a government agency that works at the county or multi­county level and assists school districts with various programs and services.28 27
​
http://www.datafox.co/blog/2014/04/educational­technology­industry­analysis­key­players­future­trends/ http://www.kentisd.org/ 28
82 9.2.2 Regulatory Restrictions Due to the fact that technology is improving and changing so quickly, it is difficult to create appropriate regulations that are broad enough to be relevant to the ever­changing technology while also specific enough to be beneficial. Regulations that do exist pertain mainly to the way in which the internet is used in the classroom29. Specifically, it relates to the means by which to protect students from various online privacy and security issues while other regulations deal with the copyright issues relating to online curriculum and virtual learning environments. 9.2.3 Significant Trends As technology becomes a more prominent part of everyday life, educational technology plays an increasingly large role in society and, more specifically, in the classroom. Some of the industry­specific trends are an increase in video use and an increase in virtual learning. Videos, whether it be from YouTube, Khan Academy, or some other education­related source, are becoming more predominant in teachers’ lesson plans as, according to a 2013 study conducted by Project Tomorrow30, a national education nonprofit group based in Irvine, California, 46% of teachers use videos as part of their lessons. According to the same survey, 33% of students use online videos as a study tool to help reinforce material presented in the classroom and 23% of students access video material created by their teachers. Virtual learning, whether in K­12 education or in online courses at the university level, is an increasingly popular alternative to the traditional classroom setting as more and more students believe that they may be able get better individualized support from an online teacher than they would through a traditional classroom. Math was the most popular subject to study online while foreign language was second and science of all types was third31. 9.2.4 Growth Rate North America is currently the largest market for educational technology as it represents approximately 60% of total revenues. However, the market in the rest of the world is expected to grow faster between 2015 and 2017 as the compound annual growth rate outside of North America is expected to increase from 24.3% in 2012 to 26.9% in 2017. In contrast, the growth rate in North America will only be 15.2% although it is expected to remain the largest market for educational technology as the industry is expected to create $89.9 billion of revenue in 201732. 29
​
https://www2.ed.gov/rschstat/eval/tech/20years.pdf ​
http://www.tomorrow.org/about/mission.html 31
​
https://thejournal.com/Articles/2014/02/03/10­Major­Technology­Trends­in­Education.aspx?Page=2 32
​
http://www.datafox.co/blog/educational­technology­industry­analysis­key­players­future­trends/ 30
83 9.2.5 Barriers to Entry and Exit Some of the primary barriers to entry into the educational technology industry are the fact that some areas of the world have difficulty getting a broadband internet connection while other parts of the world are experiencing cutbacks in government spending on education.33 9.2.6 Key Success Factors in the Industry In order to succeed in the educational technology industry, it is important for the technology to be easy to use and reliable. Educators do not want to spend a lot of time fidgeting with technology as it takes away from time spent interacting with students and, as such, technology will not be used if it is not easy and intuitive to use. As with any industry, it is also important to differentiate your product from other products within the industry. There are a good number of competitors on the market and yet there still exists a good amount of opportunity for entering into the industry. This means that marketing and market research will be crucial to success within the industry. 9.2.7 Outlook for the Future The educational technology industry is still very much in its infancy as technology represents only 1.6% of the total expenditure on education. As technology improves and plays an increasingly prominent role in society, educational technology has the opportunity to become a normal and integrated part of the classroom. 9.3 Business Strategy After having researched the educational technology industry in order to get an initial sense of how to go about forming a company, a business strategy was developed. This business strategy describes the marketing strategy for the product; it also outlines the strengths, weaknesses, and management of the company as well as the preliminary financial forecasts for the future of the company. 9.3.1 Desired Image and Position in Market The company hopes to be perceived by customers as a company that will bring interactive whiteboard technology to classrooms at a reasonable price while providing a wide array of functions and features in their products. During its startup phase, Project Brösel will not initially be thought of as a company that provides products with new, cutting edge technology. Instead, the customers will perceive the company as an organization that finds creative and reliable solutions that integrate two interactive computer controllers to reduce both the economic and environmental cost to current technologies on the market. 33
​
http://www.investopedia.com/terms/e/economiesofscale.asp 84 In order to achieve this desired image, the company will initially position itself as a newcomer into the market since it does not have the reputation and pedigree of well­established name­brand companies already on the market. However, most current solutions on the market that provide a large array of functions and features in an interactive whiteboard come with a very large price tag. The company hopes to provide the same level of functionality and a higher level of versatility as current solutions at a much reduced cost. This combination of price, function, and versatility will provide the company a competitive edge as it seeks to enter into the market. After the initial startup years and once the company controls a decent portion of the total market share for interactive whiteboards, the company will then begin expanding their product line into more cutting­edge and high­end interactive whiteboard solutions. 9.3.2 Company Goals and Objectives The goal of the company is to decrease the environmental and economic costs of interactive whiteboards. To accomplish this goal, the company created a working prototype of an interactive whiteboard that will provide many of the same functions of current interactive whiteboards at a fraction of the cost. Using this prototype as the foundation for the final design, the company would continue working and improving upon the design by iterating through several different prototypes. Ultimately, the company would have a finished product that could be put on the market. While this report only presents the design of the first prototype, the team feels confident in their design and, as such, has outlined various goals which a company based upon this product would seek to achieve should the work continue into further prototypes. 9.3.2.1 Operational Goals and Objectives Operational goals of the company include plans for the expansion of the company. In order to remain competitive in a relatively new and growing industry, the company must continuously innovate their product line. The company must expand in order to release new features and quality products in a timely fashion. This would involve the hiring of additional software programmers, engineers, and the formation of a Research and Development Department. While this may be a long term goal, the company will continually make decisions with this goal in mind. 9.3.2.2 Financial Goals and Objectives The primary financial goal of Project Brösel is to start making a profit in three years or less after having generated enough income to match the company’s initial startup cost. Another financial goal of the company is to be able to sell the final product at a low cost to teachers and educators in order to achieve the vision of making interactive technologies available to more people. Generating a maximum profit while also sticking to one of the company’s core values ­ affordable interactive whiteboard solutions ­ will require a clever company organization structure and well thought­out pricing model. As such, a third financial goal is to find this optimum combination of both profit and low­cost products. 9.3.3 SWOT Analysis Every business has its own set of strengths and weaknesses. In this next section, an analysis is provided of the internal strengths, internal weaknesses, external opportunities, and external threats of Project 85 Brösel. Doing so helps the management of the company better understand how to position itself in the market in order to achieve the highest return on investment. 9.3.3.1 Internal Strengths Project Brösel has several advantages over competing companies in the industry. One advantage is that the engineering staff is new and has many fresh ideas on how to design reliable and affordable products. Another advantage is that, during the early startup days of the company, all of the engineers and employees are recent college graduates. Since they just recently left school, they have recent personal experience in the role technology plays in education. Additionally, the company has a goal that is unique from any other company; the goal is to emphasize affordability in their products so that interactive whiteboards can be commercially available to even the poorest of schools. This unique vision may be an advantage when it comes time to market the product. 9.3.3.2 Internal Weaknesses While having young and new engineers with fresh vision is an advantage to the company, it is also one of its biggest weaknesses. An engineering department full of 21 and 22 year olds lacks the experience and the funds that competitors in the industry possess. Since the firm’s engineering staff are recent graduates, this means mistakes will most likely be made that could be avoided should the staff have more experience in the engineering industry. 9.3.3.3 External Opportunities The market for interactive whiteboards has very little to provide as alternatives to the expensive name­brand solutions. Solutions that do promote a similar suite of features do not do much to reduce the price of the system while solutions that advertise a lower price point do not provide the same level of functionality as the leaders in the industry. Very few solutions aim to solve the problem of the lack of portability of interactive whiteboard solutions and those that do often do not provide the same level of functionality as less portable systems. All of this provides an opportunity for Project Brösel’s product to be first in the industry that is competitive in both the number of features offered and price. 9.3.3.4 External Threats Possibly the largest threat to the company is the competitive advantage of the pre­existing companies in the market. The large companies in the existing market have both wide customer bases and the loyalty of their customers. Reaching a portion of that market will be difficult but through differentiation, product integration, and a lower cost, the company hopes to gain a portion of the market that is not typically in the market for interactive whiteboards: low­budget schools. Another external threat is the implications of having a system that is both portable and offers a wide range of features. To achieve this goal, the product has to contains three distinct components as detailed in Section 4 of this report. Other solutions boast fewer independent components in their system and, as such, could market their systems as requiring fewer batteries, having a lower set­up time, or providing a more user­friendly and intuitive system. 86 9.3.4 Competitive Strategy The strategy to get a metaphorical foot in the door into the industry and succeed on the market is to focus on the potential interactive whiteboard customers that cannot afford the technology at the current price. As teachers and school administrators try to figure out whether or not to purchase educational technology for schools, one of the main criterion considered is the price of the system and whether or not the system provides a sufficient benefit to students to justify the money spent. By marketing to low­budget schools and emphasizing the price of the system, the company hopes to position itself in such a way that low­budget schools will want their system. However, in order to position themselves in this manner, the company needs to consider various aspects of how they present their product. Some of these aspects are described below while others are described in the marketing strategy seen in section 9.5. 9.3.4.1 Differentiation Differentiation is of utmost importance in this market. If the company cannot provide a product that stands out in a crowded market, customers will default to the well­established name­brand competitors. As previously mentioned, the company will differentiate its product through pricing. The product will reach its customers by providing an inexpensive product compared to what is currently on the market. Additionally, the company looks to differentiate itself through providing a product that is a leader in versatility and portability while providing a large suite of features. The primary focus in the differentiation strategy is to provide a solution that not only works on whiteboards but can be easily modified to interface with projection screens as well as computer monitors through an intuitive magnetic clip system. 9.3.4.2 Response Project Brösel also looks to compete on response. The company plans on doing this once the product is initially released. In many industries, a slow response to bugs, flaws, or customer feedback can ruin a company or product’s reputation. The company looks to not only respond to customer feedback quickly but to also take that feedback and provide new quality features at a rate faster than that of the competition. 9.4 Company Products and Services This next section of the report provides a description of the products that the company would sell as well as the services associated with those products and does so in a way that would be beneficial to an individual looking to establish a business based around the finalized product. For a more technical description of the products, please refer to Section 4. 9.4.1 Description As previously described, the product is a computer interface system that simulates a virtual whiteboard. Essentially, the user holds a stylus and, using the stylus, the user will be able to simulate the act of 87 writing through the use of motion tracking technology. Software will take the user’s input movement and convert it into a live update in the form of virtual ink or the motion of the cursor on the computer. Even though there are solutions similar to the device being produced by Brösel, the device is unique in ways that make it a viable product. The main differences are the price, the portability, the minimization of hardware, and the ease of installation and use. Current solutions, such as the SMART Board, require the installation of a large board. Br​
ö​
sel, on the other hand, consists of only small peripheral sensors, a stylus, and the wireless communication with the computer. The benefit of this hardware minimization is a product that is much simpler to both install and use. Customers who come from schools with low classroom budgets will benefit from this product since they will be able to get the same functionality of a SMART Board​
​
a t a fraction of the price. Fully confident in their product, the management of Br​
ö​
sel is willing to extend a two­year warranty to anyone who purchases their product. If the device stops functioning within two years, Br​
ö​
sel will replace the product with no questions asked on the condition that the user returns the malfunctioning product. 9.4.2 Patent and Trademark Protection In order to protect the intellectual property of Project Br​
ö​
sel, a request for a design patent should be placed on the system. While this has an application cost of approximately $1,50034, the return on investment would be substantial as Project Br​
ö​
sel would have the sole rights to manufacture and sell the system for fourteen years. With the growth of the educational technology industry, it is not unreasonable to think that a competitor may wish to design a similar system to the company’s but, by placing a patent on it, Project Br​
ö​
sel is ensured that they will receive all revenue from sales of the system. 9.4.3 Future Products and Services The current prototype of the system allows for the functionality of one stylus per sensor network. Future models may be developed to accommodate for the use of multiple styluses being used simultaneously. Another possibility of the system which may be incorporated in a future prototype is the option for remote partnership. This would enable the Br​
ö​
sel system to link up with another school who, using their own Br​
ö​
sel system, would be able to communicate back and forth via messages written on the board. 9.5 Marketing Strategy In order to be successful and viable as a company in the midst of an already crowded market, a successful marketing strategy is critical. Without it, the company may be unable to reach its target demographic even though it has designed and produced a great product. This next section of the report highlights the key components of a marketing strategy that would be implemented should the final design wish to be sold. 34
​
http://www.ipwatchdog.com/2015/04/04/the­cost­of­obtaining­a­patent­in­the­us/id=56485/ 88 9.5.1 Target Market The target market for the system is the education industry, ranging anywhere from kindergarten to a university level. This means that the product must appeal to three different groups: administrators, teachers, and students of various ages. The administrators are the customers with whom the company will be dealing. They are the ones that control what the school will buy for the classrooms and, as such, are the primary target for the marketing strategy. In addition to marketing to the administrators, targeting the teachers will be beneficial since the teachers will request this technology in their classroom and put pressure on the administrators to buy it. Marketing to students could also be beneficial as the students would ask the school if they could have this “cool technology” in their classrooms. 9.5.1.1 Problem To Be Solved Brösel provides a benefit to schools by enhancing both students’ engagement and interaction in the classroom. The final Brösel system would allow the teacher to interact with the lecture slides by writing notes in the margins or even on the slides themselves. These notes can then be saved to reference at a future time if any questions arise later in class or if a student was absent that day. The extra notes written on a presentation will help the students better understand the material and thus enhance their learning experience. This device also allows the presenter to use fewer devices when presenting. The presenters do not need a clicker to change slides or a special stylus to write on the board; all of these systems are bundled into one device. Brösel aims to remove the worry of budget constraints or lack of funding from the decision to buy an interactive whiteboard. The selling price of the product will be well below the competitor’s selling price. This will provide schools with an inexpensive project while still providing the schools with digital whiteboard technology that will enhance the learning environment. 9.5.1.2 Demographic Profile The demographic breakdown of the customers will consist of the baby boomers: members of generation X and generation Y. People within these two generations range from 15 to 51 years old. The average age of administrators, according to the Institute of Education Sciences, is 50 and is made up with approximately a 50/50 split between males and females35. The administrators in the target market are late baby boomers and early generation X. This means that majority of people in the generation prefer technology that does not change the daily routine and shows an immediate improvement to one’s life without needing more of a learning curve. They also are the “explorers of technology” or the generations that did not grow up with technology but are willing to adapt to it. The demographic of the teachers will be generation X and generation Y. However, some of the students also fall within the tail end of generation Y as the youngest members of generation Y are currently 16 years old. Users within this age group will be very open to the idea of educational technology36. 35
​
https://nces.ed.gov/pubs2009/2009323/tables/sass0708_2009323_p12n_03.asp ​
http://www.socialmarketing.org/newsletter/features/generation3.htm 36
89 9.5.2 Customer’s Motivation to Buy The customers will be people that are a part of the educational or business community who frequently give presentations or lectures. They will be motivated to purchase an interactive whiteboard because the interactive functions help better convey the presenter’s ideas. The interactive whiteboard users will be motivated to purchase the Brösel system instead of the competitors’ products because Brösel offers the same functionality at a low cost. 9.5.3 Market Size and Trends The interactive whiteboard market growth has slowed down since 2009. However, the market is still expected to continue to grow by 7% between 2012 and 201637. This indicates that the interactive whiteboard technology is beginning to enter into its maturity phase and leave the growth phase of the product lifecycle. The consumer market within this industry is attracted to new functionality or features in an interactive whiteboard. Current interactive whiteboards can have multiple users and can operate with the user’s finger or a stylus; these new functions are believed to be creating growth in the interactive whiteboard market. 9.5.4 Advertising and Promotion The majority of the advertising will be conducted online to reach the students and teachers. The majority of teachers will be easier to reach through the internet because the majority of the teachers are a part of generation X or generation Y. Thus, one of the best methods for reaching them will be through online measures such as social media or google ads. The ads on google or social media will be aimed at the teachers and not the students because teachers have the buying power within the schools. Social media will be used to get Brösel’s name out on the market and get the consumer talking about the product. The school administrators will be harder to reach than the teachers and students; however, they control the majority of the buying power. The best way to reach this target market is through directly advertising to them. In other words, marketing to school administrators will involve calling them and networking with them38. 9.5.4.1 Message Brösel’s primary message will be that it cares about education. The message informs the customer that the brand cares about them. The purpose of the brand is to provide schools with a low cost interactive whiteboard solution that will better engage their students. This message fits in well with the brand name because Brösel means bread crumbs in german. The system acts like breadcrumbs which the teacher can use to leave a trail for the students to follow throughout the lesson. 37
​
https://www.blossomlearning.com/ShowResource.aspx?rid=63 ​
http://www.techlearning.com/news/0002/how­to­afford­interactive­whiteboards/61576 38
90 9.5.4.2 Medium As previously mentioned, the method that will be used to advertise to the target market will be social media, internet ads, and direct sales. The social media and internet ads will be used to target the teachers because generations X and Y are easier to target through the internet. The direct sales approach will be used for the administrators because they are the older generation and are less receptive to online marketing techniques. Also, the administrators typically ignore any fliers or advertisements they receive concerning educational materials. Thus, the best form of marketing is to directly advertise to the administrators. 9.5.4.3 Budget The marketing budget will amount to $40,000 and will be used for contracting a marketer and for advertising expenses. Approximately $25,000 will be used to contract a part time marketer who will handle and advise the majority of the marketing strategy. The marketer will control the social media image of the company. The final $15,000 will be used for marketing expenses such as online ads and any travel expenses needed for meeting potential customers. The online ads will include google ads and social media ads. After the first year, the marketing strategy will be analyzed and will be revised as necessary including revisions in the marketing budget. Marketing performance will be evaluated based on sales during different marketing campaigns. 9.5.5 Pricing The competitors’ prices range from $1,000 to $10,000. The lower price range, $1,000 to $5,000, are typically the cheaper whiteboards that require a battery operated pen and only allow one user to operate the board at a given moment. The higher priced product allows the user to operate the board with either a pen or a finger and the board can handle multiple users at once. Brösel is aimed to be priced below $200 per unit. The reason for the high cost in competitors’ products is because their product often includes a new whiteboard with their respective sensor network integrated into the board whereas Brösel will retrofit the already existing whiteboard. Brösel gives customers a low cost alternative to other interactive whiteboard technology. This low cost will help place Brösel in the customer’s hands without breaking their budget. The company is anticipating about a 30% gross profit margin at their price point of below $200. 9.5.5.1 Discount Policy Brösel is designed to have the lowest price possible while still allowing the company to remain financially stable. To achieve the goal of making this technology available to as many classrooms as possible and ensure that all students have access to the tools they need to succeed, a discount will be given to educators and educational institutions. The products would be provided at a maximum discount of 20% depending on the order size. 91 9.5.6 Distribution Strategy The company will manufacture the product and then directly sell to consumers. The product will be purchasable through the internet directly from the company’s website. This direct distribution approach is designed to reduce cost and provide the customer with whatever information they need about the product. Customer service is much easier to facilitate if the product is only being sold directly from the company making the customers and service providers happy. The system is small enough to ship to the customer via FedEx. 9.6 Competitive Analysis The main competitors of Brösel are companies who are already producing interactive whiteboard solutions including SMART Board, Promethean Board, and Epson BrightLink. These products provide similar services to one another but SMART Board and Promethean Board require a physical board to be installed and connected to the projector. The Epson BrightLink allows the user to interact with the system on any surface. The Epson product does not require a new board but requires the purchase of a new projector specially designed to make their system work. Comparisons of the three main competitors shown in Table 9 which highlights the price of each unit and projection surface. Table 9: Summary of Brösel Competition 9.6.1 Existing Competitors This section provides a list of currently existing competitors and their products. Each of their solutions are described in detail including what is required to install and operate each system and the main features of each system. 9.6.1.1 SMARTBoard SMART Board controls a significant portion of the K­12 interactive whiteboard market. SMART Board offers a variety of options including different screen sizes and resolutions, various projector options, and additional pens or erasers to increase the ease of use. SMART Board is arguably the most recognizable brand for this product in terms of interactive whiteboards. 9.6.1.2 Promethean Board Promethean Board also controls a significant portion of the K­12 interactive whiteboard market. They offer extra software specifically to work with Microsoft PowerPoint that improves the learning 92 environment in the classroom. Purchase of a Promethean board includes access to a large database of online training and software improvements. 9.6.1.3 Epson BrightLink Epson is a renowned projector manufacturer with a large consumer base and proven expertise. They offer many projector options which can work with any surface or projected on a dedicated board for increased precision. The system requires a specific projector with a special mount that can then project on any flat surface. 9.6.2 Potential Competitors This section details the potential competitors in the field. It describes their products that will either soon be on the market or are already on the market that could be potential competitors in the field of interactive whiteboards. 9.6.2.1 Apple TV Apple TV is currently involved in app and computer mirroring which could be applied in the classroom environment. They are aiming for a technology where multiple different applications can be used simultaneously. Apple TV can connect with multiple different outputs including both TVs and projectors. Apple TV has been used in classrooms to improve the classroom experience and it may be possible that Apple expands Apple TV to further innovate the classroom. 9.6.2.1 ChromeCast Chromecast, like Apple TV, offers connectivity between your smartphone or computer to a projector. Currently, Chromecast, produced by Google, allows for easy connection between the teacher or student and the classroom. They currently support seamless integration with many smartphone and computer applications and may eventually include applications useful for the classroom and teaching environment. 9.7 Management Team The section below describes the managing roles needed in the company and which individual will fit into each roll. This section also includes the skills and qualifications of each individual as it pertains to their given role. The section also details the board of directors and future roles which would be added in the future. 9.7.1 Key Managers and Employees Managers for Brösel include the four lead engineers for the project: Kegan Leberman, Nathan Leduc, Chad Malinowski, and Tom Van Noord as well as business consultant Ryan Beezhold. Beezhold, a junior engineering student at Calvin College, partnered with Brösel in order to create the business plan for the company. 93 9.7.2 Experience, Skills, and Know­How They Bring to the Company Each of these engineers bring the experience of designing digital and embedded systems from their education at Calvin College. Each also has the experience of seeing their respective middle and high schools go through the transformation from a non­interactive classroom to an interactive classroom that has an increased level of technology available for student use. 9.7.3 Future Additions to Management Team Future additions to the management team may include a plant manager if the company decides to manufacture the product on site and a new engineer to start up and oversee the Research and Development division of the company. One key addition to the team will be a marketing consultant to help the company sell its product effectively. Outside of this addition, future additions would fall into non­managerial positions as the company looks to expand the quality and feature set of the software and hardware. 9.7.4 Board of Directors At the current size of the company, Project Brösel’s board of directors will consist of the lead engineers, Kegan Leberman, Nathan Leduc, Chad Malinowski, and Thomas Van Noord. Currently, the sole advisor to the project is Ryan Beezhold who has aided in the formation of this business plan. 9.8 Operations The company being outlined in this section of the report must have a clear business structure. This next section details the form of ownership of the company, describes how the company will make its decisions, considers the compensation of its employees, describes the method of production, and describes the facilities that it will have at its disposal in the near future. 9.8.1 Legal Form of Ownership The company will be a limited liability company (LLC). This will offer limited liability protection to the owners of the company so that debts and liabilities of the company are not the personal responsibility of the owners which means that the assets of the owners are separate from those of the company. Forming this type of company will also help display a formal commitment towards the business and help establish credibility with potential customers or employees. This model also has few restrictions on leadership which means that there are no restrictions on who can be an owner or how many owners the company can have. An LLC also has fewer restrictions regarding business operations which allows for more flexibility and makes it easy to sell the company if the owners decide to exit the business. In addition, an LLC does not normally pay taxes at a business level, but rather any business income or loss is passed through to the owners and reported on their personal tax returns. 94 9.8.2 Decision Making Authority The Vice President of Finance and the Vice President of Sales and Marketing will make decisions on how to manage their respective teams. Within the engineering department, the design manager will be in charge and decide how to manage the hardware, software and prototype, and validation teams. However, large engineering decisions such as design modifications must be made in collaboration with the Vice President of Engineering. The manufacturing manager will make all decisions regarding the manufacturing process from start to finish, including logistics. Large company wide decisions are to be made in collaboration between the three Vice Presidents and the President of the company. However, the President has the authority to make the final decision. 9.8.3 Compensation and Benefits Package Due to the small size of the company and the young nature of the operation, competitive wages will be hard to match so new employees will be offered fair compensation but wages may initially be lower than those of comparable companies. For full time employees, they will receive one week vacation for the first year increasing to two weeks after the first year, and further increasing to three after the fifth year and another week added every five years that the employee stays with the company until 5 weeks of vacation time is accrued. The company will also offer full time employees medical insurance and a matching 401k retirement plan where the company will match up to 25% of the employee’s first 6% of their salary that they invest in the 401k plan. 9.8.4 Production Methodology The production process will be simple. The PCB fabrication will be outsourced to an external company. When received from the company, the products will be assembled and tested in­house before being sent to the buyer. The boards and components will be ordered in sets of 1,000 to get a bulk order discount and save on additional costs. 9.8.4.1 Raw Materials and Costs The price of raw materials is significantly reduced as the quantity bought is increased. However, this may not always be economically feasible due to customer demand. Using a demand pull strategy, it may be necessary to decrease the cost of raw materials if the demand is high enough. However, when entering the market, it may be the case that such high quantities are not always required. Table 10 demonstrates the price of each component at various quantities. 95 Table 10​
: ​
Raw materials and costs for single and bulk purchases Parts Quantity Cost (single) Cost (thousand) Total (single) Total (thousand) Camera 2 4.96 2.12 9.92 4.24 Bluetooth 4 3.03 1.41 12.12 5.64 Microcontroller 4 2.10 1.74 8.40 6.96 Accelerometer 3 2.72 1.33 8.16 3.99 Gyroscope 1 3.56 2.54 3.56 2.54 Male USB 1 0.95 0.76 0.95 0.76 Female micro 3 0.46 0.20 1.38 0.60 RGB LED 5 0.51 0.15 2.55 0.75 Push Button 5 2.67 2.13 13.35 10.65 Photodiode 1 0.99 0.26 0.99 0.26 Part Cost/Unit 61.38 36.39 Each of the parts in Table 10 were found on either Digikey or Adafruit and were each priced and added based on the prices found on those websites. This BOM is also based on a final product and not the prototype that the team has created. The final product will not be using the development boards that were used in the prototype. This means that the cost of the parts will be significantly less but the cost of engineering design will be significantly more. 9.8.4.2 Key Supply Chain Components The supply chains that are key to the components are the PCB manufacturer, the component suppliers, and the plastic case supplier. The production of new devices is dependent on having these three aspects of the product readily available. 9.8.5 Facilities The ideal location for the facility would be near Grand Rapids or Holland, Michigan. Due to the target market being schools, the facility location is based on preference and local connections with whom the company could expand their business further. The building would have office space for engineering teams as well as the manufacturing, sales, marketing and finance teams. The facility would also require a mid­to­small sized production space to produce the product. If the company were to expand, it could be the case that a headquarters location would be built in which engineering, sales, marketing, and finances operate and several separate production locations both in and out of state (or even potentially in and out of the country) would handle the manufacturing process. 96 9.9 Financial Forecasts The financial forecasts outline the predictions for the company’s financial future. 9.9.1 Key Assumptions The company is assuming that it will be able to sell approximately fifteen percent of the number of units that the largest competitor is able to sell. The largest competitor sold an estimated 40,400 units in the past year. Using this number, the company is estimating an annual sales number of 6,060 units. The assumptions that go along with this number are that the company has sufficient capability to produce the needed number of units each year and that each of these units is sold. The company also assumes that the current facilities are sufficient for the needed production volume and design space. One last assumption is that sales will increase each year by approximately five percent. 9.9.2 Financial Statements The income statement for Brösel, LLC takes into account several different factors. Since the company is an LLC, there are no direct income taxes on the company. Instead, the income of the company is claimed by the owners of the company and placed on their personal income tax statements. Therefore, there is no direct income tax reported on the company’s income statement.39 The statement focuses on the costs and profits of the company and reports several different values representing both the annual revenue and the revenue over the first three years of operation. The cash flow statement and the break­even analysis are also indicative of the company’s progress. Cash flow takes into account the operations, investing, financing, and increase in cash.40 The break­even analysis shows where and when the company moves from the red and into the black. See Appendix B for tables and values of the financial statements. In order to calculate the price of the device, the team took several financial aspects into account including, but not limited to, the overhead costs of the company, the employees’ salaries, the cost of materials, and the cost of operation for the business. These factors were all taken into account and a financial statement was produced for the first year of operation. Using these values, it was possible to find the needed amount for the startup loan of the company. These calculations can be seen in Appendix B. 9.10 Loan and Investment Proposal The original cash request for the company is $600,000. This amount will pay for the first year of operations including utilities, overhead, machines, salaries, and everything needed for the company to operate effectively during the first year of operation. 39
​
http://www.nolo.com/legal­encyclopedia/how­llcs­are­taxed­29675.html ​
http://www.investopedia.com/terms/c/cashflow.asp 40
97 The company believes that, after that first year, it will have a customer base large enough to be self sustainable and to systematically pay off the loan. The business launch plan is as follows. The launch of the business will be in early January, 2017. This launch date is early enough to begin beta testing the product during the end of the school year 2015­2016. During the summer of 2016, the design teams will be focusing on perfecting the product. The product will go on sale in July 2017 to allow educators to become familiar with the product before the new school year begins. After the first round of production, the design team will continue to make changes and updates to the software and the hardware while systematically working towards a better second revision of the product. As sales increase, the business will begin paying back loans. The company chooses to perform a principal and interest repayment strategy which means that the company will make payments toward both the principal and the interest of the loan each month. A typical small business loan has an interest rate ranging from 3.78% to 4.44%, according to regulations established by the Small Business Administration (SBA).41 While most SBA loans require the business to be in business for 2 years, Brösel’s engineering firm is hopeful that they can find one or more private investors who is willing to loan a total of $600,000 to the company. Assuming a worst­case SBA interest rate of 4.44%, it would be possible to repay the loan over a course of 7 years by paying $100,000 a year or $8,341 per month. 41
​
http://fitsmallbusiness.com/sba­loan­rates/ 98 10 Conclusion Designing a prototype of a device is a difficult task that requires teamwork, dedication, time, communication, flexibility, and lots of work. The team learned a lot about their individual strengths and weaknesses throughout the project. It was especially helpful to be able to talk through design ideas as a group and be able to work out what was feasible for the design and what would not work. Throughout the semester, the team worked hard both as a group and individually to produce the device described in this report. 10.1 Project Results The team produced a working prototype of the system described in this report. The goal of the prototype was to create a device that provided a proof of concept for the overall project. The prototype could move the cursor in two different ways. The cursor could be moved based on either the input of the gyro mouse or the cameras depending on which aspect of the system the user wanted to use. This final product proves that the team’s design is possible and feasible. The final design of the team’s device allowed the user to control a computer in the same way an interactive whiteboard would allow the user to control a computer. The device that the team designed did not require additional hardware to be installed aside from the cameras. The device functioned acceptably as a prototype but would require more work to be put into production. 10.2 Lessons Learned Some lessons learned in this project are that multidisciplinary teams are extremely helpful. It was extremely difficult ​
for a team of four electrical engineers to take on the roles of not only a team of electrical engineers but also the roles of teams of software engineers, computer scientists, and mechanical engineers. The team was able to take on these roles in order to complete the project. However, more time was taken to accomplish these goals than would have been the case had the team been multidisciplinary. Another lesson learned during the course of this project was that components of the project do not always work on the first try. However, just because things may not always work perfectly does not mean that they cannot or that the project is behind schedule. Because of this, time should be built into the schedule to allow for errors to be made. The final lesson the team learned was that many projects are not as easy as they seem. A lot of work is involved in taking an idea and turning it into a working prototype. 99 11 Acknowledgements The team would like to first and foremost thank Prof. Mark Michmerhuizen, Prof. Michael Bolt, Mr. Bob DeKraker, Mr. Chuck Holwerda, Mr. Phil Jasperse, and Mr. Eric Walstra for their role in providing insight and guidance to the team as they went through two semesters of Senior Design. Additionally, the team would like to thank the rest of the faculty and staff of the engineering department for their role in helping the team members to navigate through the various joys and challenges of the past four years at Calvin College. The team members would also like to thank their friends, family, and loved ones for their constant support and encouragement as they spent the past four years navigating the various aspects of college life and, recently, the senior design project. Finally, and most importantly, the team would like to thank God for providing the opportunity to study at Calvin College and for all of the memories with which they prepare to leave Calvin with. 100 12 Appendices Appendix A: Gantt Chart 101 Task Number Task Name Duration Start Finish Research 3D and 2D Mon 16 days 1 positioning 10/5/15 Research positioning Mon 16 days 2 with IR sensors 10/5/15 Phone App Thu (Accelerometer and 8 days 10/15/15 3 Gyro research) Thu Presentation work day 1 day 4 10/22/15 Fri Oral Presentation 1 1 day 5 10/23/15 Tue Microchip Research 3 days 6 10/27/15 Tue RFI Sensor Research 3 days 7 10/27/15 Design IR Sensor Fri 1 day 8 Circuit 10/30/15 Raspberry Pi proof of Sat concept (with IR 24 days 10/31/15 9 sensors) Raspberry Pi proof of Fri concept (with camera 14 days 11/20/15 10 as sensors) Communication Mon Research (Bluetooth 14 days 11/9/15 11 vs WiFi) Mon Website Design 7 days 12 11/9/15 Mon PPFS Report 25 days 13 11/9/15 Wed Oral Presentation 2 1 day 14 12/16/15 Thu Sensor Testing 7 days 15 12/3/15 Arduino/Raspberry Pi Sun 30 days 16 Prototype 12/13/15 Research Access Wed 7 days 17 computer controls 1/6/16 Wed Research GUIs 7 days 18 1/6/16 Mon 10/26/15 Mon 10/26/15 Mon 10/26/15 Thu 10/22/15 Fri 10/23/15 Thu 10/29/15 Thu 10/29/15 Fri 10/30/15 Wed 12/2/15 Wed 12/9/15 Thu 11/26/15 Tue 11/17/15 Fri 12/11/15 Wed 12/16/15 Fri 12/11/15 Thu 1/21/16 Thu 1/14/16 Thu 1/14/16 102 Research and Plan the 7 days 19 Screen Overlay Research and Plan the Colors/Write/Erase 21 days 20 Functionality Software Planning 1 day 21 Meeting Wed 1/6/16 Thu 1/14/16 Wed 1/6/16 Wed 2/3/16 Thu 2/4/16 Fri Parse Bluetooth Data 14 days 22 2/5/16 Fri Training a Classifier 45 days 23 2/5/16 Thu Gyro Mouse Software 14 days 24 2/25/16 Wed Gestures 7 days 25 3/16/16 Function for Fri Determining Object's 26 days 4/8/16 26 Position Mon Stylus Design 23 days 27 1/4/16 Thu PCB Fabrication 35 days 28 2/4/16 Mon Design Packaging 35 days 29 3/14/16 Mon Final Stage of Test 10 days 30 5/2/16 Tue Final Report 70 days 31 1/26/16 Tue Update Website 70 days 32 1/26/16 Thu 2/4/16 Wed 2/24/16 Thu 4/7/16 Tue 3/15/16 Thu 3/24/16 Fri 5/13/16 Wed 2/3/16 Wed 3/23/16 Fri 4/29/16 Fri 5/13/16 Mon 5/2/16 Mon 5/2/16 103 Appendix B: Financial Statements and Product Price Estimations 104 Product Price Estimation Table: 105 Appendix C: Senior Design Budget Breakdown Category Description Price Quantity Total Cost Vendor Infrared Testing TSL12T IR Sensor $1.42 5 $7.10 Digikey TSL13T IR Sensor $1.42 5 $7.10 Digikey A­to­D Converter $3.30 3 $9.90 Digikey IR LEDs $0.69 3 $2.07 Digikey Stylus Arduino Nano $20.19 1 $20.19 Adafruit Sensors Raspberry Pi Cameras $29.95 2 $59.90 Adafruit 7 IMU $19.95 1 $19.95 Adafruit 8 Bluetooth USB $11.95 1 $11.95 Amazon 9 Bluetooth Transceiver $9.99 3 $29.97 Amazon 10 LiPo Battery $7.95 3 $23.85 Adafruit 11 Lipo Charger $6.95 2 $13.90 Adafruit 12 On­Off Slide Switch $0.95 3 $2.85 Digikey 13 Stylus Buttons #1 $1.09 3 $3.27 Digikey 14 Stylus Buttons #2 $0.74 1 $0.74 Digikey 15 PCB $23.45 2 $46.90 OSH Park 16 Arduino Nano #2 $7.49 1 $7.49 Amazon 17 3D Printed Cases $26.39 2 $52.78 Calvin College 18 3­48x1/4 Screws $6.13 1 $6.13 McMaster­Carr 19 5­40x9/16 Screws $8.80 1 $8.80 McMaster­Carr 20 5­40x3/8 Screws $8.90 1 $8.90 McMaster­Carr 21 Stylus Buttons #3 $1.95 5 $9.75 Sparkfun 22 Stylus Buttons #4 $0.50 5 $2.50 Sparkfun 23 Ping Pong Ball $0 1 $0 Tom Van Noord 24 Webcams $0 2 $0 Calvin College 106 25 Paintbrushes $3.38 1 $3.38 Lowe’s 26 Blue paint $3.48 1 $3.48 Lowe’s 27 White spray paint $5.88 1 $5.88 Lowe’s 28 Expedited Shipping N/A N/A $12 N/A TOTAL COST $385.73 Appendix D: Stylus Enclosure Designs 107