Conference Paper - Department of Electrical Engineering and

advertisement

Park Sense

Evan Davidson, Afsaan Kermani, Viker

Lamardo and Scott Moriarty

School of Electrical Engineering and Computer

Science, University of Central Florida, Orlando,

Florida, 32816-2450

Abstract — Park Sense is a mainly software oriented system designed to tell the user where and which parking spaces are available in a given parking lot or parking garage.

The system is composed of a camera monitoring a parking area where the images are fed into a server. The server performs image processing and determines whether or not there is a car or not in the parking spots being monitored.

This information is then supplied to the user through a display at the entrance to the parking area, as well as being supplied via the internet to computers and mobile phones.

Index Terms — Computer learning, Haartraining, image processing, object recognition, OpenCV, parking, sensor network. illogical approach. However, in using image processing, a single camera can be used to monitor a much larger number of spots than a single sensor. So now the main option is to use the camera in an area where it is connected to a router to create a local network within the garage or lot connecting the camera, server, and display.

We would also like to add features like a website and a smartphone application for the user so that he/she can see the view of the parking lot before they enter the lot.

This would greatly help in reducing the cluttering at the parking entrance as people would know beforehand itself where there is an empty parking spot and therefore would be ready to catch that spot.

We hope this project would also be energy efficient in a way that it would help people reduce their gas consumption that they waste on looking for a parking spot.

With our senior design project ‘Park Sense’ we aim to reduce the time people spend finding parking spots and revolutionize the way people look for a parking space in parking lots.

I. I NTRODUCTION

In today’s world business centers, universities, shopping malls etc. are busting with parking lots both open and multi-level garages. Finding an empty parking spot in this large maze like parking lots is an undertaken challenge itself. On an average a person spends 2-10 minutes looking for a parking spot in urban parking places, this in turn wastes time and gas. How many people wish they would straightaway know where an empty parking spot is when they enter the parking lot?

This is where our ‘Park Sense’ senior design project comes into play. When a car enters a parking lot it will be greeted by a display at the entrance showing them where in the parking lot there is an empty space. This in itself is a huge undertaking because this idea requires for there to be a sensor or a camera with image processing in each of the parking spot so that it can detect a car entering or leaving the spot. The major part of this project is to figure out and implement what technology to use to transmit the data from the sensor to the server and its display screen at the entrance of the parking lot. With the sheer size of the parking lot running wires to each of the sensor was an

II. MOTIVATION

When we started on brainstorming ideas for our project, every one of us in our group had many different ideas for our senior design project. Our minds were just budding with ideas. Now the hard part came when we had to decide which idea we would implement to be our Senior

Design Project. Therefore we started weighing down all the ideas and started deciding the pros and cons of each idea. So we decided that we would go with the idea that had more pros than cons. So we came upon the idea of our project ‘Park Sense’. After discussing on the project for a while we all decided this would a perfect senior design project for us.

Moreover this project dealt with an everyday common problem that is faced by the students of the University of

Central Florida every day. UCF had a student population of about 55,000 students and there are just about 15,000 parking spots in UCF. So we can pretty much judge the challenge it would be to find an empty parking spot during regular school hours. After discussing our project each one of our group members was totally into the ides of

‘Park Sense’. First of all if we are successful in our project we would be helping the UCF parking services in implementing a system like ours at all the different parking garages and the open parking lots. Secondly we would be helping the students conserve their time looking for parking spaces. Thirdly all four of us in our group had

personally experienced this big parking issue at UCF and because of that we had sometimes arrived late in class or late for an exam or a meeting etc.

This above mentioned points really got us going and interested in the project. We then started o thinking of various ways in which we could implement this project.

Design ideas started flowing and after many trial and error decisions we came up with the idea of how we are going to implement it.

This project really dwelt into the excitement factor of all of us. We all were eager to take upon this challenge and see the outcome. This project also had a variety of different things to do in it, our group being a mixture of computer and electrical engineers we needed a project that would have a mixture of both. This project turned out to be a perfect blend of both software and hardware. After we started researching more on our idea we came to know that many people before us had implemented it. After hearing this many people would be discouraged but in turn we got even more encouraged to do this project as we now had benchmarks we could compare against and strive to be the best project pertaining to this idea.

So to sum it up there were many factors that motivated us towards this project. But the most important think that would help in succeeding in our goal was to be challenged and this would then bear fruits if we are successful in implementing our project to its core.

III. O VERVIEW OF THE S YSTEM from the stream and processed on the server. The output of this processing is transmitted to the display through a router.

A. Display

The display is essentially the end user devise. Going by the requirements of our project our best option for an LCD display would be a regular LCF flat screen monitor that is used in most of the computers today. This would make our project simpler as we would have to just program a

GUI to sit on top of the LCD. LCD monitors would be better than its CRT counterparts, as an LCD monitor is brighter than a CRT, giving us a nice view even in fully bright sunlight situations. Brightness is measured in nits, or one candela per square meter. A standard measurement of brightness is between 250-300 nits. The brightness of the screen directly depends on nits as such that if the value of nits is bigger than we would have to adjust the brightness to a lower value.

When choosing the right kind of monitor for our project the big factor in our decision making was the fact that the

LCD monitor is going to be out in the open as well as indoors. And moreover the angle as which people are going to view the screen should be a wide angle. In particular an LCD monitor has different specifications for the viewing angle which can be either vertical or horizontal, the specification states the degree of view ability while we slowly try to move away from the center of the screen and the screen view slowly diminishes.

There is one way to solve this issue is to keep the level of contrast always high in these LCD monitors. Viewing angles of horizontal-140 degrees and vertical-120 degrees are the most efficient. LCD screens are far more energy efficient than the CRT monitors. It is also beautiful to look at and can be crammed in the smallest of spaces due to the fact that they are very thin. The radiation emitted is always at a record low with LCD’s. Therefore an LCD screen is the perfect choice for the Park Sense project.

Below is a sample of how the output will look on a normal

15-inch LCD monitor.

Block Diagram of the Park Sense System

The above block diagram shows the layout of the Park

Sense system. The primary components that compose this design are the display, the server, and the camera. These three components map the system from its beginning to the output that is received by the user. The functionality will start at the camera with the video of the parking lot streaming onto the server. The image will be extracted

B. Server

Park Sense Output displayed on LCD Monitor

The second key piece of hardware in the system is the server. For the Park Sense system, there are a number of options for the server. The important aspect of the server is the size, processor speed, and hard drive space.

The physical size of the server is dependent on where the system is being implemented. The server does not have to be immediately adjacent to the camera because the camera is connected to a router that can be connected to the server via a local area network (LAN). The ideal setup would be to have the server in some sort of enclosure near the entrance of the garage or parking lot, to where the display can be hardwired to the server without using an excessive amount of hardwiring.

The critical aspect of the server is the speed of the processor. The server will be responsible for running the image processing on the video stream that it receives from the camera. The server must have all of the software loaded and be capable of running the software in a timely manner so that the data on the display can be updated and accurate at all times. For this, faster processors are better.

The server will also have to have a large amount of RAM.

The software uses a significant amount of RAM in its processing requiring a computer with ample memory.

For our application of the Park Sense system, the server will be a high-quality laptop computer. Ideally, the system can be implemented on a much higher performing desktop computer.

C. Camera

The main aspect of the project is detecting the cars.

There was an immense amount of discussion amongst the

Park Sense team as to which sensor would be the best way to detect cars in a parking lot or a parking garage. After weighing all of the options on price, accuracy and overall ability to be implemented into the system, the choice became clear that using a camera would be by far the most inexpensive and efficient way of identifying cars in parking spaces.

One advantage of using a camera and image processing for Park Sense is that a single camera can detect anywhere from 1 to over 30 parking spots. Most other sensor options can detect only one or two spots per sensor. This makes the price of detection per spot much more expensive for sensor options such as sonar or infrared.

With image processing and object recognition software, the Park Sense software can also be implemented in any parking garage where there is already a camera used for security purposes. There is also a variety of cameras that can be implemented in this system. The camera used in the

Park Sense system is an IP camera made by Vivotek. The camera used is the Vivotek IP7330. There are many reasons that the camera fits the needs of the Park Sense system. Two important aspects of the camera are the range and resolution. The range of this camera is ample to be mounted as high as 4 stories to view the parking lot. The resolution of the video stream is 640 x 480. This is basically the highest resolution that can be found in these types of IP cameras. The image processing is more accurate with an image that is a higher resolution.

Therefore, this camera, with the resolution it has is a good choice. It is also inexpensive and an excellent solution for the Park Sense project.

IV.

S PECIFICATIONS AND R EQUIREMENTS

One requirement of the project is low cost. We were incapable of acquiring sponsorship or any sort of financial aid for the Park Sense project. Therefore, it is critical to maintain as low of a cost as possible. While this is the primary reason for requiring a low cost, we also want to design this system to be capable of being used in real world applications or simply expandable onto a larger scale. For this to be possible, the system would have to be cost effective. For this requirement, a large portion of the research will be to find the least expensive part for each component of the project while still meeting the other requirements.

Another requirement of Park Sense is energy efficiency.

This somewhat goes hand in hand with being low cost.

Generally if a system is energy efficient, it will be low cost. This is another requirement that is meant to contribute to the scalability of the project. A system that is energy efficient is more likely to be capable of being applied on larger scales. To achieve this requirement another large portion of the research will also be devoted to finding the most energy efficient and low power part of

each component. There will also be an emphasis on researching and determining the most efficient power supply for each of the parts involved in the system

The following is a list of some of the more important specifications we are looking for in determining the sensor for this project.

 Range: o minimum 2 feet o maximum 12 feet (roughly the average length of a car)

 Cost: under $10 per unit

 Power consumption: low

 Package size: minimal – less than 10 cm x 10 cm

 Supply voltage: less than 9V

 Accuracy: 95% or better

The following list is a summary of what we are looking for in the display of the project that will be located outside of the parking lot.

 Type: LCD

 Display area (diagonal) : 15’’ to 21’’

 Resolution : 1280 * 1024

 Pixel response time : 8 ms

 Brightness : 300 cd/m2

 Contrast Ratio : 700:1

 Viewing Angle ; 150/135 (H/V)

 Video : VGA (Analog), DVI-D (Digital)

 Input Connectors : o 1 * VGA (15-pin HD D-sub) o 1 * DVI-D (24-pin Digital DVI) o 1 * Audio Line – In (Mini-Phone stereo 3.

5mm)

 Unit: Approx. 17 * 16. 6 * 6. 9’’

 Weight: Approx. 12 lbs.

 Environments : PC compatible

 Audio : Built in stereo speaker (2 * 1W)

The following list outlines some of the key aspects that will be needed for the server. The more important of these are the RAM and the amount of storage on the drive.

 Processor: Intel Core 2 Duo Processor E8400

(3GHz, 6M, 1333MHz front side bus)

 Memory: 2GB, DDR2 Non-ECC SDRAM,

800MHz (2 DIMMS)

 Boot Hard Drive: 160GB SATA 3GB/s and 8MB

Data Burst Cache

 Video Card: 256MB ATI RADEON HD 3450

(Dual DVI/VGA/1 TV-out) or comparable card that has 256MB, dual monitor support, with dual DVI out and DMS-59

 Optical Storage Device: DVD+/-RW

 Ethernet/modem

The most important part of the Park Sense Project is the image processing, which is the process of picking an image out of a video stream and recognizing objects within that image. The bulk of this process is done using

OpenCV.

OpenCV is an open source library written in C and C++ for aiding in computer vision. “Computer vision is the transformation of the data form a still or video camera into either a decision or new representation.” The main focus of OpenCV is object classification and recognition on static images and real time feeds. Currently this library is widely used for many commercial uses such as; factory product inspection, medical imaging, security, and many more. Taking advantage of the five hundred plus functions in this library one can develop their own image processing tool. Below is a table comparing OpenCV to the JAVA API, which was another option for this project.

[1]

JAVA Advanced builds

V.

IMAGE PROCESSING

RECOGNITION

Imaging API

JAVA programming

Still Developing with daily

Better for the future

AND OBJECT

SOFTWARE

OpenCv

C , C++ programming

Complete libraries

Good for the present

Not that active in updating libraries

More active in updating libraries

OpenCV is the main tool that was utilized in creating a program that can detect the desired image. There are many steps that were taken in the process to get to the point where the program can accurately detect the desired image. These steps are described in more detail below.

A. Haartraining

The first decision that had to be made in developing

Park Sense with OpenCV is determining what object is to be detected by our program. It is obvious that the software

will need to either detect cars and mark as positive or detect empty parking spaces and mark these as negative or open/available.

Haartraining is a utility that is included in the OpenCV package. For Haartraining to take place, we needed thousands of positive sample images, as well as thousands of negative sample images. A positive image is anything that includes the object that you are looking for and a negative image is anything that does not include that object.

Choosing to detect cars, the Park Sense team took hundreds of pictures, including thousands of cars from the tops of various parking garages around the UCF campus.

Once these pictures were obtained they had to be converted to bitmaps and resized for processing.

Once the positive images are in the proper format and are sized correctly, a utility called Object Marker is used.

Object Marker is a simple .exe file that loads all images in a folder one at a time, allowing the user to draw a rectangle around the object within that image that the user is interested in detecting. The output of this executable program is a text file containing all of the image names followed by how many objects are in that image and where the objects are within the image. The location of the objects is marked by pixel coordinates of where the rectangle was drawn starting with the upper left corner and ending with the bottom right corner. The Park Sense team took the time to do this a variety of different ways.

Different text files or info files were generated for positive object detection of whole cars, headlights, license plates, and windshields. An info file was also generated for rectangles for the possibility of detecting open spots and not cars.

In addition to the positive object info file, there must also be a negative info file. Fortunately there are a number of databases of thousands of negative images. Again, a negative image is any image that does not contain an occurrence of the positive object. Most databases of negative images have been designed for face recognition, which means these images may contain cars, windows, or headlight-like images. To assure that the batch of negative images were indeed negative; the Park Sense team went through all of the images, deleting any with the Haar-like features, as defined by our positive info file. The negative info file only needs the name of the image in the text file.

This can be created using the Windows command prompt.

At the end of this step we were able to create a positive info file with roughly 1500 positive images and a negative info file with roughly 4000 negative images for each of the cases of cars, headlights, license plates, and windshields.

From the positive info files, a vec file can be created.

The vec file is a vector graphic file specifying further information about the positive samples. There is another utility in the OpenCV package that aids in creating the .vec file.

The actual training of the cascade is the most time consuming and complicated part of the Haartraining. The cascade results in an XML file as well as a data folder containing image information. This XML file or cascade is the file that is referenced in the software for the object detection. The training requires the vec file for the positive images, as well as the negative info file. The training program is another part of the OpenCV package. The training takes weeks to perform running nonstop on a normal laptop computer. This was done for all of the vec files that were created for all of the sets of positive objects.

After weeks of training, the Haartraining for windshields resulted in the most accurate cascade.

After this time consuming process, the result is simply a cascade file that can be utilized in the Park Sense software as a means of object recognition.

B. Establishing a Region of Interest

In implementing the Park Sense system, it is important to mount the camera securely so that the picture is stable and constantly the same. Once the camera is installed with a view of the parking lot, the individual parking spots need to be mapped out.

Within the programing, each individual parking spot is classified as a region of interest. The region of interest is the specific area of the stream, where the program actually looks for positive objects. Once the regions of interest are established, the program will know where to look in the stream to find if there is a positive match and ultimately determine if the parking spot is occupied or vacant.

C. Receiving the Image

The IP camera streams an MJPEG file through the router where it can be accessed by any device that is on the

LAN. A MJPEG stream is a video format that consists of individual JPEG frames strung together to create the video file. The server on the network has access to this stream.

One of the programs of the Park Sense software obtains a single JPEG frame from the stream to be processed through the cascade. When the JPEG is initially retrieved from the stream, it is first converted into a bitmap because

JPEG is not an OpenCV compliant format.

The extracted bitmap image is compared to the cascade to determine whether or not the there is a positive object detected in the region of interest.

The following screenshot shows the process in action, where the program is extracting the JPEG images from the stream. As you can see in the screenshot, the program has already internally converted the JPEG to the bitmap format. The output that is shown is for internal use. This is not output that is seen by the end user. This screenshot is for the windshield cascade. The number of hits is the number of positive objects found in the given frame. The number of misses is the number of positive objects that were not detected by the program when comparing to the cascade. For a perfectly accurate system, this number should be 0, which it is for most of these frames. The numbers in the false column are the numbers of positive objects that the program has found that are not actually positive occurrences. [2]

Output of the windshield cascade showing where positive objects were detected

As you can see from this picture, there were many windshields that were detected. There are also a few non windshields that have been falsely detected. In addition to that margin of error, there are also occurrences of the other form of error where there are windshields that have not been detected by the software. Overall, from this image, the accuracy can be determined to be roughly 75% with a large number of parking spaces being detected by this single camera location.

Below is another image. In this image, the output is much more accurate.

Screenshot of JPEG stream with software displaying positive hits and misses

D. Output

In addition to the table that was described above, the program also outputs a copy of the bitmap showing where positive objects were detected and placing a rectangle around the object that was detected. This output is a good means of testing the cascade and determining what type of objects the program is detecting if is getting a large number of false positives. The image below is an example of the output image for the window cascade.

More accurate object detection comparing to the windshield cascade

The program has a tendency to be much more accurate when there are fewer cars in the image and when the picture is zoomed to a higher resolution on those few cars.

Once the program has processed the image through the cascade, the output is placed in a csv file. For each region of interest, which corresponds to a parking spot, a 1 is assigned to the csv spreadsheet for an open parking spot and a 0 is assigned to the cascade for an occupied parking spot. The csv file is constantly updated as new frames of the stream are processed so that the data is always up to date.

In the case of the image above, there would be 4 regions of interest defined, corresponding to each of the four parking spots. The program would see that no rectangle has been drawn on the spots on either end of this row, or that there is no targeted object in those regions of interest.

Because of this, a 1 would be inserted into the csv file indicating that that region of interest has no object or the parking spot is vacant. The two interior spots have detected an object. These two regions of interest would have a 0 assigned to their corresponding slot in the csv file. These values of the csv file would then be updated on the website in the corresponding graphical interface representing the parking lot.

VI.

DISPLAYING THE OUTPUT

The output of the Park Sense image processing is stored in a csv file that is also accessed by the website server.

The Park Sense website is the primary means of displaying the data to the user. The website uses PHP code to access the same csv file that is the output of the programming.

The webpage displays a diagram of the parking lot for which the Park Sense project is implemented. The layout of the parking lot is filled with either green open spaces or red occupied spaces that correspond to where the spots are in the actual parking lot.

The LCD display is placed at the entrance to the parking lot or garage. This display simply shows the website.

Using this method, the diagram of the parking lot can be viewed from any computer or mobile device with web access. This provides the user with access to the parking situation upon entrance to the garage, as well as before the user arrives at the garage and after the user is inside of the garage.

The following image is a depiction of what the user will actually see, whether it be from the display, the website, or from a mobile phone accessing the website. Each square corresponds to a location in the csv spreadsheet. As the csv file changes its values between 1 and 0, the display changes between green and red with a care in the square.

Park Sense output as viewed from the website or from the entrance display

VII.

C ONCLUSION

To accomplish the goals that have been set for this project there were many paths open to the group. Whether due to over complicating the problem, over stretching our capabilities, time constraints, or budget, various possible features for this project were chosen to not be implemented. We feel strongly that the project we are choosing to put together will be a success, and electing to not include some features will not negatively impact the end project. When we first thought of this idea at a group meeting, we were trying to come up with ideas that would make the project worthy of our senior design project.

Sensor networks are not a new concept. The idea of sensing a large number of objects and compiling the results has been implemented in many arenas. This idea has also been implemented in parking garages and lots in some places.

OpenCV is a relatively new technology. For the most part, this software package has been used in facial recognition and not much else. With successfully implementing this technology in parking lot monitoring with the Park Sense project, many new windows are opened and there are new possibilities for using OpenCV in other areas.

If the opportunity is available, the Park Sense team would like to continue to work on the idea and the software that has been developed. With more time and work, the group feels that the Park Sense project can be improved and developed into a useful system capable of being implemented in parking lots and garages all over the world.

A CKNOWLEDGEMENT

The authors wish to acknowledge the assistance and support of the OpenCV community for the help received in forums throughout the internet. This help greatly aided four hardware-oriented engineers complete a softwareoriented project.

R EFERENCES

[1] Bradski, G. & Kaehler, A (2008). Learning OpenCV:

Computer Vision with the OpenCV Library. Cambridge:

O’Reilly.

[2] “Smart Parking System using Image Processing

Techniques in Wireless

Sensor Network

Environment.” M.Y.I. Idris,

E.M. Tamil, Z. Razak, N.M.

Noor and L.W. Kin.

Information Technology

Journal. Year: 2009.

Volume: 8. Issue: 2. Page

No.: 114-127.

T HE P ARK S ENSE T EAM

Evan Davidson is a 22 year old senior electrical engineering student at UCF.

Evan currently works at ACD Telecom, an engineering consulting firm as an intern and will continue to work there full time upon graduation. He plans to eventually continue his education and pursue a Master’s degree.

Scott Moriarty plans to pursue a career in electrical engineering. He plans on accepting a job from Lockheed

Martin where he held an internship for 2 years while pursuing his bachelor’s degree. Eventually, he’d like to get out of Florida and move somewhere further north with snow.

Viker A. Lamardo is a senior student of the electrical engineering department at

University of Central

Florida. Viker will start to pursue his master degree here at the University of

Central Florida upon graduation. His current research interests are in electromagnetic, microwaves, and electro-optics.

My name is Afsaan Kermani. My major is Computer engineering and this is my final semester at UCF. I am originally from India . I came to this country almost 6 years ago to achieve what people call the 'American dream'. After graduating from UCF I might have fulfilled a part of my dream. My aims and ambitions are to help the people in solving everyday problems using my skills I learnt in college and make this world a better place to live in.

Download