Software Development Projects (1992-2005) Date: May 22, 2005 Kyungnam Kim, Computer Science Department, University of Maryland, College Park, MD 20742 This document presents the software projects which I developed 100% or I had involved in as a team member. It also details development languages used, period, contractor, etc. Some projects involve hardware development. Most software programs are operated on Microsoft Windows. The list below describes my technical skills related: TECHNICAL SKILLS Computer languages and platforms: Visual C/C++, MATLAB, Visual Basic, Borland Delphi, LISP, Fortran, Pascal, HTML, x86 Assembler, PVM, Unix, Linux, MS-Windows. Hardware design: pASIC, QuickLogic, VHDL, Verilog, Synthesis/Simulation tool, Z80 microprocessor. Image processing, vision, graphics: MATLAB, Mathematica, IDL, SGRP, OpenGL, several commercial and free toolkits. Multimedia packages: Multimedia Toolbook, Authorware, Flash, Premiere, Photoshop. Contents 1. UMD-BGS (University of MarylanD BackGround Subtraction Program) 2. Perturbation Detection Rate (PDR) Analysis Program 3. 4. 5. 6. 7. 8. 9. 10. 11. NASA Crew Performance Analyzer Multi-camera Tracking of Occluded People Video Quality Measurement Program Face Animation Face Recognition Simple Object Tracker EyeTalk – Multimodal Human-Computer Interface Facial Feature Tracking Eye Tracking 12. 13. 14. 15. 16. 17. Parallel Programming of Vision Algorithms on PVM CAD Tool for Block Interlocking Image Database for Electricity Facility Personal Information Management System – Samsung Electronics Personal Information Management System – SunJin Fleet Co. Database for Water Supply Facility 18. Score Board for TV Show 19. Internet Guide for Dummies 20. Tour Guide for Department Visitors 21. Other Projects 22. My logo artwork UMD-BGS, University of MarylanD BackGround Subtraction Program Goal This background subtraction (BGS) program processes input video data captured from a static single camera to generate foreground/background binary image outputs. Our codebook background subtraction algorithm has these key functions (i) fast and compact background modeling by a vector quantization technique, (ii) temporal filtering allowing foregrounds in the training period, (iii) a unique color model separating color and brightness evaluation efficiently, (iv) multiple background layers to handle background changes. Start/End date Aug 2001 – Aug 2004 (3 years development period) My contribution 80% of 30,000 lines of codes, Algorithms Language(Library) Platform Features Customer/User Remark Visual C/C++ (MFC, DirectShow, VisionSDK) The libraries are used for interface purpose, e.g., display, input/output, but most coding was done from scratch MS Windows Inputs can be live video (from any camera accepted by DirectShow, i.e., IEEE1394, USB camera, etc) , video files (*.avi, *.mpg, etc), or a sequence of image files (*.bmp, *.jpg, *.gif, *.ppm) Processing speed: almost 30 fps w/o post-processing Setting of parameters and post-processing options, Diagnostic displays, Event log, Binary detection images UMD students, many people in computer vision research/industry communities More details and download at http://www.umiacs.umd.edu/~knkim/UMDBGS/index.html Related publication: K. Kim, T. H. Chalidabhongse, D. Harwood and L. Davis, "Real-time Foreground-Background Segmentation using Codebook Model", Elsevier Real-Time Imaging 2005. Main control program and parameter setting Examples Perturbation Detection Rate (PDR) Analysis Program Goal The program is a comparison platform for detection algorithms. It generates a text file containing detection rates which can be converted to a PDR graph. A performance evaluation method using perturbation analysis (called Perturbation Detection Rate analysis) is proposed. It measures the sensitivity of a background subtraction algorithm in detecting low contrast targets against background as a function of contrast. It could be used as an alternative of ROC. Start/End date Sep. 2002 – Feb. 2003 (6 months development period) My contribution 60% of 20,000 lines of codes Language(Library) Visual C/C++ Platform MS Windows Features Comparison of 4 detection algorithms by adjusting parameters and simulation options Customer/User Remark Internal use only Related publication: K. Kim, T. H. Chalidabhongse, D. Harwood and L. Davis, "PDR: Performance Evaluation Method for Foreground-Background Segmentation", submitted to EURASIP Journal on Applied Signal Processing. Main control program and an example frame Graph generation NASA Crew Performance Analyzer Goal This project is to track NASA crew in the space shuttle and to analyze their motions, postures, etc. The ultimate goal of this project is to find out their cognitive states and to help designing shuttle interfaces, etc in ergonomics-view. (20012002) Start/End date 2001 – 2002 (1 year development period) My contribution 30% of 20,000 lines of codes Language(Library) Visual C/C++ Platform MS Windows Features Customer/User Remark Foster-Miller Inc. (a NASA sub-contractor) We used our background subtraction program for this application Main control program and an example frame Multi-camera Tracking of Occluded People Goal Tracking and segmentation of occluded people using multiple overlapped views. Start/End date Nov. 2004 – Jul. 2005, (9 months development period) My contribution 100% of 5,000 lines of codes Language(Library) MATLAB Platform MS Windows Features Here, an appearance model is created when an individual is detected without occlusion. The appearance model is a color density as a function of height since a human's appearance mostly changes along its height. We assume that this model is good for each view (even though each camera's output for the same realworld object is slightly different). Given the appearance model, the prior location, and occlusion information, we classify each pixel into the best-matched person class using Bayesian rules (maximum likelihood). In addition to appearance, a ground homography is used for resolving the correspondence of each person between views. Customer/User Remark Example frames Publication under preparation Video Quality Measurement Program Goal I designed a tool to assess no-reference objective quality of image or video using local statistics of pixels over space and time. It reports several quality measures – fine structure quality, clipping, entropy, etc. Start/End date Nov. 2003 – Jan. 2004, (3 months development period) My contribution 100% of 2,000 lines of codes Language(Library) Platform Features Customer/User Remark MATLAB MS Windows Q1 – noise: due to sensor sensitivity, electronic transmission, illumination fluctuation, camera vibration, etc Q2 – contrast: affected by camera optics, resolution, etc Q3 – color information: how well color values are distributed over the intensity range (entropy). Q4 – clipping: pixels brighter or darker than the representation bounds (ex, [0,255]) are clipped. Internal use only Related publication: K. Kim, and L. Davis, "A Fine-Structure Image/Video Quality using Local Statistics" IEEE International Conference on Image Processing (ICIP) 2004 EXAMPLE ============================================== Image/Video Quality Statistics ============================================== -image.height = 240 -image.width = 360 -number of frames = 50 -clipping low_bound and high_bound = [20, 235] [ Original ] clipped_low = 0.16% clipped_high = 1.46% non-clipped = 98.38% entropy = 7.423942 FIQ median = 1.680336 [ GaussianNoise: mean=0.0, std=3.0 ] clipped_low = 0.17% clipped_high = 1.47% non-clipped = 98.36% entropy = 7.444018 FIQ median = 1.036869 [ GaussianBlur: 5x5, std=3.0 ] clipped_low = 0.00% clipped_high = 0.59% non-clipped = 99.41% entropy = 7.357112 FIQ median = 1.053170 ============================================ Face Animation Goal Developed a face animation system. Start/End date Sep. 2000 – Dec. 2000, (4 months development period) My contribution 100% of 10,000 lines of codes Language(Library) Visual C/C++ (OpenGL) Platform MS Windows Features It can automatically, in real-time, animate facial feature movements such as eye-lid blinking, mouth opening/closing, and head rotation (roll, yaw, pitch) by visual tracking on the facial features from live video. Customer/User Remark Example Course Project – “Advanced Computer Graphics” Head rotation (roll, yaw, pitch) mouth opening/closing Eyeball movement Face Recognition Goal Implemented a face recognition system based on Eigenfaces papers using MATLAB and tested on the face database from AT&T Lab at Cambridge. Start/End date Sep. 2001 – Dec. 2001, (4 months development period) My contribution 100% of 1,000 lines of codes Language(Library) Platform MATLAB MS Windows Features Customer/User Remark Course Project – “Numerical Analysis I” Example recognition and Eigenfaces Simple Object Tracker Goal Implemented an object tracking software for USB cameras. Start/End date Jan. 2001 – Feb. 2001, (2 months development period) My contribution 100% of 2,000 lines of codes Language(Library) Visual C/C++ (DirectShow) Platform MS Windows Features Tracking an object in motion. This is not for multi-target tracking Customer/User Remark Example frames A private venture company $5,000 contract EyeTalk – Multimodal Human-Computer Interface Goal The goal is to build a multimodal human-computer interface integrating face tracking, eye-gaze estimation, lip-reading, and speech recognition technologies. Multimodal Human-Computer Interaction (HCI) combines natural input modes. It benefits from novel HCI modalities and from multimodal integration for tackling the problem of the HCI bottleneck. EyeTalk was a funded project to build a car navigation system in which our multimodal interface is embedded. Start/End date Feb. 1999 – Jun. 2000, (1.5 years development period) My contribution 40% of 30,000 lines of codes, a project leader/coordinator of 7 developers Language(Library) Visual C/C++ (Matrox library, HMM toolkit, Windows multimedia library) Platform MS Windows Features Multithread programming Source & version control Source code documentation Customer/User Remark System in working A government-industry sponsored project (Korea) My responsibility was coordinating regular meetings, assigning work-loads to each member, integrating each component into a single working system, and maintaining publications. User Command Control Panel System Overview Facial Feature Tracking Goal In Facial Feature Tracking, a robust, fast, and cheap scheme for locating the eyes, lip-corners, and nostrils is proposed. Head orientation-based gaze tracking is one of possible applications. Start/End date Feb. 1999 – Jan. 2000, (1 year development period) My contribution 30% of 10,000 lines of codes Language(Library) Platform Visual C/C++ (Matrox & Coreco libraries) MS Windows Features Customer/User Remark Example 5 frame per second speed Eye Tracking Goal Eye Tracking addresses the development of image processing algorithms for locating and tracking facial features, especially iris boundary (limbus). Occluded circular edge matching algorithm has been developed and tested. Start/End date Mar. 1998 – Feb 1999, (1 year development period) My contribution 100% of 7,000 lines of codes Language(Library) Visual C/C++ (Matrox library) Platform MS Windows Features Customer/User Remark Example 5 fps speed Parallel Programming of Vision Algorithms on PVM Goal Parallel Computing for Vision addresses the problem of accelerating the operations or jobs of computer vision and image processing. PVM, which I have used, is one of the most popular platforms for parallel computing. Start/End date Sep. 1997 – Feb 1998, (6 months development period) My contribution 100% of 7,000 lines of codes Language(Library) Ansi C (PVM - Parallel Virtual Machine) Platform UNIX Features A cluster of 5 different heterogeneous unix workstations Customer/User Remark System Overview Computer Vision tasks are very computationally intensive. Therefore their computations should be parallelized properly for high performance. Using existing resources, Parallel Vision Environment is built based on PVM over the heterogeneous networked workstations CAD Tool for Block Interlocking Goal It is a computer-aided design tool for the industry of concrete block interlocking. It lets a user draw blocks with different allowable interlocking options. Start/End date Sep. 1995 – Feb 1996, (6 months development period) My contribution 100% of 10,000 lines of codes Language(Library) Borland Delphi Platform MS Windows Features Zoom in/out, Copy and paste, Printing, Auto-counting of the number of blocks for each color Database: load and save user’s designs of block interlocking Customer/User Remark Snapshot Daeyang Concrete Co. LTD., Korea $10,000 contract Program Starting Logo Database Image Database for Electricity Facility Goal It is an assisting tool for the facility management team in Korea Electric Power Corporation, the only corporation handling electricity in Korea. Start/End date Mar. 1995 – Aug. 1996, (6 months development period) My contribution 100% of 8,000 lines of codes Language(Library) Visual Basic Platform MS Windows Features Capturing from digital cameras and video camcorders Comprehensive database of electricity facility Customer/User Remark Snapshot Korea Electric Power Corporation (KEPCO), Korea $30,000 contract Personal Information Management System – Samsung Electronics Goal A personal information management system was developed for Samsung Electronics Pusan Branch. Start/End date Mar. 1995 – Aug. 1996, (6 months development period) My contribution 100% of 7,000 lines of codes Language(Library) Visual Basic Platform MS Windows Features User mode VS. Administrator mode Formatted printing Customer/User Remark Snapshot Samsung Electronics Pusan Branch, Korea Personal Information Management System – SunJin Fleet Co. Goal A personal information management system was developed for SunJin Fleet Co.. Start/End date Aug. 1995 – Nov. 1995, (4 months development period) My contribution 100% of 8,000 lines of codes Language(Library) Borland Delphi Platform MS Windows Features Sailors’ personal information management Customer/User Remark Snapshot SunJin Fleet Co., Korea $5,000 contract Database for Water Supply Facility Goal It is an assisting tool for management of water supply facility. Start/End date Aug. 1995 – Nov. 1995, (4 months development period) My contribution 100% of 6,000 lines of codes Language(Library) Visual Basic Platform MS Windows Features Image data from a scanner Water supply management – damage, repair reporting Customer/User Remark Snapshot Korea Information System, Korea Score Board for TV Show Goal I developed a tool for score displaying to be used on a TV show program. Start/End date Aug. 1995 – Nov. 1995, (4 months development period) My contribution 100% of 1,000 lines of codes Language(Library) Visual Basic, Photoshop Platform MS Windows Features The artwork was also created by myself Customer/User Remark Snapshot Pusan Broadcast Co. LTD., Korea $1,500 contract Internet Guide for Dummies Goal Multimedia contents of introduction to Internet for beginners Start/End date May. 1995 – Aug. 1995, (4 months development period) My contribution 100% of 300 lines of codes Language(Library) Authorware, Photoshop Platform MS Windows Features The first Internet education content at the early age of Internet boom. Customer/User Remark Snapshot Personal project Tour Guide for Department Visitors Goal It is a tour guide for department visitors, which contains introduction to the department. Start/End date Sep. 1996 – Feb. 1997, (6 months development period) My contribution 100% of 500 lines of codes Language(Library) Platform ToolBook, Photoshop MS Windows Features Customer/User Remark Snapshot Undergraduate project Other Projects Title 8-bit CPU design and simulation using VHDL and QLogic tools Start/End date Mar. 1996 – Feb. 1997, (1 year development period) My contribution 30% of 500 lines of codes Language(Library) VHDL, Verilog, pASIC tools Platform MS Windows, Hardware (Bread boards, etc) Features Simulation on a real chip after layout Customer/User Remark Title Undergraduate project Client/server chatting software using OLE technology Start/End date Sep. 1996 – Feb. 1997, (6 months development period) My contribution 100% of 200 lines of codes Language(Library) Visual Basic (OLE) Platform MS Windows Features Client/server model of chatting, working over network Customer/User Remark Title Undergraduate project Mini operating system with boot-up function Start/End date Sep. 1994 – Feb. 1995, (6 months development period) My contribution 100% of 200 lines of codes Language(Library) Assembly Platform Mini OS which I developed Features Bootable, cursor, time/date, Customer/User Remark Title Undergraduate project Geographic Information System for Pusan City Start/End date Sep. 1996 – Feb. 1997, (6 months development period) My contribution 100% of 500 lines of codes Language(Library) Visual Basic Platform MS Windows Features Map viewing, Location registration, etc. Customer/User Remark Undergraduate project My logo artwork for Ultra fast optical network center and Samsung software associate