CSUMS………………………. 1 VISITORS .............................. 2 2008 & 2009 GRADUATES ........ 3 BI-ANNUAL REPORT 2008 & 2009 COMPUTER VISION LAB RESEARCH ............................ 4-5 COMPUTER VISION LAB PUBLICATIONS 2008 & 2009 .. 6-7 GUEST SPEAKERS ................. 8 Computational Science Training for Undergraduates in the Mathematical Sciences (CSUMS) With an Emphasis on Computer Vision and Imaging Science On September 16, 2008 the National Science Foundation awarded a grant (in the amount of $585,198 for a 3 year period with a possible extension of two more years with additional funding of $395,126) entitled “CSUMS: Computational Mathematics with Emphasis on Computer Vision and Imaging Science” Professor Mubarak Shah as PI and Profs. Constance M. Schober, Niels da Vitoria Lobo, Piotr Mikusinski and Xin Li as co-PIs. Our CSUMS has a cohort of 10 participants per year, for five years. We are currently in the second year of the program. The key distinctive elements of our approach are (1) to have a full year training in carefully designed course work so that the participants can master the mathematical and computational fundamentals, (2) to engage each participant in a meaningful research project integrated into the year-long program, (3) to present each participant with several possible project topics, so that they can feel they have chosen a project which is most interesting to them, (4) to immerse the participants in the general research environment essentially as if they were graduate students, and (5) to develop the participants’ skills in communicating scientific ideas in writing 2009 GAUSS Group (from left to right): Dr. Xin Li, Dr. Robert Muise, and oral presentations by Dr. Mubarak Shah, Devina Shiwlochan, Dr. Connie Schober, Laura Norena, Leon Guerrero, presenting at professional Dr. Piotr Mikusinski, Maria Villareal, Christopher Huff, Maria Boak, Johann Veras, meetings. Steven Schraudner, Dr. Lobo The Department of Mathematics is heading this project in which a team effort of faculty members from math, computer science, education and industry will introduce a yearlong computational mathematics research and training program based on the successes and experience of the team in undergraduate research (NSF REU) in computer vision and imaging science over the past twenty years. With a year-long training program in computational mathematics using exciting applications of mathematics in images and videos as motivating examples, it is possible to provide participants with a solid background in both mathematical theory and problem solving techniques to pursue careers and graduate study in fields that require integrated strengths in computation and the mathematical sciences. The project restructures and improves the current curriculum in computational mathematics track at UCF, making impact on a wide range of students. The first cohort of CSUMS participants have completed the inaugural year of the program. Maria Boak and Christopher Huff both graduated and were accepted to the Ph.D. program at UCF. Devina Shiwlachon also graduated and has applied to the pre-med and bioinformatics graduate programs. For more information, please visit http:// www.math.ucf.edu/csums. -1- Group Computer Vision Lab University of Central Florida Dr. Mubarak Shah Agere Chair Professor Director, Computer Vision Lab Ms. Cherry Place Laboratory Manager Research Associates Dr. Shandong Wu Ph.D. Students Subhabrata Bhattacharya Haroon Idrees Hamid Izadinia Salman Khokhar Baoyuan Liu Ramin Mehran Omar Oreifej Enrique Ortiz Ryan Patrick Kishore Reddy Vladimir Reilly Imran Saleemi Guang Shu Berkan Solmaz Imran Naveed Syed Gonzalo Vaca Yang Yang Amir Roshan Zamir B.S. Students Ada Brewton Arian Caraballo Joshua DuLac David Krauser Ryan McEachin Daniela Zicavo High School Students Nelson Tan From left to right: Ramin Mehran, Guang Shu, Shandong Wu, Kishore Reddy, Yang Yang, Arjun Nagendran, Naveed Imran Syed, Berkan Solmaz, Imran Saleemi, Vladimir Reilly, Mubarak Shah, Soumyabrata Dey, Mikel Rodriguez, Aditya Gupta, Subhabrata Bhattacharya, Jonathan Poock, Jingen Liu, Haroon Idrees, Enrique Ortiz, Josh DuLac, Omar Oreifej, Gopi, Vajravelu The picture was taken in September 2009. Visiting Scholars Zahid Riaz Xiaoguang Di Ph.D. student working with Profs. Bernd Radig and Micheal Beetz at the Technical University of Munich. Worked with the UCF Vision lab on Human Tracking and Action Recognition from December 2009 through April 2010. Riaz was thankful for the lab’s cooperation during his stay and hopes to be in contact in the future for collaboration. Associate Professor from Harbin Institute of Technology received funding from the China Scholarship Council to study abroad for twelve months. Di arrived to the Computer Vision Lab in December 2009 and has been pursuing Computer Vision and Image Processing research. Marco Zini Wang Yong Student working with Professor Rita Cucchiara at Modena University, Modena, Italy. He came to the Computer Vision Lab in November 2009 for six months to work on his M.S. thesis related to tracking in high density crowds. Ph.D. Candidate from Shanghai Jiao Tong University also received funding from the China Scholarship Council to visit the Computer Vision Lab to complete his work on “Multiple Cameras for Scene Understanding and Object Tracking”. Yong arrived in September 2009 and has recently extended his visit until December 2010. -2- Computer Vision Lab Graduates Ph. D. Saad Khan Saad Ali Arslan Basharat Topic: Multi-view Approaches to Tracking, 3D Reconstruction & Object Class Detection Graduation Term: Spring 2008 Current Affiliation: Sarnoff Corporation Topic: Taming Crowded Visual Scenes Graduation Date: Spring 2008 Current Affiliation: Sarnoff Corporation Topic: Modeling Scenes and Human Activities in Videos Graduate Term: Spring 2009 Current Affiliation: Kitware, Inc. Jingen Liu Pavel Babenko Topic: Learning Semantic Features for Visual Topic: Visual Inspection of Railroad Tracks Recognition Graduation Term: Spring 2009 Graduate Term: Spring 2009 Current Affiliation: Madison Research TechCurrent Affiliation: Sarnoff Corporation nologies M.S. Yusuf Aytar Ryan Faircloth Philip Berkowitz Topic: Semantic Video Retrieval Using High-level Context Graduation Term: Spring 2008 Current Affiliation: Oxford University Topic: Combining Audio and Video Tempo Analysis for Dance Detection Graduation Term: Summer 2008 Topic: A Statistical Approach to View Synthesis Graduation Date: Summer 2009 Current Affiliation: DRS B.S. (Honors in the Major) Brandyn White Topic: Using FPGAs to Perform Embedded Image Registration Graduation Term: Summer 2009 Current Affiliation: University of Maryland -3- UCF Researcher Developing Computer Program to Detect, Measure Brain Tumors The same techniques used to detect suspicious activity in airports, stadiums and other public places are now being used by the UCF researcher who invented them to find and measure potentially life-threatening brain tumors. Mubarak Shah, UCF’s Agere Chair professor of Computer Science and one of the world’s most eminent researchers in the rapidly developing field of computer imaging, has received $400,000 from the National Institutes of Health to develop a computer program to analyze brain scans produced by magnetic resonance imaging (MRI.). The twoyear grant funded in May 2009 is the first UCF has received from money allocated by the American Recovery and Reinvestment Act stimulus program. The funding will enable Shah and his collaborators -- Dr. Nicholas Avgeropoulos, a neuro-oncologist with Orlando Health System, and Dr. David Rippe, a neuroradiologist with Sunshine Radiology at Florida Hospital Zephyrhills -- to work together on the complex task of automatically measuring and comparing the size of a tumor in 3D from MRI scans. Nearly a decade ago, Shah approached Rippe, who at that time was chairman of the radiology department at Florida Hospital Orlando, looking for ways to use computer technology to help those in the medical profession. The alliance was “a natural fit,” Rippe said. “Radiologists use computers to look at scans, but this is taking the next step – allowing computers to help radiologists analyze the pictures and enabling an automated method to calculate the size of tumors,” he said. Radiologists are typically hindered in their analyses by a variety of factors, such as tumors that are irregular in shape or have jagged edges, tumors with liquefied centers, or surrounding tissue that is deformed or changing shape. “Not only are the changes visually hard to see, we also want numbers to quantify the types of changes we are talking about,” Rippe said. Those numbers help determine whether a particular treatment plan such as radiation or chemotherapy is working. Dr. David Rippe Automated analysis of a small data set using Shah’s preliminary method has been shown to be up to 90 percent accurate compared to the analyses provided by the radiologists. Shah said some of the challenges include making sure the typically low-resolution scans can be converted to the high-resolution images needed for computers to precisely measure tumors. He also must perform extensive experiments with a large data set to validate his method. He has partnered with a UCF biostatistician, Xiaogang Su, to ensure that the measurements are statistically correct. Shah’s work has typically focused on analyzing images for signs of suspicious or dangerous behaviors or threats. While at UCF, he has received more than $7.5 million in funding for projects ranging from visual monitoring of railroad grade crossings for the Department of Transportation to automatic classification and analysis of reconnaissance videos for the Department of the Interior. (Courtesy of UCF Newsroom) UCF-50 Dataset UCF50 is an action recognition dataset with 50 action categories, consisting of realistic videos taken from YouTube. This dataset is an extension of the YouTube Action dataset which has 11 action categories. The dataset can be downloaded from the following address: http://vision.eecs.ucf.edu/ datasetsActions.html#UCF50. Most of the available action recognition datasets are not realistic and are staged by actors. In our dataset, the primary focus is to provide the computer vision community with an action recognition dataset consisting of realistic videos which are taken from YouTube. Our dataset is very challenging due to large variations in camera motion, object appearance and pose, object scale, viewpoint, background clutter, illumination conditions, etc. For all of the 50 categories, the videos are divided into 25 groups, where each group consists of more than 4 action clips. The video clips in the same group may share some common features, such as the same person, similar background, similar viewpoint, and so on. STATESS In December of 2009, the National Science Foundation (NSF) awarded a grant of $599,973 to the University of Central Florida for a project entitled “Students Actualizing Talent at Education’s Subsequent Stages (STATESS),” Professor Mubarak Shah, as PI and Brian Moore, Niels da Vitoria Lobo and Xin Li as co-PIs. The award is effective January 1, 2010 and expires December 31, 2013. This project offers up to twenty scholarships per year in the range of $4,000 to $10,000 per year for recent undergraduate and transfer students and first year graduate students in Mathematics and Computer Science as well as in other Science, Technology, Engineering, and Mathematics (STEM) fields. These scholarships are renewable up to four years, and they are intended for students that might not have the opportunity to attend university otherwise. Scholarship recipients will be privileged to several resources and activities that are intended to ensure the student’s success in completing their undergraduate or graduate degrees. -4- AAAS Elects Shah as a Fellow Dr. Mubarak Shah was named fellow by one of the world’s largest and most wellrespected scientific societies, the American Association for the Advancement of Science (AAAS) in November 2009. He was among 531 people nationwide and 4 professors at UCF to be selected by their peers for scientifically or socially distinguished efforts to advance science or its applications. “To have our faculty members recognized by their peers for outstanding achievements in science and engineering is an honor for our entire university community,” said M.J. Soileau, Vice President for Research and Commercialization. “In classrooms and laboratories throughout our university, students are working with world-class professors who are leaders in their fields. UCF has long recognized the achievements of these scholars, and it is great to see them earn well-deserved national recognition.” Dr. Diane Chase, Dr. Debra Reinhart, Dr. Mubarak Shah Shah, UCF’s Agere Chair Professor of Computer Science, was selected for his outstanding contributions to video surveillance and monitoring, shape from shading, active contours, human action recognition and object tracking in computer vision. He was recognized on February 20, 2010 during the AAAS annual meeting in San Diego and was also included in the AAAS News & Notes section of the journal Science on December 18, 2009. (Courtesy of UCF Newsroom) UCF Teaches Aspiring Computer Vision Researchers from Around the Country In 2008, 11 students from across the country were involved in the University of Central Florida’s (UCF) annual Research Experience for Undergraduates in Computer Vision funded by the National Science Foundation. This year’s group included five students from UCF and six other undergraduates from schools such as the University of Florida, University of Southern California and Rice University. During the summer months, when most students are thinking about anything but schoolwork, UCF hosts Research Experience for Undergraduate (REU) program in disciplines ranging from nanotechnology and machine learning to computer vision and optics. The programs aim to encourage undergraduates to pursue research endeavors and graduate school. During the past two decades, nearly 200 undergraduates from schools all over the nation have taken part in the Computer Vision program, UCF’s longest running—going on 21 years. Its participants have co-authored more than 60 research papers, and six are now faculty members at universities. During 2008’s summer program participants attended introductory computer vision classes and conducted research in UCF labs. Students worked individually on projects ranging from video retrieval to image enhancement to robotics. In the fall, they continued their research projects and gathered together twice more during the school year to review and discuss their endeavors. By the end of the academic year, students had to write a comprehensive report describing their project and could choose to submit their findings to a science paper or journal. Chabra, a junior at USC who is originally from Long Island, NY, is contributing to UCF’s TRECVid project. TRECVid aims to solve the problem of locating specific segments of footage in long-running videos. According to Chabra, the UCF program has led him to consider future career paths in the industry and academia. Whatever he decides, Chabra said that “after finishing up my undergraduate studies, I would definitely like to go to graduate school.” Director Mubarak Shah believes that the Research Experience for Undergraduates program is great exposure for students who want to excel in the computer vision field. “The program has contributed to science and engineering by exposing undergraduates to the excitement and challenge of research,” Shah said. “Due to that, roughly half of the students have gone on to graduate school.” UCF’s Computer Vision Lab has received more than $1.2 million from the NSF to encourage promising students’ interests in science for the past 21 years. (Courtesy of UCF Newsroom) 2008 REU Group (from left to right): Dr. Mubarak Shah, Jason Hochreiter, Cynthia Atherton, Sarah Applegate, Steven Braeger, Ajay Chabra, Dr. Marshall Tappen, Karthik Prabhakar, Joshua Hartman, Lam Tran, Nicholas Hirsch, Alexis Oyama, Dr. Niels Lobo -5- 2008 & 2009 Publications & Invited Talks BOOK Omar Javed and Mubarak Shah, Automated Multi Camera Surveillance: Algorithms and Practice, Springer, September 2008. Jingen Liu, Saad Ali, and Mubarak Shah, Recognizing Human Actions Using Multiple Features, IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska 2008. Jingen Liu, Yang Yang and Mubarak Shah, Learning Semantic Visual Vocabularies Using Diffusion Distance, IEEE Conference on Computer Vision and Pattern Recognition, Miami, Florida, 2009. Pingkun Yan, Saad M. Khan, and Mubarak Shah, Learning 4D Action Feature Models for Arbitrary View Action Recognition, IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska 2008. Jingen Liu, Jiebo Luo and Mubarak Shah, Recognizing Realistic Actions from Videos ‘in the Wild’, IEEE Conference on Computer Vision and Pattern Recognition, Miami, Florida, 2009. BOOK CHAPTER Yaser Sheikh, Omar Javed and Mubarak Shah, Object Association Across Multiple Cameras, in Multi-camera Networks: Concepts and Applications, Elsevier, editors Hamid Aghajan and Andrea Cavallaro, 2009. CONFERENCES Mikel D. Rodriguez, Javed Ahmed, and Mubarak Shah, Action MACH: A Spatiotemporal Maximum Average Correlation Height Filter for Action Recognition, IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska 2008. Saad M. Khan, and Mubarak Shah, Reconstructing Non-stationary Articulated Objects in Monocular Video using Silhouette Information, IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska 2008. Jingen Liu, and Mubarak Shah, Learning Human Actions via Information Maximization, IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska 2008. Yusuf Aytar, Mubarak Shah, and Jiebo Luo, Utilizing Semantic Word Similarity Measures for Video Retrieval, IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska 2008. Arslan Basharat, Alexei Gritai, and Mubarak Shah, Learning Object Motion Patterns for Anomaly Detection and Improved Object Detection, IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska 2008. Andrew Miller, Mubarak Shah, and Don Harper, Landing a UAV on a Runway Using Image Registration, International Conference on Robotics & Automation, 2008. Jun Xie, Shahid Khan, and Mubarak Shah, Automatic Tracking of Escherichia Coli Bacteria, 11th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI, September 6-10, New York City. 2008. Saad Ali and Mubarak Shah, Floor Fields for Tracking in High Density Crowded Scenes, European Conference on Computer Vision, Marseille, France, October 12-18, 2008. Min Hu, Saad Ali, and Mubarak Shah, Detecting Global Motion Patterns in Complex Videos, International Conference on Pattern Recognition, December 2008. Min Hu, Saad Ali and Mubarak Shah, Learning Motion Patterns in Crowded Scenes Using Motion Flow Field, International Conference on Pattern Recognition, December 2008. Jun Xie, Min Hu, and Mubarak Shah, Unfolding Warping for Object Recognition, International Conference on Pattern Recognition, December 2008. -6- Ramin Mehran, Alexis Oyama and Mubarak Shah, Abnormal Crowd Behavior Detection using Social Force Model, IEEE Conference on Computer Vision and Pattern Recognition, Miami, Florida, 2009. Kishore Reddy, Jingen Liu, and Mubarak Shah, Incremental Action Recognition Using Feature-Tree, International Conference on Computer Vision, September 2009. Arslan Basharat and Mubarak Shah, Time Series Prediction by Chaotic Modeling of Nonlinear Dynamical Systems, International Conference on Computer Vision, September 2009. Mikel Rodriguez, Saad Ali and Takeo Kanade, Tracking in Unstructured Crowded Scenes, International Conference on Computer Vision, September 2009. Yang Yang, Jingen Liu, and Mubarak Shah, Video Scene Understanding Using Multi-scale Analysis, International Conference on Computer Vision, September 2009. http://www.vision.eecs.ucf.edu/publications INVITED TALKS (by Professor Shah) JOURNALS Arslan Basharat, Yun Zhai, Mubarak Shah, Content Based Video Matching Using Spatiotemporal Volumes, Computer Vision and Image Understanding, Volume 110, Issue 3, June 2008, Pages 360-377. J. Ahmed, M.N. Jafri, M.Shah, M. Akbar, Real-time edge enhanced dynamic correlation and predictive open-loop carfollowing control for robust tracking, Machine Vision and Applications (2008) 19:1-25. Pingkun Yan, Xiaobo Zhou, Mubarak Shah, and Stephen T.C. Wong, Automatic Segmentation of High Throughput RNAi Fluorescent Cellular Images, IEEE Trans. Information Technology in Biomedicine, Volume 12, Number 1, January 2008. Jun Xie, Pheng-Ann Heng, and Mubarak Shah, A Shape Matching Approach Using Skeletal Features and Context Descriptor, Pattern Recognition, Volume 41, Issue 5, May 2008. Yaser Sheikh, and Mubarak Shah, Trajectory Association Across Multiple Airborne Cameras, IEEE Transactions on PAMI, Volume 30, No. 2, February 2008, Pages 361-367. Omar Javed, Khurram Shafique, Zeeshan Rasheed and Mubarak Shah, Modeling intercamera spacetime and appearance relationships for tracking across nonoverlapping views, Computer Vision and Image Understanding, Volume 109, Issue 2, February 2008, Pages 146-162. A. Yilmaz and M. Shah, A Differential Geometric Approach To Representing the Human Actions, Computer Vision and Image Understanding Journal, Vol. 109 No. 3, pp.335-351 2008. Pavel Babenko and Mubarak Shah, MinGPU: A Minimum GPU Library for Computer Vision, Journal of Real-Time Processing, (2008) 3:255-268. J. Xie, P.A. Heng and Mubarak Shah, Image Diffusion Using Saliency Bilateral Filter, IEEE Transactions on Information Technology in Biomedicine, Volume 12, Number 6, 768-771, 2008. Saad M. Khan and Mubarak Shah, Tracking Multiple Occluding People by Localizing on Multiple Scene Planes, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume: 31, Issue: 3, pp 505-519, March 2009. Pingkun Yan, Ashraf A. Kassim, Weijia Shen, and Mubarak Shah, Modeling Interaction for Segmentation of Neighboring Structures, IEEE Trans. on PAMI, Volume 31, Number 3, pp 505-519, March 2009. I. Saleemi, K. Shafique, M.Shah, Probabilistic Modeling of Scene Dynamics for Applications in VIsual Surveillance, IEEE TPAMI 2008, Vol 31, No. 8, August 2009. Jun Xie, Shahid Khan, and Mubarak Shah, Automatic Tracking of Escherichia Coli in Phase-Contrast Microscopy Video, IEEE Trans. on Biomedical Engineering, Vol. 56, no2, pp. 390-399, 2009. Alexei Gritai, Yaser Sheikh, Cen Rao and Mubarak Shah, Matching Trajectories of Anatomical Landmarks under Viewpoint Anthropometric, and Temporal Transforms, International Journal of Computer Vision (IJCV), Volume 84, Issue 3, Pages: 325-343, September 2009. -7- Visual Analysis of Crowded Scenes, invited talk at DARPA Workshop on Cityscapes, Reno, Nevada, March 8-9, 2008. Video Surveillance and Monitoring, Klagenfurt University, Klagenfurt, Austria, July 15, 2008. Human Action Recognition Using Bag of Video Words, Vienna University of Technology, Vienna, July 17, 2008. Video Surveillance and Monitoring, Austrian Research Centers GmbH, Vienna, August 1, 2008. Tracking Across Multiple Moving Cameras, Missouri University of Science and Technology, ACM Distinguished Lecture, Rolla, Missouri, October 10, 2008. UCF VIRAT efforts, Audacity/VIRAT workshop, Lockheed Martin, Herndon, VA, February 18, 2009. Taming Crowded Visual Scenes, ECE department, Wayne State University, Detroit, MI, April 7, 2009. Taming Crowded Visual Scenes, EECS department, University of Michigan Ann Arbor, MI, April 8, 2009. Taming Crowded Visual Scenes, DARPA ISAT workshop, Berkeley, CA, May 6, 2009. Visual Analysis of Crowded Scenes, International Workshop on Video, Barcelona, Spain, May 27, 2009. An Overview of Visual Tracking in EO and IR Imagery, keynote talk, IEEE Workshop on Object Tracking & Classification in and Beyond the Visible Spectrum (OTCBVS), CVPR 2009, Miami Beach, FL, June 20, 2009. Seminars http://www.eecs.ucf.edu/index.php?id=research/seminars Naresh Cuntoor University of Maryland Activity Modeling Using an SVD-like Decomposition September 10, 2008 Amit Roy-Chowdhury University of California, Riverside From Single Images to Camera Networks: Modeling and Inference Strategies September 15, 2008 Hierarchical Graph-based Representations for Segmentation, Walter Kropatsch Vienna University of Technol- Tracking and Shape Matching ogy December 3, 2008 Leandro Loss University of Nevada, Reno An Iterative Multi-Scale Tensor Voting Scheme for Perceptual Grouping of Natural Shapes in Cluttered Backgrounds December 5, 2008 Monique Thonnat INRIA Semantic Activity Recognition for Visual Surveillance and Healthcare Monitoring December 9, 2008 Jean-Marc Odobez IDIAP Research Institute Analysis of the Visual Focus of Attention in Group Conversation December 12, 2008 Pingkun Yan Philips Research Segmentation of Prostate for Image Guided Targeted Biopsy February 9, 2009 Dinesh Manocha University of North Carolina at Chapel Hill Bringing Realism to Virtual Environments: Sounds and Crowds February 11, 2009 Hanan Samet University of Maryland Sorting in Space February 19, 2009 Jim Rehg Georgia Tech Towards a Theory of Cascaded Detectors March 20, 2009 Arnold Smeulders University of Amsterdam Object Class Recognition March 26, 2009 Donald Geman Johns Hopkins University Stationary Features and Cat Detection March 30, 2009 Ronald Coifman Yale University Wavelets and Applications: Past and Future April 14, 2009 Martial Hebert Carnegie Mellon University Some Steps in Modeling and Understanding in a User’s Environment from Vision Data April 20, 2009 Bernhard Rinner Klagenfurt University Challenges and Opportunities of Distributed Smart Cameras May 20, 2009 Computer Vision Lab Electrical Engineering & Computer Science University of Central Florida 4000 Central Florida Blvd. Orlando, FL 32816-2362