Robotics Research at CCNY
John (Jizhong) Xiao
Robotics and Intelligent Systems Lab
Department of Electrical Engineering
City College of New York
Tel: 212-650-7268
Email: jxiao@ccny.cuny.edu
Website: http://134.74.16.73
City College of New York
1
Current Projects
• NSF MII Planning Project:
– Center of Perceptual Robotics and Intelligent
Systems (PRISM Center at CCNY)
– Funding Agency: NSF Minority Institutional
Infrastructure Program
• Wall-climbing Robot project
– Funding Agency: Army Research Office
• Smart Brain project
– Funding Agency: NSF Major Research Instrument
Program
City College of New York
2
Wall-climbing Robot Project
• Project Title:
– Cooperative Wall-climbing Robots in 3D
Environments for Surveillance and Target Tracking
• The objective:
– develop a modular, re-configurable, wall-climbing
robotic system and to investigate intelligent control
methods and vision algorithms to control and
coordinate a team of such robots to perform various
defense, security, and inspection missions.
• Principle Investigators:
– John Xiao (EE), Zhigang Zhu (CS)
– Ali Sadegh (ME)
City College of New York
3
Wall-climbing Robot Project
• Dream:
– transform the present 2-D world of mobile rovers into a new 3-D
universe.
– move on ground, climb walls, walk on ceilings, transit between
surfaces.
• Applications:
– Urban warfare applications: surveillance and reconnaissance,
weapon delivery, guiding perimeter around a building, etc
– Security and counter-terrorist applications: intelligence
gathering about a hostile situation within a building, etc.
– Inspection and maintenance applications: routine inspection
of buildings, nuclear containment domes, and other hard-toreach places, inspection of aircraft, sand blasting of ship hulls,
etc.
– Other Civilian applications: assistance in firefighting, search
and rescue operations, etc.
City College of New York
4
Wall-climbing Robot Project
• Challenges:
– Adhesive mechanism
• strong attraction force on various wall surfaces
(brick, wood, glass, stucco, plaster, and metal)
• without sacrificing mobility
– Transition Mechanism
• wheeled robot to achieve quick motion
• articulated structure for smooth transition
• modular design combine two
– Control/coordination of multiple robot modules
– Vision research for surveillance applications
City College of New York
5
Existing Technologies and Robots
• Existing Technologies
– magnetic attraction devices
– vacuum suction techniques
– biologically inspired
• gecko foot
• limbed devices
– aerodynamics attraction
• vortex attraction technique
• attraction generated by propeller
City College of New York
6
Existing Technologies and Robots
JPL-Stanford rock climber
Avionic Instruments Inc.
Vortex attraction technique
MSU “Flipper” & “Crawler”
City College of New York
CMU gecko inspired climber
• Existing wall-climbers:
iRobot’s Mecho-Gecko
7
Wall Climber: Adhesive Mechanism
• Design alternatives:
– vacuum pumps (MSU climber)
– vortex attraction device
– vacuum rotor package
air out
Exhaust
(inner)
air out
Exhaust
(outer)
air in
City College of New York
Vacuum
motor
Vacuum
impeller
8
Wall Climber: Vacuum Chamber Seal
• Inflated Tube Skirt
Seal
attraction force is so strong that it
anchored the device to wall surfaces
• Flexible Bristle Skirt
Seal
trade-off between sealing and mobility
City College of New York
9
Wall Climber: Selected Design
• Selected Design
–
–
–
–
vacuum rotor package
flexible bristle skirt seal
differential drive
pressure force isolation rim (re-foam)
• improves mobility, & enhances sealing by reducing the
deformation of the skirt
Re-foam
Plate
Vacuum Off
Vacuum On
Re-foam
PressureForce
Skirt
Reaction forces from weight
and pressure force on
Drive wheel
outer rim area only
Reaction forces from weight
City College of New York
10
Wall Climber: Transition Mechanism
• Transition Mechanism
• Modular Design
Four wall-climber modules are configured
to form a larger wall-climbing robot which
can carry heavy payload
City College of New York
11
DSP-based Control System
6 Digital I/O Sensors
ChA
• Actuator and sensor
suite
•TMS320F2812 DSP
from Texas Instruments
Inc.
QEP1
PWM1
GPIOB2
PWM2
QEP2
ChB
ChA
M2
Encoder
PWM3
GPIOB3
PWM4
QEP4
ChB
ChA
IN1
EN
IN2
FB
OUT1
33887
MotorolaOUT2
QEP3
CAP3
IN1
EN
ChB
M1
Drive Motor
OUT1
33887
IN2 Motorola
OUT2
FB
M3
Encoder
M2
Drive Motor
CAP6
PWM5
GPIOB4
• 32-bit Processor
• Target for control
applications
GPIOF, 8,9,10,11,12,13
M1
Encoder
PWM6
Ultrasonic
Sensor
F2812 DSP
Trig
GPIOB7
eco
Presssure
Sensor
P-Sensor2
P-Sensor1
Valve1
IR
Sensor
(SHARP)
OUT1
33887
IN2
MotorolaOUT2
M3
FB
Lift Motor
PWM7
XINT1
IN1
EN
GPIOB5
PWM8
IN1
OUT1
33887
OUT2
IN2 Motorola
EN
ADCINB3
M3
Vacuum Motor
ADCINB4
ADCINB0
ADCINB1
ADCINB2
Magnetic
ADCINA0
ADCINA1
ADCINA2
ADCINA3
Accelerometer
GPIOB6
ADCINA7
ADCINB5
ADCINB6
ADCINB7
Host Computer
ADCINA4
ADCINA5
ADCINA6
SCI-A
RS232
GYRO
MARG
SCI-B
RS232
City College of New York
Decoder
Receiver
12
Wall Climber: Software Structure
Climbing Robot
Environment
Microcamera
Initial & goal
configurations
Motion
Planner
MARG
sensor
Feasible motion
sequence
Command
Interpreter
Task level
commands
Task Level
Scheduler
Motor1
Motion Status
Sonar
Sensor
Encoder
Motor 2
Infrared
Sensor
Encoder
Motor 3
Motion steps
Desired angles
Motor
Controller
Remote
Controller
Encoder
Valve
Motor
Operator
Commands
Trajectory
Planner
Pressure Sensors
Wall Surface
City College of New York
13
CCNY Wall Climber Prototypes
Prototype I, vortex attraction
Prototype III, vacuum rotor package
Prototype II, inflated tube seal
Prototype II, flexible bristle skirt seal
City College of New York
14
CCNY Wall Climber Prototypes
• Video
City College of New York
15
Smart Brain Project
• Project Title:
– Smart Re-configureable Miniature Robot Systems
Based on System on Programmable Chip Technology
– NSF MRI Instrument Development
• The objective:
– to develop highly-adaptive computation module based
on SoPC technology (FPGA) for ultra-small robots
– to realize onboard sensor processing, advanced motion
control, and reliable wireless communication
• Principle Investigators:
– Umit Uyar (EE), John Xiao (EE)
City College of New York
16
Project Overview
• FPGA technology
– programmable logic  programmable systems
– integrate FPGA logic, embedded high-performance
processors, digital signal processor (DSP) blocks,
and multi-gigabit transceivers, making FPGA a
versatile technology for high-end research and
commercial products.
• FPGA Device Features
–
–
–
–
Xilinx Virtex-II Pro family FPGA device
two 32-bit IBM PowerPC 405 cores
FPGA logic, DSP blocks
10M of block RAM, off-chip memory as a gap-stop
measure
– Xilinx Intellectual Property (IP) Core library
City College of New York
17
Project Overview
• Benefits
– flexibility, reconfigureability
• hardware reconfigureable, software reprogrammable
– hardware/software partitioning
• high-speed logic implementation in FPGA fabric &
high-flexibility software code in Power PC
– IP core library to achieve basic robotic functions
• pre-verified, reusable
• satisfy the requirements for control, communication,
and onboard vision processing capability of miniature
robots
City College of New York
18
FPGA-based Multiprocessor
RF
Module
OPB
Arbiter
Power
Amplifier
Encoder
architecture
Motor
On-Chip Peripheral BUS (OPB)
PWM
PWM
Logic
PLB to OPB
Bridge
INT
OPB
Timer/
Counter
OPB
Timer/
Counter
Encoder
Counter
CAPT0 CAPT1
Phase A
Quadrature
Logic
PowerPC
405 Core
Phase B
A/D
• FPGA fabric is used for
custom logic and
interfaces.
Virtex-II Pro
FPGA
GPIO
Auto-Scan
Circuitry
PLB
Arbiter
Sensors
OPB
Arbiter
Processor Local BUS (PLB)
• A processor-centric
IIC Bus
Interface
UART
16
SDA
SCL
8
Y0-Y7
UV0-UV7
HREF
PCLK
VSYNC
8
CLK
Q0
Binary
Q16
Counter
(V6.0)
ACLR
17
1D0
2D0
1D15
2D15
Dual Port
BRAM
1A0
(V5.0) 2A0
1A16
2A16
1OE
2OE
1WE
2WE
City College of New York
Fast Fourier
Transform
(FFT V2.0)
Processor Local BUS (PLB)
PowerPC
405 Core
I 2C
Controller
Color Camera
Module
1D0
2D0
1D15
2D15
1A0
PLB to OPB
Bridge
• Single board FPGAbased multiprocessor for
robotics applications
PLB Block
RAM
Interface
Controler
Dual Port
BRAM
(V5.0) 2A0
On-Chip Peripheral BUS (OPB)
IIC Bus
Interface
SDRAM
Controller
Discrete Cosine
Transform
(2D DCT V2.0)
PLB Block
RAM
Interface
Controler
1A16
2A16
1WE
2WE
1OE
2OE
PLB Block
RAM
Interface
Controler
DMA
Controller
PLB
Arbiter
19
Recent Progress
• Logic design for
motor control (PWM,
encoder reading)
• Demonstration of
virtual backbone
concept for reliable
server pooling
Virtex-II Pro ML300 Evaluation board
City College of New York
20
Thank you!
City College of New York
21