Antarctic Survey Telescope x 3 (AST3)
On-site Real-time Data Processing
Bin Ma（马斌）
NAOC
On behalf of Collaborations at
NAOC: 商朝晖、刘强、胡义、Jason Quinn、…
TJU：孙济洲、于策、肖健、李继良、康文东、杨
沐津、余骏…
2012-11-29
China-VO 2012 @ YiChang
OUTLINE
AST3:
• On-stie real-time pipeline & database
• Onsite Data Processing in 2012
• Onsite Data Processing in the future
Antarctic Survey Telescope x 3 (AST3)
• Three 50/68cm modified Schmidt Telescopes (NIAOT)
• CCD Camera (STA 1600-FT)
10k x 10k, 9 micron/pixel , Plate Scale: 1 arcsec/pixel
FOV: ~ 2.93*1.47 ~ 4.3 square degree
AST3 Survey Control and Data System
Survey Area Selection
Survey Scheduling
Data Acquisition and Storage
Real-time Pipeline
Database (onsite & in Beijing)
Onsite Real-time Data Processing
Requirements
•
•
•
•
Fast
Stable
Fully automatic with configuration
Automatic recover from error/crash
Onsite Real-time Data Processing System
1. Daemon running on background
2. Real-time pipeline
3. Database for Real-time Photometry Results
Daemon
• Detect new image
• Start & manage pipeline task
• log
Daemon
Protect
service
Cron
Daemon
Real-time Pipeline
Open source software + programming
• Image Reduction
Flat, Bias, cosmic ray……
• Photometry & Astrometry
x, y, mag ... (by SExtractor)
(x, y)  (ra, dec) (by SCAMP)
• Template Mosaics
registration, coadding (by SWarp)
• Image Subtraction (by HOTPANTS)
• Transients Detection & Classification
Test, Test, Test, …
Data Acquisition
Image
Process
Pipeline
Flow Chart
Image reduction
(bias/flat/cosmic..)
Simulations
Aperture
Photometry
(source extractor)
• Aperture
photometry
=> all sources
• Image
subtraction
photometry
=> transients
Source
Catalog
Aperture
Photometry
Astrometry
Data
Quality
Build
Template
(swarp)
(scamp)
Template exists
Template
Image
Subtraction
Image
Subtraction
(isis/hotpants)
Transien
t Catalog
PSF fitting
Photometry
Source
Catalog
Light Curve Analysis，Stamp Image，Alert
Database, Follow up
Data
Analysis
已有
待完善
图像预处理
Bias，flat，mask
精确的改正图像
测光、位置定标
Sextractor，Scamp
精度，测光定标
模板制作
Swarp
模板精度
图像相减
Hotpants
相减精度
变源探测、判断
Sextractor
效率、准确性，利用AST3
图像测试
变源分类、SN识别
传回切图
切图传输数据量
传回后的人工判断
光变曲线
Transit？
Real-time Pipeline – Performance Test
• 10K*5K simulated image, based on
Guide Star Catalog 2.2, 8*2 subimages
• Intel® i7-620M and Intel® HM55
2 cores / 4 threads, 2660MHz, 4M
Cache, TDP 35W
• 8GB Memory
• Ubuntu 10.04 64bit
Real-time Pipeline – Performance Test
Function
Combine
3-exposure
Photometry
&
Astrometry
Template
Mosaic
Subtraction
Total
Time (sec)
33.1
9.7
5.8
57.0
105.6
Database for Real-time Photometry Results
• Database at Dome A, server in China
• Low bandwidth: OpenPorts ( 128kbps) allows us to
retrieve light catalogs
• Light, open-source database: MYSQL
Database for Real-time Photometry Results
• Observation information:
time, airmass, temp, sky, FWHM...
Monitor the running status
• All sources:
catalogs by Sextractor
position, brightness, shape...
• Variable sources:
light curves
classification
Database for Real-time Photometry Results
• All sources requirements:
i~20:
star 3200/deg2 ,b~60°
(Allen‘s Astrophysical Quantities )
galaxies 2100/deg2 (Yasuda et al. 2001)
total ~ 5.3K/deg2 * 4.3deg2 ~ 23K /field
500 fields ~ 11.5M sources
110d, 3min/field 79K images ~ 1.8G records
30entries/record  > 200 GB
Database for Real-time Photometry Results
In Memory
DB
Upate
Generate a new
AST3_ID
No
Source
Catalog
Generate
HEALPix_ID
Cross
Matching
Matched
Yes
Remote
Storage
Local
Storage
Archive
11M sources ~1GB memory
23K/field ~40sec
Add AST3_ID
to the record
Onsite Data Processing in 2012
Obs in 2012: late March ~ early May
Targets: SNe, LMC, transit, QSO…
Onsite Data Processing in 2012
• Pipeline was not used
designed for automatic SN survey
filename format, FITS header, obs. Mode …
• Simple scripts
aperture photometry
image quality: FWHM, sky brightness…
astrometry calibration
Onsite Data Processing in 2012
• Limiting Mag. ~ 19mag (1min exposure )
AST3-1 60sec
Onsite Data Processing in 2012
• Focal plane
AST3-1 2012-3-17
Onsite Data Processing in 2012
• Astrometry calibration
AST3-1 2012-4-17
Onsite Data Processing in 2012
• Telescope Pointing Correction (TPoint)
Onsite Data Processing in the future
• In 2013:
interactive，script-based，automatic
test；observation
• High performance computing in the future
specific sciences
processing in detail
Thanks!