AstroGrid and the
Virtual Observatory
Concept
(www.astrogrid.org)
Nicholas Walton
AstroGrid Project Scientist
Insitute of Astronomy
University of Cambridge
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p1
Printed: 29/04/02
Presentation Overview
Virtual Observatories and why are they needed
Technological drivers
Astronomical drivers
Examples of 'AstroGrid' science
The Grid as an enabling technology
AstroGrid and the UK escience programme
AstroGrid in an International Context
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p2
Printed: 29/04/02
What is a Virtual Observatory?
Modern astronomy: matching multisourced data, e.g:
Find a distant luminous galaxy in an IR survey
Look for it in an Xray database: is it a hidden quasar
Aim to make all archives, 'speak the same language':
All searchable and analysable by the same tools
All data sources accessible through a uniform interface
All data held in distributed databases that appear as one
Form the 'Digital Sky'
Eventual interface to real observatories
Achieving this transparent 'Virtual Observatory'
involves solving many technological and
sociological problems. But others have these as well ...
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p3
Printed: 29/04/02
Supernova Remnant
CassiopeiaA – a 300
year old Supernova
The Challenge and
Opportunity of multi
Wavelength data:
Shocks seen in the Xray
Chandra image
Dust shows in the IR
Mapping es in
the magnetic
field as revealed
by Radio data
Heavy elements
seen in the optical
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p4
Printed: 29/04/02
AstroGrid Drivers: Sociology
Astronomy focuses now on major science themes
e.g. The 'Origins' programme > evolution of the
Universe, formation and evolution of the Solar system
Continuing Collectivization: evolution to 'big'
projects involving many coworkers
Facility class (large common user) instruments
Central development of supporting software
Calibrated archives and access tools (e.g. IPAC)
Information services (e.g. ADS, NED, astroph)
Consortium projects (e.g. MACHO, SLOAN, VISTA)
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p5
Printed: 29/04/02
Drivers: Technological Change I
The Growth of Data
Number and size of telescopes
Optical: ESO's 4x8m VLT, 2x8m Gemini
Xray: XMMNewton, Chandra
submm: ALMA
Increase in size and multiplex
capabilities of instrumentation:
InfraRed: VISTA > 100 GB/nights
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
VIMOS on the VLT
~800 spectra per field
Radio: eMerlin > data rates of 320 Gbps
p6
Printed: 29/04/02
Drivers: Technological Change II
The Growth of Data Archives
Many observatories hold multiTB archives
New initiatives setup to support new observing
capabilities (e.g. TeraPix for CFHT imaging surveys)
In the optical, need to ingress allsky survey's
Whole sky at 0.1 arcsec/pix is 100TB
Increasing Importance of Archival Data
Time on expensive facilities (e.g. HST) only awarded
if the archive has been searched before hand
This trend is continuing, driving observatory and user
demand for more accessible archival data
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p7
Printed: 29/04/02
Astronomical Drivers: Enabling New Science
Linking the near and distant Universe
Creating the 'Digital Sky'
Comparison of rest frame samples to study evolution of
galaxies
e.g. Combination of Xray, UV, optical and IR datasets
Temporal data measures motions in the Galactic centre,
probes the creation of our Galaxy
The search for extrasolar planets
The planettransit technique using federated survey data for
millions of stars
Key driver – lightcurve+event generation
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p8
Printed: 29/04/02
Empowering Science Driven
Observational Astronomy
Break down the 'wavelength' barriers
e.g. Greater focus on science driven
proposals encouraging the use of data sets
from across a wide range of wavelengths
Increase Coordination across countries
EU support: AVO, EGSO, AstroWise, etc.
Increase Access
East European countries may not be able to
fund a major new telescope but can improve/
access 'Virtual Observatories'
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p9
Printed: 29/04/02
Astronomical Drivers: New Era of Surveys
SuperCOSMOS (UK: till 2002) – optical
Based on Schmidt plates Science database ~2TB
Sloan Digital Sky Survey (US: till 2005) – optical
Dedicated CCD survey telescope: Science database ~10TB
UKIDSS (UK: from 2003) – large infrared survey
IR camera on 4m UKIRT: Science database ~30TB
VISTA (UK: from 2006) – large infrared survey
IR camera on 4m VISTA: Science database ~300TB
LSST (US: from 20078?) large optical survey
Dedicated ~8m telescope: All sky/few nights:~5000TB/yr
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
29/04/02
p10
Printed:
New and Rare Objects
Huge data sets open up the
possibility to find rare objects
Those that are hard to find:
Brown dwarfs, red colours
Those that are rare
Highz quasars, stand out due
to suppression of their blue
colour by the Lyα forest
10's from billions
Hiz QSO's found from SDSS
multicolour data: the shaded
area is domain of QSO's, solid
line is track for increasing z
(Fan et al, 2001, ApJ, 121, 31)
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p11
Printed: 29/04/02
Rare Objects: High Redshift Quasars
Recent results of a 4TB data
mining exercise
Datamining phase took 5 years
4TB of pixel data from 515 year
old UKST data
108 stars+gals found in 2
wavebands
Identified 59 quasars with z>4
Goal: Measure the total amount of
Neutral Hydrogen in the Universe 1
From StorrieLombardi, Irwin,
Billion years after the Big Bang.
McMahon
and Hook, 2001, MNRAS
Next Stage:
10 times larger samples at higher z
Use VST/WFCAM/VISTA data
Use of AstroGrid tools will
be vital to speed process
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p12
Printed: 29/04/02
Astronomical Drivers: The Xray Universe
AstroGrid will enhance the ability to
classify new Xray objects and speed
object sample creation
Issue: federating large scale multi data
!"
!" #$ %!
The XMMNewton team is using
XMM, optical imaging
and IR data to study a number of
fields: see Watson et al (2001)
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
&' (
)* +,) ) +,)
+ !
-' (
)*
+,) .) /
% 0& 1
! & ! +,) 2
, 34 !
!5! & ) 6 4!" 2 )+,!
p13
Printed: 29/04/02
Astronomical Drivers: Cosmology thru SN Ia
TB's of data
AstroGrid will enable rapid typing
of search fields via galaxy photoz's
in support of UK/EU groups, e.g.
EC RTN SN network.
Issue: large scale image deconvolution
matching candidates to z
SCP data from Perlmutter, 2001
snap.lbl.gov/pubdocs/SAUL.pdf
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p14
Printed: 29/04/02
Astronomical Drivers: WD's and Dark Matter
AstroGrid products will enable
the federation of multiepoch
data to increase the number of
White Dwarf's known. With IR
data cool objects will be found
impacting on the Dark Matter
problem.
Issue: TB's of multiepoch, multi
colour data, finding moving objects
WD0346+246, moving at 1.3''/
year, was discovered by Hambly
et al., (2000, Nature, 403, 57).
It is cool, but has blue colours in
the IR due to collision absorption
by H2
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p15
Printed: 29/04/02
Astronomical Drivers: Solar Complexity
SolarB will be launched
in 2005, aimed at studying
the interaction between
the Sun's magnetic field
and its corona
SolarB will be supported
by complementary space/
ground data.
Science often targets
events with many multi
observations required
Image from Marshall Space
Flight Centre, NASA
science.msfc.nasa.gov/ssl/pad/solar/Beyond_SolarB.htm
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p16
Printed: 29/04/02
Solar/STP Drivers: A SolarTerrestial Model
ISTP (ILWS) – NASA/ESA/ISAS – employing a
multimission approach to understand how solar
activity powers mass, momentum, energy flows and
how these impact the earth's upper atmosphere.
Requires Space and Earth data: eg SOHO, Cluster etc
SOHO/EIT – EUV
Yohkoh – Xray
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
NASA: Living With a Star http://lws.gsfc.nasa.gov
p17
Printed: 29/04/02
AstroGrid and the eScience Program
DTI/OST escience programme (~ £120M/3yr)
PPARC funding two major (GridPP – supports
CERN/LHC experiments, AstroGrid) and a few
smaller programmes (e.g. Gravitational Waves)
DTI/OST core escience programme: £50M
National eScience Centre (www.nesc.ac.uk)
Other Grid programs (e.g. MyGrid – Bioinformatics)
Applications leverage Grid technologies
UK contribution to the Globus project
Interaction through e.g. Data Base Task Force and
iVDGL (an international virtual laboratory)
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p18
Printed: 29/04/02
The UK eScience Program
e.g AstroGrid
eScience Centres Application Pilots
IRCs …
eScientists, Grid users, Grid services & Grid Developers
GNT
DBTF
ATF
TAG
NeSC
GSC
UK Core Directorate
eSI
CS Research
Global Grid Forum …
Rapid formation with quick and effective delivery
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p19
Printed: 29/04/02
The Grid: an emerging technology
The Grid seeks to enable dynamic 'virtual organisations'
by integrating distributed compute/storage/resources.
In astronomy Grid Technologies will enable the creation
of Virtual Observatories
the Globus project www.globus.org
Key Grid technologies
Globus toolkit addresses issues of:
security, information discovery, resource management
data management, communication, fault detection, portability
New initiative is the Open Grid Services Architecture
Merges Grid and Web services initiatives
Web services
XML, SOAP, WSDL, .NET
Industry support
IBM, Compaq, Sun, MS etc.
Overview at Global Grid Forum: www.gridforum.org
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p20
Printed: 29/04/02
The Grid Concept: in a picture
Graphic: Dennis Gannon – see http://www.extreme.indiana.edu/~gannon/
Peer Creation
& resolution
Services
Information
Routing
Discovery
service
Event/Mesg
Service
Information/
Naming
Services
(co)scheduling
Service
Security
Service
Accounting
Service
User Help
Services
Monitoring
Service
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p21
Printed: 29/04/02
Astronomical eScience Initiatives
There are now a number of initiatives in astronomy which are
aiming to create Virtual Observatories (or parts of) using
these new Grid developments (and exploiting escience funds!)
AstroGrid
www.astrogrid.org
European Grid of Solar Observations
www.mssl.ucl.ac.uk/grid/egso
Astrophysical Virtual Observatory
www.eso.org/avo
AstroVirtel
ecf.hq.eso.org/astrovirtel
National Virtual Observatory
www.usvo.org
The Digital Sky
The Virtual Sky Project
virtualsky.org
Sloan Digital Sky Survey: SkyServer
skyserver.fnal.gov/en
Skyview
skyview.gsfc.nasa.gov
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
www.cacr.caltech.edu/SDA/digital_sky.html
p22
Printed: 29/04/02
The AstroGrid Project
The AstroGrid project is a distributed project.
News, Forum, Wiki > all interactive to enable
community participation
www.astrogrid.org
news.astrogrid.org
forum.astrogrid.org
wiki.astrogrid.org
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
>
>
>
>
static portal
news and announcements
discussion site
documents, development
p23
Printed: 29/04/02
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p24
Printed: 29/04/02
AstroGrid Development Activities
Workpackages concentrating key areas:
Science and Architecture
Grid technologies
Database interfaces
Network, Storage and Compute resources
Pilot Programme: Opt/nearIR/XMM, solar etc
AstroGrid is also a partner in the Astrophysical
Virtual Observatory (www.eso.org/avo)
AstroGrid works with Opticon's Interoperability
working group on standards such as VOTable
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p25
Printed: 29/04/02
AstroGrid: framework for applications
Enable future development
and deployment of
AstroGrid and External (e.g
NVO) tools:
Automated detection of outliers
in SDSS two colour data
(Connolly et al, 2001, AJ in press)
A NVO prototype of an automated
discovery tool for arcs developed
by A Szalay
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p26
Printed: 29/04/02
Architecture Layer Diagram
Focus on usability,
improving the
users workflow
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p27
Printed: 29/04/02
AstroGrid: the AVO and NVO
AstroGrid is a major partner in the EC's Astrophysical
Virtual Observatory project – http://www.eso.org/avo
AVO partners: ESO, AstroGrid, ESA, CDS, Terapix,
Jodrell Bank and eMerlin
AstroGrid works closely in an international context with
the US National Virtual Observatory initiative. Regular
coordination between AstroGrid, NVO, AVO.
Major functionality demonstrators planned for release in
Jan 2003 by the NVO (at Jan AAS) and AVO.
June Munich Virtual Observatory conference sees launch
of International initiative – http://www.eso.org/vo2002
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
p28
Printed: 29/04/02
Concluding Remarks
N A Walton: AstroGrid and VO's: NeSC Apr 25, 2002
AstroGrid is a major
new UK funded e
science initiative
In partnership with
EU centres it will
play a lead role in the
realisation of a
European Wide
Virtual Observatory
AstroGrid is poised
to significantly
enhance the
opportunities of the
UK astronomical
research community
p29
Printed: 29/04/02