

Store
Devices Microsoft Surface PCs & tablets Xbox Virtual reality Accessories Windows
phone Microsoft Band Software Office Windows Additional software Apps All apps
Windows apps Windows phone apps Games Xbox One games Xbox 360 games PC
games Windows games Windows phone games Entertainment All Entertainment
Movies & TV Music Business & Education Business Store Education Store Developer
Sale Back-to-school essentials Sale Products
Software & services Windows Office Free downloads & security Internet Explorer
Microsoft Edge Skype OneNote OneDrive Microsoft Health MSN Bing Microsoft
Groove Microsoft Movies & TV Devices & Xbox All Microsoft devices Microsoft
Surface All Windows PCs & tablets PC accessories Xbox & games Microsoft Band
Microsoft Lumia All Windows phones Microsoft HoloLens For business Cloud
Platform Microsoft Azure Microsoft Dynamics Windows for business Office for
business Skype for business Surface for business Enterprise solutions Small business
solutions Find a solutions provider Volume Licensing For developers & IT pros
Develop Windows apps Microsoft Azure MSDN TechNet Visual Studio For students
& educators Office for students OneNote in classroom Shop PCs & tablets perfect
for students Microsoft in Education Support
Sign in


Research Research
o Research Home
o Research areas
 Algorithms
 Artificial intelligence and machine learning
 Computer systems and networking
 Computer vision
 Data visualization, analytics, and platform
 Ecology and environment
 Economics
 Graphics and multimedia
 Hardware, devices, and quantum computing
 Human-centered computing
 Mathematics







o
o
o
o
o



Medical, health, and genomics
Natural language processing and speech
Programming languages and software engineering
Search and information retrieval
Security, privacy, and cryptography
Social Sciences
Technology for emerging markets
Products & Downloads
Programs & Events
 Academic Programs
 Events & Conferences
People
Careers
About
 About
 Microsoft Research blog
 Asia Lab
 Cambridge Lab
 India Lab
 New England Lab
 New York City Lab
 Redmond Lab
 Applied Sciences Lab
Research areas
o Algorithms
o Artificial intelligence and machine learning
o Computer systems and networking
o Computer vision
o Data visualization, analytics, and platform
o Ecology and environment
o Economics
o Graphics and multimedia
o Hardware, devices, and quantum computing
o Human-centered computing
o Mathematics
o Medical, health, and genomics
o Natural language processing and speech
o Programming languages and software engineering
o Search and information retrieval
o Security, privacy, and cryptography
o Social Sciences
o Technology for emerging markets
Products & Downloads
Programs & Events
o Academic Programs
o



Events & Conferences
People
Careers
About
o About
o Microsoft Research blog
o Asia Lab
o Cambridge Lab
o India Lab
o New England Lab
o New York City Lab
o Redmond Lab
o Applied Sciences Lab
Batch is back: CasJobs, serving multi-TB
data on the Web
February 1, 2005

Download PDF

BibTex
Authors

Jim Gray
Publication Type
TechReport
Pages
7
Number
MSR-TR-2005-19

Abstract

Related Info
Abstract
The Sloan Digital Sky Survey (SDSS) science database describes over 140 million objects and is
over 1.5 TB in size. The SDSS Catalog Archive Server (CAS) provides several levels of query
interface to the SDSS data via the SkyServer website. Most queries execute in seconds or
minutes. However, some queries can take hours or days, either because they require non-index
scans of the largest tables, or because they request very large result sets, or because they
represent very complex aggregations of the data. These “monster queries― not only take a
long time, they also affect response times for everyone else – one or more of them can clog the
entire system. To ameliorate this problem, we developed a multi-server multi-queue batch job
submission and tracking system for the CAS called CasJobs. The transfer of very large result sets
from queries over the network is another serious problem. Statistics suggested that much of this
data transfer is unnecessary; users would prefer to store results locally in order to allow further
joins and filtering. To allow local analysis, a system was developed that gives users their own
personal databases (MyDB) at the server side. Users may transfer data to their MyDB, and then
perform further analysis before extracting it to their own machine. MyDB tables also provide a
convenient way to share results of queries with collaborators without downloading them.
CasJobs is built using SOAP XML Web services and has been in operation since May 2004.
Related Info
Related Files
 tr-2005-19.doc
Follow Microsoft Research


Follow @MSFTResearch

Share this page


Tweet

Learn

Windows

Office

Skype

Outlook

OneDrive

MSN
Devices

Microsoft Surface

Xbox

PC and laptops

Microsoft Lumia

Microsoft Band

Microsoft HoloLens
Microsoft Store

View account

Order tracking

Retail store locations

Returns

Sales & support
Downloads

Download Center

Windows downloads

Windows 10 Apps

Office Apps

Microsoft Lumia Apps

Internet Explorer
Values

Diversity and inclusion

Accessibility

Environment

Microsoft Philanthropies

Corporate Social Responsibility

Privacy at Microsoft
Company

Careers

About Microsoft

Company news

Investors

Research

Site map

English (United States)

Contact us

Privacy & cookies

Terms of use

Trademarks

About our ads

© 2016 Microsoft
​