NOAA’s
NWS Telecommunication Gateway
RTH Washington
ICM MTN ET-OI , Sep 2008
Office of the Chief Information Officer
NOAA’s National Weather Service
Data Input to the NWSTG
 MPLS
 IP / Sockets
 X.25
 Asynchronous
 FTP – http://weather.gov/tg/ftpingest.html
 Email – http://weather.gov/tg/emailingest.html
 Web - http://weather.gov/tg/bullguid.html
“Taking the pulse of the planet”
Replacement & Backup RTHs
 Full functional replacement of existing capabilities
 Expanded capacity and capability
– Input data growth from 200 to 800 GB/day
– Output data growth from 800 to 2400 GB/day
 Transition to new technology
– Middleware for internal transport
– Network-centric systems interconnectivity
– Central switching engine with relational database
– SAN and NAS storage solutions
 Highly scaleable architecture
 Hardware refresh
“Taking the pulse of the planet”
RTG/BTG Performance
Performance Metric
Threshold
System Availability (averaged monthly)
Warning Message Latency (averaged monthly)
Routine Message Latency (averaged monthly)
Daily Traffic Volume (averaged monthly)
99.90%
10 seconds
60 seconds
1.2TB
Typical Performance Results (averaged monthly)
Performance Category
System Availability
Warning Message Latency
Routine Message Latency
Daily Traffic Volume
Threshold Actual
99.90%
10 seconds
60 seconds
1.2TB
100%
< 1 second
< 1 second
1.2 - 1.5TB
“Taking the pulse of the planet”
RTH Washington
 Can route more than 50 routine messages per second
with reliability for all dissemination to all of its users of
99.9 percent
 Latency for high priority traffic of 10 seconds or less and
routinely disseminates 1.2 terabytes of information per
day
 On Main Trunk Network (MTN) of the WMO Global
Telecommunication System (GTS)
– GTS delivers tsunami data and warnings to connected MTN
centers within two minutes
 On AFTN network (KWBCYMYX) & ICAO OPMET
Databank
“Taking the pulse of the planet”
NOAANet Logical Design
“Taking the pulse of the planet”
Primary / Backup
NWSTG Access
“Taking the pulse of the planet”
“Taking the pulse of the planet”
Major Software Components
Core software components of the NWSTG replaced in 2006
 Data switching software
–
Core software rewritten and implemented in IBM WebSphere Message Broker
–
Message Broker provides an application execution environment, threading, and fourth
generation language support (ESQL), integrated with middleware services
 Data transport infrastructure (middleware)
–
Shared data stores replaced with IBM MQSeries Middleware
–
Applications send data to each other without concern for target location
–
Data delivery guaranteed even during failures
 Data storage infrastructure (relational database)
–
All data stores moved into Sybase relational database
“Taking the pulse of the planet”
Internal Redundancy
Internal redundancy implemented for critical processes
 Active – Warm Standby
–
Standby system running at all times, system disks moved to standby system during
failover
–
EXAMPLES: Individual front-end processors, application servers
 Active – Hot Standby
–
Data replicated to Hot Standby system in realtime, automatic failover
–
EXAMPLE: Sybase relational database servers
 Active - Active
–
Multiple systems active in parallel, data shared/routed between systems groups
–
EXAMPLES – Switching system, HTTP/FTP server groups
“Taking the pulse of the planet”
Architecture Overview
“Taking the pulse of the planet”
Architecture Overview
“Taking the pulse of the planet”
System Extensibility
Virtualized server hardware platform selected for extensibility
 Virtualization
–
System resources (CPUs, Memory, Internal Storage, Network Interfaces, etc.)
available within hardware frame
–
Individual resources selected to create virtual servers
–
Major storage implemented in Storage Area Network
–
Virtual disks allocated to virtual servers
 Extensibility
–
Unused resources can be allocated to virtual servers, as needed
–
Underused resources can be removed from a virtual server
–
Virtual disks can be extended or reduced as storage requirements change
“Taking the pulse of the planet”
System Extensibility
Virtualized server hardware platform selected for extensibility
 Advanced Virtualization
–
Network and SAN I/O interfaces can be shared through virtual I/O
–
CPUs can be carved into fractional components down to 1/10th of a CPU
 Advanced Extensibility
–
Resources (CPU, Memory) included within system that were not purchased
–
If additional resources are needed, resources are “turned on” via software key after
purchase and are immediately available
“Taking the pulse of the planet”
RTH Washington
WIS approach
RTG/BTG upgrades made with knowledge of coming WIS/GISC
requirements
– Production and Replicated core systems
• All functions not essential to mission critical operations (i.e.
switching) execute against replicant
– SOA Approach to design
– Integral file, message and application switching
– RDBS core design
• ~ 2000 destinations defined
• ~ 400K defined products
• ~ 10M entries (rows) in switching table
– Many tables used for managing bulletin, file and report storage
including dissemination and services metadata.
“Taking the pulse of the planet”
RTH Washington
WIS approach
We view Data Access & Retrieval (DAR)
service as the only new service provision
– Plan to lay this service onto existing architecture
– Will break up DAR functions
• DAR catalog creation & update
• discovery & access servicing
– DAR will not execute on primary production system
(at least in early stages of implementation
“Taking the pulse of the planet”
RTH Washington
WIS approach
Develop metadata catalog first in conjunction with national &
regional partners
– Will leverage knowledge expertise of partners
– Will expand to all RTH (GISC) data holdings and holdings of appropriate
Centers (RSMC/DCPC/NC) and willing partners.
– Work will be made available to all Members
– Will collaborate with WIS Project Team, WMO ETs and developers in
other Regions
– Regional WIGOS Development Project will meet WIS requirements
• RA-IV Integrated Atmosphere Observing System
• With initial emphasis on upper-air observations
“Taking the pulse of the planet”
Meteorological Assimilation Data Ingest
System (MADIS) Transition to Operations
SFC-LAND
SFC-MARINE
U/A-IN SITU
MADIS
U/A-REMOTE
SENSING
Information
Bases
(QCed
Datasets)
Collection,
QC, and
Distribution
SATELLITE
GRIDS
METADATA
Quality Control
Information
(Data QC Flags)
And Metadata
12/12/05
“Taking the pulse of thePMplanet”
MADIS Ingests and Performs QC on
Meteorological Observational Data
 Meteorological Surface
– METAR
– Airways
– Maritime
– Modernized NWS
Cooperative Observer
– UrbaNet
– Integrated Mesonet
• State DOT
• Mesonets
• AWS
 NOAA Profiler Network
 Hydrological Surface
 Automated Aircraft
 Multi-Agency Profiler
 Cooperative Agency Profiler
 Radiosonde
 Radiometer
 Satellite Wind
– NOAA GOES Products
 Satellite Radiances and
Soundings
– NOAA POES
 Snow
 RSAS Surface Grids
“Taking the pulse of the planet”
Gateway Customers (NCDC, NCF, GTS etc)
?
NOAA Customers
FTP
Non-NOAA
Observations
Gateway
Individual
Data
Products
LDM
Internet
FTP
Tunnel
Web Server
?
Web Server
External ISP
Storage
(SAN)
LDM
Integrated
Datasets
?
Data Processing
LDM
Automated
QC processing
MADIS-T
TOC Environment
“Taking the pulse of the planet”
RTH Washington
Other Issues
– Foundation of Programs is Observing Systems & Data
Exchange
– Transition of GTS-MTN
• Two existing clouds to ???
– Cloud 1 - frame relay
– RMDCN – satellite MPLS
– Addition of Guam/Hono to AP-RARS
– Future operations locations
• Backup first then Primary
“Taking the pulse of the planet”
GTS Link with NOAA-TOC Washington
Inbound flow may be inhibited due to the saturated Outbound flow being unable to return
IP sockets ‘ack’ packets quickly enough.
© Crown copyright Met Office
Additional Slides
“Taking the pulse of the planet”
NWSTG Functional Overview
 The NWSTG is:
 the central communication facility of the NWS;
 the primary acquisition and distribution center for NWS
data and products;
 the primary acquisition and distribution center for
international data and products to meet WMO, ICAO and
bi-laterally agreed US requirements;
 A major data exchange hub for NOAA and other agency
data and products.
“Taking the pulse of the planet”
NWSTG Functional Overview
The NWSTG includes
 WMO Regional Telecommunication Hub (RTH)
Washington
 ICAO OPMET Databank
 the ASOS Operations and Monitoring Center
•
Operational oversight of U.S. federal automated surface observing
systems
 the AWIPS Network Control Facility
•
AWIPS is the main NWS’ system which supports the NWS forecast and
warning mission requirement
•
The NCF is the central communications hub and technical support center
for AWIPS
“Taking the pulse of the planet”
Dissemination Systems
International
National
 GTS
 NWWS
 ISCS
 NWR
 EMWIN
 LDAD
 Internet
 GMDSS
 NWS Telecom.
Gateway
 Family Of Services (FOS)
 Interagency Connections
 AWIPS SBN / NOAAPORT
“Taking the pulse of the planet”
Dissemination and
Distribution
NESDIS
Specialized
Customers
•Commercial Weather Services
•Research Institutions
•FAA, etc.
WMO
GOES
Imagery
Commercial
Satellite
Satellite Legend:
EMWIN
NOAAPORT
NWWS
EMWIN
FOS
(> 1 min)
SBN/NOAAPORT
• GOES
•NCEP Product Suite
• Observations
ISCS
ICAO
Designated
Local
Customers
Telecommunications Operations Center
NCEP
NWSTG
NCF
NWWS
Media &
Other
Customers
(10 sec)
LDAD
WAN
Other
Agencies
Data
Servers
Private line
Dissemination
GTS
Internet-Based
Dissemination
Field
Offices
WSR-88D
EMWIN
NWR
Public
“Taking the pulse of the planet”
“Taking the pulse of the planet”
Replacement / Backup
TG Description
NWS users
• Nat’l Centers
for Environ.
Prediction
• NWS Regional
Offices
• Domestic/int’l
observation &
forecast offices
• AWIPS
120GB/46
600 GB
90GB/46
375 GB
NWS Telecom Gateway
• Located in Silver Spring
Worldwide
users
80GB/38
200 GB
• New message switching
system allows future upgrades
• 2x upgradeability
• Much improved response time
700GB/38
• Redundancy ensures
uninterrupted service
2000 GB
• Full configuration management
Geographically separated
backup system
• ICAO / WMO
• Govt. agencies
• Family of
services
• Internet users
• Foreign
countries
• Emergency
mgrs
Daily Throughput/# of circuits
Legacy over Replacement
“Taking the pulse of the planet”
System Extensibility
“Taking the pulse of the planet”
Backup NWSTG
“Taking the pulse of the planet”
Government Networking
Requirements - NOAANet
• IP-based networking solution
− Any-to-any connectivity
− High degree of bandwidth scalability
− Optimum redundancy and survivability
− IP convergence (i.e., voice, video and data over IP)
− High-end performance
• Network security remains paramount, particularly
in light of today’s socio-political threats
• Segmentation from the public Internet
• Minimizes risk of security or privacy breaches
“Taking the pulse of the planet”
Sprint Peerless IP Network Map
Tacoma
Stockton
Chicago
Cheyenne
New York
Pennsauken, NJ
San Jose
Relay, MD
DC
Kansas City
Anaheim
Atlanta
OC48 (significant portions
upgrading to OC192)
Fort
Worth
Internet Transport Node
“Taking the pulse of the planet”