Experiments with Remote
Access to Scientific Data using
HDF5
Kun Yan
Robert E. McGrath
(mcgrath@ncsa.uiuc.edu)
NCSA
May 15, 2002
Today’s Talk
1. Background
2. Experiments with JSP, XML, HDF4,
HDF5, etc.
3. Experiments with CORBA, Java, HDF5
4. Discussion
Acknowledgements
Kun Yan conducted these experiments as a Research Assistant at
NCSA.
This report is based upon work supported in part by a Cooperative
Agreement with NASA under NASA grants NAG 5-2040 and NCC5599. Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the author(s) and do not
necessarily reflect the views of the National Aeronautics and Space
Administration.
Other support provided by NCSA and other sponsors and agencies.
HTML
Web
browser
Java
Java C
Java
Server
Platform
Java
Native
Interface
CORBA
Server
C++
HDF
Library
and File
C
XML
Any
Applet
Java
Native
Interface
C
Java
Java
Other
App.
Other
App.
H5view,
etc
Any
Any
Client/Remote
Distributed Product
Demonstrated in Research
Should work, but not demonstrated
Server/Local
Java
2. Experiments with JSP
• Experiments with JSP, XML, XSL, etc.
– Web access to Data and services.
– Demonstration of browsing HDF5 datasets
• Generate HTML or XML
• Convert data from HDF4 to HDF5 on demand
http://arabica.ncsa.uiuc.edu:7070/hdf5/
Tomcat
• is a standalone web server that supports
the Servlet and JSP specifications.
• developed under the Jakarta project at the
Apache Software Foundation.
• integrated into the J2EE Reference
Implementation.
• Tomcat 4.x (Catalina, Servlet 2.3, JSP 1.2)
• Tomcat 3.3
JavaServer Pages
• JavaServer PagesTM (JSPTM) technology
• rapidly develop and easily maintain,
dynamic web pages
• uses XML-like tags and Java scriptlets
• application logic can reside in serverbased resources
The Flow of a Request
response
request
HTML
Xalan.jar
Xerces.jar
client
JSP
response
XML
beans
beans
beans
Application
XSL
DB
lib
How JSP and Beans Work together
data
3. Experiments with CORBA
• Created a demonstration CORBA server
that reads HDF5
– Written in C++, linked directly to HDF5 library.
• Created a pure-java client
– No CORBA required on the client
– Can be used from an applet within a standard
Web browser
Technology used
• Orbacus 4.1.0 (C++) (free, full source)
• Java 1.3 or above
• HDF5.1.4.3, HDF5 C++ interface
HDF5 CORBA Experiment -- Design Diagram
CORBA Object Hierarchy
HDF5 CORBA Experiment – IDL File
1.
module HDF5{
2.
3.
4.
5.
6.
7.
8.
9.
10.
struct ObjInfo{
long oid;
long fid;
long type;
string name;
long numOfMembers;
};
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
struct DatasetInfo{
string name;
long numOfDim;
Sizes dimSizes;
Sizes dimMaxSizes;
long dataTypeClass;
long dataTypeSize;
long long storageSize;
};
26.
27.
interface H5Obj{
void getObjInfo(out ObjInfo obji);
typedef sequence<long> Sizes;
struct DatatypeInfo{
string name;
long type;
long size;
};
28.
29.
30.
31.
void getNumAttributes(out long attrNum);
void openAttributes (out long attr);
};
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
interface H5DatasetObj:H5Obj{
void getDatasetInfo(out DatasetInfo dsinfo);
long long getStorageSize();
};
53.
54.
55.
56.
interface H5GroupObj:H5Obj{
ObjInfo getGMember(in long fid, in string gName, in long index);
void openGroup(in string gName, out H5GroupObj h5gobj);
void getDatasetInfo(in string dsName, out DatasetInfo dsinfo);
void openDataset(in string dsName, out H5DatasetObj h5ds);
void getDataTypeInfo(in string ttName, out DatatypeInfo dtinfo);
};
interface H5FileObj{
void getTOC(out ObjInfo obji);
void openRootGroup(out H5GroupObj h5gobj);
};
interface H5FileAccess{
void openH5File(in string fileName, out H5FileObj h5fobj);
void numOfGMembers(in long fileId, in string gName, out long
gNum);
string closeH5File(in long fileId);
};
};
HDF5 CORBA Experiment -- Design Diagram
C++ Servant Object Hierarchy
H5FileAccess
H5Obj
H5FileObj
H5DatasetObj
H5GroupObj
H5Obj_impl
ObjInfo
poa_
H5FileAccess_impl
H5FileObj_impl
H5::H5File
poa_
H5::H5File
ObjInfo
poa_
openFile:
closeFile:
getTOC
openRootGroup
setFileName:
getObjInfo
setObjName:
H5GroupObj_impl
H5::H5Group
poa_
openGroup
getGMembers
openDataset:
getDatasetInfo:
getDatatypeInfo:
H5DatasetObj_impl
H5::H5Daraset
DatasetInfo
getDatasetInfo
getStorageSize
HDF5 CORBA Experiment -How to Use the Client Applet
•
•
•
•
•
•
•
•
•
•
Select file name from drop-down list (file resides corba server side)
Select Flags and Access Property of the selected file
Click on “open file” to open the file
Click on “get file TOC” to get the file object info
Click on “open root” to open the root group
Click on “get group TOC” to get the root group object info
Click on “iterate group” to get the group member’s info of the root and each
subgroup
Input dataset name including path, then click on “get dataset” to get the dataset
object
– Click on “get dataset TOC” to get the dataset object info
– Click on “get storage size” to get the storage size of the opened dataset object
Input dataset name including path, then click on “get dataset info” to get object
info of the given dataset
Input datatype name including path, then click on “get datatype info” to get object
info of the given datatype
URLs
• JSP experiments:
http://hdf.ncsa.uiuc.edu/HDF5/XML/JSPExperiments/
•
•
•
•
http://hdf.ncsa.uiuc.edu
http://hdf.ncsa.uiuc.edu/h4toh5/
http://hdf.ncsa.uiuc.edu/HDF5/XML/
http://hdf.ncsa.uiuc.edu/hdf-java-html
4. Discussion
HTML
Web
browser
Java
Java C
Java
Server
Platform
Java
Native
Interface
CORBA
Server
C++
HDF
Library
and File
C
XML
Any
Applet
Java
Native
Interface
C
Java
Java
Other
App.
Other
App.
H5view,
etc
Any
Any
Client/Remote
Distributed Product
Demonstrated in Research
Should work, but not demonstrated
Server/Local
Java
HDF5 CORBA Experiment -- User Interface
HDF5 CORBA Experiment -- Examples of Object Info window
Other HDF Tools
HDF4
HDF5 file
h4toh5
XML
HDF5 file
Viewer/
Editor
XML
ASCII
HDF4
HDF5 file
XML
Generator
Source code
HDF DDL
HDF5 file
Dumper
XML
HDF4 file
h4toh5
HDF5
H4 to H5 LIbrary
HDF5 file
h5toh4
HDF4
GIF file
gif2hdf
HDF5
flat file
fp2hdf
HDF5
Silly XMLTricks
Netcdf to HDF5, via XML using XSL
netcdf
ncxdump
nc.xml
netcdf.dtd
xalan
nctoh5.xsl
hdf5
h5gen
h5.xml
URLs
• JSP experiments:
http://hdf.ncsa.uiuc.edu/HDF5/XML/JSPExperiments/
•
•
•
•
http://hdf.ncsa.uiuc.edu
http://hdf.ncsa.uiuc.edu/h4toh5/
http://hdf.ncsa.uiuc.edu/HDF5/XML/
http://hdf.ncsa.uiuc.edu/hdf-java-html