Java Native Interface: - UIC

Design and Implementation of CoreWall’s Session Database TABLE OF CONTENTS 1. INTRODUCTION .................................................................................................................... 3 1.1 CoreWall .......................................................................................................................... 3 1.2 Background & Need Identification .................................................................................. 3 1.3 Related Work ................................................................................................................... 4 2. THE COREWALL SUITE: GEOSCIENCE CORE DATA INTEGRATION AND VISUALIZATION .................................................................................................................... 6 2.1 Core Workflow Database ................................................................................................. 6 3. PROJECT DESCRIPTION AND WORK BREAK DOWN STRUCTURE ........................... 8 4. DATABASE ........................................................................................................................... 11 4.1 Database Design [13] ..................................................................................................... 11 Annotation_Table ....................................................................................................... 13 Annotation_Relationship_Table ................................................................................. 14 Database_Table ........................................................................................................... 14 DataFile_Table ............................................................................................................ 14 Graph_Table ............................................................................................................... 15 Graph_Annotation_Mapper_table .............................................................................. 15 Graph_Image_Mapper_Table ..................................................................................... 16 Group_Table ............................................................................................................... 16 Group_Database_Mapper_Table ................................................................................ 16 Image_Table ............................................................................................................... 17 Image_Annotation_Mapper_table .............................................................................. 17 Image_Manipulation_Non-Uniform_Scaling ............................................................. 18 Image_Manipulation_Uniform_Scaling ..................................................................... 18 Manipulation_sequencer_table ................................................................................... 19 Session_Annotation_Mapper_table ............................................................................ 19 Session_table............................................................................................................... 20 Session_Relationship_Table ....................................................................................... 20 Track_table ................................................................................................................. 20 Track_Annotation_Mapper_table ............................................................................... 21 Track_Image_Mapper_table ....................................................................................... 21 Track_Session_Mapper_Table ................................................................................... 21 Users_Table ................................................................................................................ 22 4.2 XML DTD[8] ................................................................................................................. 22 5. TECHNOLOGIES USED / CONCEPTS APPLIED.............................................................. 25 5.1 XML DOM Parsing [8,10] ............................................................................................. 25 5.2 XML to RDB Mapping [2] ............................................................................................ 26 Introduction to the Problem of XML to RDBMS Mapping ....................................... 26 Pros And Cons Of Xml Over Relational Model ......................................................... 26 Need To Convert Xml Data Into Relational Model .................................................... 28 Challenges And Complexities To Efficient Storing Of Xml Databases ..................... 28 Storage ........................................................................................................................ 29 Creating a XML structure that corresponds to a database schema ............................. 29 Referential Integrity and storing of XML documents to a RDBMS ........................... 30 5.3 Java Prepared Statements 14] ........................................................................................ 30 Department of Computer Science, UIC Page 1 of 58 Design and Implementation of CoreWall’s Session Database 5.4 Java Native Interface [4] ................................................................................................ 31 6. WORKFLOW [7] ................................................................................................................... 32 6.1 Save Session: ................................................................................................................. 32 6.2 Retrieve_Session ............................................................................................................ 39 6.3 Queries ........................................................................................................................... 44 UserSessions: .............................................................................................................. 45 SessionTracks: ............................................................................................................ 47 ImageSessions: ............................................................................................................ 48 SessionTrackImages: .................................................................................................. 49 ImageAnnotations: ...................................................................................................... 50 ImageGraphs: .............................................................................................................. 51 6.4 User Management [12,13] ............................................................................................. 51 7. SAMPLE SCREENSHOTS .................................................................................................... 53 8. SYSTEM REQUIREMENTS ................................................................................................. 56 9. TABLE OF ACRONYMS ...................................................................................................... 57 10. REFERENCES ....................................................................................................................... 58 Department of Computer Science, UIC Page 2 of 58 Design and Implementation of CoreWall’s Session Database 1. 1.1 INTRODUCTION CoreWall CoreWall Suite(CWS) is a real-time stratigraphic correlation, core description (CD) and data visualization system to be used by the marine, terrestrial and Antarctic science communities. It is aimed to significantly alter and enhance the current approaches used for core description and analysis of sediment and rock cores by providing an integrated environment for these activities, for both field and repository environments. 1.2 Background & Need Identification A primary need for marine, lake, and polar studies of sediment and rock cores, is an integrated environment for stratigraphic correlation, visual core description, and data visualization. Within the last few years, digital images of cores, microfossils, thin sections, etc. are being routinely collected, replacing traditional photography. This dramatic change to digital imagery is creating a huge change and opportunity in the way scientific measurements are captured, collected, analyzed, and distributed by the scientific community. These images provide the fundamental template for all sediment descriptive work, including annotations about structures, lithologic variation, macroscopic grain size variation, bioturbation intensity, chemical composition, and micropaleontology, among other features.The integration of core-section images with discrete data streams and ‘nested’ images (e.g., smear slides, photomicrographs, thin sections) provides a robust approach to the description of sediment and rock cores. Integration of value-added data, later acquired in a timely manner by individuals and teams of scientists is another major priority. The real-time and/or simultaneous display of multiple integrated databases, with all the data rectified (co-registered) to the fundamental template of the core image, is sorely needed to expedite the process that currently takes months to years. The cost of wasted time and lost opportunity for science caused by the lack of integrated data and image management is enormous and the challenge exists for all current geoscience’s coring programs. Currently, visual core description (VCD) within IODP(Integrated Ocean Drilling Program) is primarily carried out in manual fashion with observations carried out with pencil and paper and entered into a variety of 3rd Department of Computer Science, UIC Page 3 of 58 Design and Implementation of CoreWall’s Session Database party software (e.g. AppleCore), and Excel template spreadsheets to create summary “barrel sheets” (see Figure 1 for example) of basic lithology, structures, color, and grain size, etc. This is perhaps the most labor-intensive activity with up to 8 to 10 people allotted per expedition. The VCD summaries are also now supplemented by digital line scans of the split-cores, color reflectance measurements, and other data sets. However, the automated measurements are not captured by the “barrel sheets” and certainly not integrated into the summaries. Only the interest of individual scientific party members drives combining these data into an overall site summaries capture any of this analysis. Essentially, there is no data visualization system within IODP or any other drilling program that allows easy, interactive data visualization and integration. The ability to easily map and analyze these diverse datasets onto multiple depth scales interchangeably has now become a requirement. The need for a data visualization system that is scaleable, integrated, linked to outside sources, and allows for collaborative research both onboard and onshore is absolutely a necessity to improve the overall science. CWS will be an integrated interpretive environment to facilitate all scientists that use geosciences data as a component of their research. 1.3 Related Work LacCore is an integrated system that provides training and service to all researchers regardless of funding source, and that aims to function as a role model for handling of and primary analysis of lacustrine sediment cores. Its facilitates interdisciplinary studies of lake sediments and environmental history by bringing together researchers in an efficient workspace that provides standard and specialized equipment, knowledgeable staff, and training. A networked computer operates as a link to the collections and acts as a server to host the various databases. However, a quantum leap in data display capability is needed to better facilitate researchers who use the LacCore. CoreWall Suite will fill this glaring need. Department of Computer Science, UIC Page 4 of 58 Design and Implementation of CoreWall’s Session Database Figure 1: Example Barrel Sheet Department of Computer Science, UIC Page 5 of 58 Design and Implementation of CoreWall’s Session Database 2. THE COREWALL SUITE: GEOSCIENCE CORE DATA INTEGRATION AND VISUALIZATION The central component of the CWS is the Core Workflow Database (CWD) whose role is to intercept queries from end-users and retrieve the relevant pieces of data and meta-data, and stage it for rapid and frequent access by the users. Figure 2 depicts how CWD associates with other CoreWall Suite components. Figure 2: Components of CoreWall Suite 2.1 Core Workflow Database The data required for a core interpretation session can be very large. The data for an entire IODP core, including core images, multi-sensor scans, smear slides and all related data, can amount to many gigabytes. If a scientist wants to simultaneously compare multiple cores, the size of the needed data can indeed be large. To compound this problem, many users will be interpreting at locations with slow Internet connections; in the field or drawing data from databases which themselves are on slow networks. In addition, users will be interpreting data from databases that are often designed as read-only archives and not designed to hold “works in progress” of investigators. The Core Workflow database will be developed to address these two problems. First it will have interfaces to enable the CWD to retrieve user selected data from established databases such as JANUS, LacCore, dbSEABED, and PaleoStrat. It will also be able to pull data through the emerging portals such as CHRONOS which will allow much more diverse data to be accessible to the CWS than through any single database. This gives the user instant web-services based access to multiple data sources . Second, the CWD captures the results of analyses and interpretations made through accessing and correlating data Department of Computer Science, UIC Page 6 of 58 Design and Implementation of CoreWall’s Session Database retrieved from these databases while the user is working to analyze the cores. As the workflow is captured other collaborators can access it locally or remotely. For example, the work of one scientist can be shared from one shift to the next and biologists in a separate lab can watch as other labs in the ship interpret a specific core segment. In a higher bandwidth environment, such as a core lab or a university office, a group of collaborators could track the work of one-another as they work on the same cores. A primary feature of the CWD is the co-registration of the data across multiple coordinate systems. For instance, once several wire length control points have been assigned depths, the remainder can be extrapolated, and depth is now stored as an alternate coordinate system for the well log. Networks of Core Workflow Databases can be connected to facilitate mirroring of data, and permissions can be set at each institution’s database to control the degree of sharing. A central Core Integration Database can also be established (for example, on a ship or a core lab on shore) to collect all contributions made by the community. Department of Computer Science, UIC Page 7 of 58 Design and Implementation of CoreWall’s Session Database 3. PROJECT DESCRIPTION AND WORK BREAK STRUCTURE Corewall includes a database which stores data related to the work done by geoscientist in the form of sessions for e.g. core manipulations and observation related data . In general, Session could be thought as an aggregated study done by a geoscientist upto the point of exiting Corelyzer, which is an integrated visualization tool for the study of lake and ocean sediment . Corelyzer allows a geoscientist to load multiple cores and associate some text with them in the form of text(called as annotations). Apart from this textual information, a user's work in Corelyzer could also contain graphs, images and there associated annotations These annotations are notes about the core study. This study could contain the hierarchical associations between the annotations in terms of parent child relationship amongst themselves. Currently geoscientists are not able to build on or reuse previous core studies . This restricts them in terms of retrieving their older work (old sessions), extending their work and/or integrating work with other geoscientists and analysis on the basis of different CoreWall specific data attributes e.g finding out what sessions referred a particular image, what users used a particular image in the session. All this data is beneficial to their core studies and research. CoreWall database is the main focus of this project which allow the researchers to save, retrieve and query the session information. This information could then be used to go back in time to see what studies were performed or to analyze the studies at different points of time. In addition to this, it also allows execution of any ad-hoc queries on this session data With the integration of database into Corelyzer , CoreWall system was separated into two independent components: Corelyzer rendered the core data and met other visualization needs of the geoscientist and session database acted as a repository for the core study data from which user could retrieve, save or query on specific data attributes. Department of Computer Science, UIC Page 8 of 58 Design and Implementation of CoreWall’s Session Database Figure 3: CoreWall Demonstration Their was a need of bidirectional data communication (see figure 4) between session database and Corelyzer. XML was chosen as the medium for data exchange due to the following reasons: Network CoreWall Session Database Server CoreWall Client CoreWall Session Database Server Figure 4: High Level CoreWall Design Firstly, XML could represent the hierarchical structure of CoreWall's data better than a relational model and secondly it avoided CoreWall from getting tied up to a specific database Department of Computer Science, UIC Page 9 of 58 Design and Implementation of CoreWall’s Session Database back-end, which gave the flexibility to switch to any other proprietary databases with minimal efforts. The work break down to build the session database included the following tasks:  Identification of database needs for the CoreWall suite (Section 1 and 2).  Design a database that meets those needs.(Section 4.1)  User Management Interface for CoreWall session database(6.4).  Design a DTD to simulate the flat structure of CoreWall session database(Section 4.2) .  Parse the XML state files to store information into the database. This involves intricacies of XML to RDBMS mapping(Section 5.2) .  Retrieve existing session information as small XML files (Section 6.2)  Generate query results as XML files(Section 6.3).  Integration of database into CoreWall as its backend. This database serves as a repository of data to meet the data integration objectives of CoreWall Suite. Department of Computer Science, UIC Page 10 of 58 Design and Implementation of CoreWall’s Session Database 4. 4.1 DATABASE Database Design [13] The database schema for CoreWall session database is depicted below. A detailed description about each table’s columns, their data types, data constraints and the integrity constraints imposed on each table is described in the following pages. Session_table PK Session_ID Session_Relationship_table Session_Name User_ID Time_started Time_Last_Modified Permission PK Session_Parent_ID Session_Child_ID Track_Table Track_Session_Mapper_table Users_Table PK User_ID User_Name First_Name Last_Name Group_ID Password email PK PK Session_Annotation_Mapper_Table Session_ID Annotation_ID Track_ID Track_Name Track_Description Position_X Position_Y Track_Annotation_Mapper_Table Track_Image_Mapper_table PK PK PK PK PK PK Track_ID Session_ID Track_ID Image_ID Session_ID PK PK Track_ID Annotation_ID Position_X Position_Y Position_X Position_Y Position_X Position_Y Annotation_relationship_table PK Image_Table1 Annotation_Parent_ID Annotation_Child_ID Annotation_Table PK Annotation_ID PK Image_ID Image_URL Image_Description User_ID Annotation_Author Annotaton_Text Annotation_Type Timestamp Image_Annotation_Mapper_table PK PK PK Annotation_ID Session_ID Core_Image_ID_1 Core_Image_ID_2 Annotation_Image_ID Position_X_1 Position_Y_1 Position_X_2 Position_Y_2 Position_X_Ann Position_Y_Ann Database Schema Department of Computer Science, UIC Page 11 of 58 Design and Implementation of CoreWall’s Session Database Session_table PK Track_Table Session_ID PK Session_Name User_ID Time_started Time_Last_Modified Permission Track_ID Track_Name Track_Description Annotation_Table PK Image_Table Annotation_ID PK User_ID Annotation_Author Annotaton_Text Annotation_Type Timestamp Image_ID Image_Name Image_URL Image_Description Graph_Annotation_Mapper PK PK PK Annotation_ID Session_ID Graph_ID_1 Graph_ID_2 Position_X_1 Position_Y_1 Position_X_2 Position_Y_2 Position_X_Ann Position_Y_Ann Image_Manipulation_Uniform_Scaling Manipulation_Sequencer PK PK Graph_Table PK Graph_ID PK Image_Manipulation_ID Session_ID Image_Manipulation_US_ID scale_x scale_y Image_ID Manipulation_Type Order_of_operation Data_File_ID Field_Name_1 Field_Name_2 Image_Manipulation_Non_Uniform_Scaling PK Image_Manipulation_NUS_ID DataFile_Table PK DataFileID Graph_Image_Mapper Data_File_Name Data_File_URL Data_File_Description PK PK Graph_ID Image_ID Start_Row End_Row depth Session_ID x_value y_value width height scale_x scale_y Database Schema Cont…. Department of Computer Science, UIC Page 12 of 58 Design and Implementation of CoreWall’s Session Database User_Table PK Group_Database_Mapper User_ID PK PK User_Name First_Name Last_Name Group_ID Password email Group_ID Database_ID user_login user_password Database_Table PK Group_Table PK Database_ID Database_Name IP_address admin_email Group_ID Group_Name Description Database Schema Contd….. Annotation_Table This table is used to store annotation related information. These annotations could belong to session, track, image or graph. Column_Name Null Data Type Annotation_ID User_ID Annotation_Author Annotation_Text No No No Yes bigint(20) bigint(20) Varchar(25) Text(500) Annotation_Type No Varchar(25) Timestamp No Timestamp Description A unique ID assigned to every annotation. ID of annotation’s owner Name of the author Annotation text This tells if an annotation is related to an image, graph track or session. Time when the annotation was last updated. Primary Key : Annotation_ID Foreign Key : User_ID REFERENCES track_table(track_ID) on DELETE CASCADE on UPDATE RESTRICT Department of Computer Science, UIC Page 13 of 58 Design and Implementation of CoreWall’s Session Database Annotation_Relationship_Table This table is used to store parent –child relationship between annotations. Column_Name Null Annotation_Parent_ID Annotation_Child_ID Data Type No No bigint(20) bigint(20) Description A unique ID assigned to every annotation A unique ID assigned to every annotation Primary Key : Annotation_Parent_ID, Annotation_Child_ID Foreign Key : Annotation_Parent_ID REFERENCES Annotation_Table(Annotation_ID) on DELETE CASCADE on UPDATE RESTRICT Annotation_Child_ID REFERENCES Annotation_Table(Annotation_ID) on DELETE CASCADE on UPDATE RESTRICT Database_Table The table is used to store information about external databases. Column_Name Database_ID Database_Name IP_Address Admin_email Null No No No Yes Data Type Description bigint(20) Varchar(255) Varchar(20) Varchar(25) A unique ID assigned to every external database Name of the database IP address of the machine hosting the database Email address of the database administrator Primary Key : Database_ID DataFile_Table The table is used to store information about datafiles that are used to generate graphs Column_Name DataFileID File_Name_1 File_Name_2 Null No No No Data Type bigint(20) Varchar(255) Varchar(255) Description A unique ID assigned to every data file Name of the first file Name of the second file Primary Key : DataFileID Department of Computer Science, UIC Page 14 of 58 Design and Implementation of CoreWall’s Session Database Graph_Table This table is used to store graph information. Column_Name Graph_ID Data_File_ID Field_Name_1 Field_Name_2 Null No No No No Data Type Description bigint(20) bigint(20) Varchar(255) Varchar(255) A unique ID assigned to every annotation Id of the file which stores data for the graph. Name of graph’s first field Name of graph’s second field Primary Key : Graph_ID Foreign Key : Data_File_ID REFERENCES datafile_table(datafile_ID) on DELETE CASCADE on UPDATE RESTRICT Graph_Annotation_Mapper_table This table is used to store information that relates annotations with graphs Column_Name Null Data Type Annotation_ID Graph_ID_1 Graph_ID_2 No No Yes bigint(20) bigint(20) bigint(20) Session_ID No bigint(20) Position_X_1 Position_Y_1 Position_X_2 Position_Y_2 Position_X_Ann Position_Y_Ann No No Yes Yes No No Decimal(10,4) Decimal(10,4) Decimal(10,4) Decimal(10,4) Decimal(10,4) Decimal(10,4) Timestamp Yes Timestamp Description A unique ID assigned to every annotation ID of the first graph ID of the second graph Session with which this graph_annotation is associated with. X Position where graph 1 is laid out. Y Position where graph 1 is laid out. X Position where graph 2 is laid out. Y Position where graph 2 is laid out. X Position where annotation is laid out. Y Position where annotation is laid out. Time at which this graph_annotation was created. Primary Key : Annotation_ID, Graph_ID_1, Graph_ID_2 Foreign Key : Annotation_ID REFERENCES annotation_table(Annotation_ID) on DELETE CASCADE on UPDATE RESTRICT Session_ID REFERENCES session_table(session_ID) on DELETE CASCADE on UPDATE RESTRICT Graph_ID_1 REFERENCES Graph_table(Graph_ID) on DELETE CASCADE on UPDATE RESTRICT Graph_ID_2 REFERENCES Graph_table(Graph_ID) on DELETE CASCADE on UPDATE RESTRICT Department of Computer Science, UIC Page 15 of 58 Design and Implementation of CoreWall’s Session Database Graph_Image_Mapper_Table The table is used to store information about mappings between graphs and images during a user session. Column_Name Graph_ID Image_ID Session_ID Position_X Position_Y Null No No No No No Data Type bigint(20) bigint(20) bigint(20) Decimal(10,4) Decimal(10,4) Description Identification for the Graph Identification for the Image ID of the session X position of the graph in the image Y position of the graph in the image Primary Key : Graph_ID, Image_ID Foreign Key : Graph_ID references Graph_Table(Graph_ID) on DELETE CASCADE on UPDATE RESTRICT Image_ID references Image_Table(Image_ID) on DELETE CASCADE on UPDATE RESTRICT Session_ID references Session_Table(Session_ID) on DELETE CASCADE on UPDATE RESTRICT Group_Table The table is used to store information about the user groups of core wall. Each user is assigned to a group. Column_Name Group_ID Group_Name Description Null No No Yes Data Type bigint(20) Varchar(255) Varchar(255) Description A unique ID assigned to every group Name of the group A brief description about the group Primary Key : Group_ID Group_Database_Mapper_Table The table is used to store associate groups and the databases that group have access to. Column_Name Group_ID Database_ID User_login User_password Null No No No No Data Type bigint(20) bigint(20) Varchar(255) Varchar(255) Description A unique ID assigned to every group A unique ID assigned to every database Login for the user Password for the user Primary Key : Group_ID, Database_ID Foreign Key : Group_ID REFERENCES Group_Table(Group_ID) on DELETE CASCADE on UPDATE RESTRICT Database_ID REFERENCES Database_Table(Database_ID) on DELETE CASCADE on UPDATE RESTRICT Department of Computer Science, UIC Page 16 of 58 Design and Implementation of CoreWall’s Session Database Image_Table This table is used to store information about images that are used in a corewall session. Column_Name Image_ID Image_Name Image_URL Image_Description Null No No No Yes Data Type bigint(20) Varchar(255) Varchar(255) Varchar(255) Description A unique ID assigned to every image Name of the image. URL where the image is stored A brief description about the image Primary Key : Image_ID Image_Annotation_Mapper_table This table is used to store annotations associated with images. Column_Name Null Data Type Description Annotation_ID No Bigint(20) Session_ID No Bigint(20) Core_Image_ID_1 Core_Image_ID_2 Position_X_1 Position_Y_1 Position_X_2 Position_Y_2 Position_X_Ann Position_Y_Ann No Yes No No Yes Yes No No Bigint(20) Bigint(20) Decimal(10,4) Decimal(10,4) Decimal(10,4) Decimal(10,4) Decimal(10,4) Decimal(10,4) Timestamp Yes Timestamp A unique ID assigned to annotation. Session to which the image and annotation belong ID of the first image IDof the second image X Position where image 1 is laid out. Y Position where image 1 is laid out. X Position where image 2 is laid out. Y Position where image 2 is laid out. X Position where annotation is laid out. Y Position where annotation is laid out. The time when this annotation was last saved. Primary Key : Annotation_ID,Core_Image_ID_1, Session_ID Foreign Key : Annotation_ID REFERENCES Annotation_table(Annotation_ID) on DELETE CASCADE on UPDATE RESTRICT Session_ID REFERENCES session_table(session_ID) on DELETE CASCADE on UPDATE RESTRICT Core_Image_ID_1 REFERENCES Image_table( Image_ID) on DELETE CASCADE on UPDATE RESTRICT Core_Image_ID_2 REFERENCES Image_table( Image_ID) on DELETE CASCADE on UPDATE RESTRICT Department of Computer Science, UIC Page 17 of 58 Design and Implementation of CoreWall’s Session Database Image_Manipulation_Non-Uniform_Scaling This table is used to store non-uniform scaling related image manipulation information. Column_Name Null Data Type Image_Manipulation _NUS_ID No bigint(20) Image_ID No bigint(20) X_value Y_value Width Height Scale_x Scale_y No No No No No No Decimal(10,4) Decimal(10,4) Decimal(10,4) Decimal(10,4) Decimal(10,4) Decimal(10,4) Description A unique ID for every Non uniform manipulation. ID of the image that is manipulated. X value Y value Width Height Scaling ratio X Scaling ratio Y Primary Key : Image_Manipulation_NUS_ID Foreign Key : Image_ID REFERENCES Image_Table(Image_ID) on DELETE CASCADE on UPDATE RESTRICT Image_Manipulation_Uniform_Scaling This table is used to store uniform scaling related image manipulation information. Column_Name Null Data Type Image_Manipulation_US_ID No bigint(20) Image_ID Scale_x Scale_y No No No bigint(20) Decimal(10,4) Decimal(10,4) Description This is a unique ID for every uniform scaling manipulation. ID of the image that is manipulated. Scaling ratio X Scaling ratio Y Primary Key : Image_Manipulation_US_ID Foreign Key : Image_ID REFERENCES Image_Table(Image_ID) on DELETE CASCADE on UPDATE RESTRICT Department of Computer Science, UIC Page 18 of 58 Design and Implementation of CoreWall’s Session Database Manipulation_sequencer_table This table is used to store the order of manipulations on a image in a particular session. Column_Name Null Data Type Sequencer_ID No bigint(20) Session_ID No bigint(20) Image_ID No bigint(20) Image_Manipulation_ID No bigint(20) Manipulation_Type No Varchar(255) Order_of_operation No bigint(20) Description A unique ID for each manipulation record. Session to which this specific manipulation is done. Unique ID of the image Manipulation ID from one of the manipulation table(Uniform Scaling or Non-Uniform Scaling) Indicates the manipulation type. This records the sequence in which modifications are made to the image. Primary Key : Sequencer_ID Foreign Key : Session_ID references Session_table(Session_ID) on DELETE CASCADE on UPDATE RESTRICT Image_ID REFERENCES Image_Table(Image_ID) on DELETE CASCADE on UPDATE RESTRICT Session_Annotation_Mapper_table The table is used to store information about mappings between annotations and sessions. Column_Name Null Data Type Description Annotation_ID No bigint(20) Session_ID No bigint(20) Position_X_Ann Position_Y_Ann No No Decimal(10,4) Decimal(10,4) A unique ID assigned to every annotation Session with which this track_annotation is associated with. X Position where annotation is laid out. Y Position where annotation is laid out. Primary Key : Annotation_ID, Session_ID Foreign Key : Annotation_ID REFERENCES annotation_table(Annotation_ID) on DELETE CASCADE on UPDATE RESTRICT Session_ID REFERENCES session_table(session_ID) on DELETE CASCADE on UPDATE RESTRICT Department of Computer Science, UIC Page 19 of 58 Design and Implementation of CoreWall’s Session Database Session_table The session table stores information about a user session. Column_Name Null Data Type Session_ID No bigint(20) Session_Name User_ID No No varchar(255) bigint(20) Permission No tinyint(4) Time_Created Time_Last_Modified No No Timestamp Timestamp Description It is used to uniquely identify every user session. Automatically generated by mysql. A unique name for the session. This is used to associate a user with each session This determines the accessibility of the session. Value 1: Session is private, Value 2: Session is accessible to group for rw. 3 means the public can read this session. The time when the session was started. Time when the session was last modified Primary Key : Session_ID Foreign Key : User_ID REFERENCES Users_Table (User_ID) on DELETE CASCADE on UPDATE RESTRICT Session_Relationship_Table This table is used to store parent–child relationship between sessions. Column_Name Session_Parent_ID Session_Child ID Null Data Type No No bigint(20) bigint(20) Description A unique ID assigned to every session. A unique ID assigned to every session. Primary Key : Session_Parent_ID, Session_Child ID Foreign Key : Session_Parent_ID REFERENCES Session_table (Session_ID) on DELETE CASCADE on UPDATE RESTRICT Session _Child_ID REFERENCES Session_table(Session_ID) on DELETE CASCADE on UPDATE RESTRICT Track_table The table stores information about session tracks. Column_Name Null Data Type Track_ID No bigint(20) Track_Name No varchar(255) Track_Description Yes varchar(255) Description It is used to uniquely identify every track irrespective of the session. User assigned name for the track This contains detailed description about the track. Primary Key : Track_ID Department of Computer Science, UIC Page 20 of 58 Design and Implementation of CoreWall’s Session Database Track_Annotation_Mapper_table The table is used to store information about mappings between annotations and tracks. Column_Name Null Data Type Description Annotation_ID No bigint(20) Session_ID No bigint(20) Track_ID Position_X_Ann Position_Y_Ann No No No bigint(20) Decimal(10,4) Decimal(10,4) A unique ID assigned to every annotation Session with which this track_annotation is associated with. ID of the track X Position where annotation is laid out. Y Position where annotation is laid out. Primary Key : Annotation_ID, Track_ID Foreign Key : Annotation_ID REFERENCES annotation_table(Annotation_ID) on DELETE CASCADE on UPDATE RESTRICT Session_ID REFERENCES session_table(session_ID) on DELETE CASCADE on UPDATE RESTRICT Track_ID REFERENCES track_table(Track_ID) on DELETE CASCADE on UPDATE RESTRICT Track_Image_Mapper_table The table is used to associate tracks and images. Column_Name Track_ID Image_ID Session_ID Position_X Position_Y Null No No No No No Data Type bigint(20) bigint(20) bigint(20) Decimal(10,4) Decimal(10,4) Description ID of the track ID of the image ID of the session X Position where the image is laid out. Y Position where the image is laid out. Primary Key : Track_ID, Image_ID Foreign Key : Track_ID REFERENCES Track_Table(Track_ID) on DELETE CASCADE on UPDATE RESTRICT Image_ID REFERENCES Image( Image_ID) on DELETE CASCADE on UPDATE RESTRICT Track_Session_Mapper_Table The table is used to associate tracks with session. Column_Name Track_ID Session_ID Position_X Position_Y Null No Yes No No Data Type bigint(20) bigint(20) Decimal(10,4) Decimal(10,4) Description ID of the track ID of the session. X Position where track is laid out. Y Position where track is laid out. Primary Key : Track_ID, Session_ID Foreign Key : Track_ID REFERENCES Track_Table(Track_ID) on DELETE CASCADE on UPDATE RESTRICT Session_ID REFERENCES Session_table( Session_ID) on DELETE CASCADE on UPDATE RESTRICT Department of Computer Science, UIC Page 21 of 58 Design and Implementation of CoreWall’s Session Database Users_Table The table is used to store information about users of core wall. Column_Name User_ID User_Name First_Name Last_Name Group_ID Password Null No No No Yes No No Data Type bigint(20) Varchar(255) Varchar(255) Varchar(255) bigint(20) Varchar(255) Description A unique ID assigned to every user User Name First name of the user Last Name of the user Group the user belongs to User’s password Primary Key : User_ID Foreign Key : Group_ID REFERENCES Group_Table(Group_ID) on DELETE CASCADE on UPDATE RESTRICT 4.2 XML DTD[8] A DTD for the state that is used to pass session information(state file) . For information on how a DTD is created that conforms to a particular relational database schema refer to section 5.2. <?xml version="1.0" encoding="UTF-8"?> <!ELEMENT Session_Database(Data_File_Table*,Session_Table+) <!ELEMENT Data_File_Table EMPTY> <!ATTLIST Data_File_Table Data_File_Name Data_File_URL Data_File_Descrip Data_File_Index CDATA #REQUIRED CDATA #REQUIRED CDATA #IMPLIED CDATA #IMPLIED //used to relate to graphs > <!ELEMENT Session (Track+, Annotation*)> <!ATTLIST Session Session_ID CDATA #REQUIRED User_ID CDATA #REQUIRED Time_Started CDATA #REQUIRED Time_Last_Modified CDATA #REQUIRED Permission CDATA #REQUIRED SessionParent_ID CDATA #IMPLIED (If Null this session is not based on any other session) > <!ELEMENT Annotation EMPTY> <!ATTLIST Annotation Annotation_ID CDATA #REQUIRED Annotation_Author CDATA #REQUIRED Annotation_Text CDATA #REQUIRED Annotation_Type CDATA #REQUIRED //session Position_X CDATA #REQUIRED Position_Y CDATA #REQUIRED Pointer_Ann_X CDATA #REQUIRED Pointer_Ann_Y CDATA #REQUIRED Department of Computer Science, UIC Page 22 of 58 Design and Implementation of CoreWall’s Session Database Annotation_ParentID index parent_index CDATA #IMPLIED (If Null this annotation does not have a parent) CDATA #IMPLIED //a unique ID for each annotation CDATA #IMPLIED //used to relate annotations within the current session. > <!ELEMENT Track(Image*,Annotation*)> <!ATTLIST Track Track_ID CDATA #REQUIRED Track_Name CDATA #REQUIRED Track_Description CDATA #REQUIRED Position_X CDATA #REQUIRED Position_Y CDATA #REQUIRED > <!ELEMENT Annotation EMPTY> <!ATTLIST Annotation Annotation_ID CDATA #REQUIRED Annotation_Author CDATA #REQUIRED Annotation_Text CDATA #REQUIRED Annotation_Type CDATA #REQUIRED //track Position_X CDATA #REQUIRED Position_Y CDATA #REQUIRED Pointer_Ann_X CDATA #REQUIRED Pointer_Ann_Y CDATA #REQUIRED Annotation_ParentID CDATA #IMPLIED (If Null this annotation does not have a parent) index CDATA #IMPLIED parent_index CDATA #IMPLIED > <!ELEMENT Image(Graph?,Annotation*)> <!ATTLIST Image Image_ID CDATA #REQUIRED Image_Name CDATA #REQUIRED Image_Description CDATA #REQUIRED Image_URL CDATA #REQUIRED Position_X CDATA #REQUIRED Position_Y CDATA #REQUIRED > <!ELEMENT Annotation EMPTY> <!ATTLIST Annotation Annotation_ID CDATA #REQUIRED Annotation_Author CDATA #REQUIRED Annotation_Text CDATA #REQUIRED Annotation_Type CDATA #REQUIRED //image Position_X CDATA #REQUIRED Position_Y CDATA #REQUIRED Pointer_Ann_X CDATA #REQUIRED Pointer_Ann_Y CDATA #REQUIRED Annotation_ParentID CDATA #IMPLIED (If Null this annotation does not have a parent) index CDATA #IMPLIED parent_index CDATA #IDREFS > <!ELEMENT Graph <!ATTLIST Graph Graph_ID EMPTY> CDATA #REQUIRED Department of Computer Science, UIC Page 23 of 58 Design and Implementation of CoreWall’s Session Database Data_File_Index Field_Name1 Field_Name2 Position_X Position_Y Position_X_Ann Position_Y_Ann Depth > IDREF #REQUIRED CDATA #REQUIRED CDATA #REQUIRED CDATA #REQUIRED CDATA #REQUIRED CDATA #REQUIRED CDATA #REQUIRED CDATA #REQUIRED <!ELEMENT Annotation EMPTY> <!ATTLIST Annotation Annotation_ID CDATA #REQUIRED Annotation_Author CDATA #REQUIRED Annotation_Text CDATA #REQUIRED //graph Annotation_Type CDATA #REQUIRED Position_X CDATA #REQUIRED Position_Y CDATA #REQUIRED Pointer_Ann_X CDATA #REQUIRED Pointer_Ann_Y CDATA #REQUIRED Annotation_ParentID CDATA #IMPLIED (If Null this annotation does not have a parent) index CDATA #IMPLIED parent_index CDATA #IDREFS > Figure 5: XML DTD that corresponds to the Relational Schema of section 4.1 Department of Computer Science, UIC Page 24 of 58 Design and Implementation of CoreWall’s Session Database 5. TECHNOLOGIES USED / CONCEPTS APPLIED This section contains short introduction to the concepts and technologies that were employed by the project and reasons for the same. These concepts/technologies are referenced throughout this report in different sections. 5.1 XML DOM Parsing [8,10] CoreWall state files are written in a well-structured XML document. XML was chosen as a technology to store state files as they prevent CoreWall from getting tied up to a specific database from the visualization end. This state file data needs to be stored to the database in appropriate tables. This data retrieved from the state file needs to be parsed before it is could be used by the SQL queries. Create a JAXP document builder. This is a wrapper around a specific XML parser. DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance(); DocumentBuilder builder =builderFactory.newDocumentBuilder(); Invoke the parser to create a Document representing an XML document Document document = builder.parse(someInputStream); Obtain the root node of the tree. This returns an Element, which is a subclass of the more general Node class that represents an XML element. Element rootElement = document.getDocumentElement(); Examine various properties of the node using some of the methods provided by DOM API. getNodeName: Returns the Name of the Node getNodeType: Returns the Type of the Node .Returned value is a predefined constants in the Node class getNodeValue: Returns the Value of the Node getAttributes: Returns the attributes used by the element's start tag getChildNodes: Returns the elements contained between the current element's start and end tags Modify properties of the nodes using some of the methods provided by DOM API. appendChild: Add child Node to the Current Node removeChild: Remove a Child node from the current Parent Node setNodeValue: Set a node’s value Figure 6: Steps in DOM processing Figure 7 show an overview of how this is achieved in the CoreWall Session Database. As the order in which XML document elements are to be accessed is determined by the order in which the primary- Department of Computer Science, UIC Page 25 of 58 Design and Implementation of CoreWall’s Session Database foreign key relationships are designed in the database schema therefore it was imperative to use a parser that could allow us to access any part of the XML document out of the sequence in the actual XML file. To accomplish this DOM was used as it stores the XML document in memory and allows for faster access to specific element in the XML document. The code specific details about parsing a DOM is shown in Figure 6 Parsing using DOM involves the following two steps: a) Generate a DOM data structure that represents the structure of the XML document. b) Access and modification of data of interest using the methods provided by the DOM API. Figure 7 shows the detailed steps in DOM processing 5.2 XML to RDB Mapping [2] XML was chosen as a medium for data exchange between the session database and the visualization component. The was done to prevent it from getting tied up to a specific database. For example if MySQL is not able to handle the data loads , the database could be changed for MS-SQL Server or Oracle with minimal effort. Now we discuss about the XML to RDB mapping issues. Introduction to the Problem of XML to RDBMS Mapping XML is getting increasingly popular for data exchange and storing for the web applications because of its portability and ease of data exchange features. It is useful when applications must communicate with other applications or integrate information from several other applications. XML documents are self-contained i.e. they contain the data as well its presentation format. However querying of XML data is difficult because of its hierarchical structure. Over the past few years a lot of work has been done to figure out optimal ways of storing XML data in a relational model and take advantage of its query processing abilities. Pros And Cons Of Xml Over Relational Model XML is self-describing in nature: An XML document represents the data along with the metadata. In a relational model data and the metadata exist at different places. Department of Computer Science, UIC Page 26 of 58 Design and Implementation of CoreWall’s Session Database XML data is portable while a data in relational model is not. For example to exchange data between MSAccess and Oracle we need to do some data transformations even both of them are based on relational model. Document DOM Session_ Database DocumentBuilder Factory XML Data Document Builder Data_ File Session Track Annotatio n Image Get required data from the DOM and store It into the Database CoreWallDB Graph Annotatio n Annotatio n Annotati on Figure 7: Depicts how a state file is processed before its data could be stored into the session database Department of Computer Science, UIC Page 27 of 58 Design and Implementation of CoreWall’s Session Database Adaptable to Changes: The format of an XML document is not rigid we can easily add new tags as per our requirements. However doing the same thing in relational model is not possible because of the restrictions imposed by the relational model constraints. Good for storing tree or graph based structure: XML’s hierarchical structure makes it possible for us to store tree or graph structured data in XML documents. Databases using relational model do not allow us to store hierarchical data. XML is considered to be a database in a weak sense and suffers from the limitations with respect to: Efficient access to data due to parsing and conversion, Efficient Storage, Indexes, Security, Multi-user access, Triggers, Transaction Management, Data Integrity, Queries across multiple documents. Given these limitations XML documents work good in an environment consisting of small amounts of data, few users, and modest performance requirements. However it will fail in an environment, which has many users, requires good performance and needs strict data integrity. Need To Convert Xml Data Into Relational Model We saw above that XML is really good way of exchanging data over the web, however looking at its limitations above. There is a need to convert XML data into a relational model. So that data can take advantage of both XML and relational model features i.e. “take advantage of portability of XML” and “be able to exercise powerful database features like querying, efficient storage and security”. Challenges And Complexities To Efficient Storing Of Xml Databases In order to store XML document in a relational database we need to map the XML schema to a database schema. A database is represented by a set of flat tables linked together by one-to-one, one-to-many and many to many relationships. Each table has a fixed number of columns, which represent the attributes of the data model. Relationship between different tables is represented using the Primary and Foreign Key relationship. XML data on the other hand is tree structured and relationship between data is represented by containment i.e. a sub-element or attribute stored within another element. Thus we see that XML documents and relational databases have a tree and flat structure respectively. Therefore, we need to map XML’s tree structure so that it fits into an equivalent flat relational databases structure. Department of Computer Science, UIC Page 28 of 58 Design and Implementation of CoreWall’s Session Database Storage Whenever an XML document is to be stored in a relational database, there are two points of consideration namely optimal generation of relations to store the XML data and preservation of order information present in an XML document. We discuss both of them below. a) Optimal Relation Generation When converting XML documents we need to identify what relations should be created out of elements present in an XML document. Creating a relation for each element means we are able to capture all the data and there is no loss of semantics and data when it is mapped from XML to RDBMS. However they (large number of relations) are inefficient for querying because they increase the amount of join processing to be done during querying. Since join is the most costly operation to be performed on a relational database. Therefore, for performance reasons we need to create an optimal number of relations from an XML schema/DTD. b) Order preservation Order is an important feature of XML documents and there is an implicit ordering of elements according to the document order. This ordering also determines the semantics/interpretation of data. The relational model on the other hand is not ordered. To ensure that the mapping of data is lossless (no loss of information) we need to make sure that order is preserved during the conversion between XML and relational form. There are two basic features desired from any ordering scheme: a) We should be able to reconstruct an XML document using the ordering information preserved in relational schema. b) This ordering information should facilitate translation of ordered XML queries to SQL queries. Creating a XML structure that corresponds to a database schema The following steps are used to convert a RDBMS structure to an equivalent XML DTD  For each table in the database create an element only content because element only tags allow us to deal with data points.  For each column in the table create an attribute. They constrain the number of values that can be contained unlike elements thus help in simulating data points in a XML’s tree structure. Department of Computer Science, UIC Page 29 of 58 Design and Implementation of CoreWall’s Session Database Referential Integrity and storing of XML documents to a RDBMS Referential Integrity means that foreign keys always point to valid primary keys . In other words, a row containing the primary key must be inserted before inserting the row containing the foreign key e.g. a session row must be created before creating the tracks, annotations that relate to it. Thus referential Integrity imposes a certain order, which is needed for transferring the data from a XML document to the database and also decide the order in which XML elements are to be accessed. For e.g. in our saveSession example in the workflow section the session row is created first then the session ID generated is used throughout the save session process. 5.3 Java Prepared Statements 14] Java prepared statements are pre-complied SQL statements .The SQL statement contained in a Java PreparedStatement could have more than one IN parameters. IN parameters are the parameters whose values are not specified when the SQL statement is created. Instead the SQL statement has a ? mark which acts as a placeholder for each IN parameter.Values for all the IN parameters must be supplied or set before executing the prepared statement. An example is shown in Figure 8. These prepared statements provide the following three benefits 1) Increased Portability: With prepared statements the responsibility of handlijng the conversion and formatting issues of data in queries is shifted onto the implementing JDBC driver. e.g. the problems of handling date values, how to handle ‘ are all handled by the prepared statements independent of the programmer’s code. Thus, making the code more portable. 2) Security: Prepared Statements provide protection against SQL Injection attacks i.e. they prevent malicious users from running the code that was not intended. 3) Performance benefits: Prepared Statements are more efficient than the string statements. They reduce the amount of work the JDBC driver and the database has to do when the statements are reused. At the first execution these prepared statements are parsed and compiled and likely put into the statement cache. This caching allows to save the execution path which cannot be accomplished with string statements which have variable parameters .As SQL statements with different parameters appear different to the database thus caching does not happen. Department of Computer Science, UIC Page 30 of 58 Design and Implementation of CoreWall’s Session Database public static int userSessions(String uname) { PreparedStatement prepareGetUserSessions=null; String GetUserSessions=null; GetUserSessions="Select session_name,session_id,ST.user_id from users_table UT,session_table ST where UT.User_ID=ST.User_ID and User_Name=?"; prepareGetUserSessions=con.prepareStatement(GetUserSessions); prepareGetUserSessions.clearParameters(); prepareGetUserSessions.setString(1,uname); rsGetUserSessions=prepareGetUserSessions.executeQuery(); …………………………….. …………………………….. } Figure 8: Prepared Statement example that executes the userSessions query 5.4 Java Native Interface [4] Java Native Interface (JNI) is a feature provided by the Java platform.. It allows us to build applications that have use both native code in programming languages like C and C++ with code written in Java. It allows programmers to combine the power of Java platform without having to abandon their investments in the legacy code. JNI allows to handle two way situations where we need to combine Java applications with native code(refer to Figure 9). Thus, JNI provides for two types of native code. 1) Native libraries : We could write Java code that invokes functions written in other languages. These native methods are called in the same way as the Java methods. These native methods reside in native libraries. 2) Native Applications : JNI’s invocation interface allows us to embed Java Virtual Machine implementation into native applications. Native applications can link with a native library that implements the JVM and use the invocation interface to execute software components written in Java. Java application and library Java virtual machine implementation Native application and library Figure 9: JNI acts as communicating channel between Java and legacy code Department of Computer Science, UIC Page 31 of 58 Design and Implementation of CoreWall’s Session Database 6. WORKFLOW [7] In this section, detailed diagrammatic descriptions of CoreWall’s workflow are shown. They depict the high level view in terms of the user activity and the way system handles a user activity. The detailed diagrams show low level details in terms of the sequence of tables affected during a particular activity. These user activities could include saving of a session from the state file, retrieval of a specific session, executing queries to get the desired data. 6.1 Save Session: This is used to store a session from a state file into the database. The name of the XML file that contains the session information is passed by the C function. The data to be inserted into the database from the XML file is parsed using the DOM parser.Figure10 below gives a high level view of the save session Corewall Client Machine graph data User Saves Session XML document containing (references to spatial data + session related info) created Execute JNI C code SAVE SCENARIO microscopic data Remote Database Execute Java Code. Geoscientist image data Populate Database System Activity CoreWall Session Database Server User Event Figure 10: High Level process of Save Session Department of Computer Science, UIC Page 32 of 58 Design and Implementation of CoreWall’s Session Database process in term of system events and user activity . JN I C o d e(invo ke_saveS ession) int invoke_saveS ession(JN IEnv* env,char *fnam e) { jint check_save_flag; jclass save; jm ethodID saveM ethod_M ain,saveM ethod_check; jobjectArray saveM ethodArgs; …………………… … … … … … … … … .. if(check_save_flag ==1) { (*env)->C allS taticV oidM ethod(env, save, s aveM ethod_M ain, saveM ethodA rgs); } return check_save_fla g; } in voke ifS e ssionE xist S ho w an error m essage an d e xit. YES A se ssion exists in th e d ata base w ith the sa m e na m e a s in the ne w state file . NO invoke sa veM ethod _m ain S tart saving the sessio n b y parsin g th e X M L file usin g DOM. Figure 11: Save Session Workflow-1 Department of Computer Science, UIC Page 33 of 58 Design and Implementation of CoreWall’s Session Database Get all the data file elements from the XML file. 1. Get all the session elements from the XML file. - Insert into DataFile_Table XML to relational Mapping CoreWall Database 1. Insert into session_table 2. Insert into session_relationship_table 3. Insert into annotation_table Insert into session_annotation_mapper_table 4 Insert into annotation_relationship_table XML to relational Mapping 2. If session has a parent. CoreWall Database 3. Get all the session annotations from the XML file 4. If the annotation has parent -Generate global sessionID -generate local ID for each session annotation Session_ID 1. Get one track element at a time XML to relational Mapping 2. Get all the track annotations from the XML file CoreWall Database 3. If the annotation has parent 1. Insert into track_table Insert into track_session_mapper_table 2. Insert into annotation_table Insert into track_annotation_mapper_table 3. Insert into annotation_relationship_table -generate local track_id -generate local ID for each track annotation track_ID Figure 12: Save Session Workflow-2 Department of Computer Science, UIC Page 34 of 58 Design and Implementation of CoreWall’s Session Database track_id 1. G et one im age elem ent from the X M L file at a tim e. X M L to relational M apping 1. Insert into im age_table Insert into the track_ im age_m apper_table 2. Insert into annotation _table Insert into im age _annotation _m apper_table 3. Insert into annotation _ relationship _table 2. G et all the im age annotations from the XM L file C oreW all D atabase -G enerate local im ageID -G enerate local ID for each im age annotation 3. If the annotation has parent im age _id 1. G et one graph elem ent from the X M L file at a tim e. 1. Insert into graph_table Insert into the graph _im age _m apper_table 2. Insert into annotation_table Insert into graph _annotation _m apper_table 3. Insert into annotation _relationship _table X M L to relational M apping 2 . G et all the graph annotations from the X M L file C oreW all D atabase 3 . If the annotation has parent -G enerate local graphID -G enerate local ID for each graph annotation Figure 13: Save Session Workflow-3 Figure 14 below depicts the tree structure of the state file and what tables each node affects, when the file is parsed and used to store the session information into the session database Department of Computer Science, UIC Page 35 of 58 Design and Implementation of CoreWall’s Session Database Session_ Database Populates Populates Data_ File Session session_table session_relationship_table datafile_table Populates Populates Track track_table track_session_mapper_table Annotati on annotation_table annotation_relationship_table session_annotation_mapper_table Populates Annotati on Image Populates Populates graph_table graph_image_mapper_table annotation_table annotation_relationship_table track_annotation_mapper_table Graph Annotati on annotation_table annotation_relationship_table image_annotation_mapper_table Populates Annotati on annotation_table annotation_relationship_table graph_annotation_mapper_table Figure 14: Tree structure of XML file and what tables are affected by each element Department of Computer Science, UIC Page 36 of 58 Design and Implementation of CoreWall’s Session Database Figure 15: Sample state file Figure 16 below shows tables contained in CoreWall’s session database . Department of Computer Science, UIC Page 37 of 58 Design and Implementation of CoreWall’s Session Database Department of Computer Science, UIC Page 38 of 58 Design and Implementation of CoreWall’s Session Database Figure 16: Database Schema Snapshot 6.2 Retrieve_Session Retrieve session is used to retrieve a session belonging to an authorized user. It retrieves the session information as a number of small XML files(see Figure 20 and 21). These files are then used to recreate the session on the fly. For information on what files are generated during session retrieval and there naming conventions please refer to table in Figure 19.Figure 17 below gives a high level view of the retrieve session process in term of system events and user activity. The low-level data and logic flow during session retrieval is depicted in Figure 18. Corewall Client Machine User requests Session Execute JNI C code Execute Java Code. Send Retrieve Session SQL Query CoreWall Session Database Server Geoscientist Text and Session Data Session Recreation Information Text and Session Data Get retrieved data Spatial Data System Activity User Event Figure 17: High Level process of Retrieve Session Department of Computer Science, UIC Page 39 of 58 Design and Implementation of CoreWall’s Session Database JNI Code(invoke_retrieve("5","Corewall_ID_000") int invoke_retrieve(JNIEnv* env,char *userid ,char* sessName) { jint check_retrieve_flag; jclass retrieve; jmethodID retrieveMethod_Main,retrieveMethod_check; jobjectArray retrieveMethodArgs; ............................... ........................................... if(check_retrieve_flag==1) { (*env)->CallStaticVoidMethod(env, retrieve, retrieveMethod_Main, retrieveMethodArgs); } return check_retrieve_flag; } No XML files are generated and 0 is returned If a session exist with the passed userid and session name NO YES Create a folder with the session name Save generated XML files in this folder. These files contain tracks, images, graphs, annotations information belonging to the session Corewall_ID_000 Figure 18 : Process of retrieving session information for a given session name and user id. File Generated Rule Example Session information Session annotations Session annotation +parent child relationship Session tracks Track Annotations +parent child relationship Track images Image graphs Image Annotations +parent child relationship Graph Annotations +parent child relationship Session Name +”retrieve_session.xml” Session Name +”_sessionannotations.xml” Session Name +”_Session_”+SessionID+”_Ann_”+ AnnotationID+”_session_relations.xml” Session Name+”_tracks.xml” Session Name +”_Track_”+trackID+”_Ann_”+ AnnotationID+”_Track_relations.xml” Corewall_ID_000_retrieve_session.xml Corewall_ID_000_sessionannotations.xml Corewall_ID_000_Session_229_Ann_1226 _session_relations.xml Session Name +”_trackimages.xml” Session Name +”_imagegraphs.xml” Session Name +”_Image_”+imageID+”_Ann_”+ AnnotationID+”_image_relations.xml” Session Name +”_Graph_”+GraphID+”_Ann_”+ AnnotationID+”_graph_relations.xml” Corewall_ID_000_trackimages.xml Corewall_ID_000_imagegraphs.xml Corewall_ID_000_Image_216_Ann_1233 _image_relations.xml Corewall_ID_000_Tracks.xml Corewall_ID_000_Track_276_Ann_1231 _track_relations.xml Corewall_ID_000_Graph_140_Ann_1243 _graph_relations.xml Figure 19 : Naming conventions for xml files that contain the session information Department of Computer Science, UIC Page 40 of 58 Design and Implementation of CoreWall’s Session Database Department of Computer Science, UIC Page 41 of 58 Design and Implementation of CoreWall’s Session Database Figure 20: Retrieved XML file that shows the parent-child relationship between annotations Figure 21: Retrieved XML file that shows graphs associated with an image in a particular session Department of Computer Science, UIC Page 42 of 58 Design and Implementation of CoreWall’s Session Database Figure 22 below represents the snapshot of an example of retrieve session. Example with the following data : Session ID= 229 Session Name= Corewall_ID_000 Track : 276,277 Image : 216,217 Graph: 140,141 Figure 22 : XML files generated by retrieve session information Department of Computer Science, UIC Page 43 of 58 Design and Implementation of CoreWall’s Session Database 6.3 Queries In addition to retrieve session and save session features, there is a querying facility available to the CoreWall user. This could become handy when the geoscientists are trying to analyze their previous studies or when they are trying to integrate information from different sessions. The query feature is highly extensible and more queries could be easily added following the template of the following queries. In this section, we explain the workflow of some of the queries(refer Figure 23 for High Level Query Process). The queries parameters are passed through C functions and the results are generated as XML files. Corewall Client Machine User Query Execute query specific JNI C code Execute Java Code. Send SQL Query CoreWall Session Database Server Geoscientist Text and Session Data Get retrieved data Spatial Data System Activity User Event QUERY SCENARIO Figure 23: High Level process for any query execution Department of Computer Science, UIC Page 44 of 58 Design and Implementation of CoreWall’s Session Database UserSessions: This query is used to retrieve information about sessions that are associated with a given user name. The result is produced as an XML file(refer Figure 25). Figure24 below shows the workflow of the process and the query that is executed to retrieve this information. JNI Code(invoke_userSessions) int invoke_userSessions(JNIEnv* env,char *user_name) { jclass uname; jint userSessions_flag; jmethodID unameMethod; ................ uname = (*env)->FindClass(env, "retrieve_ps"); unameMethod = (*env)->GetStaticMethodID(env, uname, "userSessions", "(Ljava/lang/String;)I"); ........................ } No XML files are generated and 0 is returned NO Check if sessions exist for a given username YES Generate XML file that contain information about sessions belonging to a given username. This is stored in the "queries" directory GO TO "queries" Executes Create an XML file as : username+_user_sessions .xml "Select session_name,session_id,ST.user_id from users_table UT,session_table ST where UT.User_ID=ST.User_ID and User_Name=?"; Figure 24: UserSessions query workflow Department of Computer Science, UIC Page 45 of 58 Design and Implementation of CoreWall’s Session Database <?xml version="1.0" encoding="UTF-8" ?> <result xmlns:xsi="http://www.w3.org/2001/XMLSchemainstance"> <row> <session_name>Corewall_ID_765</session_name> <session_id>218</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_001</session_name> <session_id>219</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_002</session_name> <session_id>220</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_120</session_name> <session_id>221</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_007</session_name> <session_id>222</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_004</session_name> <session_id>223</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_010</session_name> <session_id>224</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_011</session_name> <session_id>225</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_111</session_name> <session_id>226</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_777</session_name> <session_id>227</session_id> <user_id>5</user_id> </row> <row> <session_name>Corewall_ID_000</session_name> <session_id>229</session_id> <user_id>5</user_id> </row> - <user_id>5</user_id> </row> </result> Figure 25: Sample XML file generated by userSessions query Department of Computer Science, UIC Page 46 of 58 Design and Implementation of CoreWall’s Session Database SessionTracks: This query is used to retrieve information about tracks that are associated with a given session name. Result is generated as XML file(refer to Figure 27). The workflow and query executed is shown in Figure 26 below. JNI Code(invoke_sessionTracks) int invoke_sessionTracks(JNIEnv* env,char *strack_name) { jclass strack; jint sessionTracks_flag; .................................. strack = (*env)->FindClass(env, "retrieve_ps"); strackMethod = (*env)->GetStaticMethodID(env, strack, "sessionTracks", "(Ljava/lang/String;)I"); ................................ return sessionTracks_flag; } No XML files are generated and 0 is returned NO Check if tracks exist for a given session name YES Generate XML file that contain information about tracks belonging to a specific session . This is stored in the "queries" directory GO TO "queries" Executes Create an XML file as : sessionname+_session_tracks.xml "Select TT.track_name ,TT.track_id from track_table TT,Session_Table ST,Track_Session_Mapper_Table TSMT where TT.Track_ID=TSMT.Track_ID and ST.Session_ID=TSMT.Session_ID and ST.Session_Name=?"; Figure 26: A figure depicting how to get tracks associated with a given session name Department of Computer Science, UIC Page 47 of 58 Design and Implementation of CoreWall’s Session Database <?xml version="1.0" encoding="UTF-8" ?> <result xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <row> <track_name>Corewall_track_01</track_name> <track_id>276</track_id> </row> <row> <track_name>Corewall_track_02</track_name> <track_id>277</track_id> </row> </result> Figure 27: Sample XML file generated by sessionTracks query ImageSessions: This query is used to retrieve information about sessions that are associated with a given image name. Result is generated as an XML file. The workflow of process and corresponding query executed is shown in Figure 27 below. JNI Code(invoke_imageSessions) int invoke_imageSessions(JNIEnv* env,char *image_name) { jclass sti; jint sti_flag; jmethodID stiMethod; ................................... sti = (*env)->FindClass(env, "retrieve_ps"); stiMethod = (*env)->GetStaticMethodID(env, sti, "imageSessions", "(Ljava/lang/String;)I"); stiMethodArg0 = (*env)->NewStringUTF(env,image_name); ............................. } No XML files are generated and 0 is returned NO Check if sessions are associated with a given image name. YES Generate XML file that contain information about sessions associated with a specific image . This is stored in the "queries" directory GO TO "queries" Executes Create an XML file as : imagename+_session.xml "Select distinct ST.Session_ID, Session_Name,IT.Image_Name from Session_Table ST,Track_Image_Mapper_Table TIMT, Image_Table IT where ST.Session_ID=TIMT.Session_ID and TIMT.Image_ID=IT.Image_ID and Image_Name=?"; Figure 27 : Get sessions associated with a given image name Department of Computer Science, UIC Page 48 of 58 Design and Implementation of CoreWall’s Session Database SessionTrackImages: This query is used to retrieve information about tracks that are associated with a given imagename. The workflow of process and corresponding query executed is show in the Figure 28 below. JNI Code(invoke_sessionTracksImages) int invoke_sessionTracksImages(JNIEnv* env,char *sti_name) { jclass sti; jint sti_flag; jmethodID stiMethod; ....................... sti = (*env)->FindClass(env, "retrieve_ps"); stiMethod = (*env)->GetStaticMethodID(env, sti, "sessionTracksImages", "(Ljava/lang/String;)I"); stiMethodArg0 = (*env)->NewStringUTF(env,sti_name); ..................... } No XML files are generated and 0 is returned NO Check if tracks are associated with a given image name. YES Generate XML file that contain information about track and sessions associated with a specific imagename . This is stored in the "queries" directory GO TO "queries" Executes Create an XML file as : imagename+_track_images.xml "Select distinct TT.*,IT.Image_ID,IT.Image_Name,TIMT.Session_ID,ST.Session_Name from Track_Image_Mapper_Table TIMT, Session_Table ST, Image_Table IT, Track_Table TT where ST.Session_ID=TIMT.Session_ID and ST.Session_Name=? and IT.Image_ID=TIMT.Image_ID and TIMT.Track_ID=TT.Track_ID" Figure 28: Get tracks associated with a given image name Department of Computer Science, UIC Page 49 of 58 Design and Implementation of CoreWall’s Session Database ImageAnnotations: This query is used to retrieve information about annotations that are associated with a given image name across different sessions. The workflow of process and corresponding query executed is show in Figure 29 below. JNI Code(invoke_imageAnnotations) int invoke_imageAnnotations(JNIEnv* env,char *image_name) { jclass sti; jint sti_flag; ........................................... sti = (*env)->FindClass(env, "retrieve_ps"); stiMethod = (*env)->GetStaticMethodID(env, sti, "imageAnnotations", "(Ljava/lang/String;)I"); return sti_flag; } No XML files are generated and 0 is returned NO Check if the image has annotations associated with it YES Generate XML file that contain information about annotations associated with an image name . This is stored in the "queries" directory GO TO "Queries" Executes Create an XML file as : imagename+_annotations.xml "Select distinct TT.*,IT.Image_ID,IT.Image_Name,TIMT.Session_ID,ST.Session_Name from Track_Image_Mapper_Table TIMT, Session_Table ST, Image_Table IT, Track_Table TT where ST.Session_ID=TIMT.Session_ID and ST.Session_Name=? and IT.Image_ID=TIMT.Image_ID and TIMT.Track_ID=TT.Track_ID" Figure 29: Get annotations associated with a given image name Department of Computer Science, UIC Page 50 of 58 Design and Implementation of CoreWall’s Session Database ImageGraphs: This query is used to retrieve information about graphs that are associated with a given image name. The workflow of process and corresponding query executed is show in Figure 19 below. JNI Code(invoke_imageGraphs) int invoke_imageGraphs(JNIEnv* env,char *image_name) { jclass sti; jint sti_flag; ............................................... sti = (*env)->FindClass(env, "retrieve_ps"); stiMethod = (*env)->GetStaticMethodID(env, sti, "imageGraphs", "(Ljava/lang/String;)I"); ................................... return sti_flag; } No XML files are generated and 0 is returned NO Check if the image has graphs associated with it Generate XML file that contain information about graphs associated with an image name . This is stored in the "queries" directory YES GO TO "Queries" Executes Create an XML file as : imagename+_graphs.xml "Select distinct GT.Graph_ID,IT.Image_Name from Image_Table IT, Graph_Image_Mapper_Table GIMT, Graph_Table GT where GIMT.Image_ID= IT.Image_ID and GT.Graph_ID=GIMT.Graph_ID and IT.Image_Name=?"; Figure 30: Get graphs associated with a given image name Figure 19 : Get graphs associated with a given image name 6.4 User Management [12,13] There was a need to include the user and database management queries. The former was needed to manage the users of CoreWall in terms of users, permissions and group. Latter was needed for integration of session database with other database. Department of Computer Science, UIC Page 51 of 58 Design and Implementation of CoreWall’s Session Database This involved writing C code (example code in Figure 31) that could be used to perform standard user management functions which include a) Creation of CoreWall user account b) Deletion of a CoreWall user account c) Password change feature for a CoreWall user d) Creation of a group e) Deletion of a group f) Creation of a database references and their association with session database. int sdb_create_user(char *uname,char *fname,char *lname,char *grp, char *pwd) { char query_create_user[xlong_query]; char *query1="insert into users_table(User_Name,First_Name,Last_Name,Group_ID,password) Values('"; int grp_check=-1,create_query_status=-1; grp_check=sdb_check_grp_exist(grp); if(grp_check<1 || grp_check ==1) { printf("Group ID could not be found\n"); return -1; } else { sprintf(query_create_user,"%s%s%s%s%s%s%s%d%s%s%s",query1,uname,"','",fname,"','",lnam e,"','",grp_check,"','",pwd,"')"); create_query_status=mysql_query(conn,query_create_user); if(create_query_status==0) return 0; else { if(mysql_errno(conn)==1062) { return 2; } return 1; } } Figure 31 : Code that creates a new CoreWall user account } Department of Computer Science, UIC Page 52 of 58 Design and Implementation of CoreWall’s Session Database 7. SAMPLE SCREENSHOTS In this section a few screenshots generated by the visualization component are included. These screenshots depict how the annotations, images, tracks, graphs are presented to the user.For e.g. Figure 32 shows graphs ,tracks and images from a session. This screenshot also details the tables that are affected when the user saves this session. Track Graph Image Populates 1) Track Table 2) Image Table 3) Track Image Mapper Table 4) Graph Table 5) Graph Image Mapper Table Figure 32 : Track ,Image and Graph and what tables they affect when the session is saved Department of Computer Science, UIC Page 53 of 58 Design and Implementation of CoreWall’s Session Database Track Track annotation Populates 1) Track Table 2) Annotation Table 3) Track Session Mapper Table 4) Annotation Relationship Table 5) Track Annotation Mapper Table Figure 33: Track and its annotation and what tables they affect when the session is saved Department of Computer Science, UIC Page 54 of 58 Design and Implementation of CoreWall’s Session Database Image Annotation Image Populates 1) Image Table 2) Annotation Table 3) Image Annotation Mapper Table 4) Annotation Relationship Table Figure 34: Image and its annotation and what tables they affect when the session is saved Department of Computer Science, UIC Page 55 of 58 Design and Implementation of CoreWall’s Session Database 8. SYSTEM REQUIREMENTS This section specifies the system requirements for both the server machine running the MySQL server and the client machine containing the Session Database along with visualization code. Client Side JNI version 1.4 Java Run Time Env. jre1.5.0_05 Server Side MySQL 4.1.12-standard Java Development Kit JDK 5.0 Visual Studio.Net 2003 mysql-connector-java-3.1.10 RS2DOM Class (included in the CD) Department of Computer Science, UIC Page 56 of 58 Design and Implementation of CoreWall’s Session Database 9. TABLE OF ACRONYMS CD CWD CWS DOM DTD IODP JDBC JNI JVM MS-SQL RDBMS SQL VCD XML Department of Computer Science, UIC Core Description Core Workflow Database CoreWall Suite Document Object Model Document Type Definitions Integrated Ocean Drilling Program Java Database Connectivity Java Native Interface Java Virtual Machine Microsoft SQL Server Relational Database Management System Structured Query Language Visual Core Description Extensible Markup Language Page 57 of 58 Design and Implementation of CoreWall’s Session Database 10. REFERENCES [1] Michael, kofler. MySQL. 2001 ed. : a! Apress, [2] Ronald, Bourret. "Data Transfer Strategies." at URL : http://www.rpbourret.com/. 2001. [3] Liang , Sheng . Java(TM) Native Interface: Programmer's Guide and Specification . : Sun Microsystems,1999 [4] Java Native Interface at URL : http://java.sun.com/docs/books/tutorial/native1.1/ [5] JDBC2XML/ at URL : http://www.jeckle.de/freeStuff/JDBC2XML/ [6] LRC Core Facility - barrel sheets at URL :http://lrc.geo.umn.edu/barrelsheets.htm [7] CoreWall at URL : http://www.evl.uic.edu/cavern/corewall/ [8] Extensible Markup Language (XML) at URL: http://www.w3.org/XML/ [9] Java Technology at URL : http://java.sun.com/ [10] org.w3c.dom (Java 2 Platform SE v1.4.2) at URL : http://java.sun.com/j2se/1.4.2/docs/api/org/w3c/dom/package-summary.html [11] MySQL AB :: Download Connector/J 3.1 at URL http://dev.mysql.com/downloads/connector/j/3.1.html [12] MySQL 5.0 Reference Manual :: 23.2 MySQL C API at URL http://dev.mysql.com/doc/refman/5.0/en/c.html [13] MySQL AB :: The World's Most Popular Open Source Database at URL http://www.mysql.com/ [14] Using Prepared Statements at URL : http://java.sun.com/docs/books/tutorial/jdbc/basics/prepared.html Department of Computer Science, UIC Page 58 of 58

Java Native Interface: - UIC

Related documents

Products

Support

Java Native Interface: - UIC

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib