EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE Olivier MARTIN Project Leader Isabel FERNANDEZ-GONZALEZ Analyst CERN - Geneva - SWITZERLAND Summary CERN collaborates with physics institutes from all around the world. The aim of our project is to provide worldwide access to our human resources database. This internet application is made of a pure JAVA client which interacts with ORACLE*HR through a JDBC bridge. The JAVA client provides a highly interactive interface to business objects. This permits the non-specialist user to manipulate HR data with no understanding of the underlying database structure and constraints. On the server side, data are encapsulated into PL/SQL APIs. An extensive use of dynamic SQL makes this layer compact, multi-purpose and reusable. We will present the architecture of the application, the techniques used, and the experience we gained from this project and other web applications developed by our department. EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 2/12 Introduction CERN CERN is the European Laboratory for Particle Physics, the world's largest particle physics centre located near Geneva. Founded in 1954, the laboratory was one of Europe's first joint ventures, and has become a shining example of international collaboration. From the original 12 signatories of the CERN convention, membership has grown to the present 19 Member States. The Laboratory provides state-of-the-art scientific facilities for academic research. Around 3000 staff welcome 6500 scientists (so called CERN Users) each year from 500 universities and over 80 nationalities. The Web was born at CERN in the early 90s from the need for sharing information among the large community of physicists. The CERN staff designs and builds CERN's intricate machinery and ensures its smooth operation. CERN is currently building the world largest machine: the Large Hadron Collider, a 27-km circumference super-conducting accelerator sited about 100m underground under Swiss/French countryside around Geneva. The physicists come to CERN to run big experiments that may require a few thousand man-years of work and last several years. These experiments function as independent structures within the laboratory. Administrative Information Services Within the administrative sector, the Administrative Information Services is the IT department in charge of maintaining CERN corporate applications in the following business areas: accounting and purchasing, stock and assets management, human resources, electronic document handling (routing of administrative documents), data warehousing tools... These areas have been covered by both commercial products (ORACLE*HR, BAAN, ORIAC/SIRIAC) and in-house developments. The department employs 30 IT professionals plus 10 temporary staff. Our applications are used by around 6000 people (business specialists, normal employees or CERN users). Our medium-term strategy is to move all our applications to the web. We have tried several strategies; two of them will be detailed in this paper. Human resources: context Five years ago, personnel data were maintained in more than 10 separate and redundant databases. To integrate these sources of information into a single database (Oracle*HR) has been a constant objective during the past years. Most of the human resources areas have been covered and rationalised, except the follow-up of the volatile population of CERN users. Depending on the experiment(s) that they work for, physicists are registered following different procedures, sometimes in several databases. This situation leads to multiple data entry and inconsistencies and sketchy administrative follow-up. Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 3/12 Considering that CERN, as hosting organisation, is responsible for requesting work permits from the host states (France and Switzerland) and for the security on the site, the poor quality of the personnel data for the user community made the situation unacceptable. Finally, the existing IT solutions provide a "CERN centric" view of users which does not suit to experiments as independent structures. PIE PROJECT PIE stands for Person Institutes & Experiments and has the following objectives: gather data from all sources into a single database (Oracle*HR) standardise procedures and enforce rules by a common workflow maximally decentralise data entry As the PIE data must be maintainable from anywhere in the world (from collaborating institutes), we will provide web-based user interfaces. The developments should cover various needs: a standard data entry Interface for experiment/institute administrative services a data warehousing tool for experiment/institute management a public physic dictionary on the web Data entry interface: How ? Technical choices PIE data will be stored into ORACLE*HR. At the time, the project was launched (Q2 1997); ORACLE did not proposed any web-based solution that would meet our requirements. ORACLE*HR is a highly parameterisable product: most of the tables are simply empty repositories to contain user-defined data. The table structure is often not related to the business object that the user wants to manage. This gap would make a standard design approach (such as a form generation using designer 2000) almost impossible. We had to define a complete architecture based on user's business objects. This component based model wraps the HR complex structure through PL/SQL APIs. PIE has been chosen as a pilot project to test the JAVA technology. The client consists of JAVA applets that interact with the database through the ORACLE JDBC thin driver. It is currently configured in a 2tier architecture. Layered approach The PIE framework is based on a stack of independent layers. This approach minimises the maintenance work and allows the reuse of components at any level (such as reusing PL/SQL APIs to write data loading or bulk update programs). The next figure gives an overview of the structure of PIE. The following sections details the role and techniques implemented in each layer. Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 4/12 Low level APIs The low-level APIs directly encapsulate HR tables. There are roughly four API's per HR data table : a create API an update API a remove API a query API that allow queries on any table field. The query statement is built and executed dynamically with the data returned as parameters (PL/SQL arrays). These APIs perform some basic verifications such as foreign key checks. A basic concurrent access detection is made (based on row timestamp). No business-oriented checking can be performed at that level (APIs are table-oriented). For those of you that know Oracle HR, good example are the APIs that allow us to manipulate any type of "special info type". High level APIs The high-level APIs implement a procedural interface to the business objects. At most there are 5 APIs per business object: a create API an update API a remove API a query API that allow queries on any object item. The result is stored in a temporary buffer (PL/SQL table) on the server side. a fetch data API that compresses the result of a query into a stream to be sent to the client. These APIs make the correspondence between business objects and raw data. Most of the business-oriented validations are made at this level (such as static, dynamic or contextual list of values, lookups…). Security is enforced at this level. This layer also implements some features related to the temporal nature of HR data. This point will be addressed later. Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 5/12 JDBC layer The JDBC layer provides an object-oriented view of the business objects. They are implemented as Java objects with all necessary methods: a Query() method a FetchNextObject() method a constructor, a modifier and a destructor For each field, a set, get and validate method All objects are formatted in the same way, so that the upper layer can handle them. In addition, this layer is in charge of the connection to the database, the decompression of the query results and the transaction processing. Events and beans processor The event and bean monitor is the client organiser. It is made of three main components: The generic grid The grid monitor The event monitor The generic grid may be compared to a spreadsheet made of several occurrences of the same business object (a table, a buffer, an SQL*Forms block…). This component manipulates business objects as beans; the generic grid is able to handle any type of business object as long as they respect a predefined format. The generic grid provides all navigation methods within the business object buffer (skip, next, previous, last, first…) The grid monitor is able to handle an unlimited number of grids and their relationships. Its main purpose is to manage the master-details relationships between grids (for instance to populate a detail grid when the reference data changes). It may react to internal events (transmission of data from the server) or external events (navigation to next row, click on buttons). It refreshes the screen using a unique refresh() function provided by the screen layout. The GridMonitor is also in charge of some standards features such as displaying list of values, error and debug boxes. The Event monitor centralises all events generated by the user interface. It has been designed following the "old" JDK 1.0 event model (but using the JDK 1.1 events). We found the new event model much more difficult to control and tune, and we had to go back to a more centralised approach. Screen layout Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 6/12 The Screen layouts have been designed to require as little work and knowledge as possible from the developer. The layout instanciates the graphical objects and the grids by in the applet, and provides a refresh() method to the grid monitor. This function makes the link between the graphical objects and the underlying grids. Development tools When the project started, no suitable solution existed on the market: The standard design tool provided by ORACLE did not offer a sufficient level of flexibility. Furthermore, the Designer was inadequate to handle the "semantic gap" between table structure and business objects. The Java design tools offered a good environment development for Java programming, but the JDBC facilities were still in their infancy. We opted for a simple design tool for the screen layouts (KAWA from Tek.Tools). We built up our own design tool to develop intermediate layers. We started from the definition of the business objects. From this definition, we automatically generate the high level APIs and the JDBC layers. The client main part (the event and beans processor) has been designed to be able to handle any business object. This layer has been written once, and is reused everywhere. The "semantic gap" between database structure and business objects has been solved by manual programming. Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 7/12 Data entry interface: Features Look and feel Just to give you an idea of what the screen looks like. They are made of three areas: A search panel which allows queries on several key fields. These searches may tolerate typing mistakes. The button panel. The number of actions that may be performed there depends on the methods associated to the current business object. A data entry area, which may include several zones (may work on several objects in the same screen). Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 8/12 Security Common authentication system Since PIE is to be open for worldwide access, it was essential to have a secure way to identify our users. We have defined a common login mechanism for all our web-based applications This mechanism uses the https protocol. Once registered, the user has to download a certificate and can use any of our application he has access to (logging once per browser session). Preventive security One of the objectives of PIE is to delegate as much as possible the data entry from central services to the experiment secretariats and institute administrative services. Consequently, we have many operators, and therefore had to invest a lot in a security mechanism that enforces everyone's responsibilities. We opted for a security mechanism based on the actions a category of user may perform (static), and the data the user may consult/alter (dynamic). The security system acts preventively: buttons and entry fields are disabled when the user cannot perform an operation. Integration with ORACLE*HR Finally, PIE must coexist with HR and some of our users use both interfaces. The security system and validations are fully integrated with Oracle*HR: PIE reads the definition of the access rights the validations (so-called value sets and quickcodes) at run time. Date-tracked logic If you ask to any ORACLE*HR user what is the least intuitive feature in the system, he would probably answer the "Date-track logic". All data in ORACLE*HR are dated. The core data are "date-tracked", which means that every time a change is made, a row is inserted in the database with the new values, and the old record is ended. For instance, let us consider the following personal file: It just means that the person will be emancipated on the first of July. If now that the person got married the 5th of May, his personal file should be modified as follows: Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 9/12 Through the standard HR interface, this operation has to be performed in four steps: 1) Set the session date at the 5th of May 2) Set the marital status to "Married". This operation creates a new period between the 5th of May and the next change (the 1st of July). It does not propagate the change to the next periods 3) Set the session date to the 1st of July 4) Set the marital status of the next period to "Married". This operation is not intuitive, complicated and error prone. In PIE, the change can be performed in a single operation as follow: The user changes the marital status on the last period, and tells the application from which date the information is valid. The system will automatically split the existing period and roll forward the changes. Another nice feature we have implemented is the automatic compression of similar periods. A user may be interested in only a subset of the information stored into the table. If a change exists on a field which is not displayed (the field 'C' in our example), the user would see two identical periods on the screen, which is a bit annoying and difficult to explain. PIE automatically merges the periods in such a case. In reality, the user does not see the rows, but the period of validity of the pieces of information he his interested in. At the same time, values that change between to consecutive periods are highlighted. Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 10/12 Analysis tool Architecture The data-warehousing tool on PIE data is based on a three-tier architecture: The database server contains the data pre-structured for multidimensional analysis The middle tiers is made of Java servlets that query the database through the ORACLE JDBC thin driver and generates dynamic HTML pages with embedded JavaScript. The client can be any browser that understands JavaScript. Features The tool allows multidimensional queries such like the EXCEL “Pivot tables”. User can make various styles of charts (we use the NetCharts library from Visual Mining Inc.). The user can download the result of his analyses in formatted text files, HTML and EXCEL spreadsheets for further processing. Thereby offering a means of recovering the data he has entered in our system. He can save personal queries/report into his user folder. Developments are based on reusable components: SQL queries and layout definition are isolated from the code. Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 11/12 Our experience Comparison of WWW technologies Our department has experienced three different strategies to move its applications to the web: Java in the client has been tested for the PIE project and the electronic document handling application, Java on the middle tiers (servlets) is used for our data warehousing tools. The servlets generate HTML pages with embedded JavaScript. ORACLE Designer 2000 has been used for medium-size, in-house developments. The next figure makes the synthesis of our experience: The servlets offers a high level of reliability and portability. This solution is not appropriate for applications that require a very high level of interactivity - as any other HTML/JavaScript solution such as Designer 2000 -. The servlet solution is more extensible than the designer, albeit with a higher development cost. The applet solution enables you to design better interfaces in term of interactivity, look and feel. On the other hand, this solution remains risky (portability, stability) and expensive. The Designer approach is very suitable for rapid application development: it is the leat expensive once you know the tool. It is very much appropriate for new applications. All solutions are slower than the old, character-mode user interfaces. Nevertheless, response times are similar and acceptable if the applications are correctly optimised. Olivier Martin / Isabel Fernandez-Gonzalez CERN EASY MANAGEMENT OF ORACLE*HR DATA USING A JAVA INTERFACE 12/12 Java in the client Developers found Java simple and intuitive compared to C++. It is a pure object-oriented language, when C++ is a mix of procedural and object oriented programming. Portability was a problem for a long time. Loading of JVMs at runtime, use of JVM from the same vendor (in our case: MicroSoft) everywhere have been solutions to work around this problem. Performance is acceptable although slower than compiled programs on more traditional architectures. We noticed big differences between Java Virtual Machines. Our preferred one - in term of performances is the MicroSoft Internet Explorer JVM. The way you program may have a large impact on performances, especially when your application generates many events. Bean programming (dynamic method invocation) did not affect performances of our application. Stability is not better than acceptable. We faced memory leaks with Netscape 4.0. Our general impression is that the situation has progressed since beginning of last year. Use of archive files highly reduces the network traffic. The use of bean techniques greatly reduces the size of code: with around 20 screens, the PIE client represents 1.5Mb of code. To conclude we can say that the PIE project has demonstrated that complex applications can be built using JAVA. On the other hand, this technology is getting mature slower than expected. This has increased the overall cost of the project. Olivier Martin / Isabel Fernandez-Gonzalez CERN