WEGA-GDM-Server-Information Copyright© 1997-2002 M.O.S.S. Computer Grafik Systeme GmbH Address: Hohenbrunner Weg 13, 82024 Taufkirchen Telephone: (089) 666 75 100 Fax: 666 75 180 Managing Director: Dipl.-Math. Hans Braun HR B: 82009 München - Registry Authority office All rights reserved M.O.S.S. only, reserves the rights to modify the specifications and other contents of this document without notice. This publication may not be copied without agreement of M.O.S.S. and is intended for the exclusive use by M.O.S.S. customers. (1) WEGA-GDM stands for Web Enabled Graphics Architecture Geo Document Management System. WEGA-GDM is a Web Enabled Graphics Architecture for a geo-documents content management system. The document management system stores application independently documents with geo-reference. For every document-type the presentation or editor program can be defined and automatically started. WEGA-GDM is based on an client server architecture and it makes possible the management of any document type. The access to the documents is by means of factual or graphical data. Documents, e.g. graphic documents (raster and vector graphics), text documents (e.g. Ms Office documents) or also web pages can be archived, selected, processed and displayed. Common feature of all documents is their geo-reference, that means, all documents have a position in space by co-ordinates and thus they are spatial selectable. Apart of this, the documents are stored with additional application-dependent information (factual data). Documents can be selected by two different ways: By graphical/spatial means on the map or by a query of their alphanumerical data. WEGA-GDM is the solution for managing documents and it is used not only within the areas of the surveying but also in other technical documentation departments like energy supply and municipalities. (2) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Maps and Plans Integration Starting point: Large number of different documents • Building requests • Permission documents • Surveying documents • Damage reports • Images/photographs • Multimedia • Films, Audio • HTML-documents • ... Common feature: location in space Aim: Spatial search on documents (3) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH Computer Grafik Systeme Geoinformationssysteme GDM – Document type: Description of Leveling Points M.O.S.S. Computer Grafik Systeme Geoinformationssysteme (4) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH GDM – Document type: Description of TP (Topographic Points) M.O.S.S. Computer Grafik Systeme Geoinformationssysteme (5) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Document type: Cadastral Field Sheet Computer Grafik Systeme Geoinformationssysteme (6) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH GDM – Document type: Gravity Fix Point Description M.O.S.S. Computer Grafik Systeme Geoinformationssysteme (7) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Document type: Photo Computer Grafik Systeme Geoinformationssysteme Photo for the documentation of the position of a gravity fix point (SFP) (8) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Request • Large number of documents (in practice > 100,000 documents) Computer Grafik Systeme Geoinformationssysteme • Central access to the documents • Selective access on the basis of: - Geometrical information (point, rectangle, polygon,...) - Factual data (date of creation, responsibility, land registry,...) • Use of (available) Windows PC's • Use of existing software for visualisation and handling: Office products (e.g. WinWord) Graphic applications (e.g. RoSy, ImageViewer, AutoCAD). • Access control: Access limitations, logging, accounting,... (9) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Function Distribution WEGA-GDM Computer Grafik Systeme Geoinformationssysteme consisting of GDM-Server • serves all inquiries, • administers all documents, • monitors the accesses, • controls the of data (data base, file system,...), • defines a specific client communication protocol (not HTTP). GDM Client • regulates server communication, • controls access to documents (cache, data exchange with the server), • activates assigned visualisation or processing programs, • permits bringing new documents into the central storage of data, • regulates communication with VB application modules (user administration, customised forms,...). (10) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – GDM-Client-GUI Computer Grafik Systeme Geoinformationssysteme Documentviewer Window for selecting the document types Coordinate information Logging and Connecting information List of all found documents = hit list (11) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH GDM – Separation of Factual from Graphical Data M.O.S.S. Computer Grafik Systeme Geoinformationssysteme Factual data Graphical data (12) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH GDM - Input of the Factual Data by Form M.O.S.S. Computer Grafik Systeme Geoinformationssysteme Depending upon type of document, different factual data are defined which are to be provided for creating a document. The forms can be created by way of VB applications after own needs or a form can be dynamically generated on the basis of the factual data definition. Example of customised form. (13) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH GDM - Searches of the Factual Data by Dialogue Box M.O.S.S. Computer Grafik Systeme Geoinformationssysteme A dialogue box can be customised using VB applications. If no dialogue box is defined, then a default dialogue box is used. Example of customised dialogue box (14) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Geo-Reference of Documents Computer Grafik Systeme Geoinformationssysteme The geo-reference of a document is made by means of pass points, i.e. by assigning internal document co-ordinates to absolute Cartesian co-ordinates. If no geo information is available, then one can use the absolute Cartesian coordinates determined with the help of a VB Script: • On the basis of factual data • On the basis of the file name • By inquiry with the GDM server (15) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Support of Foreign Formats Computer Grafik Systeme Geoinformationssysteme Documents with foreign formats are displayed with the help of external programs (e.g Viewer,MS Office). (16) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Administration of the GDM-Client Computer Grafik Systeme Geoinformationssysteme In order to simplify the administration of the users, their rights and the document types to, an administrator can execute certain functions for the controlling of the GDM server from the GDM Client: • User administration • Activating and deactivating users • Reactivating users, who were deactivated for having tried to log in with a false password or having used unauthorised privileged instructions • Executing GDM-Protocol commands • Regulating the access to document types • Enabling and disabling logins for maintenance work • ... (17) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Data Output at the GDM Client GDM data can be printed or delivered as a file: Printout with customised framework (18) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH Computer Grafik Systeme Geoinformationssysteme M.O.S.S. GDM – Data Output at the GDM Client GDM data can be printed or delivered as a file: Printout with customised framework (19) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH Computer Grafik Systeme Geoinformationssysteme M.O.S.S. GDM – Data delivery at the GDM Client Computer Grafik Systeme Geoinformationssysteme With delivery of the data in digital form an export channel can be activated, which does not only get the files from the GDM Server, but converts them also into e.g. tiff. On the basis of available document formats, a VB script can control the conversion of the files e.g. from TFW to tiff. Available geo information can be brought in into a tiff file in the data base. The necessary converter programs are transferred with the Login from the GDM Server to the GDM Client (if on the GDM server newer versions are available). There is this: • geotifcp Conversion of compression type and loading of the geoinformation, • layer2pal Conversion from MultiLayer tiff to pallet tiff, • cvim2tiff Conversion from CVIM to pallet or RGB tiff, • tifftags Output of the tiff tags such as data format and color specification, (20) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Coloured Documents Internal representable colour images. Computer Grafik Systeme Geoinformationssysteme •Windows-BMP, •CVIM (M.O.S.S. image format), •b/w TIFF •256-Colours-TIFF(= palette TIFF), • colour separated Multi-Layer-TIFF •24-bit-RGB-TIFF, •JPEG, • further formats by link to FreeImage Graphics library. The large amount of data with 24-bit colour images require special methods: •Colour reduction of 24 bits (app. 16 million-colours) on 256 colours, if colour separation is not possible. •Filtering of palette tiff images (8-bits) to reduce quantity of data. •Colour separation of the image into app. 8-16 colours (depends upon original) and storage of the colour layers as frames in tiff file and output in GDM client by overlay of the colour layers. (21) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Data Reduction by Colour Separation Computer Grafik Systeme Geoinformationssysteme No, repeat Result acceptable? WEGA-WOW: • predefined parameter records • filter functions • produce / modify parameter records • activates ImgConvert respectively other programs activate ImgConvert, etc. Original image (22) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH Resultant image M.O.S.S. GDM – Fitting Colour Documents Computer Grafik Systeme Geoinformationssysteme GDM Fitting Client Colour separated TIFF Bmp, TIFF 24Bit Preview TIFFfile Take over of results out of WEGA-WOW WEGA-WOW Colour separation and filtering (24-Bit, Palette) (23) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH PaletteTIFF To the GDM-Server Scanner M.O.S.S. GDM – Use of WEGA Components Computer Grafik Systeme Geoinformationssysteme Create filter chains WEGA-IMG WEGA-IMG Image Server Export Filter Workflow Edit documents WEGATwain WEGA-WOW Raster TIFF, JPEG, BMP, ... file Scanner To fit, search and administer documents edited raster images WEGA-GDMClient (24) Workflow © Copyright by M.O.S.S. Computer Grafik Systeme GmbH WEGA-GDMServer Database M.O.S.S. GDM – Client/Server Technologies Computer Grafik Systeme Geoinformationssysteme GDMClient Win9x GDM-Client-side Scripting User Interface GUI) (ActiveX) VB-Script, (J-Script) OLE-Automation, e.g. Office WinNT Win2K GDM-Client base module TCP/IP GDMServer GDM-Server-side Application WinNT GDM-Server-side Scripting (Tcl, EASI) Tcl-Tk, Servlets e.g. documentspecific DB-Access Win2K Unix Tcl-Packages GDM-Server base module e.g. for different DB TCP/IP DBServer Server-side Application (stored procedures) DB-Server base module (25) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH DB M.O.S.S. GDM – GDM Server Structure External data source, e.g. DASY Internal data, Application data, GDM Client Script Repository Archiv User administration Access protection Factual data of documents Logging Document administration Module for different methods of storage File system GDM Server Spatial access Computer Grafik Systeme Geoinformationssysteme Communication via TCP/IP Databank Informix, Oracle, MS-Access, Databank Server, e.g. DBSRV Databank Spatial access (SGD or AVR) (26) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH GDM Clients (+„Debugger“) M.O.S.S. GDM – Realisation (1) Computer Grafik Systeme Geoinformationssysteme GDM Server • can be used under UNIX (HP-UX, IRIX, Solaris, Linux) and Windows™ (WinNT, Win2k, WinXP), • has been realised with Tcl, • for the spatial access EASI is used (therefore WinNT must be used at least), • for the data base access these Packages are used: • Isqltcl for Informix, • Oratcl for Oracle, • tclodbc for ODBC (under Windows) • special Package for the server „EcmDoRis“ (e.g. for encryption and dates). (27) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Realisation (2) Computer Grafik Systeme Geoinformationssysteme GDM Client • can be used on Windows™ platforms only, • has been realised with C++, • interfaces have been programmed in MFC (Microsoft Foundation Classes), • for the development of client applications an ActiveX interface is available, • with certain GDM client functions it is examined whether VB Scripts are present. If so they are automatically executed. • All scripts, programs, VB applications are loaded from the GDM server during login, so far as the GDM server possesses more current versions (automatic application update). (28) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Server folder structure (1) <server> Computer Grafik Systeme Geoinformationssysteme The GDM server with all necessary shared libraries and scripts, particularly ecmdoris.dll, ecmrun.dll and ecmzvfs.dll (Extension depends on the operating system). actlib contains all action routines for the GDM server commands. lib Contains general routines (bas _ basic routines, clg _ geometry routines, dba _ DB access, doc _ document handling, geo_ GDM server routines. The procedures are located in the BTL library (Binary Tcl LIBRARY). sql SQL instructions, in order to install a data base. base All Scripts with SQL instructions and data necessary for the GDM server. <name> Scripts with SQL instructions for application data. sachsen ... Saxonia GDM server with document types and factual data definitions. srvfiles application-specific data and scripts, arranged by the kind of the server: paths, license information, special procedures for the generation of the DokID. <name> Scripts and data for named application. sachsen ... all files for Saxonia GDM server. All red folders will be delivered, the blue are necessary for user development and server extensions. (29) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Server folder structure (2) Computer Grafik Systeme Geoinformationssysteme <server cont‘d> sgd Folder for GeoBase-file for georeferenced searching and logging information for all server actions. files Folder for all documents if storage in filesystem was chosen. The place for document storage can be defined in the configuration file. doc Server documentation if chosen by installation. clntscripts Folder for all programs, DLLs, scripts and further files needed by the GDM-Client for this installation. Can be structured in subfolders to run different serverinstances. During the login-process the clientscripts on the client side are compared with these files and downloaded to the client if missing or different. clntmodules If present this folder contains all modules, programs, scripts needed to update the GDM-Client-Installation. During the login-process the GDM-Client checks the neccessity for an update. Folders marked in red are distributed, all marked in grün are optional (30) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Configurationfile „...srv.dat“ (1) The folder <server> / srvfiles / <name> Computer Grafik Systeme Geoinformationssysteme contains the configurationfile with informations concerning the used DB, licenses, pathes to serverfiles, serverdocuments ... The file is a Tcl-script but with extension „.dat“. Usually thr application is included in the filename, e.g. „cityguidesrv.dat“. Most lines of this script define components of the array geoSetup: e.g. serverID, licenseOwner, licenseCode, licenseFeat. The component licenseFeat is a list of special features like LZWcompression, matchpointdialog, GeoTIFF-support, annotation etc. Depending on the license these features are listed here and so activate these functionalities in the GeoServer: set set set set set set set (31) geoSetup(serverID) geoSetup(applName) geoSetup(licenseOwner) geoSetup(licenseCode) geoSetup(licenseFeat) geoSetup(xfrReadBlockSize) geoSetup(xfrWriteBlockSize) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH 111 democd "Demo CD" AQ2PLSSYINC70S-CK%MF.WT "1 2 4 6" 8192 ;# Blocksize for fileread 4096 ;# Blocksize for filewrite M.O.S.S. GDM – Configurationfile „...srv.dat“ (2) Computer Grafik Systeme Geoinformationssysteme The component srvSources lists application specific Tcl-scripts loaded at start of the GeoServers. These procedures are present in mainmemory from beginning on. They are not loaded by the library mechanism. set geoSetup(file2ppGenerator) set geoSetup(dokidGenerator) set geoSetup(srvSources) dasy_marTif2PP dasy_marDokID "geomar mhdokdb mhgeschbuch„ The component dbConnect defines the databank to be used by the GeoServer. The server works in "autocommit"-mode. set geoSetup(dbConnect) {ODBC DemoCDDB} set geoSetup(dbDatetimeFmt) "%d-%b-%Y %H:%M:%S„ Further components define more filenames and pathes. set set set set (32) geoSetup(gdmPath) geoSetup(progPath) geoSetup(sgdFilename) geoSetup(filePath) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH [geoCheck4EnvVar WEGA_GDMPATH $pf 1] [file join $geoSetup(gdmPath) server] $geoSetup(gdmPath)/sgd/democd.sgd $geoSetup(gdmPath)/Files/democd M.O.S.S. GDM – Configurationfile „...srv.dat“ (3) Computer Grafik Systeme Geoinformationssysteme The licencing-codes are generated with this tool and the data are entered into the configuration-file by Cut& Paste. This tool can also be made available for clients owning a (state) licence. (33) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – DB- Structure (1) Computer Grafik Systeme Geoinformationssysteme • The GDM-Server is identified by ServerID. This must be unambiguous on the Net. • By ServerID the data-bank-tables are assigned to the server-instance. • All relevant tables are deposited in a dictionary. • On the GDM- server the access to the tables is made by tableID. • Server-intern tables have tableID 100. • Not only the tables names, but also the column names and types are deposited. • Easy assign of GDM- types to specific DB- data types. • Possibility of depositing extra information, for example column-names in the DB and in the form. • Possibility of defining a virtual table, it means, that the data are loaded or manipulated by procedure (access on extern DBs, calculated data…). (34) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – DB- Structure (2) Computer Grafik Systeme Geoinformationssysteme DoSysDataT tblWegaGDM SysServerID syServerGeo syTypTableID 98 DemoServer 11 ... tblID tblName ... Dictionary or tblWegaGDM, if only one GDM-Server 0 Dictionary or tblWegaGDM tblID tblName tblDescription 11 500 DoTypDefT HotelSD Tablewith document types Datatable Hotels ... Owner tblWegaColumns (35) HotelSD (project specific) Zip Code ... tblWegaTypes tabID colID colname colGDMType 500 500 1 2 Owner Zip Code 1 3 © Copyright by M.O.S.S. Computer Grafik Systeme GmbH ... GDMTypeID 1 3 GDMtypeName DBtypeName GDMSTRING GDMLONG CHAR INTEGER ... M.O.S.S. GDM – DB-Datatypes Computer Grafik Systeme Geoinformationssysteme Wega provides ist own GDM data-types, in order to be independent from the data-types of any database system. In database table tblWegaTypes the data types are deposited, as well as their conversion to the active used database system. Since the attribute-data-tables are described in the Dictionary, one can here also define “computed types”. While storing them into the database or selecting them from the database, they will be converted accordingly (e.g. the type GDMTRANSFORM). • GDMSTRING character string, possibly with certain length • GDMSHORT 6-Bit Integer • GDMLONG 32-Bit Integer • GDMFLOAT decimal number with simple accuracy • GDMDOUBLE decimal number with double accuracy • GDMBOOLEAN yes/no field • GDMDATE Date 1583...2199 as day-number • GDMDATETIME Date and Time as day-number + seconds • GDMENUM referenced entry in catalog • GDMTRANSFORM 5 digits/character transformed Date- and time-data are represented in an own GDM type. For that purpose there is the command ”ecmdoris datetime...”. The reference day is 1 January 1900 and can be also negative (before 1900). The data type corresponds to the MS-VB-Script and MS ACCESS data type Date time (double: integral part day; decimal part seconds of the day). (36) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Document-types Computer Grafik Systeme Geoinformationssysteme One can use different kinds of documents in GDM: Document types. Definition of the characteristics of the document-type in database table doTypDefT: • with attribute data (one table/view). • without attribute data (only the document itself). • with complex attribute data (data in several tables, data from external source/database,...). • only attribute data, but document. • whether write protected or not. • Kind of storage, in file system, in database-blobs, by external source of file... • Document with or without georeference. • Document can be a map, that can be merged to form larger maps (e.g. tiles of an ALK). These maps can be used again for spatial inquiries about documents. • Document is a folder, i.e. it refers to other documents in an hierarchical structure or may be registered into a folder. • Access restrictions on certain users. • Kind of the document identification, e.g. derived from attribute data or automatically generated. (37) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – attribute data per document-typ Access doTypDefT Computer Grafik Systeme Geoinformationssysteme TypID TypName sgdTyp ... 1 framemap layer_nn ... >0 tableID? tableID n id tableName =0 Descr Name (38) layer_nn img4711_56... tblWegaProcs procID procedureName Name ... Attribute-data table „Name“ col2 MapDokID id Documenttyp without attribute-data col1 sgdTyp <0 tblWegaGDM tableID SGD-file: attribute ... © Copyright by M.O.S.S. Computer Grafik Systeme GmbH Procedural attribute-data Tcl-Prozedur „Name“ übernimmt die Sachdatenverwaltung ... ... M.O.S.S. GDM – External/complex attribute data Computer Grafik Systeme Geoinformationssysteme Get/put attribute data for document type no Get/put data from/into DB (39) dokDBHandler ? yes Process attribute requirements from external source or compute complex attribute data. © Copyright by M.O.S.S. Computer Grafik Systeme GmbH Occasionally attribute data are needed that reside not in the same DB in which the GDM-Server holds its data, e.g. data of the ALB, that are stored on a different computer and/or database. Furthermore attribute data may be to complex to be provided by the standard functionality of the GDMServer. Whenever The GDM-Client or an application requests attribute data for a documenttype the presence of a so called dokDBHandler is checked. If present the request is forwarded to the handler that must than process the request. M.O.S.S. GDM – Install a new document type (1) • Specify the characteristics (conception), Computer Grafik Systeme Geoinformationssysteme • Register the characteristics in data base table doTypDefT (online), • Insert new line into SQL-script <server>/sql/<app>/dotypdeftvalues.tcl: set values { {1, 'Rahmenkarte', 'tif', ‚Raka aus DASY', 'rahmenkarte', ..., ... } {3, 'doc', 'doc', 'WinWord', 'Folie_1', 35954.52, ..., ... } { ..., ..., ..., ..., ..., ... } {id, 'doktypName', ‚'fmt', ‘description', 'Folie_n', 36449.3812963, ..., ... } } • If there are attribute data for the document, make entries in the database and corresponding SQL-scripts or write new SQL-scripts depending on the type of attribute data access, • Create attribute data tables „CREATE TABLE name ...“ <server>/sql/<app>/name.tcl, • Enter new line(s) and tableID in table tblWegaGDM, • Update SQL-script <server>/sql/<app>/tblwegagdmvalues.tcl accordingly: set values { {100, 'mhRissTable', 'Datentabelle Risse Mülheim a.d.Ruhr', '1', 36317 } {101, 'mhFlurModifTable','Tabelle der veränderten Flurstücke MH', '1', 36332 } { ..., ..., ... } { tabid, 'tableName', 'description', '1', date} } (40) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Install a new document type (2) Computer Grafik Systeme Geoinformationssysteme • Insert tableID and columID in table tblWegaColumns and provide column information, • Update SQL-script <server>/sql/<app>/tblwegacolumnsvalues.tcl and enter tableID and Columns, • Possibly enter tableID and procID in table tblWegaProcs and define procedure, • Update SQL-script <server>/sql/<app>/tblwegaprocsvalues.tcl: set values { { -1, 'mh_DokDBhandler', 'Für Datentyp RissDatei DB-Daten-Funktionalität'} { -n, 'myOwnDokDBhandler', ‚ 'description'} } • Write specified Tcl-procedure myOwnDokDBhandler, • Define a document identification (DokID): • that corresponds with the filename (default), e.g. in case of topographical maps, • that is generated occording to attribute data, e.g. geomar.tcl, • that is computed by an unique ID-generator. Accordingly procedures have to be written that will be loaded as additional sources (see component srvSources in the configuration file). (41) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Install a new document type (3) Computer Grafik Systeme Geoinformationssysteme • If the type of document is a map, then consider, how the geo information can be brought in: • on the basis the file name, procedure necessary, in order to derive geo info, • on the basis of passpoint generator procedure necessary, • on the basis of geo information in the file, e.g . GeoTIFF. • Specify access to documents: • Storage in file system; define path, search sequence, ... • in BLOBs of the data base, asynchronous transfer necessary, • by handler that regulates the access to external sources; write handler. • Possibly write and adapt VB-scripts for the GDM Client, so that e.g. the representation of hits or the export of documents of this type are treated separately. • Consider whether patch script is necessary for existing installation. (42) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Databank Initialization (1) Computer Grafik Systeme Geoinformationssysteme Before one can put on the data base, one must specify the document types: • Which attribute data are to be stored? table definition? • Which document formats are permitted TIFF, JPEG, BMP? • Are the documents georeferenced? • Can the georeference be derived, e.g. from DB information or from the document name (e.g. from map name)? • Are special features, e.g. annotations or attachments, permitted? • May the document be changed or deleted? • Is the document a map, so that it can be used for the selection of other documents? • How is the document stored? as a Blob, in a file system, as external source? (43) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Databank Initialization (2) Computer Grafik Systeme Geoinformationssysteme A data base for the GDM server is initialized by Tcl-scripts, which contain SQL instructions for creating and loading of DB tables. The scripts are deposited in the folder <server>/sql/... The file names are subject to a name convention: <tablename>.tcl Create the DB table(s), inclusive indices, <tablename>values.tcl Load the table with data. E.g. dotypdeft.tcl creates the table DoTypDefT (definition of the document types) using the „CREATE ...“-sqlCommand, whereas dotypdeftvalues.tcl contains the „INSERT ...“- sqlCommand to fill the table. In order to be able to use these scripts the script dbinitialize.tcl has to be run in a Tclshell. file.There particularly the sqlCommand is defined that is used in the scripts. One should pay special attention to the settings present in dbinitialize.tcl (kind of the DB, deletion of the tables (!!) or just initialization?). (44) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Databank Initialization (3) The initialization is done in 2 steps: Computer Grafik Systeme Geoinformationssysteme 1. First defining and loading the GDM base tables (these scripts are in the folder <server>/sql/base) 2. Than defining and loading own tables and extending the base tables by definitions of types of documents, by pre-defined users,... (these scripts are stored in the application-specific folder <server>/sql/<appl>) Thus producing tables and loading of values takes place as follows: % % % % % % tclsh source dbinitialize.tcl cd base or cd ../<appl> #depending upon table/data source dotypdeft.tcl source dotypdeftvalues.tcl sqlDisconnect Also the DB-initialization can be done automatically, so that the DB is accordingly set up. (45) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Server starting procedure Computer Grafik Systeme Geoinformationssysteme The starting procedure is done in phases, in some phases one can intervene by so-called phased servlets. Phases 0 Load the configuration file with server-specific data. 1 Initialize the internal caches, timers, create the log file, “learn” the command set (communication protocol). 2 Connect with the database, open and initialize the SGD. The Tcl packages EcmDoRIS, Ecmzvfs, ...will be loaded. 3 Load the server information (with the help of serverID from the srv-file), as well as the system tables, the document definitions and examine the license. 4 Install GDM server socket, disconnect the SGD, until a user logs in, and wait for client requests. (46) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Login Process GDM-Client GDM-Server connect CONR relVers protocolID sid LGIN userName password GDM Client examines whether the password has expired. CONN Rights + Password expire time Change Password CPWD DWPC new password expiration date GDM Client sends requests Main work loop GDM Server sends results, error messages GDM Client terminates the connection (47) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH Computer Grafik Systeme Geoinformationssysteme connect GDM server examines the connecting data and the number of active clients. If there are too many clients, it will reject. Otherwise it will assign a SessionID for that Client. accept GDM server examines user data, password and determines the user rights on the basis of the user identification and the computer address. Reset number of login trials. Update password in database Depending upon mode of the client connection instructions are permitted or change the mode. disconn GDM server closes the connection and stores all user-relevant data in database. M.O.S.S. GDM – DB-Access for user rights Computer Grafik Systeme Geoinformationssysteme User login with username doUsersT Username uid demo password aclID (coded password) aclID doUserRightsT UserID rightsID uid doAccListT Add/Remove Flag aclID 1 or –1 aclID rid Base right + FeatureID facilID doRightsEnumT rightsID rid doFeaturesT rightsName faclID Feature User rights facilID FeatureName doTrustedHosts IP-Address Mask User is on computer with IP-address (48) 192.9.200.0 © Copyright by M.O.S.S. Computer Grafik Systeme GmbH aclID/rightsID aclID processed user rights UserID M.O.S.S. GDM – Instruction loop in the GDM server Computer Grafik Systeme Geoinformationssysteme Wait for client requests Send error to GDM-Client decode Send answer to GDM-Client N Command known? N Command permitted? Process result Perform command QBOX .... ... ... QNXT ... ... ... Error? (49) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH J Privilege violations are counted. If too many force Logout. M.O.S.S. GDM – Processing of a client request (1) Computer Grafik Systeme Geoinformationssysteme 1. GDM Client demands documents for a certain range (rectangle). Search all documents in the range of P(lower left) ... P(upper right) GDM client request: QBOX {2561018 5698236 2561508 5698625} 2. GDM server carries out the selection with the help of the spatial access method: The result is a collection with the found hits. (50) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Processing of a client request (2) Computer Grafik Systeme Geoinformationssysteme 3. The GDM server communicates to the GDM Client, how many hits are found: XOBQ colhd2 2 Than waits for the next request. 4. The GDM Client begins to query the hits in sequence (Query NEXT one): QNXT colhd2 5. The GDM server examines, whether there are still hits in the collection, and supplies result TXNQ colhd2 150 {2561500 5698250 Rahmenkarte 6198.0 {... } 6198_0.tif} 6. The GDM Client analyzes answer: •Draws symbol in the overview map. •Updates the list of hits. (51) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Logging of the Actions Computer Grafik Systeme Geoinformationssysteme These actions are logged with the GDM client and with the GDM server: id=0x00b5 (12:02:45, 15255596): QueryBox: lu=(2561018.19, 5698236.77), ro=(2561508.85, 5698625.82), docTypes='', loadDate='', docDate='' id=0x00b5 (12:02:45, 15255626): <- QueryBox: strHandle='colhdl2', count=2 id=0x00b5 (12:02:45, 15255636): QueryNext: handle='colhdl2' id=0x00b5 (12:02:45, 15255696): <- QueryNext: isn=150, posX=2561500.00, posY=5698250.00, typ='Rahmenkarte', docId='6198.0', docName='6198_0.tif', 4 Koordinaten: (2561005.000 5698005.000) (2561995.000 5698005.000) (2561995.000 5698495.000) (2561005.000 5698495.000) id=0x00b5 (12:02:45, 15255726): QueryNext: handle='colhdl2' id=0x00b5 (12:02:45, 15255746): <- QueryNext: isn=151, posX=2561500.00, posY=5698750.00, typ='Rahmenkarte', docId='6198.9', docName='6198_9.tif', 4 Koordinaten: (2561005.000 5698505.000) (2561995.000 5698505.000) (2561995.000 5698995.000) (2561005.000 5698995.000) id=0x00b5 (12:02:45, 15255766): QueryNext: handle='colhdl2' id=0x00b5 (12:02:45, 15255766): <- QueryNext: isn=0, posX=0.00, posY=0.00, typ='', docId='', docName='', 0 Koordinaten: id=0x00b5 (12:02:45, 15255766): QueryDone: handle='colhdl2' id=0x00b5 (12:02:45, 15255776): <- QueryDone GDM-Client 00/02/28 00/02/28 00/02/28 00/02/28 00/02/28 00/02/28 00/02/28 (52) 11:06:27: 11:06:27: 11:06:27: 11:06:27: 11:06:34: 11:14:19: 11:14:19: Try to connect to sgdfile... Using SGD-File 'D:/Wega-GDM/sgd/mhpc.sgd' SGD-File delete coll and context Server goes to main loop 014:sock216:1: file read cancelled Incoming connect request sock200, 192.9.200.48, 1073 Server goes to main loop © Copyright by M.O.S.S. Computer Grafik Systeme GmbH GDM-Server M.O.S.S. GDM – Setting up a Document (1) GDM client announces to GDM server to set up a document. GDM Client received release and wants now to transfer the document block by block. GDM Client sends the data block by block. (53) ADOK KODA WDOK KODW iohandle Computer Grafik Systeme Geoinformationssysteme GDM server approves this: Checks whether type of document is permitted, whether document is available already, or at present locked, i.e. in use by another user. A temporary LOCK is put on this (not yet existing) document. Provisional entries in the database and the SGD-file and in doGeoXTableT. GDM server prepares the datentransfer: Determines, where the document has to be stored, (examination of free storage location), store file, supply an I/O-handle to refer to file and send to GDM Client. WRBL LBRW © Copyright by M.O.S.S. Computer Grafik Systeme GmbH GDM server receives data: Store the data in file. Every transfer is confirmed by the GDM server. M.O.S.S. GDM – Setting up a Document (2) Computer Grafik Systeme Geoinformationssysteme Store the DokID in the document cross reference table: DokID, DokTyp doGeoXTableT DokTypID ... ... Date ... pntCnt n pntID pid doGeoXPntsT Entry into SGD-file pntID No of point x y ... ... ... ... Enclosing polygon X, Y, ... Enclosing polygon with attributes DokID and, DokTyp doGeoHotSpT (54) pntID x y ... ... ... © Copyright by M.O.S.S. Computer Grafik Systeme GmbH Hotspot hotSpotX, hotSpotY In case document is georeferenced DokID M.O.S.S. GDM – States of a Document (1) Computer Grafik Systeme Geoinformationssysteme Document is set up into the system Document was removed from the system DokID in Cross Reference Table Document is locked to be changed LockMode Document entry into lock table All information about - setting up into the system, - changes, - removal is documented with respect to time and user. (55) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH LockMode 0 (locked) in editing 1 (done) document has been edited. 2 (reset) editing of the document was registered, but was reset by the administrator 3 (cancelled) editing of the document was registered, but the process was cancelled by the editor. M.O.S.S. GDM – States of a Document (2) Computer Grafik Systeme Geoinformationssysteme Document not locked, allow all release Released lock, document updated Lock conversion Lock reset by Admin Lock cancelled Restore document by BFIM, release lock (56) locked document, read only for other users reset cancel Exclusive lock, document locked for other users © Copyright by M.O.S.S. Computer Grafik Systeme GmbH All changing transactions are recorded in the database with the help of the lock mechanism. tblWegaLock DokID DokTyp LockMode LockID ... ... mode ... M.O.S.S. GDM – Adding Server Command • Time for roundtrip (Time between GDM-Client-Request and GDM-Server-Reply) has to be considered. Computer Grafik Systeme Geoinformationssysteme • If possible commands should be split up into smaller commands. • A command can consist of several subCommands , the commandname will be part of the protocoll, the subcommands are implemented and checked in the action routine. • Consideration of the mode of client connection (accept, update, connect, ...). • Definition of rights that are necessary. • Insert the command into actionDefineCommands: list of commands is extended by name, mode before and after the execution of the command and assigened action routine, that implements the command. Convention: Protokol command consists of 4 capital letters, if reply-name is necessary it consists of the reverse of the original command name, e.g. ANNO => ONNA • DB-relations: If the command needs new DB-relations, these have to be defined and have to be provided by SQL-scripts in <server>/sql/base. • actionNeuerBefehl: Provide the action routine in a separate Tcl-file, the file is stored in the folder <server>/actlib. The Tcl-index has to be updated. • geoClient.tcl: Define the command for the monitor for debugging • geocmds.doc: document the command in the list of server commands. (57) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Access to External Documents (1) • All Maps, that are administered in DASY (Digitales Auskunftssystem LvermGeo Rheinland-Pfalz) are to be used by WEGA-GDM . • Problem: • The maps are stored as QT- or CBIM-rasterdata in SGD-files. GDM cannot directly present SGD-files. • The files must be converted into TIFF (computing time-intensively). Due to the large data sets no double data storage of the maps is to be used for WEGA-GDM : • At access the files must be converted into TIFF and transferred to the GDM client. • In order to have an acceptble reply time an accordingly large cache is used at the GDM server. • • (58) Computer Grafik Systeme Geoinformationssysteme Problem: • Application programs run on the volume of data in each night, which to a large extent change nothing, but update the time stamp of the files, which can lead to unnecessary transfers. © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Access to External Documents (2) DASY-computer DASY Transfer Server SGD GDM-Server computer GDM-Client computer Computer Grafik Systeme Geoinformationssysteme docHandler RelayServer Cache Cache • GDM-Client demands document from GDM-Server • GDM-Server asks by docHandler the Relay-Server whether document is available and possibly in updated state. • Relay-Server requests the document-information from DASY-GDM TransferServer: • Examines whether document is still up-to-date in the cache. • If available and current, then announce its release to the GDM server. • Otherwise transfer the SGD file to the server and convert it into TIFF or convert the SGD file into TIFF on the DASY computer and transfer the finished tiff file (depending upon settings in control file). •Place transferred TIFF-file into the cache and announce release to the GDM-Server. (59) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (1) Computer Grafik Systeme Geoinformationssysteme With the help of a demo server the necessary steps to install a new GDM-server will be shown. 1. Copy an existing server - new server name: GDMServerKarlsruhe - provide folders <server>/srvfiles/karlsruhe <server>/sql/karlsuhe - copy and adapt the files of existing folders 2. Locate area: Karlsruhe - locate the lower left and upper right corner coordinates - these values are inserted into the component mcWorldBox in the srv-file karlsruhesrv.dat. 3. Provide SGD-Geobase - with the help of the EASI-Skripts newsgdfile.tcl a “blank“ SGD-file can be provided: easish source newsgd.tcl newSGD karlsruhe.sgd "3439400 5419500“ "3475000 5442500“ file copy karlsruhe.sgd karlsruhe.org ;# Org-Datei für Recovery - the new SGD file contains the pass points and the attribute table for GDM (60) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (2) Computer Grafik Systeme Geoinformationssysteme 4. Provide overview map(s) -there are different possible raster formats, e.g. karlsruhe.jpg. -the map must be georeferenced, e.g. GeoTIFF, or this information is deposited in the srv-file in the components „overviewFile“ and „overviewFile, worldBox“. -both methods can be used simultanuously (World file: karlsruhe.jpgw) and entry in srv-file. 5. „Complete“ the srv-file by editing components. -allocation of a unique server ID (unique within the data base): serverID=98 -inscribe licensed data in the components licenseOwner, licenseCode and licenseFeat (the licensed data can be acquired by using the licensing tool). 6. Connect data base: -determine which data base is used (ODBC, Oracle, Informix, ...). That defines the DB-package the server must load. -depending on „blank“ attach Datenbank/DB-Space/etc. define DB. -depending on DB, determine DB Access e.g.. ODBC-Datasource or DB-Servername. -GDM-Server should obtain the appropriate rights of access (attach and delete tables, attach data records, modify and delete...) -accommodate the Tcl-script dbinitialize.tcl, so that the data base immediately takes one access here: accommodate variable dbName. (61) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (3) Computer Grafik Systeme Geoinformationssysteme cont 6. Connect data base - for example: ODBC-Connection MyDemoDB with ACCESS database karlsruhe.mdb. Dialogue accessible by: Start/Einstellungen/Systemsteuerung Auswählen ‚Datenquellen (ODBC)‘ Add new database with MS-Access-driver (if server is to run as service, then select system-DSN, else user-DSN is sufficient). Add System-DSN „MyDemoDB“ In ODBC-Setup, create or select data base respectively. -now test the connection to the new created database with the help of the Tcl-script dbinitialize.tcl in the folder <server>/sql tclsh source dbinitialize.tcl # there should be no errors (62) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (4) cont 6. Connect data base Computer Grafik Systeme Geoinformationssysteme - if everythings is ok, then we we can provide and fill the specific GDM tables. tclsh source dbinitialize.tcl cd <server>/sql/base createDB 1 ;# 1 table produced ,0 tables only be filled sqlDisconnect exit - now information appears, which tables were provided and loaded with which data. Here it is to be examined whether all tables and data were correctly loaded. - all load actions should be confirmed before end of the program by sqlDisconnect. 7. Server specific SQL-Data - the Tcl script „dosysdatatvalues.tcl“ must be available and possibly editef in the folder <server>/sql/karlsruhe - enter serverID and server identification in this file and load as with 6. set values { { 98, 1, 1, 1, 'GDM-Server Karlsruhe', 'karlsruhe', 0, 0, 0, 0, 11, '1' } } .... tclsh source dbinitialize.tcl cd <server>/sql/karlsruhe> source dosysdatatvalues.tcl ;# load ServerID (63) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (5) Computer Grafik Systeme Geoinformationssysteme 8. Register all path data and DB-connections in the srv file - Check the components serverID and overviewFile - in dbConnect the data base connection is deposited, dbConnect ={ODBC MyDemoDB} - than the paths are to be specified to the files applPath: defines, where the SGD file and log file can be found filePath: defines, where the setup files are to be stored (only if storage in the file system) - Define file names for SGD file, log file and SGD master file (= org file) logFile, sgdFilename and sgdMasterFile - Define possibly necessary specific Tcl Scripts and procedures srvSources: the indicated sources are loaded immediately (in phase 1) file2ppGenerator: Procedure name to compute passpoints from file names dokidGenerator: Procedure name for generating a unique document identification (DokID) on the basis of attribute data and type of document. (64) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (6) 9. Under Unix adapt the Enable-script „wega.enb_ksh“ Computer Grafik Systeme Geoinformationssysteme - actually everything is defined in the srv file, however some DB systems need still environment variables:, this can one either Either they can be defined in a special script. Then however this shell script must be run before the Enable-script, e.g. in the login script ”.profile” Or they can be defined in the subfolder “setup” of the server in the shell script ”wega.enb_ksh” (caution: can be overwritten by updates) - specify, which srv file is to be used, see WEGA_GDMPARAMFILE 10. Test wether the server starts - activate terminal with appropriate environment, e.g. WEGAshell or RoSyShell under Win32, or new login under Unix. - change to the server folder and start the server cd <server> GDMServerKarlsruhe start srvfiles/karlsruhe/karlsruhesrv.dat StartServer –console Win32 Unix - thereafter the server should be shut down again (65) GDMServerKarlsruhe stop srvfiles/karlsruhe/karlsruhesrv.dat ^Z Win32 Unix ShutServer Unix © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (7) Computer Grafik Systeme Geoinformationssysteme 11. Analyzing the output at the terminal and the protocol files - Check whether the access to database is correct and whether the correct server was activated. - Check the license. - Check which features are activated: 02/09/18 02/09/18 02/09/18 02/09/18 02/09/18 02/09/18 02/09/18 02/09/18 02/09/18 02/09/18 13:18:51 Logging of process 217 started: WEGA-GDM GeoServer 13:18:51: (setup) Starting geoserver in directory 'D:/Wega-GDM/server‚ 13:18:51: (setup) Try to connect to database... 13:18:51: (setup) ODBC connect to MyDemoDB 13:18:51: (setup) Try to connect to sgdfile... 13:18:52: (criti) Server ID 98 Name 'GDM-Server Karlsruhe' with (1, 1) 13:18:52: (setup) Checking License... 13:18:52: (setup) License yields 20 max. Clients 13:18:52: (setup) Activated Features GeoTIFF,CPTdialog,Attachments 13:18:52: (setup) Setup listener socket... 12. Testing the GDM-Client connection - As soon as all seems to be o.k then start the server again - Activate the GDM-Client and connect to the server Examine the connection: computername and portnumber Check if the overview map appears and the coordinates are plausible (66) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (8) Computer Grafik Systeme Geoinformationssysteme 13. Set up the clientscripts and the OCX-based forms - Get scripts and OCX files from an existing environment e.g. <server>/clntscripte/democd <server>/clntscripte/karlsruhe remove all unneccessary files and in particular adapt the following VBS-files: + init.vbs + login.vbs + add_hit.vbs - First you should begin with the dynamic files GDM_DYNAMISCH.ocx 14. Check the path of the clientscripts in the srv-file - entry “lappend geoSetup(clientScriptsPath) [file ..../karlsruhe]“ 15. Start server again and establish the connection to the client again - The clientscripts should be transferred to the client during login - Examine whether OCX registers correctly by activating the searchmask (67) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (9) Computer Grafik Systeme Geoinformationssysteme 16. Obtaining server as service in Win32 - Copy file GDMServerDemo.exe to GDMServerKarlsruhe.exe - Copy file GDMServerDemo.ini to GDMServerKarlsruhe.ini - Adapt parameters in the sektions of the ini-file (SeviceName and paths): [System] ServiceName=GDM-Server-Karlsruhe .... [GDMStart] Params=start srvFiles\karlsruhesrv.dat ... [GDMStop] Params=start srvFiles\karlsruhesrv.dat - then install service and start start WEGAshell or RoSyShell cd <path where GDMServerKarlsruhe.exe is stored> GDMServerKarlsruhe.exe –i ; install service GDMServerKarlsruhe.exe –s ; start service (68) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH M.O.S.S. GDM – Installing a New Server (10) Computer Grafik Systeme Geoinformationssysteme 17. Check and verify active state of service - At the control panel of the dialog „Services“ activate and verify the existence of service in the list. 18. Setting up more document types - With the help of the TCL-script „dotypdeftvalues.tcl“expand the SQL-tables „dotypdeft“ - Define corresponding attribute data tables and save them as Tcl-scripts <tablename>.tcl in folder <server>/sql/karlsruhe, e.g. hotelsd.tcl - If attribute data exist, then dictionary-tables tblWegaGDM and tblWegaColumns are extended and saved in the corresponding TCl-scripts. - Then it is required to restart the server! 19. „Enlarging“ the Server and Client further - Applications, forms, EASI/Tcl-procedures (69) © Copyright by M.O.S.S. Computer Grafik Systeme GmbH