Search Engine for Online Physiologic Databases by Jack V. Chung Submitted to the Department of Electrical Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degrees of Bachelor of Science in Computer Science and Engineering and Master of Engineering in Electrical Engineering and Computer Science at the Massachusetts Institute of Technology K December 14, 2000 Ebce 2ocAi jOF Copyright 2000 MIT. All rights reserved. BARKER MASSACHUSETTS INSTITUTE TECHNOLOGY JUL 1 1 2001 LIBRARIES Author DepF tment of ElecricaZ Engineering and Computer Science December 19, 2000 Certified by ',.Roger G. Mark Thosis Supervisor Accepted by_ Arthur C. Smith Chairman, Department Conmittee on Graduate Thesis Search Engine for Online Physiologic Databases by Jack V. Chung Submitted to the Department of Electrical Engineering and Computer Science December 14, 2000 In Partial Fulfillment of the Requirements for the Degrees of Bachelor of Science in Computer Science and Engineering and Master of Engineering in Electrical Engineering and Computer Science at the Massachusetts Institute of Technology ABSTRACT PhysioNet is an online collection of free physiologic databases and signal processing software. The search engine for these physiological databases is built on the following web system architecture: Linux operation system, Apache server with modAOLServer, and Postgres relational database. The administrator has the capability to index physiologic databases and records using XML files generated by PERL scripts. Users of the PhysioNet web site can search for physiologic databases and records that have been indexed. Thesis Supervisor: Roger G. Mark Title: Distinguished Professor In Health Science & Technology 2 1. Introduction PhysioNet is an online collection of free physiologic databases and tools to analyze these databases. Currently, this collection includes databases of multi-parameter cardiopulmonary, neural, and other biomedical signals from both healthy subjects and subjects with a variety of conditions that have major public health implications including sudden cardiac death, congestive heart failure, epilepsy, gait disorders, sleep apnea, and aging. Thousands of researchers in the biomedical community world-side visit http://www.physionet.org to search for patient records that are appropriate for their studies. In addition to providing the information free for download, PhysioNet is a central location for researchers to share their physiologic records with the community by submitting databases from their research. A physiologic database is an archive of well-characterized digital recordings of physiologic signals and related data indicating the patient's demographics and medical conditions. The more than forty gigabytes of physiologic databases presently archived on PhysioNet illustrate a variety of physiologic phenomena from patients with a vast range of diseases, health conditions, ages, and ethnicities. For a majority of the databases, the records are formatted in the following manner: * A Header file stores the patient's demographic, medical conditions, and medications. " An Annotation file provides labels for significant features or events in the data (ECG beat labels, rhythm changes, sleep stages in the EEG, ST-T changes in ECG, etc.). 3 * A Signal file is a binary file containing one or more digitized signals, which can be displayed by the tools provided by PhysioNet. 2. Problem Statement Currently in PhysioNet, there are no search engines for the forty gigabytes worth of records in the physiologic databases. A search engine accepts one or more search criteria from the user, usually through a web form, and looks through all possible records to find the ones that fit the criteria. Without a search engine, a user needs to read the description or abstract of each database to determine which can potentially apply to his study. Then depending on his search criteria, he may need to examine the header file of each individual record in the database for the subject's demographic information and statistical summaries, and then one or more annotation files to see if specific events (e.g., types of heart beats, cardiac rhythms, or respiration patterns) occur. The header files and the annotation files are not intended to be read directly by human readers. To extract the information in a format that can be easily understood, the user needs to run various PhysioNet programs. After selecting suitable records, he might use other PhysioNet tools to analyze the signal files. A possible search could be like this: "Find records of all male patients between the ages of 40 and 50 with atrial fibrillation and using atenolol (medication)". The process may begin by looking at the descriptions of each database to see if any of them contain data that fit the search criteria. With the list of suitable databases, the user then needs to examine all of the records' header files for each of the databases. This process can easily 4 take a minimum of 10 minutes for experienced users and more for new visitors. Since the size of the PhysioNet archives will continue to grow as more researchers contribute their data, the time needed for a manual search will grow at the same rate. Therefore it is obvious that a search engine is needed. 3. Design Criteria This section lays out the requirements for my Master of Engineering Thesis. The major issues that need to be considered or solved include the following: 3.1 Relational Database A relational database is a fundamental requirement in order to perform many of the possible searches. Examples of relational databases include: Oracle, Microsoft Access, MySQL, and Postgres. (Details of the capabilities of a relational database will be discussed later.) Once the information for each record is stored, a standard Structured Query Language (SQL) query is used to select the appropriate records from the relational database. An SQL statement for the search criteria mentioned in the Problem Statement looks like this: select from records where gender = 'm' and age between 40 and 50 and condition = 'Atrial Fibrillation' and medication = 'Atenolol'; 5 3.2 Automatic Indexing of Records Currently there are on the order of a thousand records in PhysioNet, which are stored as flat files in the file system. In order to use a search engine, information about each of these records needs to be indexed in a relational database. Having a secretary manually index these records one by one is a waste of resources. New physiologic databases are also continuously added to the PhysioNet archive. In addition to uploading the flat files for the new records, these records also need to be indexed in the relational database. As a result, more manpower is necessary to index these records. Human errors, such as accidentally forgetting to index a few records, are unavoidable. A protocol needs to be developed to assist in the automatic indexing of records in the relational database. The protocol requires the contributor of a new physiologic database to submit an Extensible Markup Language (XML) file containing the list of records and the information to be indexed for each record, along with the flat files. Once PhysioNet retrieves this XML file, it is parsed and the records are inserted into the database. More details regarding the automatic generation of the XML file and the protocol used to upload the XML files to a relational database will be mentioned later. 3.3 Open Source Software The software used for this project needs to be open source, which means the source code for the program is released for free. The main reason to go open source is because it is free. If a user wants to mirror the site, we can simply give him the required software without having to deal with any licensing issue. Another advantage of using open source software is that interested users (not necessarily limited to the authors of the software) 6 can examine the source code for errors, correct them, and even modify the software to suit their needs better. 3.4 Search Engine Easily Extensible The scope of my thesis is developing the protocol to store the information located in the header and annotation files and providing a search engine for this information. My search engine needs to be easily extended to allow for other searches, which may be applied to the signal files. 4. Implementation Details The implementation for the search engine is discussed in the following manner. First, I begin with the description of the web system architecture. Then I describe the features of a relational database along with how the information from a set of physiologic records is stored in the database. Next, I explain in detail the protocol used to update the relational database automatically with this information. Finally, the last section describes the design of a friendly user interface for searching. 4.1 Web System Architecture PhysioNet currently runs under the Linux operating system and the Apache web server. (As of December 2000, PhysioNet used RedHat Linux 6.2 and Apache 1.3.14.) A web server, such as Apache, handles Hyper-Text Transfer Protocol (HTTP) requests. When you enter a URL into a web browser (e.g. Netscape or Internet Explorer), you are essentially performing an HTTP request to retrieve the web page. 7 Another option for the web server is AOLServer, with which I am more comfortable. With AOLServer, you have the ability to generate quick web pages using TCL and ADP. (Generating web pages using TCL and ADP is described in section 4.3.) Interacting with a relational database is simple in AOLServer. We cannot simply abandon Apache since part of the current web site requires Apache. Originally I thought that only one web server could be installed on a single computer, but as it turns out, there is an Apache module called modAOLServer, which can emulate AOLSever under Apache. The final decision was to install modAOLServer. All of the major relational database products run under Linux, including commercial products such as Sybase, Oracle, and Informix, as well as open source databases such as MySQL, mSQL, and Postgres. We are restricted to use one of the open source databases due to one of the requirements mentioned in section 3.3. During the time of development, out of the three open source databases listed above, AOLServer supported only Postgres. For this reason, Postgres was chosen. For instructions on installing the necessary components of the web system architecture, please refer to Appendix A. Figure 1 outlines the web system architecture by showing the interactions between the different software components. 8 Client's Browser (Netscape or Internet Explorer) Relational Database Postgres Web Server - Apache with modAOLServer I Operating System RedHat Linux Figure 1: Diagram of the Web System Architecture 4.2 Relational Databases Here is an example that illustrates how a relational database is used. John Doe runs a business with a thousand employees. As a good manager, he wants to have a way of organizing the employees' information. To keep the example simple, we are only concerned about an employee's department and contact information. First we create a table to store the information for each department. The columns of the department's table are: Department ID, Name, and Description. create table departments ( deptid integer primary key, name varchar(100), description varchar(1000) 9 This create statement specifies the Department ID as an integer, Name as a string that can contain up to 100 characters, and Description as a string that can contain up to 1000 characters. The phrase "primary key" will be explained later. create table employees empid ( lastname firstname phone integer primary key, varchar(100), varchar(1000) varchar(10), deptid integer references departments The employees table is created in a similar manner. The main difference is that the department is a reference to another table and not the full name of the department. When referencing the departments table, you store the "primary key" of that row. This is shown with the following statements. insert into departments (deptid, name, description) values (1, 'Payroll', 'handles paying the employees'); insert into departments (deptid, name, description) values (2 'IT', 'handles computer equipment'); insert into employees (empid, last-name, firstname, phone, deptid) values (1, 'Jones', 'Susan', '617-222-3333', 2); The first two statements add two departments to John's company. The last statement places Susan Jones in the department reference by dept-id of 2, which is the IT department. If we decide to change the name of the IT department to "Information Technology", we do this: update departments set name = 'Information Technology' where dept-id = 2; 10 The statement above updated the name of the department. We do not need to update Susan's information since we only store the Department ID. The beauty of referencing a table with primary keys is that changes to the main table (departments in this case) will propagate to the other tables automatically. Had we stored the name of the department in the employees table, we would have had to update all the records that contained "IT" and change them to "Information Technology". After a year, John wants to store the employees' phone numbers in addition to the work numbers. We can easily achieve this by altering the employees table: alter table employees add home phone varchar(10); This approach is not scalable if we decide to store cell phone numbers, pager numbers, and fax numbers. Another approach is to have a "mapping" table, so-named because the "employee and phone" relationship is a one to many mapping. First, let us go back and assume that employees table does not contain the phone number field. Instead, this table is created: create table phone-numbers empid integer references departments primary phone number type varchar (10), varchar(100) key, insert into phonenumbers (empid, phonenumber, type) values (1, '617-222-3333', 'work'); insert into phonenumbers (empid, phonenumber, type) values (1, '617-222-4444', 'home'); 11 These two insert statements store the two phone numbers for Susan Jones. If Susan gets married to Bob Williams, we will update the last name for Susan in the employees table and leave the phone-numbers table alone since the phonenumbers table uses the employee ID as a reference. The definition of these tables is called a "data model". In order to have the least trouble in maintaining the relational database, we need to capture the information accurately in the data model. One benefit of using a relational database is that the information is stored in an organized fashion. The other benefit is that the "referential integrity" of the database is preserved during inserts, updates, and deletes. This means that when a column is specified as a "primary key", there is only one occurrence of this number in the table. Also, when a row in table A refers to a row ID in table B, that ID must exist in table B. Using the example shown earlier, when a row is inserted into the phonenumbers table, this row refers to the employees table by specifying the emp-id of 1. Before this row is inserted, the relational database checks if the empid of i is a valid row in the employees table. Before this row (empid of 1) of the employees table is deleted, the relational database checks to make sure that this row is not referenced by other tables. Since the information is stored in an organized fashion, querying from the database should be simple. The language for querying is called Structured Query Language (SQL), which reads like English. Here are a few examples: select phone number, type from phone-numbers where emp-id = 1; 12 select from phone_numbers, type from phone-numbers, employees where phone number.empid = employees.emp_id and employees.lastname = 'Jones'; The first query assumes that you know the employee ID. This query returns a list of phone numbers and the type. The second query is a little more complicated. It tries to select from two tables and to join them by the employee ID. Finally the query filters the phone numbers of all the employees with last name "Jones". The data model for PhysioNet is much more complicated than the Employee scenario. A complete data model is provided in Appendix B. Pieces of the data model are discussed later in the appropriate sections of the Implementation Design. 4.3 HTML and TCL 4.3.1 HTML Web Pages A majority of Web pages are formatted using a language called Hyper-Text Markup Language (HTML). When you request a web page from your Netscape or Internet Explorer browser, the server will send an HTML page to your browser. The HTML page consists of text surrounded by tags. Here is a sample of an HTML page: <html> <header> <title>My Homepage</title> </header> <body> <hl>Welcome to My Homepage</hl> <font color=red>Hello everyone</font> <P> Hope you enjoy my web page. </body> </html> 13 An HTML page begins with the opening tag </html>. <html> and closes with an ending tag The header section, which is surrounded by the <header> and </header> tags, contains some information about the page, such as the title. The body of the HTML is what the user sees in the browser window. Text between the <hi> and </hl> tags makes the phrase big and bold. A <p> tag tells the browser to place a paragraph break. HTML programming is static. Whenever you come to this page, you will see the same content. As you can see, plain HTML is not sufficient for the purpose of displaying dynamic content. If I want to have web pages that show the description for each of the records, I need to type up an HTML file for each record. Then if I decide to add another piece of record information, I need to go through each of these thousand files and edit them. When there is a new record, I need to type up a new HTML file for this record. Another example is generating web pages to display weather reports. If I want to see a web page with the latest weather report, I do not want the report as of when the HTML file was last updated. 4.3.2 Dynamic Web Pages using TCL An alternative style of generating web pages is to make them dynamic. The content that appears on the page is determined by the "parameters" that are appended to the URL. Here are two URLs from PhysioNet Search: http://physionet.org/search/physiobank/record-view.tcI?record-db-id=83 1 and http://physionet.org/search/physiobank/record- view. tcl?record-db-id=837 There is - only one file, record-view.tcl, for these dynamic web pages, but these URLs also include a parameter, record db id (the database ID of the record), and its value. 14 By passing in different IDs, you are shown different records. This approach is scalable since you only need to maintain one file even though you may add thousands more records in the future. You may wonder why the file, record-view.tcl, ends with a .tcl extension rather than the normal .html extension. Before I continue describing dynamic programming in detail, I need to explain TCL. TCL is a scripting language that helps in parsing and manipulating text strings. Since HTML is essentially text strings surrounded by tags, TCL is great for generating HTML code. Here is an example of a TCL file called greetings. tci: ad_page variables { name time } if { $time == "morning" } { set greetings "Good morning $name" } elseif ( $time == "night" } { set greetings "Good evening $name" } else ( set greetings "Hello $name" } nsreturn 200 text/html $greetings If I type in this URL, http://my.server.com/greetings.tcl?name=John&time=night, this will produce a web page that says "Good evening John". The procedure ad-page.variable extracts the two variables, name and time, from the URL. $time is the value of the variable time and since the time is night, it sets the variable greetings to "Good evening John". ns return returns a normal code of 200, which basically means 15 that this page is fine and does not contain any error. The format (MIME type) of this file is text/html (i.e., an HTML page) and the content of this file is the greeting message ($greetings). Other file formats may be plain text, images, or PDF files. As you can see with the simple example, this script can generate an infinite number of web pages since one can pass different names and times within the URL. For more information about the TCL language, please refer to the documentation from Scriptics, the creator of TCL: http://scriptics.coml. 4.3.3 Interacting with the Database On the PhysioNet web server, indices of the physiologic records are stored in the Postgres relational database, which can be accessed using modAOLServer. TCL needs a way of asking modAOLServer to retrieve the data from the relational database. Here is another file, employee. tcl, which illustrates how this is done. ad page variables { empid } set sqlquery "select e.lastname, e.firstname, d.name as dept name from employees e, departments d where e.deptid = d.deptid and e.empid = $emp id # get a database handle to connect to the database set db [nsdb gethandle] #perform the SQL query to retrieve a row from the database set selection [ns-db 1row $db $sqlquery] #set the TCL variables for each column, where the name of the # variable is the column name and the value of the variable is # the value of that column setvariablesafter_query #return the HTML to the user nsreturn 200 text/html "$firstname $last-name works $dept name " 16 for When the URL, http://my.server.com/employee.tcl?emp id=1, is queried, "Susan Jones works for IT" is returned. The process of retrieving this data from the database occurs in the following few steps. From section 4.2, "Relational Databases", the data for employee ID I was inserted to the database. First the emp_id is extracted from the URL and incorporated into the SQL query. Then a database handle is used to connect to the Postgres Database. Next Postgres processes the query and sets the TCL variables to the value of the column of the query. (e.g. TCL variable lastname is set to Jones.) Finally the TCL script returns the information to the browser in HTML format. We can make this page more complex by retrieving Susan's phone numbers and displaying them in a similar fashion. Following is a revised version of the script to display Susan's phone numbers. adpage variables { empid } set sqlquery "select e.lastname, e.firstname, d.name as dept name from employees e, departments d where e.deptid = d.deptid and e.emp id = $empid # get a database handle to connect to the database set db [ns_db gethandle] #perform the SQL query to retrieve a row from the database set selection [nsdb irow $db $sqlquery] #set the TCL variables for each column, where the name of the # variable is the column name and the value of the variable is the value of that column # setvariablesafter_query #set the page_content variable to hold the current content of this web page # set pagecontent "$firstname $last-name works for $dept name" 17 set phone _sqlquery "select phonenumber, type from phonenumbers where empid = 1 #perform the phonesqlquery to retrieve all the phone numbers set selection [nsdb select $db $phone sqlquery] #loop through each row that is returned from the database while ( [nsdb getrow $selection] } { setvariables_afterquery #append the phone number to the content of this web page append pagecontent "<p>$type: $phone-number" } #return the content of this web page to the user nsreturn 200 text/html $page_content After the first query is processed, the page content contains "Susan Jones works for IT". The second query may return 0, 1, or many rows. For each row that is returned by the query, the script adds the phone number and type to the page content. 4.3.4 TCL and ADP In Web site design, there tend to be two classes of people. Programmers make up the first class and their job is to write up SQL queries and scripts. The other class consists of graphic designers; they make the site beautiful by adding graphics and laying out the information nicely in the web page. When I implemented the site, I took this into consideration. Associated with every web page are two files, one TCL and one ADP. The TCL file is similar to the examples from above. Instead of using ns_return to return the file to the user, the ad returntemplate procedure is used. Using the "greetings" example, the user still 18 types in the same URL, http://my.server.com/greetings.tcl?name=John&tme=night. The modified version of greetings. tcl is shown below. ad-page variables ( name time } if { $time == "morning" } set greetings } elseif ( "Good morning" ( $time == "night" } { set greetings "Good evening" } else { set greetings "Hello" } adreturn template ad returntemplate uses the template file greetings. adp to generate the HTML and returns the HTML to the user. Following is the code for greetings. adp: <html> <header> <title>Greetings</title> </header> <body> <hl><=% $greetings => <%= $name %></hl> I hope that you will enjoy my web page. </body> </html> ADP is an acronym for AOLServer Dynamic Pages. From the looks of the ADP file, it resembles HTML. Having ADP files resemble HTML files is one of the benefits of using two files to deliver one web page. The scripting and queries will not change much through the life of a web site, but the look and feel of the pages will. Graphic designers are a lot more comfortable working with files that look similar to HTML than editing TCL files. 19 To insert "dynamic" content to the ADP pages, you simply wrap the variable with an opening "<=%" and closing "=>" tags. Another powerful feature of ADP is creating new tags that are not included in the HTML language. Since HTML files begin with the same <html> and <header> tags, a new tag can be created to abstract the header section of the HTML. Also, most websites include the email address of the webmaster on the bottom of every page. It is annoying to type this in every single ADP file. Later when you need to change the email address, it will be frustrating to go back and change all the files. So another tag can be created to abstract the footer section of the HTML page. Before the tags can be used, they must be created. Here are two procedures that demonstrates how the header and footer tags are created for PhysioNet, but they have been modified slightly for the purpose of keeping the code short: ad register-styletag pn-header "" { set title [uplevel [list subst [ns_set iget $tagset title]]] return " <html> <head> <title>$title</title> </head> <body> } adregister styletag pnfooter return "" ( " <hr> <font size=-1><p> Please e-mail your comments and suggestions to <a href=\ "mailto:webmaster@physionet .org\"> <tt>webmaster@physionet.org</tt></a>, or post them to: <P> <i><address> PhysioNet<br> MIT Room E25-505A<br> 77 Massachusetts Avenue<br> Cambridge, MA 02139 USA<br></address></i> </font> 20 </body> </html> } Now I can modify the code for greetings.adp to use the new header and footer. <pn header title="Greetings"></pn header> <body> <h1><=% $greetings => <%= $name %></hl> I hope that you will enjoy my web page. </body> <pn-footer></pnfooter> The pn header tag adds the <html>, <header>, and <body> tags along with the title of this web page. The footer closes with the </body> and </header> tags and adds the email address and physical address of the administrator of the web site. When the address changes, only the pn-f ooter procedure needs to be edited. (pn is short for PhysioNet.) By using ADP tags, you reduce the amount of HTML code that needs to be written. Figure 2 describes the interaction between a user's browser and the web server. The browser first sends an HTTP request for a web page. When the web server retrieves this request, it will use TCL and ADP to generate and deliver the HTML web page to the user. When scripting the page using TCL, the web server may interact with the relational database to provide the dynamic content for the web page. 21 Client's Browser (Netscape or Internet Explorer) iL web page request TCL/ADP to generate HTML Relational Database Postgres Web Server - Apache with modAOLServer Operating System RedHat Linux Figure 2: Requesting and Delivering Web Pages 4.4 Administrator Interface Users with "administrator" privileges (verified by a login procedure) can describe a physiologic database and index records for each of the physiologic databases. From this point forward, "database" can mean two things. When I refer to physiologic databases, I mean collections of physiologic records. Relational database refers to the collection of tables in Postgres that will store these physiologic databases. Details of how each administrative page is implemented will also be mentioned in this section. 4.4.1 Describing a Physiologic Database In the data model (found in Appendix B) only these columns are explicitly defined for the pndatabases table: name, URL for the database, and abstract of the database. One of 22 the goals was to make the description of the database very flexible. In order to achieve this, we need a minimum number of predefined columns and a mechanism to add more columns in the future. The two tables, pn sections and pn-fields, help in describing the database dynamically. pn-f ields stores additional colunm definitions such as the number of records in the database, age range of the subjects in this database, type of subject, and health condition of the subjects. Each of these three columns can be identified as a number, a range of numbers, radio boxes, or check boxes. The number of records in the database is thus identified as a number. The age range is a range-typed field, with the possible units of years, months, and days. The subject type is defined using radio boxes. (Radio boxes allow only one selection, while check boxes allow multiple selections simultaneously.) The choices for the subject type are currently "human", "animal", "cell", and "molecule". The field for health conditions is an example of the last main choice of field type, check boxes. Since I want to keep the description of the physiologic databases ordered, I map fields to sections. For example, the subject section contains the above list of fields. I can also define a medication section, which may list the drugs. Associated with each section and field are sort orders. If a low sort order number is specified, this item will appear at the beginning of the physiologic database entry form. If the subject section has the lowest sort order, it will appear at the beginning of the entry form. Within the subject section, the field with the lowest sort order will appear first in the subject section. Please refer to Appendix B for the data model of the pn-f ields and pn sections tables. The source 23 code of the web pages that allow an administrator to create and edit the sections and fields can be found in Appendix E. After a good number of fields and sections are defined, an administrator can begin adding physiologic database descriptions to the relational database. The web form to add the physiologic databases, located at http://physionet.org/search/physiobank/admin/databaseadd.tcl, is dynamically generated from the pn_sections and pn_fields tables. (Figure 3 is a screenshot of the web form to add a physiologic database. For the source code, please refer to Appendix E, Part 5.) Thus, whenever these two tables are updated, the web form will reflect the changes. There is another web form for the administrator to edit the physiologic database description. This web form, located at http://physionet.org/search/physiobank/admin/database-edit.tcl, looks similar to Figure 3, but is pre-populated with the information that was uploaded earlier so the administrator does not need to enter the information again. 24 Add a New Database You are here: Home > Physiobank Search > Admin > Add a Database Please enter the following info Name required URL required r Subect Type oceli r human - animal r Disease F Healthy 7 CHF F AID r Subjects molecule Age Range r Activities r meditation r walking F marathon F resting r swlmming r Number of Records in Database Record r Annotations F tOeats F Abstract - C html Lenath of Record Signal 0 quality text w :Dosw 'Dom Figure 3: Screenshot of Web Form to Describe a PhysiologicalDatabase. This form will eventually be located at http://physionet.org/search/physiobank/admin/database-add.tcl. (The source code for this form can be found in Appendix E, part 5.) 4.4.2 Indexing Records in the Relational Database After adding a physiologic database to the relational database through the web interface, an administrator can then begin indexing the physiologic records. One of the design requirements was to make this indexing process simple and automatic. We do not want to have a human being sitting next to a computer and filling in a web form for each of 25 these physiologic records. The protocol for indexing the physiologic records consists of the following two steps. First, an extensible markup language (XMIL) file (containing all of the information about the records that is to be added to the relational database) needs to be created. Then the administrator can upload this XML file through a web form, located at http://physionet.org/search/physiobank/admin/database-upload-xml.tcl, and have the server insert the individual records from the XML file into the database. An XML file resembles an HTML file. HTML is actually a class of XML since XMIL is simply text wrapped around by tags. Please refer to Appendix C for an example of an XML file for a specific physiologic database. The structure of the XML file is as follows: " List of subjects or patients, one followed by another. " The beginning of each subject section contains the subject ID, subject type, sex, and birth date. If any of the information is unknown, then the corresponding field value will be given as an x " Following the description of the subject is a listing of records for this particular subject. " The record section contains the record detail, subject detail, signal detail, and annotator detail subsections. " The record detail contains the following fields: record ID, record source, record type, record URL, start date, start time, duration, and notes. * The subject detail contains the information regarding the subject at the time of recording. The fields for the subject detail subsection are age, list of diagnoses, and list of medications. 26 " A record can have more than one signal, so the signal details section contains a list of signal information. The signal information list contains these fields: signal number, signal type, sample intervals, sampling frequency, bandwidth, gain, ADC resolution and ADC zero. * A record can also have more than one annotator, so the annotator detail subsection is further divided into a series of annotator sections. Each annotator section begins with the annotator name, URL, and source. Following these three fields are all the different annotation categories used by the annotator. Within each category, it specifies the types and either the number of events (e.g. beats) or episodes for that type. A scripting language, PERL, is used to collect this information from the header and annotation files for each record and to store them in the XML format above. A specific PERL script must be written for each of the physiologic databases, although these PERL scripts are very similar. The script first calls the wfdbcat program from PhysioToolkit with the name of the physiologic database as a parameter to this program. This program lists all the record IDs and annotators for this database. Then two other programs, wfdbdesc and sumann, with the record IDs and annotator names as parameters, are used to extract the information from the records. (wf dbdesc reads the header file to obtain the subject, record, and signal details, and sunann reads the annotation files and summarizes their contents.) After the contributor of the database examines the XML file that is returned from the PERL script, this XML file is uploaded to the server using the web form located at 27 http://physionet.org/search/phvsiobank/admin/database-upload-xm.tcl. When the server receives this file, a TCL script parses and translates the XML to SQL insert statements, which Postgres uses to insert these records into the relational database. "Parsing" is the action used to describe reading a file and selecting certain useful text. In an XMIL file, it is very clear what the text is by looking at the surrounding tags. So the script iteratively selects some of these texts and inserts them as rows into specific tables of the relational database. Other texts are selected and inserted into other tables. The TCL script mentioned above first calls an XMIL parser. The XML parser, also written in TCL, reads the file and returns the same information in a TCL list structure. (You can search through the internet to find an XML parser. The one I used was from Arsdigita, http://www.arsdigita.com.) After looking at the format of this TCL list, a procedure is written to convert the TCL list to SQL insert statements, which then are inserted into the relational database. This TCL script works for all XML files as long as these files are in the format described earlier in this section. This script can be found at /home/physionet/search/physiobank/admin/database-upload-2.tcl on the PhysioNet server. Before using the XML parser to insert the records into the relational database, the annotation categories and types need to be registered. I actually do not store the name of the annotation category for the record, but I use the category_id field instead because the names of the categories may change over time. There are a few pages that allow users with "administrator" privileges to add new categories and types within these categories. 28 The step by step process of indexing the physiologic databases and records can be found in Appendix D. 4.5 User Search Interface There are two primary methods of searching. The first method is searching on the physiologic database level. When a user comes to PhysioNet, he probably does not know what physiologic databases are available on PhysioNet. By searching on the physiologic database level, he can find out which databases may be appropriate to his study. After narrowing down the choice of databases, he has the option of searching for particular physiologic records within these databases. The other method is a physiologic record level search. The user can bypass the database level search and directly search through the records in any or all of the physiologic databases. Figure 4 is a screen shot of the user interface for the physiologic database level search. This search page mainly consists of a big select box listing all the attributes of the databases. Some of the example attributes are: Subject Type (Human), Subject Type (Animal), Disease (Healthy), and Disearch (CHF). After selecting the attributes for the searchable criteria (demonstrated in figure 5), you can specify whether the database needs to contain all of the selected attributes ("AND" search) or just any one of them ("OR" search). Beneath the select box are additional parameters (e.g., age range and searching through key words of the abstract) where the user can refine the search ("AND" only). 29 Search Databases You are here: Home > Physiobank Search > Search Databases Use the right arrow Use the left arrow [ to select the field to search for to remove the field from the search list. Search I'll I' ll z DocuftrA: Dom Figure 4: Physiologic DatabaseLevel Search Web Page. This page will eventually be located at http://physionet.org/search/physiobank/database-search. tcl (The source code for this page can be found in Appendix E, part 6.) 30 zj Search Databases You are here: Home > Physiobank Search > Search Databases Use the right arrow Use the left arrow Fl to select the field to search for. [-l to remove the field from the search list. Subject Search These Criteria Subject Type (human) Disease (Healthy) Searchable Criteria Type Subject Type (animal) Subject Type (cell) F Subject Type (molecule) Disease Disease (CHF) [ - Dis-aqP (AIDq) and AND or OR or In addition to the criteria above, you can further refine your search with these folowing parameters. (blank values will be ignored.) AbstrartSearch Searchc Aostract i sepr te t exact match a space. of the words wcrds with ' any Age Range to | Length of Record to e all of the words al Search | Figure 5: Physiologic DatabaseLevel Search Web Page with CriteriaSelected The administrator of PhysioNet controls the user interface for the physiologic database search. In section 4.4.1, "Describing a Physiologic Database", when adding new fields to the pn-fields table, you have the option of specifying whether the field is searchable. If the field is searchable, then this field will appear on the search form. The type of the field (a number, range of numbers, radio boxes, or checkboxes) is important in how we present this searchable field. (Please refer to Figure 4.) If the field type is either radio boxes or check boxes, this field will appear in the big select box. For example, the subject type field is a radio box, so the select box on the user search form contains the different options for the subject type: Subject Type (Human), Subject Type (Animal), 31 Subject Type (cell), and Subject Type (molecule). For fields tagged as a number or ranges of numbers, the search form will present these parameters below the big select box. These fields are presented as two text boxes for the user to type in the range. For example, if the user wants to search for databases containing records in which the subject age is between 50 to 60 years old, the user will type in 50 in the first text box, 60 in the second select box, and select the word "years". After specifying the search criteria in the database level search, the search engine will return the list of databases that match the criteria (shown in Figure 6). At this point, the user has two options. The first is to refine or broaden his search, and the other is to search for records within these databases. Refining the search allows the user to narrow the number of databases. Broadening the search has the opposite effect. If the user chooses to search for the records within these databases, a record search form will be displayed. Database Search Results You are here: Home > Physiobank Search > Database Search > Results Database Search Criteria You are searching for physiological databases with any of these criteria: .ubjecIt T yp Ihuis r Disease ( ealthyi * Age Rang 50 to 60 yeats + * Search Results + + BIDMC Congestive Heart Failure Database MIT-BIH Arrhythmia Database * European ST-T Database " Long-Term ST Database + search records within these databases * refine or broaden search criteria Figure 6: Physiologic Database Level Search Results (The source code for this page can be found in Appendix E, part 7.) 32 The record search form separates the searchable criteria into different sections. To the right of each section is a short description of the section, and to the left is a checkbox. (This is shown in Figure 7.) Fk Ed& Vew G~o Zomn"c"o Help Search Records You are here: Home > Physiobank Search > Search Records Hints for this search page: * For the fields that you are concerned about, click the checkbox next to the field. To remove the criteria, click the checkbox again. e Do not restrict the search with too many criteria Start out broadly since you will have the option to refine your search later. * For boxes with a list of options: to select an item, click the item once; to deselect an item, just click it again. r Databases F Record Type E Recorder Type The selection of the physiclogicai databases The type of physiolog!cal reord T? e ype ofnstrument that is used for the recording Not all databases have thercorter type The-sa SSI-X Th4& rSubjet Aae g -ofthe sub ar,ge of the subjet-,. f Record Diration The durationvength of the recordr r Diagnoses Thed dinse of the subect. Medications The medicatio n taken by the FAubject The syMPftms of the subject The sgnals types contained tn the records. The anotations cateory and beat/epIsode Symptoms Signals Armatations type contned reords Search '!D raeaM.JDot Figure 7: InitialPhysiologic Record Level Search Web Page. This page will eventually be locatedat http://physionet.org/search/physiobank/record-search.tcl. (The source code for this page can be found in Appendix E, part 8.) When the checkbox is selected, that section of the search form will be expanded to allow the user to define his search parameter. For example, when the checkbox next to "Databases" is selected, the section of the search form is expanded to present a list of possible databases to search through. (This is shown in Figure 8.) 33 Search Records You are here: Home > Physiobank Search > Search Records Hints for this search page. * For the fields that you are concerned about, click the checkbox next to the field- To remove the criteria, click the checkbox again " Do not restrict the search with too many criteria. Start out broadly since you will have the option to refine your search later. * For boxes with a list of options: to select an item, chck the item once; to deselect an item, just click it agan. Figure 8: Physiologic Record Level Search with "Database" Checked When the checkbox next to "Subject Age" is selected (shown in Figure 9), that section will be expanded to reveal two text boxes to enter the age range and a select box to select the unit (years, months, or days). 34 Search Records You are here: Homa > Phystobank Search > Search Records Hints for this search page: " For the fields that you are concerned about, click the checkbox next to the field. To remove the criteria, click the checkbox again. * Do not restrict the search with too many criteria Start out broadly since you will have the option to refine your search later. * For boxes with a list of options: to select an item, chck the item once, to deselect an item, just click it again. IMTBHPalysomnolgraphic Database European ST-T Database JLong-Term ST Database r Recard T yp The type of physiotogirecord. SRecorder Typ The type of instrurnen thatis used for the r Sex The | ocr The diagnoses Diagnoses r- Medicatiuns Symptoms F g r Annotatinns type i-- % -iye rs to Record Duration The duraofon/ength F all databaseshave th-rcorder sex of the subject. M eo F-- Sulrect Age Not r mnths or the subiec dLys 4 The redications taken by the subjct.. . The syrnptorns of the subject. The sgnals types contained i the records. The annotationscat r and btepsde pes contind t reords. Search Figure 9: PhysiologicRecord Level Search with "Subject Age" Checked To remove a criterion, the user simply needs to click the checkbox again. Then this section of the search will disappear from the search form. Whenever a section is expanded or collapsed by the clicking of the checkbox, the search page will automatically reload. This feature is accomplished through Javascript. Javascript is an extension of HTML that allows the page to be interactive since HTML itself is static. (The scripting languages I mentioned earlier, TCL, ADP, and PERL, can generate HTML pages, but these scripting languages cannot interact with the HTML pages after these pages are delivered to the user.) Whenever the page reloads, the information that was already entered is remembered so the user does not need to type the 35 information again. (This behavior can also be implemented using PERL and CGI.pm, which may be preferable since some users disable Javascript for security reasons, and not all browsers support Javascript.) Some sections of the search form are a few levels deep. Under the annotation section, you first select the annotation category and then the type within the category. This is illustrated in Figure 10. SBR - sinus bradycardia F SVTA - apraveptncu r tachyarrhythm F T - ventrecuiar treminy "VFIB - ventncular fibnllaten? " V F - Va, trcutar 1-utter-/fibraxan " VFL - Ventncular Ptter 1 VT - V hycardiA signal qualty search_ Figure 10: Physiologic Record Level Search - "Annotations" Section After the user specifies all the search criteria, he clicks the "Search" button to perform the search. After the resulting records are displayed, the user has the option of refining or broadening his search (see Figure 11). 36 Record Search Results You are here: Home > Physiobank Search > Record Search > Results Record Search Criteria You are searching for physiological records with these criteia e located in these databases. BIDMC Congestive Heart Failure Database, IfT-BIH Arrhythmia Database * age between 50 and 55 Y * Number of Signals between I and 99 0 At least 1 ECG * Between 1 and 999999 ECG rhythm annotations Search Results - mitdb/112 mitdb/119 mitdb/122 mitdb/214 " chfdb/chf04 * 5 records found . refine or broaden search criteria OerrmeeAt Done *~ 7Z~ Figure 11: Physiologic Record Level Search Results (The source code for this page can be found in Appendix E, part 9.) In addition, the SQL query that was used to perform the search is displayed in a text box. This allows this user to edit the SQL query manually and to perform the search again if he understands the SQL language (see Figure 12). + refine or broaden search criteria SQL Query select record dbid, record-id from pnrecords r where databaseid in (4,3) and age unit,'50;60;Y) ';' pnin-range(age = 't' and I II exists (select I from pnannotations a where a.recorddbid = r.recorddbid and category id = 48 and number between I and 999999) and exists (select I from pn annotationstype map a_tmap, pn_annotations a where a. recorddb id = r.recorddbid and a-t-map.annotationid = a.annotationid and a-t-map.type = 'N'and (a_t_map.number between 1 and 999999 or a_t_map.episodes between I and 999999)) Figure 12: Physiologic Record Level Search - Updating SQL Query 37 When the user clicks on the "refine or broaden search criteria" link, this will take the user back to the record search form with a list of options to "refine", "broaden", or "complement" the previous search. Please refer to the screen shot in Figure 13 for descriptions of these options. Search Records You are here: Home > Physiobank Search > Search Records Hints for this search page: * For the fields that you are concerned about, click the checkbox next to the field. To remove the criteria, click the checkbox again. * Do not restrict the search with too many criteria. Start out broadly since you will have the option to refine your search later. * For boxes with a list of options: to select an item, click the item once; to deselect an item, just click it again. Figure 13: Physiologic Record Level Search - Refining or BroadeningSearch If this record search form is displayed as the result of the database level search, the "Databases" select box will be highlighted with the results of the database level search. 4.6 Extending Search Capability Currently, these two search features, database level search and record level search, only deal with what has been extracted from the XML files and added to the relational database. Another requirement of this project was the capability to extend the search 38 engine to handle signal or annotation processing by external programs. Such searches can be achieved in the following manner: " The user will perform the record search as described in the previous section. * In the revised result page, there will be a link to perform external processing on the resulting records. " Let us assume the user wants to find records in which the subject's blood pressure is greater than n. The server will pass the record IDs from the search result to an external program. After it processes the data files for these records, the external program will return the records IDs that passed the test. Finally, these record IDs will be returned to the user. 5 Results and Conclusion One of the hardest parts of this project was to understand enough of the physiology to capture the information correctly in the data model. Designing the user interface for searching was also pretty difficult. We wanted to provide a simple, clean search form that novice web users can understand. There were many iterations of the search form before we came up with the current design. Here are a few suggestions for improving the search engine. When the results are returned from the record search, we can offer the user links to view the signal files through Emily Liu's Java Applet implementation of Wave. This applet should be completed by the middle of 2001. As mentioned in the previous section, we can add external post-processing of the search results to allow additional user-defined search 39 criteria on information not included in the relational database. The design of the search engine can be revisited to offer more search criteria. 6 Bibliography 1. http://www.scriptics.com/, documentation on TCL. 2. http://www.pgsql.com/, documentation on Postgres. 3. http://www.aolserver.com/, documenation on AOLServer. 4. http://www.arsdigita.com/books/tcl/, tutorial for TCL. 5. http://www.arsdigita.com/books/sql/, tutorial for SQL. 40 Appendix A: Installation of the Web Server and Database The software and code for the search engine is currently located at pc I1.ecg.mit.edu. 1) From a linux machine that is within the lab's fire wall, type ssh pcll.ecg.mit.edu 2) The tar file can be found at /home/physionet/search. 12102000 .tar.gz 3) Copy this file to the /home/physionet / directory on your own machine. 4) Untar this file by typing tar xvfz search.12102000.tar.gz 5) After this file is untarred, the following directories and files are added: /search/physiobank: tcl files for the search engine /sql : data model /tcl : tcl functions /parameters : some ACS parameter used in the site : software used for the search engine /software : perl script to generate the XN1L for the physiologic /perl-script databases : XML files for the physiologic databases nsd. ini : AOLServer parameter file search-readme.txt : instructions to install the software. /xml 41 Appendix B: Data Model of PhysioNet -- physionet.sql -- data model to store the database information 10/16/2000 flattop@mit.edu -- create sequence pnsectionid-seq; create table pn sections ( sectionid integer primary key, description varchar(100), sortkey int2 create sequence pnfieldid-seq; create table pn fields ( field id integer primary key, description varchar(100), int2, sortkey modifier varchar(10) check(modifier in ('radio','checkbox','range','url','text','number')), --used only if it is radio, checkbox, or range -- if radio or checkbox ==> semicolon separated list -- if it is range ==> stores the units modifiervalues text, integer references pn-sections sectionid searchablep varchar(l) check(searchable-p in ('t','f')), create sequence pndatabaseidseq; create table pn-databases ( integer primary key, database id name varchar(1000), url varchar(1000), abstract text, --whether the abstract is in html or plain text varchar(l) check(html-p in ('t','f')), htmlp creationdate date create table pndatabasefield map databaseid integer references pn-databases, fieldid integer references pnfields, -- for checkboxes ==>there will be multiple rows -- for range ==> begin vale;end value;unit extravalue text -- constraing not needed due to checkboxes --primary key(database_id, field id) 42 -** *** ********* **** --Subject informaton --most of the table have a _db _id which is used for indexing -- and referencing, also for performance reason create sequence pndb_id_seq; create table pnsubjects ( subjectdbid integer primary key, varchar(100) unique, subject id varchar(200), subject type sex varchar(1), --birthdate is stored as this: yyyy/mm/dd -if the number is not known, an "x" will be used varchar(20), birthdate upload-date date create table pnrecords ( integer primary key, recorddbid record id varchar(100) unique, subjectdbid integer references pnsubjects, integer references pn-databases, database id recordsource varchar(200), varchar(200), record_type recordertype varchar(200), recordurl varchar(200), -- startdate is stored as this: yyyy/mm/dd -- starttime is stored as this: hh24:mm:ss -- duration is stored as this: h:m:s.ms -if the number is not known, an "x" will be used startdate varchar(20), starttime varchar(20), duration varchar(20), notes text, --age is represented like 44Y age int4, age-unit varchar(5) create table pn recorddiagnosis map ( record_db_id integer references pn-records, diagnosis varchar(200) create table pnrecordmedicationmap ( recorddbid integer references pn-records, varchar(200) medication create table pnsignals ( integer primary key, signal id recorddb id integer references pn-records, 43 int eger, var char(200), var char(200), signaltype int eger, sample-intervals -- samplingfrequency is in hertz int 4, samplingfrequency flo at4, bandwidthfrom flo at4, bandwidthto -- gain is specified as < amount> <unitl>/<unit2> int 4, gain var char(100), gain unit1 var char(100), gain unit2 -- adcresolution is in b its int 2, adcresolution int 2 adczero signal-number signal name create table annotator pnannotators id ( int eger primary key, recorddb id int eger references pnrecords, annotatorname annotatorurl var char(200), var char(200), annotatorsource var char(200) -- pn_annotationcategories and pnannotationdescriptions -need to be defined before uploading any records to the database -create table pnannotationcategories ( integer primary key, categoryid varchar(100) unique categoryname create table pnannotation descriptions descriptionid ( integer primary key, categoryid integer references pn-annotationcategories, type varchar(5), description varchar(1000) create table pnannotations ( integer primary key, annotationid integer references pnannotators, annotator id -- record_db_id is denormalized to allow quicker search -- using recorddbid integer references pn-records, recorddbid categoryid integer references pnannotationcategories, integer number 44 create table pnannotationstype map ( annotationid integer references pn annotations, -- some annotation type may not be symbols -but actual text -- for symbols, look them up in the pn annotationdescriptions varchar(1000), type -- depending on the type -some will only have the *number* field -othes will only have the *episodes,time* fields number integer, integer, episodes -- time is stored as this: hh:mm:ss.ms varchar(20) time 45 Appendix C: Sample Copy of XML (This only includes two records of the same patient from the MIT-BIH Polysomnographic Database) <subjects> <one-subject> <subject-description> <subjectid>slpdb/slpO 1 a</subject id> <subject-type>human</subject-type> <sex>M</sex> <birthdate>xxxx/xx/xx</birth_date> </subject-description> <records> <onerecord> <recorddetails> <record-id>slpdb/slpO Ia</recordid> <record source>MIT-BIH Polysomnographic Database</recordsource> <record-type>Polysomnograph</record type> <recordurl>http://www.physionet.org/physiobank/database/slpdb/slpOla.hea</record-url> <startdate>1989/01/19</startdate> <starttime>23:07:00</starttime> <duration>2:00:00.000</duration> <notes> </notes> </record-details> <subject-details> <age>44Y</age> <diagnoses> </diagnoses> <medications> </medications> </subject-details> <signal-details> <one-signal> <signal-number>0</signal-number> <signal name>ECG</signal_name> <signal_type>ECG</signal_type> <sample intervals>1800000</sample-intervals> <sampling-frequency>250</sampling-frequency> <bandwidth></bandwidth> <gain>-200 adu/mV</gain> <adcresolution>12 bits</adcresolution> <adc_zero>0</adc_zero> </one-signal> <one-signal> <signal-number> 1 </signal-number> <signal-name>BP</signalname> <signaltype>BP</signal_type> <sample-intervals> 1800000</samplejintervals> <sampling-frequency>250</sampling-frequency> <bandwidth></bandwidth> <gain>4.77778 adu/mmHg</gain> <adcresolution>12 bits</adcresolution> <adc_zero>0</adc_zero> 46 </onesignal> <one-signal> <signal-number>2</signal_number> <signaln ame>C4-A 1 </signal-name> <signaltype>EEG </signal-type> <sample-intervals> 1800000</sampleintervals> <sampling-frequency>250</sampling-frequency> <bandwidth></bandwidth> <gain>-6430 adu/mV</gain> <adcresolution>12 bits</adcresolution> <adc_zero>0</adc_zero> </one-signal> <one-signal> <signal-number>3</signal_number> <signal-name>sum</signal-name> <signal-type>Resp </signal-type> <samplejintervals>1800000</sampleintervals> <sampling-frequency>250</sampling-frequency> <bandwidth></bandwidth> <gain>690 adu/l</gain> <adcresolution>12 bits</adcresolution> <adc_zero>0</adc_zero> </one-signal> </signal-details> <annotatordetails> <oneannotator> <annotator_name>ecg</annotatorname> <annotatorurl>http://www.physionet.org/physiobank/database/slpdb/slpO 1a.ecg</annotatorurl> <annotatorsource>reference</annotatorsource> <annotation-categories> <oneannotation-category> <annotationscategory-name>QRS</annotation-categoryname> <number>7806</number> <annotation-types> <oneannotationtype> <type>N</type> <number>7806</number> </oneannotation type> </annotation types> </oneannotationcategory> <oneannotationscategory> <annotation-category_name>ECG rhythm</annotationcategory-name> <number>0</n umber> <annotation-types> </annotation-types> </oneannotationcategory> <oneannotationscategory> <annotation-category-name>signal quality</annotation-category-name> <number> 1</number> <annotationjtypes> <oneannotationjtype> <type>cc</type> <episodes> 1</episodes> <time>2:00:00</time> </oneannotationjtype> </annotation-types> 47 </oneannotationcategory> </annotation-categories> </oneannotator> </annotatordetails> </onerecord> <onerecord> <record-details> <recordid>slpdb/slpO Ib</record_id> <recordsource>MIT-BIH Polysomnographic Database</recordsource> <record-type>Polysomn ograph</recordjtype> <recordurl>http://www.physionet.org/physiobank/database/slpdb/slpO lb.hea</record url> <startdate> 1989/01/20</startdate> <start time>02:14:00</start time> <duration>3:00:00.000</duration> <notes> </notes> </recorddetails> <subject-details> <age>44Y</age> <diagnoses> </diagnoses> <medications> </medications> </subject-details> <signal-details> <one-signal> <signal-number>0</sign alnumber> <signal-name>ECG</signal-name> <signal_type>ECG</signaLtype> <sample-intervals>2700000</samplejintervals> <sampling-frequency>250</sampling-frequency> <bandwidth></bandwidth> <gain>-200 adu/mV</gain> <adcresolution>12 bits</adcresolution> <adc_zero>0</adc_zero> </one-signal> <one-signal> <signal-number> 1</si gnal_number> <signalname>BP</signal_name> <signal-type>BP</signal-type> <sample-intervals>2700000</samplejintervals> <sampling-frequency>250</sampling-frequency> <bandwidth></bandwidth> <gain>4.77778 adu/mmHg</gain> <adcresolution>12 bits</adcresolution> <adc_zero>0</adc_zero> </one-signal> <one-signal> <signal-number>2</signal-number> <signal-name>C4-A1</signal-name> <signal-type>EEG </signal-type> <samplejintervals>2700000</sampleintervals> <sampling-frequency>250</sampling-frequency> <bandwidth></bandwidth> <gain>-6430 adu/mV</gain> <adcresolution>12 bits</adcresolution> 48 <adczero>O</adczero> </one-signal> <one-signal> <signal-number>3</signal-number> <signal name>sum</signalname> <signal-type>Resp </signal-type> <sampleintervals>2700000</sample-intervals> <sampling-frequency>250</sampling-frequency> <bandwidth></bandwidth> <gain>-690 adu/l</gain> <adcresolution>12 bits</adcresolution> <adczero>O</adczero> </one-signal> </signal-details> <annotatordetails> <oneannotator> <annotatorname>ecg</annotator name> <annotatorurl>http://www.physionet.org/physiobank/database/slpdb/sIp0lb.ecg</annotatorurl> <annotatorsource>reference</annotatorsource> <annotation-categories> <oneannotation category> <annotation-category-name>QRS</annotation-categoryname> <number> 11467</number> <annotation-types> <oneannotation-type> <type>N</type> <number>1 1465</number> </oneannotation type> <oneannotation type> <type>S</type> <number>2</number> </oneannotation type> </annotation-types> </oneannotationcategory> <oneannotationcategory> <annotationscategory_name>ECG rhythm</annotationcategory-name> <n umber>O</n umber> <annotation-types> </annotation-types> </oneannotationcategory> <oneannotationcategory> <annotationscategory-name>signal quality</annotationcategory-name> <number> 1</number> <annotation-types> <oneannotationjtype> <type>cc</type> <episodes> 1</episodes> <time>3:00:00</time> </oneannotationjtype> </annotation-types> </oneannotationcategory> </annotation-categories> </oneannotator> </annotatordetails> </onerecord> </records> 49 </one-subject> <subjects> 50 Appendix D: Indexing Physiologic Databases and Records 1) Go to the administrative section: http://<server-name>/search/physiobank/admin (the server name for Physionet is www.physionet.org) 2) The "Form Section" and "Form Field" links allow you to add and edit new sections and fields. This is described in section 4.4.1. 3) To index a new database, click on the "Add a new database" link. A web form will be displayed for you to fill out. (Ideally, the flat files for this database should have already been uploaded to the server before you do this step.) 4) To edit the database information, click on the "Database List" and then select the database that you want to edit. One of the options in editing the database is to upload the XML file that describes the records in this database. 5) Write a PERL script that returns an XML file for the descriptions of the records. Sample files can be found at the /home/physionet/perl-script/ directory. (Please refer to Appendix A for instructions to download the tar file.) You need to make changes to the sample script for your new database. 6) When the PERL script is executed, an XML file will be returned. Use the edit database feature to upload this XML file. 7) Another link in the database edit page is a link to "Delete Records in this Database". If you made a mistake in the PERL script or XML file, you may use this option to delete the records. Then you can start over with step #5. 51 Appendix E: Source Code Part 1: Adding a New Section jX Add a New Section - Netscape- Fie Edt View Go Communcator Help Add a New Section You are here: Home > Physiobank Search > Admin > Section List > Add a New Section Section Name Add ## section-add.tcl set db [ns-db gethandle] set sectionid [databasetojtcl string $db "select nextval('pn-sectionidseq')"] nsdb releasehandle $db set contextbar [pn-searchadmincontext-bar [list "section-list" "Section List"] "Add a New Section"] adreturntemplate ## section-add.adp <pn-header-plain title="Add a New Section"></pnheaderplain> <form method=post action=" section-add-2"> <%= [export formvars section-id] %> <table> <tr> <pnjinput pretty-name="Section Name" name="description" size="40" maxlength="100"></pn_input> </tr> </table> <p><center><input type=submit value="Add">'</center> </form> <pn-footer-plain></pn-footer-plain> 52 ## section-add-2.tcl ##processes the datafrom the forn ad-page-varables { sectionid description set description [string trim $description] if { [empty-string-p $description]} adreturncomplaint 1 "You forgot to enter a Section Name." return set db [ns-db gethandle] set exist-p [database to tcl string $db "select count(*) from pn sections where section id = $section id"] if {$exist-p} { nsdb releasehandle $db nsreturnredirect "[pnsearch admindir]/section-list" return } set insertsql "insert into pn sections (sectionid, description) values ($section_id, '[DoubleApos $description]') ns-db dml $db $insert-sql nsdb releasehandle $db nsreturnredirect "[pn-searchadmin_dir]/section-list" return 53 Part 2: Editing a Section Edit Section You are here: Home > Physiobank Search > Admin > Section List > Edit Section Section Name jSubjects ## section-edit.tcl ad-page-variables {sectionjid description} set contextbar [pn-searchadmincontext-bar [list "section-list" "Section List"] "Edit Section"] adreturntemplate ## section-edit.adp ad-page-variables {section-id description} set contextbar [pn-searchadmincontext-bar [list "section-list" "Section List"] "Edit Section"] adreturntemplate ## section-edit-2.tcl ##process the datafrom the form ad-page variables {section id description} set contextbar [pn-search admincontext-bar [list "section-list" "Section List"] "Edit Section"] adreturntemplate 54 Part 3: Adding a New Field Add a New Field You are here: Home > Physiobank Search > Admin > Field List > Add a New Field Field r Subjects r Signals r Record C Annotations - Medications r- none C radio C checkbox C range r url r te xt C number Section Modifier Radio/CheckboxfRange Values r yes r no Searchable --_ F__ - if you selected Iradi* or *checkox, please enter 0be options separated by semcolons, hke this: hgh; mefium;iow (klr *range* enter the units separated < >& by sern&icons)-- Please do not use these characters: Whether this field is .a search criterna or rno-t - -- ----- .... " """' -- ,__ - , - ;- " "',,"" .......... Documer ; DaFaa ##field-add.tcl set db [ns-db gethandle] set field-id [databasetotcl-string $db "select nextval('pnfieldidseq')"] set sectionlist [databasejto_tcllistlist $db "select description, sectionid from pnsections order by sort-key"] nsdb releasehandle $db set sectionvaluelist "" set sectiondefaultvalue foreach section $section-list { if {[empty-string-p $sectiondefault value]} { set sectiondefaultvalue [lindex $section 1] lappend sectionvaluelist "[lindex $section O],[lindex $section 1]" } set sectionvalue [join $sectionvaluelist ";"] set contextbar [pn-searchadmincontext-bar [list "field-list" "Field List"] "Add a New Field"] adreturntemplate 55 7TI.... ##field-add.adp <piuheader-plain title="Add a New Field"></pn~headerplain> <form method=post action="field-add-2"> <%= [export_formvars fieldid] %> <table> <tr> <pninput pretty-name= "Field" name="description" size="40" maxlength="100"></pnnput> </tr> <tr> <pnjradio pretty-name="Section" name="sectionid" value=$section value default value=$sectiondefaultvalue></pnjradio> </tr> <tr> <pn-radio pretty-name="Modifier" name="modifier" value="none,NULL;radio,radio;checkbox,checkbox;range,range;url,url;text,text;number,number" defaultvalue="NULL"></pnjradio> </tr> <tr> <pnjinput pretty-name="Radio/Checkbox/Range Values" name="modifiervalues" size="40" maxlength=" 1000" desc="If you selected *radio* or *checkbox*, please enter the options separated by semicolons, like this: high;medium;low (for *range*, enter the units separated by semicolons)-- Please do not use these characters: , . &lt; &gt; &amp;"></pn_input> </tr> <tr> <pn-radio pretty-name= "Searchable" name=" searchable-p" value="yes,t;no,f' defaultvalue=f desc="Whether this field is a search criteria or not"></pn_radio> </tr> </table> <p><center><input type=submit value= "Add"></center> </form> <pn-footer-plain></pn-footer-plain> 56 ##field-add-2.tcl ##processes the datafrom the form #check if they use . , <> & in there field # also check in field-edit-2 adpage-variables { field-id description sectionid modifier modifiervalues searchable-p} set description [string trim $description] if { [empty-string-p $description]} { adreturncomplaint 1 "You forgot to enter a Field Name." return } set db [ns-db gethandle] set exist-p [databasetotcl-string $db "select count(*) from pn_fields where fieldid = $fieldjid"] if {$exist-p} { nsdb releasehandle $db nsreturnredirect "/holter/field-list.tcl" return } if {$modifier == "url" 11$modifier == "text" $modifier set modifiervalues I == "number" if {$modifier != "NULL"} { set modifier "'[DoubleApos $modifier]" set insert-sql "insert into pnfields (field-id, description, modifier, modifier-values, section-id, searchable-p) values ($field-id, '[DoubleApos $description]', $modifier, '[DoubleApos $modifier-values]', $section-id, '$searchable-p') ns_db dml $db $insert-sql ns_db releasehandle $db ns_returnredirect "[pn-search admindir]/field-list" return 57 Part 4: Editing a Field Edit Field You are here: Home > Physiobank search > Admin > Field List > Edit Field Field Isubject Section r Subjects C Signals C Record - Annota tions C Medications Modifier C Type none r radio C checkbox ( range C url C text r number human; animal, cell; molecule Radio/CheckboxfRange Values if you se/ected *radio* or *checkbo< please enter the options separatedby semicolons, like this: high;medium lew (for *range* enter the units separated 8: < by sermicolons) -- Please do not use these characters: r- yes r no Searchable W4hether th. Docume rt: ,eid s a search crteria or not Done 5 K / ##field-edit.tcl ad-page-variables {field-id} set db [ns_db gethandle] set selection [nsdb Irow $db "select description, modifier, modifiervalues, section-id, searchable-p from pnfields where fieldid = $field-id"] setvariablesafter-query if {[empty-stringp $modifier]} I set modifier NULL set modifiervalues } elseif {$modifier == "text" $modifier == "number"|1 $modifier == "url"} set modifiervalues set sectionlist [databasejto_tcllistlist $db "select description, section-id from pn-sections order by sort key"] nsdb releasehandle $db set sectionvalue_list set sectiondefaultvalue $section-id foreach section $sectionjlist { if {[empty-string-p $sectiondefault-value]} { set sectiondefaultvalue [lindex $section 1] 58 lappend sectionvaluelist "[lindex $section 0],lindex $section 1]" set section-value Uoin $sectionvaluelist ";"] set context-bar [pn-search-admincontext bar [list "field-list" "Field List"] "Edit Field"] adreturn-template ##field-edit.adp <pn-header-plain title="Edit Field"></pn header-plain> <form method=post action="field-edit-2"> <%= [exportformvars field-id] %> <table> <tr> <pn-input pretty-name="Field" name="description" value=$description size="40" maxlength="100"></pnjnput> </tr> <tr> <pnradio pretty-name= "Section" name="sectioni d" value=$section value default value=$sectiondefaultvalue></pnjradio> </tr> <tr> <pn-radio pretty-name= "Modifier" name= "modifier" value="none,NULL;radio,radio;checkbox,checkbox;range,range;url,url;text,text;number,number" default-value=$modifier></pnrradio> </tr> <tr> <pninput pretty-name="Radio/Checkbox/Range Values" name="modifier_.values" value=$modifiervalues size="40" maxlength=" 1000" desc="If you selected *radio* or *checkbox*, please enter the options separated by semicolons, like this: high;medium;low (for *range*, enter the units separated by semicolons) -- Please do not use these characters: , . &lt; &gt; &amp;"></pninput> </tr> <tr> <pn-radio pretty-name= "Searchable" name="searchable-p" value="yes,t;no,f' default value=$searchable_p desc="Whether this field is a search criteria or not"></pn_radio> </tr> </table> <p><center><input type=submit value="Edit"></center> </form> <pn-footer-plain></pn-footer-plain> 59 ## field-edit-2.tcl ## processes the datafrom the form ad-page-variables { field id description modifier modifiervalues section_id searchable-p} set db [ns-db gethandle] set description [string trim $description] if { [empty-string-p $description]} { adreturncomplaint 1 "You forgot to enter a Field Name." return if {$modifier == "url" || $modifier == "text" J|$modifier set modifiervalues if {$modifier != "NULL"} { set modifier "'[DoubleApos $modifier'" } set update-sql "update pnfields set description = '[DoubleApos $description]', modifier = $modifier, modifiervalues = '[DoubleApos $modifiervalues]', section-id = $section-id, searchable-p = '$searchable-p' where field-id = $field-id nsdb dml $db $update-sql nsdb releasehandle $db nsreturnredirect "[pn-searchadmindir]/field-list" return 60 == "number") Part 5: Adding a New Database The screenshot for this web page is Figure 3. ## database-add.tcl set db [nsdb gethandle] set fieldinfolist [databasetotcllist_list $db "select f.fieldid, f.description, f.modifier, f.modifiervalues, s.description as sectionname from pn-fields f, pn-sections s where f.sectionid = s.section id and s.sortkey is not null and f.sort-key is not null order by s.sort-key, f.sort-key"] nsdb releasehandle $db set fieldinputjlist set currentsection foreach fieldinfo $field-info list set fieldid [lindex $fieldinfo 0] set description [lindex $field_info 1] set modifier [lindex $field-info 2] set modifiervalues [lindex $field-info 3] set sectionname [lindex $field-info 4] if {$current-section != $sectionname} if {![empty-string-p $currentsection] append fieldjinputjlist "</dl> </td> </tr>" { I append field-inputjlist "<tr> <th align=\"left\" valign=\"centerV><font color=\"blue\">$section-name</font></th> <td bgcolor=\"#cccccc\"> <dl>\n " set current-section $sectionname set modifierhtml if {$modifier == "radio" { append modifierhtml [ns-adp-parse -string -local "<pnjradio-no-pretty-name name=\"${ field-id }_value\" value=\"$modifiervalues\" script=VonClick=CheckBox($fieldd)\"></pnadio-no-pretty-name>"] } elseif {$modifier == "checkbox" { 61 append modifierhtml [ns-adp-parse -string -local "<pncheckbox-no-pretty-name name=\"${ field-id} value\" value=V'$modifiervalues\" script=\"onClick=CheckBox($field-id)\"></pnscheckbox-no-pretty-name>"] } elseif {$modifier == "range" I{ append modifierhtml " <input type=\"text\" name=\"${field_id}_valuel\" size=1 1 maxlength=10 onFocus=\"CheckBox($field-id)\"> to <input type=\"text\" name=\"${fieldd }_value2\" size=1 1 maxlength=10 onFocus=\"CheckBox($field-id)\"> [ns-adp-parse -string -local "<pn-selectnopretty-name name=\"${ field-id }_value3\" value=\"$modifiervalues\"></pn-select-no-prettyname>"] } elseif {$modifier == "number" { append modifierhtml " <input type=\"text\" name=\"${field-id}_value\" size=1 1 maxlength=10 onFocus=\"CheckBox($field-id)\"> <font color=\"red\" size=\"-1\"><b> - number</b></font>" } elseif {$modifier == "text"} { append modifierhtml " <input type=\"text\" name=\"${field_id}_value\" size=1 1 onFocus=\"CheckBox($field-id)\"> <font color=Vred\" size=\"-1\"><b> - text</b></font>" } else { append modifierhtml if {$modifier == "range"}{ append field-inputjlist "<dt><input type=checkbox name=field-ids value=\"$field-id\" onClick=\"ClearField('${ field id }valuel');ClearField('${ fieldid }_value2')\"> $description</input></dt> <dd>$modifier_html</dd>\n" else { append field-inputjlist "<dt><input type=checkbox name=fieldids value=\"$fieldid\" onClick=\"ClearField('${ field-id }value')\"> $description</input></dt> <dd>$modifierhtml</dd>\n" append fieldinputlist "</table> </td> </tr>" set contextbar [pn-search-admincontextbar "Add a Database"] set script " <script language=\"JavaScript\"> function CheckBox(id) { var nelement = document.forms\[O\I.length; for (var i = 0; i < n_element; i++) { var element = document.forms\[0\].elements\[i\; if (element.name == \"fieldids\" && element.value element.checked = 1; == id) } function ClearField(inputname) { var nelement = document. forms\[O\]. length; for (var i = 0; i < n_element; i++) { var element = document.forms\[O\]. elements\[i\]; if (element.name == input-name) { if (element.type == \"checkbox\" |1 element.type == \"radio\") element.checked if (element.type == \"text\") element.value = \"; 62 = 0; </script> adreturnjtemplate ## database-add.adp <pnheader-plain title="Add a New Database"></pn headerplain> Please enter the following info: <p> <form method=post action="database-add-2"> <table cellspacing=10> <tr> <pninput pretty-name= "Name" name="name" size="40" required-p="t"></pn-input> </tr> <tr> <pnjinput pretty-name="URL" name="url" size="40" required-p="t"></pn-input> </tr> <%= $fieldinputlist %> </table> <font color="blue"><b>Abstract<Ib></font> &nbsp;&nbsp;&nbsp; <pn radio-no-pretty-name name="html-p" value="html,t;text,f defaultvalue="f'></pn-radio-no-pretty-name> <br> <textarea name="abstract" wrap=soft rows=20 cols=80></textarea> <p> <p><center><input type=submit value=" Add" ></center> </form> <pn-plain-footer></pn-plainfooter> 63 ## database-add-2.tcl ## processes the datafrom the form set db [ns-db gethandle] set field_id_values "" set selection [ns db select $db "select field id, modifier from pnfields where sort-key is not null"] while { [ns-db getrow $db $selection] { setvariablesafter-query if {$modifier == "range"} { append fieldidvalues "\{ ${ field-id }valuel \"\"\} \{ ${ fieldid _value2 \"\"\} \{ ${ field_id}_value3 \"\"\} " else { append fieldidvalues "\{${field-id}_value -multiple-list\} ad-page variables [subst {name url {fieldids -multiple-list} abstract html p $fieldjid values}I set exceptiontext set exception-count 0 set name [string trim $name] if { [empty-string-p $name] I { append exceptiontext "<li>You forgot to enter a name for the database." incr exceptioncount } set url [string trim $url] if { [empty-string-p $url]} { append exceptiontext "<li>You forgot to enter the url." incr exceptioncount } elseif {![philg urlvalid_p $url] } I # there is a URL but it doesn't match our REGEXP append exceptiontext "<li>You URL doesn't have the correct form. A valid URL would be something like \"http://www.physionet.orgA"." incr exceptioncount } set abstract [string trim $abstract] if { [empty-stringp $url]} { append exceptiontext "<li>You forgot to enter an abstract for the database." incr exceptioncount } elseif {[string length $abstract]> 8000}{ append exceptiontext "<li>The abstract contains [string length $abstract] characters. Please limit it to 8000." incr exceptioncount } if { $exception count > 0 adreturncomplaint $exception-count $exceptiontext return } 64 set database id [database to-tcl-string-or-null $db "select databaseid from pn-databases where UPPER(name) = UPPER('$name')"I if {![empty-string-p $databasejid]} { ns db releasehandle $db ns returnredirect "[pn-search admindir]/database-edit.tcl?[export-urlvars database-id]" return } set database id [database tojtcl-string $db "select nextval('pn-databaseid-seq')"] set insert-sql "insert into pndatabases (database id, name, url, abstract, html-p, creation-date) values ($database-id, '[DoubleApos $namef', '[DoubleApos $url]', '[DoubleApos $abstract]','$html-p', sysdateo) ns db dml $db $insert-sql foreach fieldid $field-ids f set selection [nsdb irow $db "select modifier, modifiervalues from pnfields where fieldid $field id"] setvariablesafter-query if {$modifier == "range" { set extravalue [subst "$${field-id}_valuel"] append extravalue ";" append extra-value [subst "$${field_id}_value2"] #add the units append extravalue append extra-value [subst "$${field-id}_value3"] nsdb dml $db "insert into pn-database fieldmap (database id, field-id, extra-value) values ($database-id, $field-id, '[DoubleApos $extravalue]') else #might be a checkbox, so it might have multiple values # in the other cases, there are only one value set extravalues [subst "$${field-id}_value"] if {[llength $extra-values] == 0} { #no modifer values nsdb dml $db "insert into pn-database field map (database id, field-id, extra-value) values ($databaseid, $fieldid, ") } else foreach extravalue $extra-values nsdb dml $db "insert into pn-databasefield-map (database-id, field-id, extra-value) values ($databaseid, $fieldid, '[DoubleApos $extravalue]') 65 = } nsdb releasehandle $db ns returnredirect "[pnsearch admindir]/database-edit.tcl?[export-url-vars database-id]" 66 Part 6: Search Databases The screenshot for this web page is Figure 4. ## database-search.tcl ad-page-variables {{ databaseidj ist ""}} set db [ns-db gethandle] #don't hande ranges, text, number, url for now set fieldinfolist [databasetotcllistlist $db "select f.fieldid, f.description, f.modifier, f.modifiervalues, s.description as sectionname from pn-fields f, pn-sections s where f.sectionid = s.sectionid and s.sort-key is not null and f.sort-key is not null and f.modifier not in ('url','text','number','range') and searchable-p = T order by s.sort-key, f.sort key"] set ncount 0 set leftselectoptions "" foreach field _info $field-infolist I set field-id [lindex $field-info 0] set description [lindex $fieldinfo 1] set modifier [lindex $fieldinfo 2] set modifiervalues [lindex $field info 3] set sectionname [lindex $fieldinfo 4] append left-select-options "<option value=\"$fieldjid\">$description</option> \n" incr ncount if {$modifier == "radio" 11$modifier == "checkbox"} { set valuelist [split $modifiervalues ";"] foreach valueelement $value-list { append leftselectoptions "<option value=\"${fieldjid}~${value-element}\">&nbsp&nbsp&nbsp $description ($value-element)</option> \n" incr ncount #values from before since the modifier can change set old value-list [databasetotcllist $db "select distinct extravalue from pn-database field-map ""] where field-id = $fieldid and extra-value foreach valueelement $old-value-list I if {[lsearch -exact $value-list $valueelement] == -1} append leftselect-options "<option value=\"${ field-id }-${ value-element }\">&nbsp&nbsp&nbsp $description ($value-element)</option> \n incr ncount 67 set selection [ns db select $db "select f.field-id, f.description, f.modifier, f.modifiervalues from pn-fields f, pn-sections s where f.sectionid = s.sectionid and s.sort key is not null and f.sort-key is not null and f.modifier in ('number','range') and searchable-p = T order by f.modifier, s.sort-key, f.sort key"] set modifiersearch "" while { [ns-db getrow $db $selection] { setvari abl esafter-query if {$modifier == "number" append modifiersearch "<tr> <td><font color=\"blue\"><b>$description</b></font></td> <td> [ns-adp-parse -string -local "<pn-select-no-pretty-name name=\"${field-id }_valuel\" value=\"greater than;less than;exactly\"></pn-select-no-pretty-name>"] <input type=\"text\" name=\"${ fieldid }_value2\" size=1 1 maxlength=10 onFocus=\"CheckBox($field-id)\"></td> </tr> } elseif {$modifier == "range"} { append modifiersearch "<tr> <td><font color=\"blue\"><b>$description</b></font></td> <td><input type=\"text\" name=\"${fieldid}yvaluel\" size=11 maxlength=10> to <input type=\"text\" name=\"${field-id}_value2\" size=1 1 maxlength=10> [ns-adp-parse -string -local "<pn-selectno-pretty-name name=\"${field-id}_value3\" value=\"$modifiervalues\"></pn-select-no-pretty-name>"]</td> </tr> set n-longest 50 set spaces "" for {set i 0} {$i <= $njlongest} {incr append spaces "&nbsp;" i} { } set right-null-options for {set i} {$i < $n-count I {incr i}{ append right-null-options "<option value=\"null\">&nbsp;</option> \n if {$ncount > 30} { set ncount 30 } 68 set leftselect " <select name=Vleft\" size=$ncount> $left-select-options <option value=Vnull\">$spaces</option> </select> set right-select <select name=VrightV size=$n-count> $right-null-options <option value=\"null\">$spaces</option> </select> if { [empty-string-p $databaseidjlistl} { set databaseidtext else { set databasenames [databasetotcl_list $db "select name from pn-databases where databaseid in (Uoin $database id list ","])"] set databaseidtext "<tr> <td colspan=3> <table> <tr> <th align=\"left\"><font color=\"blueV>Databases</font></th> <td><em>[join $databasenames ", "]</em> <font size=-1><a href=Vdatabase-search.tcl\">clear these databases</a></td> </tr> <tr> <td colspan=2><input type=radio name=databaseid-listboolean value=and checked> <b>Refine</b>:Search only through the databases listed above <em>or</em> <br><input type=radio name=databaseidlistboolean value=or> <b>Broaden</b>: Do another search and Union the results with the above databases </tr> </table> </td> <tr> <td colspan=3><br> </tr> set javascript <script language=javascript> function moveObject(direction,selectbox) I /direction = up or down //selectbox = nameof theselectbox = left or right selectedindex = document. theForm\[selectbox). sel ectedIndex; if (selected-index != -1) { oldText = document.theForm\[selectbox].options\[selectedindex].text; oldValue = document.theForm\[selectbox].options\[selectedjindex].value; if (selected-index != -1 \&\& oldValue != \"null") if (direction == \'up\") 69 // move table up if (selectedindex > 0) // the table was in the interior of a page, so moving up means swapping with the table above document.theForm\[selectbox].options\[selected_ index].text = document.theForm\[selectbox].options\[selectedindex-1].text; document. theForm\[selectbox].options\[selectedindex]. value = document.theForm\[selectbox].options\[selectedindex-I].value; document.theForm\[selectbox ].options\[selected index-1].text = oldText; document.theForm\[selectbox].options\[selected index-1].value = oldValue; document. theForm\[selectbox]. selectedlndex--; } else if (direction == \"down\") // move table down // calculate the index of the last element in the current page (needed to check for interior moves or moves to new pages) real-length = 0 x = \"continueV while (x == \"continue"){ if (document.theForm\[selectbox].options\[real-length].value==VnullV) x = \"stop\" reallength--; else { real_length++; if (selectedindex < real-length) // move within the page, so just swap values with the table below document.theForm\[selectbox].options\[selected-index].text = document.theForm\[selectbox].options\[selectedindex+1].text; document. theForm\[selectbox]. options\[sel ectedindex]. value = document. theForm\[selectbox].options\[selectedindex+ 1]. value; document.theForm\[selectbox].options\[selectedindex+I ].text = oldText; document.theForm\[selectbox].options\[selected_index+i ].value = oldValue; document. theForm\[selectbox]. selectedIn dex++; else // nothing was selected alert(\"Please select a element first.\"); return false; function slide(selectbox) // selectbox=nameofthetheselectbox if (selectbox == \"left\") { newselectbox = \'rightV; else { newselectbox = \"left\'; selectedindex = document. theForm\[selectbox]. selectedlndex; if (selected-index != -1) { oldText = document.theForm\[selectbox].options\[selected-index].text; 70 oldValue = document.theForm\[selectbox].options\[selected_index].value; else { alert(\"Please select a element first\"); return false; } if ( oldValue==V'nullV') { alert(\'Please select a element first\"); return false; real-length = 0 x = \"continue\" while ( x == \"continueV) // calculate the last entry in the destination page if (document.theForm\[newselectbox].options\[real-length]. value==Vnull\") x = \'stopV else t reallength++; / table to the bottom of other side of page document.theForm\[selectbox].options\[selected_ index].text = document.theForm\[newselectbox].options\[real-length].text; document.theForm\[selectbox].opti ons\[selectedindex]. value = document.theForm\[newselectbox].options\[real- ength ]. value; document.theForm\[newselectbox].options\[reallength]. text = oldText; document.theForm\[newselectbox].options\[realjlength].value = oldValue; // get the length of the originating page real-length = 1 x = \"continueV while ( x == \"continue\") { if (document.theForm\[selectbox].options\[reallength].value==\"null\") x = \"stopV } else { reallength++; } // shift everything below the moved element up one in the original selectbox counter = selected index while (counter < reallength) { oldText = document.theForm\[selectbox].options\[counter].text oldValue = document. theForm\[sel ectbox]. option s\[coun ter].value document. theForm\[selectbox].options\[counter].text = document.theForm\[selectbox].options\[counter+ l].text; document. theForm\[sel ectbox] .options\[counter]. value = document. theForm\[sel ectbox]. option s\[counter+ 1]. value; document.theForm\[selectboxj.options\[counter+ 1].text = oldText; document. theForm\[selectbox].options\[counter+ 1]. value = oldValue; counter++; return false; function doSubO { 71 // Loads the string of elements on a page into hidden variables left and right // These are used on the latter page for the update. document.theForm\[\"searchjinfo\"].value += '{'+doSubInfo(VrightV')+'}' document.theForm\[\"right-side\"I.value += '{'+doSubSide(VrightV')+'} return true; } function doSubSide(side) val = V\"; for (i=O;i<document.theForm\[side].length;i++) newval = document. theForm\[side].options\[i]. value; if (newval != \"nullV') { val += newval; val += \"V;;; } return val; function doSublnfo(side) val = \"\; for (i=0;i<document.theForm\[side].length;i++) newval = document.theForm\[side].options\[i] value; newtext = document.theForm\[side].options\[i] .text; if (newval != \"null\") val += newtext; val += \";;\"; return val; </script> nsdb releasehandle $db set contextbar [pnsearch context bar "Search Databases"] adreturnjtemplate 72 ## database-search.adp <pnheader-plain title="Search Databases" javascript=$javascript></pn headerplain> Use the right arrow <img src=images/right.gif> to select the field to search for.<br> Use the left arrow <img src=images/left.gif> to remove the field from the search list. <p> <form action =database-search-2.tcl method=post name=theForm> <input type=hidden name="searchinfo" value="" > <input type=hidden name="right-side" value=""> <%= [export-formvars databaseidlist] %> <table align=center bgcolor=cccccc width=85% cellspacing=O cellpadding=4 border=O> <%= $databaseidtext %> <tr> <td bgcolor=cccccc align=center valign=bottom><font color="blue"><b>Searchable Criteria</font></td> <td bgcolor=cccccc align=center valign=bottom>&nbsp;</td> <td bgcolor=cccccc align=center valign=bottom><font color="blue"><b>Search These Criteria</font></td> </tr> <tr> <td bgcolor=cccccc align=center valign=top><%= $left-select %></td> <td bgcolor=cccccc align=center valign=center> <table cellpadding=0 cellspacing=O border=O> <tr> <td align=center><a href="#" onClick="return slide('left')"><img src=images/right.gif border=O alt="Right"></a></td> <td>&nbsp;</td> <td>&nbsp;</td> <td>&nbsp;</td> <td>&nbsp;</td> <td align=center><a href="#" onClick="return slide('right')"><img src=images/left.gif border=O alt="Left"></a></td> </tr> </table> </td> <td bgcolor=cccccc align=center valign=top><%= $right-select %> </td> </tr> <tr> <td colspan=3> <table> <tr> <pn-radio prettyname="AND or OR" name="andor" value="and,and;or,or" defaultvalue="or" desc="match all the criteria above or any of the criteria"></pnradio> </tr> <tr> <td>&nbsp;</td> </tr> <tr> 73 <td colspan=2><br><br>In addition to the criteria above, you can further refine your search with these following parameters. (blank values will be ignored.)</td> </tr> <tr> <pnjinput pretty-name="Abstract Search" name="textsearch" size="30" desc="please separate the words with a space."></pn input> </tr> <tr> <pn-radio pretty-name="&nbsp;" name="search-option" value="exact match,exact;any of the words,any;all of the words,all" default_value="exact"></pnradio> </tr> <%= $modifiersearch %> </table> </td> </tr> </tr> <tr bgcolor=ffffff> <td colspan=3 align=center><input type=submit value="Search" onClick="return doSubo;"></td> </tr> </table> </center> </form> <pn-plain-footer></pn-plainfooter> 74 Part 7: Database Search Results The screenshot for this web page is Figure 6. ## database-search-2.tcl set db [ns-db gethandle] set field_id_values "" set fieldinfolist [databasetotcllist_list $db "select f.fieldid, f.modifier, f.description from pn fields f where f.modifier in ('number','range') and searchable-p = 't"I foreach fieldinfo $field-info list { set field id [lindex $fieldinfo 0] set modifier [lindex $fieldinfo 11 if {$modifier == "range" { append field_id_values "\{ ${ field-id }valuel \"\"\} \{ $ {field-id} value2 \"\"\} \{ $ {fieldid }_value3 \"\"\} \{${fieldjid}_value2 \"\"\} \"\"\} " else { append fieldidvalues "\{${field-id}_valuel ad-page-variables [subst {searchinfo right-side andor textsearch search-option $fieldidvalues {databaseid_listboolean "or"} {databaseidlist ""}}] #right side store the field ids set whereclause-listl [list] set text-search [DoubleApos [string trim $text-search]] set textsearchlist [split $text_search ""] if {! [empty-string-p $text-search] I { if {$search-option == "exact"} { lappend whereclausejlisti "upper(abstract) like upper('%${tex tsearch }%')" set abstracttext "<br>&nbsp;&nbsp; where the abstract contains exactly <font color=\"brown\">&quot;$text-search&quot;</font><br>&nbsp;&nbsp;" } else { set searchquerylist foreach text $text-search-list { lappend search-query-list "upper(abstract) like upper('%${ text}%')" if {$searchloption == "any"} I lappend whereclausejlistl [join $search-query-list " or "I set abstracttext "<br>&nbsp;&nbsp; where the abstract contains any of <font color=\"brown\">&quot;$text-search&quot;</font><br>&nbsp;&nbsp;" } else { lappend whereclausejlistl [join $search-query-list " and "] 75 set abstracttext "<br>&nbsp;&nbsp; where the abstract contains all of <font color=\"brown\">&quot;$text-search&quot;</font><br>&nbsp;&nbsp;" else set abstracttext if {$andor == "and"} { set andortext "all of' else { set andortext "any of' set searchinfolist [split [lindex $search-info 0] ";;"] set searchlist "<ul>" set search-count 0 foreach search $search-infolist { if {![empty-stringp [string trim $search]]}I incr search count append searchlist "<li><font color=\"brown\">$search</font>\n" } #the dynamically generated ranges and number fields foreach fieldinfo $field-infolist { set field id [lindex $fieldinfo 0] set modifier [lindex $field_info 1] set description [lindex $field-info 2] set fieldvaluel ${field-id}_valuel set fieldvalue2 ${ field-id}_value2 set fieldvalue3 ${field-id}_value3 if {$modifier == "range" I { set rangel [subst $$field-valuel] set range2 [subst $$fieldvalue2] set unit [subst $$field-value3] if {![emptystring-p $rangel] I incr searchcount append searchlist "<li><font color=\"brown\">$description $rangel to $range2 $unit</font>\n" field-id lappend whereclauselistl "databaseid in (select databaseid from pn-database fieldmap where = $fieldjid and pn-inrange('$rangel;$range2;$unit',extra-value) =T)" } else set relation [subst $$field-valuel] set value [subst $$field-value2] if { ![emptystring-p $value]} { 76 switch $relation "greater than" set sql-relation ">" I "less than" set sqL]relation "<" default set sq]-relation = } incr searchcount append searchlist "<li><font color=\"brown\">$description $relation $value</font>\n" lappend where clauselistl "databaseid in (select databaseid from pn-database fieldmap where field-id = $fieldid and tointeger(extra-value) $sql-relation $value)" if {$search_count == 0} append searchlist "<li><i>no criteria specified</i>\n" } append search-list "</ul>" #gathering the searchable fields set whereclause list2 [list] set fieldinfolist [split [lindex $rightside 0] ";;"] foreach fieldinfo $field-info-list { if { [empty-string-p [string trim $field-info]]} { set field-pair [split $fieldinfo if { [llength $field-pair] == 1} "~"] { lappend whereclauselist2 "databaseid in (select database_id from pn-database fieldmap where field-id = $field-info)" else { lappend whereclauselist2 "databaseid in (select database_id from pn-database fieldmap where field-id = [lindex $field-pair 0] and extravalue = '[lindex $field-pair 1]')" } if {![empty-string-p $whereclauselistl] && ![empty-string-p $where clauselist2]} { set whereclauses "[join $wheresclauselistl " and \n"] and \n([join $whereclausejlist2 } elseif {![empty-string-p $wheresclausejlistl]} { set whereclauses "[join $wheresclauselist 1 " and \n"]" I elseif {!fempty-string-p $where clausejlist2l} { set whereclauses "[join $where clause list2 " $andor \n"]" } else { set whereclauses } if {![empty-string-p $databaseidlist] { if { [empty-string-p $whereclauses] I 77 " $andor \n "])" set whereclauses "database_id in ([join $databaseidlist ","])" else { set whereclauses "($whereclauses) \n $databaseidlistboolean databaseid in ([join $databaseidlist "")" if {![emptystring-p $where-clauses]} set whereclauses "where $where clauses" } #search in database set sqlquery "select database-id, name from pn-databases $whereclauses set resultlist "<ul>" set resultcount 0 set databaseidlist [list] set selection [ns-db select $db $sql-queryl while {[nsdb getrow $db $selection]){ setvariables after.query incr resultcount append resultlist "<li><a href=\"database-view.tcl?[exporturlvars databaseid textsearch search-option]\" target=database>$name</a>\n" lappend databaseidlist $databaseid } nsdb releasehandle $db if {$result_count == 01 { append resultlist "<li><i>no records match</i>\n <p> <li>Click your browser's back button to broaden search criteria</a> } else { set databaseids $databaseidlist append result-list "<p> <li><a href=\"record-search.tcl?databasesvisible-p=t&[export urlvars database_ids]\">search records within these databases</a> <p> <li><a href=\"database-search. tc I ?[expor turl-vars databaseidlist]\">refine or broaden search criteria</a> } append resultlist </ul> set context-bar [pn search contextbar [list "database-search.tcl" "Database Search"] "Results"] adreturntemplate 78 ## database-search-2.adp <pn-headerplain title="Database Search Results"></pn-header-plain> <b>Database Search Criteria</b> <br> You are searching for physiologic databases <%= $abstract text %> with <font color="brown"><%= $andortext %></font>these criteria: <%= $search-list %> <p> <b>Search Results</b> <br> <%= $result-list %> <p> <br> <b>SQL Query</b> <br> <pre> <%= $sql-query %> </pre> <pn-plain-footer></pn-plainfooter> 79 Part 8: Search Records The screenshot for this web page is Figure 7. ## record-search.tcl ad-page-variables { {databaseids -multiple-list} {recorddbid-list ""I {recorddbidlistboolean "or" record-types -multiple-list} {recordertypes -multiple-list} {sex "M"} {agel "0"} {age2 "999999"} {ageoption "Y"} {duration1 "O"} {duration2 "999999"} {duration-option "minutes"} {diagnoses ""} {diagnoses-option "or"} {medications ""I {medications-option "or"} {symptoms ""} {symptoms-option "or" I {signal-types -multiple-list I {signalnames -multiple-list} {signal-numl "1" {signalnum2 "99"1 {annotationcategory-ids -multiple-list} {annotationdescription-ids -multiple-list} {databasesvisible-p "f'} {record-type-visible-p "f"I {recorder-type-visible-p "f"} {sexvisiblep "fl {subject-age-visible-p "Y" {durationvisible-p "f"} {diagnosesyvisibleqp "Y"} {medications-visible-p "Y" {symptoms-visiblep "f"} {signals-visible-p "f"} {annotations_visible-p "f" I set db [ns-db gethandle] set databasename id list [databaseto tcl list $db \ "select name ||',' databaseid from pndatabases"J set databasevalues "[join $databasename_idlist ";"J" set databasedefaultvalues [join $database-ids ";"] set recorddefault-types [join $record-types ";"] set record-type-list [databasetotcl-list $db \ "select distinct record_ type from pn-records"] set record-types [join $record_typelist ";"] set recorderdefault-types [join $recorder_types ";"I set recorder-type-list [databasetotcl_list $db \ "select distinct recorder-type from pn-records"] set recorder-types [join $recorder-typejlist ";"] 80 if {$signals-visible-p == "f" set signals-section else { set signal-default-types Uoin $signal-types ";"] set signal-defaultnames Uoin $signal-names ";"I set signal-typejlist [databasetotcl_list $db \ "select distinct signal-type from pn-signals"] foreach signal-type $signal-type-list { if {[string match "*$signal_type*" $signal-default-types]} { set checked-option "checked" set signalnamelist [databasetojtcljlist $db \ "select distinct signal-name from pn-signals where signal-type='[DoubleApos $signal_type]"'] set signalnamecheckboxes foreach signal-name $signal-namejlist { #have the value as a pair(name,type) in case the name # can be in multiple types set signalnamevalue "${signal-name}-${signal-type}" if {[string match "*$signal-namevalue*" $signal-default_names]} { set checked-option2 "checked" } else ( set checked-option2 append signal-namecheckboxes "&nbsp;&nbsp;&nbsp; <input type=checkbox name=signal-names value=\"$signal namevalue\" $checked-option2> $signal-name <br>\n" } #grab this dynamic variable value #get rid of the spaces since variable names cannot contain spaces regsub -all " "$signal-type "_"type ad-page variables [list [list "signal_${typenum" "1"]] set signaltype-display "${signal-type}: contains at least <input type=text name=signal_${ type} num value=\"[set signal${ type}Lnum]V size=\"2\"> $signal type signal(s) \n" } else { set checked-option set signalnamecheckboxes set signal-type-display $signal-type } append signalssection "<input type=checkbox name=signal-types value=\"$signal-type\" $checked-option onClick=ReloadFormo> $signal-type-display <br>\n $signaL-namecheckboxes" if {$annotations visible-p set annotations_section I else { == "f" 81 set category-idnamelist [databasejto_tcl list_list $db \ "select category-id, category-name from pnannotation-categories c where exists (select pn-annotations a where a.categoryjid = c.category-id)"] 1 from foreach categoryid-name $categoryidnamelist { set category-id [lindex $categoryidname 0] set category-name [lindex $categoryid-name 1] if { [lsearch -exact $annotation category-ids $categoryjid] >=O set checked-option "checked" { set description-idtypejlist [database-totcllistjlist $db \ "select description-id, type, description from pn-annotation-descriptions where category-id $categoryjid"3 = set annotationdescription-checkboxes foreach description-id-type $descriptionjid-typejlist { set description-id [lindex $descriptionidjtype 0] set type [lindex $descriptionjid-type 1] set description [lindex $description-id-type 2] if { [lsearch -exact $annotation-description-ids $description_id] >=O} I set checked-option2 "checked" adpage variables [list [list "annotation d_${descriptionid}_numi" "i'] [list "annotation-d_${ description-id}_num2" "999999"]] set type range "<br> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; contains between <input type=text name=annotationd-${ description id}num1 value=\"[set annotationd$ {description_id }numl ]\" size=\"7\"> to <input type=text name=annotationd_${ descriptionid }_num2 value=\"[set annotationd_${ description-id }num2]\" size=\"7\"> annotation(s) \n" } else { set checked-option2 set type-range } append annotation descriptioncheckboxes "&nbsp;&nbsp;&nbsp; <input type=checkbox name=annotation-description _ids value=\"$description_id\" onClick=ReloadFormo $checkedoption2> $type <em><font size=-1>- $description</font></em> $type-range <br>\n" I ad-page-variables [list [list "annotation${ category-id }numl """1"] [list "annotation_${categoryid}_num2" "999999"]] set annotationcategory-display "${category-name}: contains between <input type=text name=annotation_${category-id}_num1 value=V[set annotation_${category id }num1]\" size=\"7\"> to <input type=text name=annotation_${category-id}_num2 value=\"[set annotation_${ category-id }_num2]\" size=\"7\"> $category-name annotation(s) \n" } else { set checked-option "" set annotationdescription-checkboxes set annotationcategory-display $category-name 82 append annotationssection "<input type=checkbox name=annotationwcategoryjids value=V'$categoryid\" $checked-option onClick=ReloadFormo> $annotationscategory-display <br>\n $annotationdescriptionscheckboxes" } #if user wants to refine or broaden search if { [empty-string-p $recorddbidlist] set record db_id_text else { set recordids [databasetojtcljlist $db "select recordid from pnjrecords where recorddb_id in ([join $recorddb_id_list ","])"] set recorddbidlistbooleanand set record dbidlistbooleanor " set recorddbidlistbooleannot #check the default set recorddb_id_list _boolean_${recorddbidlistboolean} "checked" set recorddbidtext "<tr> <th align=\"left\"><font color=\"blue\">Record IDs</font></th> <td>[join $record-ids ", "] <font size=-1><a href=\"record-search.tcl\">clear these records</a></td> </tr> <tr> <td>&nbsp;<td><input type=radio name=recorddb_id_listboolean value=and $recorddbidlistbooleanand> <b>Refine</b>: Search only through the records listed above <em>or</em> <br><input type=radio name=recorddbidlistboolean value=or $recorddbidlistboolean or> <b>Broaden</b>: Do another search and Union the results with the above records <br><input type=radio name=recorddbidlistboolean value=not $recorddb_id_listbooleannot> <b>Complement</b>: Do another search and the results will not include the above records </tr> <tr> <td colspan=2><br> </tr> } nsdb releasehandle $db set contextbar [pn-search-contextbar "Search Records"] set script " <script language=javascript> var choicewin; function getchoices() var url = \"annotation-list\" var selectindex = document.theForm.categoryid. selectedlndex; 83 var selectvalue = document.theForm.category-id.options\[select-index\].value; if ( select-index > 0) { url = \"annotation-view.tcl?categoryjid=V + selectvalue; if (choice-win != null && !choicewin.closed) { choicewin.closeO; choicewin = window.open(url, 'choice','toolbar=yes,location=no,directories=no,status=no,scrollbars=yes,resizable=yes,copyhistory=no,wi dth=400,height=500', true); choicewin.focusO; } function ReloadFormo { document. theForm. action = \"record-search.tcl\"; document.theForm.submito; I </script> adreturntemplate 84 ## record-search.adp <pn-header-plain title="Search Records"></pn header-plain> Hints for this search page: <ul> <li>For the fields that you are concerned about, click the checkbox next to the field. To remove the criteria, click the checkbox again. <li>Do not restrict the search with too many criteria. Start out broadly since you will have the option to refine your search later. <li>For boxes with a list of options: to select an item, click the item once; to deselect an item, just click it again. </ul> <form action=record-search-2.tcl method=post name=theForm> <%= [exportformvars recorddb_id-list] %> <center> <table align=center bgcolor=cccccc width=95% cellspacing=O cellpadding=4 border=O> <%= $record dbidtext %> <tr> <th valign=top align="left"> <pn-checkbox-no-prettyname name= "databases-visible-p" value=" , defaultvalue="$databases-visible-p" script="onCl ick=ReloadFormo "></pn-checkbox-no-pretty-name> <font color= "blue">Databases</font> </th> <td><pn-select multiple-no-pretty-name name="databaseids" value=" $database-values" defaultvalue="$database-defaultvalues" size=5 visible-p="$databases-visible-p" visibletext="The selection of the physiologic databases." desc="Please select one or more physiologic databases. The search result will return records within the selected databases. "></pn-selectmultiple-no-pretty-name></td> </tr> <tr> <th valign=top align="left"> <pn-checkbox-no-pretty-name name= "record-type-vi sible-p" value=" ,t" defaultvalue="$record_typevisiblep" script="onClick=ReloadFormo"></pn-checkbox_no-pretty-name> <font color="blue">Record Type</font> </th> <td><pn-sel ect-multipl eno-pretty-name name="recordjtypes" value="$record-types" defaultvalue="$record-defaultjtypes" visible-p="$record type-visible-p" visible text="The type of physiologic record." desc="Please select one or more record types. The search result will return records that contain any of the selected types. "></pn select multiple-no-pretty-name></td> </tr> <tr> <th valign=top align="left"> 85 <pn-checkbox-no-prettyname name= "recordertype-visible-p" value=" ,t" defaultvalue="$recorderjtype-visiblep" script="onClick=ReloadFormo"></pn-checkbox-no-pretty-name> <font color="blue">Recorder Type</font> </th> <td><pn-select-multiple no-pretty-name name="recordertypes" value="$recorder-types" defaultvalue="$recorder default-types" visible p="$recorderjtype-visible-p" visibletext="The type of instrument that is used for the recording. Not all databases have the recorder type listed." desc="Please select one or more recorder types. The search result will return records that contain any of the selected types. "></pn-select-multiple-no-pretty-name></td> </tr> <tr> <th valign=top align="left"> <pncheckbox no-prettyname name="sexvisible-p" value=" ,t" default-value="$sex-visible-p" script= "onClick=ReloadFormo "></pncheckbox_no-pretty-name> <font color="blue">Sex</font> </th> <td><pn-select-no-prettyname name="sex" value="male,M;female,F" defaultvalue="$sex" visiblep="$sex visiblep" visibletext="The sex of the subject." desc="Please select the sex of the subject. "></pn-selectno-pretty-name></td> </tr> <tr> <th valign=top align="left"> <pn-checkbox-no-prettyname name="subject-agevisible-p" value=" ,t" defaultvalue="$subject-ageyvisible-p" script="onClick=ReloadFormo"></pncheckboxnopretty_name> <font color="blue">Subject Age</font> </th> <td><pnrangeno-prettyname namel="agel" valueI="$agel" name2="age2" value2="$age2" size=7 maxlength=10 visible-p="$subject-age-visible-p" desc="Please enter the age range of the subject. "></pnrangenoprettyname> <pn-select-no-pretty-name name= "age-opti on" value=" years,Y;month s,M;days,D" defaultvalue="$age-option" visible-p="$subject-age-visible-p" visiblejtext="The age range of the subject. "></pnselectnoprettyname> </td> </tr> <tr> <th valign=top align="left"> <pn-checkbox-no-prettyname name="duration-visible-p" value=" ,t" defaultvalue="$duration visiblep" script="onClick=ReloadFormo"></pn-checkbox-no-prettyname> <font color="blue">Record Duration</font> </th> <td><pn-range-no-prettyname name 1 ="duration 1" value 1="$duration 1" name2="duration2" value2="$duration2" size=7 maxlength=10 visiblep="$duration visible_p" desc="Please enter the range for the duration of the records."></pnrangenoprettyname> <pn-select-no-pretty-name name="durationoption" value="seconds;minutes;hours" default_ value="$duration-option" visible-p="$duration visible-p" visibletext="The duration/length of the record. "></pnselectnoprettyname> </td> </tr> 86 <tr> <th valign=top align="left"> <pncheckboxnoprettyname name="diagnoses-visible-p" value=" ,t" defaultvalue="$diagnoses.visible-p" script="onClick=ReloadFormo"></pn-checkboxnnopretty-name> <font color="blue">Diagnoses</font> </th> <td><pn-input-no-pretty-name type="text" name="diagnoses" value=" $diagnoses" size=30 visibleIp="$diagnoses-visible-p" desc="Please separate each diagnosis with a semicolon. "></pn-input-no-pretty-name> <pn-radio-no-pretty-name name= "diagnoses-option" value=" contains any,or;contains all,and" defaultvalue="$diagnoses-option" visiblep="$diagnoses-visible-p" visibletext="The diagnoses of the subject. "></pn radio-no-pretty-name> </td> </tr> <tr> <th valign=top align="left"> <pn-checkbox-no-prettyname name= "medications_visible-p" value=" , defaultvalue="$medications visible _p" script="onClick=ReloadForm()"></pn-checkbox-no-pretty-name> <font color= "blue">Medications</font> </th> <td><pn-input-no-pretty-name type="text" name="medications" value="$medications" size=30 visiblep=" $medications_visible-p" desc="Please separate each medication with a semicolon. "></pn-inputnno-pretty-name> <pn-radiono-pretty-name name="medications-option" value= "contains any,or;contains all,and" default value="$medications-option" visible.p='$medications.visible p" visibletext="The medications taken by the subject. "></pn radionopretty-name> </td> </tr> <tr> <th valign=top align="left"> <pn-checkbox-no-prettyname name= "symptoms-visible.p" value=" ,t" defaultvalue="$symptomsvisible-p" script="onClick=ReloadForm"></pn-checkbox-no-pretty-name> <font color="blue">Symptoms</font> </th> <td><pn-input-no-pretty-name type="text" name="symptoms" value=" $symptoms" size=30 visible-p="$symptoms-visible-p" desc="Please separate each symptom with a semicolon. "></pnj nputnoprettyname> <pn-radioqno-pretty-name name="symptoms-option" value="contains any,or;contains all,and" defaultvalue="$symptoms-option" visible.p="$symptoms.visible-p" visibletext="The symptoms of the subject."></pn radioqno pretty-name> </td> </tr> <tr> <th valign=top align="left"> <pn-checkbox-no-prettyname name=" signals-vi sible-p" value=" ,t" defaultvalue="$signals-visiblep" script="onClick=ReloadFormo"></pn checkbox no pretty name> <font color="blue">Signals</font> </th> <td><pn-staticjtextnopretty-name visibletext="The signals types contained in the records." visiblep="$signals-visible-p" desc="Please first enter the number of total signals. Then select the signal 87 category. Once you selected a category, please specify the minimum number of signals for that category. If you wish, you may specify the signal names which the record must contain. If you select more than one signal category, we will return the records that contain all the selected categories."></pn-statictext-no-prettyname> <p> <pn-static-text-no pretty-name visible-p="$signals visible p" value="Contains between "></pn static textno-pretty-name> <pn-range-no-pretty-name name 1 ="signal-num 1" valuel="$signalnum1" name2="signalnum2" value2="$signalnum2" size=2 maxlength=10 visible p="$signals visible-p"></pn-range-no-pretty-name> <pn-static-text-no-pretty-name visible-p="$signals visiblep" value="total signals"></pnstatic textno_prettyname> <p> <%= $signals_section %> </td> </tr> <tr> <th valign=top align="left"> <pn-checkbox-no-prettyname name="annotations visible-p" value=" ,t" defaultvalue="$annotations visible-p" script="onClick=ReloadFormo"></pnscheckbox-no-pretty-name> <font color="blue">Annotations</font> </th> <td><pn-static-text-no-pretty-name visibletext="The annotations category and beat/episode types contained in the records." visible-p="$annotations visible-p" desc="Please select the annotation category. Once you selected a category, please specify the minimum number of annotations for that category. If you wish, you may specify the annotation beat/episode types which the record must contain. If you select more than one annotation category, we will return the records that contain all the selected categories. "></pn statictext-no-prettyname> <p> <%= $annotations_section %> </td> </tr> <tr> <td colspan=2 align=center><input type=submit value="Search"></td> </tr> </table> </center> </form> <pn-plainjfooter></pn-plainfooter> 88 Part 9: Record Search Results The screenshot for this web page is Figure 11. ## record-search-2.tcl ad-pageyvariables { {databaseids -multiple-list} {recorddb_id-list ""} {recorddb_id_listboolean "or" I {record-types -multiple-list} {recorderjtypes -multiple-list} {sex "MI {agel "0"} {age2 "999999"1 {age-option "Y"} {durationl "0"1 {duration2 "999999"} {durationoption "minutes"} {diagnoses "" I {diagnoses-option "or" I {medications ""I {medications-option "or"} {symptoms ""} {symptoms-option "or"} {signal-types -multiple-list I Isignal-names -multiple-list} {signal-numl "1" I{signalnum2 "99"1 {annotation category-ids -multiple-list} {annotationdescriptionjids -multiple-list} {databasesvisible-p "f} {record-type-visible-p "f"} {recorder-type-visible-p "f} {sexvisible_p "f} {subject-ageyvisible-p "f"} {duration visible-p "f"} {diagnoses-visible-p "f"} {medications-visible-p "f} {symptomsyvisible-p "f} {signals-visible-p "f} Iannotationsvisible-p "f} } set agel [string trim $agell set age2 [string trim $age2] set duration 1 [string trim $duration I] set duration2 [string trim $duration2] set medications [string trim $medications] set medications [string trim $medications ";"] set symptoms [string trim $symptoms] set symptoms [string trim $symptoms ";"] set diagnoses [string trim $diagnoses] set diagnoses [string trim $diagnoses ";"I set signal-numi [string trim $signal-numl] set signal-num2 [string trim $signal-num2] set db [ns-db gethandlel set exception-text set exception-count 0 89 if { $exception count > 0 adreturncomplaint $exception-count $exceptiontext return set whereclause-listl [list] set searchlist "<ul>" if {$databasesvisible-p == "f' 1 [empty-string-p $databaseids] 01 { lappend whereclauselistl "1=1" append search-list "<p><li>located in any databases \n" [lsearch -exact $databaseids "any"] >= } elseif {![empty-string-p $databaseids]} { lappend whereclauselistl "database-id in (Ujoin $databasejids ","])" append search-list "<p><li>located in these databases: [join [databasetotcljlist $db "select name from pn-databases where database-id in ([join $databasejIds ","])"1 ", "] \n" if {$record-type-visible-p == "t" && ![empty-string-p $recordjtypes] { lappend whereclauselistl "record_type in ('[join [DoubleApos $recordjtypes] "','"]')" append search-list "<p><li>record type: [join $recordjtypes ", "] \n" if {$recordertype-visiblep == "t"&& ![empty-string-p $recorderjtypes]} lappend whereclauselistl "recorder-type in ('Uoin [DoubleApos $recorder_types] ""]')" append search-list "<p><li>recorder type: [join $recorderjtypes ", "] \n" } if {$sex-visible-p == "t" && ![empty string-p $sex] && $sex != "any" { lappend whereclauselistl "subject-dbid in (select subject-dbid from pn-subjects where sex = '$sex')" append search-list "<p><li>sex: $sex \n" } if {$subject-age-visible-p == "t"} { # # if {$agel =="0"}{ set agel #1} # # if {$age2 =="999999"} { set age2 #1} if {![empty string-p $agell && ![empty-string-p $age2]} { lappend whereclausejlisti "pnjin-range(age 11';' 11age-unit,'$agel;$age2;$age-option') =T" 90 append searchlist "<p><li>age between $agel and $age2 $age-option\n" } elseif {![empty-string-p $agel]} { lappend whereclausejlistl "pnjinrange(age |I';' age-unit,'$agel;9999999999;$age-option') append searchjlist "<p><li>age greater than or equal $agel $age-option\n" elseif {![empty-string-p $age2j} { lappend whereclausejlistl "pnin-range(age |I';' age-unit,'O;$age2;$age-option') =T' append search-list "<p><li>age less than or equal $age2 $age-option\n" =T" } if {$durationvisible-p == "t"} { if {$durationl == "0"1 set duration 1 } if {$duration2 == "999999"} { set duration2 } # # # # # # if {![empty string-p $durationl] && ![empty string-p $duration2]} { lappend where clauselistl "pnin-range(durationtoseconds(duration) | ';seconds','$duration1;$duration2;$durationoption') = T" append searchlist "<p><li>duration between $durationl and $duration2 $durationoption\n" elseif {![empty-string-p $duration1]} { lappend whereclausejlistl "pn-in-range(durationtoseconds(duration) | ;seconds','$duration1;9999999999;$duration-option') =T" append search-list "<p><li>duration greater than or equal $durationl $duration-option\n" } elseif {![empty-string-p $duration2]} { lappend whereclausejlistl "pnjin-range(duration toseconds(duration) j ;seconds','O;$duration2;$duration option') =T" append search-list "<p><li>duration less than or equal $duration2 $duration-option\n" } if {$diagnosesyvisible-p == "t"I if {![empty-string-p $diagnoses]} { set diagnosis-list [split $diagnoses ";"] set diagnosis search_list [list] foreach diagnosis $diagnosisjlist set diagnosis [string trim $diagnosis] if {![empty-string-p $diagnosis]} { lappend diagnosissearchlist "(select count(*) from pnjrecord-diagnosis-map m-map where m_map.recorddbid = r.recorddbid and upper(m-map.diagnosis) like upper('%[DoubleApos $diagnosis]%')) > 0" lappend whereclauselistl "([join $diagnosissearchlist " $diagnoses-option "])" append search-list "<p><li>diagnoses: [join $diagnosisjfist ", $diagnosesoption "] \n" i if {$signalsvisiblep == "t" }{ 91 if {$signal_num # == "0"1 # set signal-numi" # # } if {$signal num2 == "99" # # set signal-num2 } if {![empty-string-p $signal numll && ![empty stringp $signaLnum2l} I lappend whereclauselistl "((select count(*) from pnsignals s where s.recorddbid = r.record db id) >= $signal numl and (select count(*) from pn-signals s where s.recorddbid = r.record-dbjid) <= $signal-num2)" append search-list "<p><li>Number of Signals between $signalbnuml and $signalnum2\n" I elseif { ![empty-string-p $signal-numl]} { lappend where _clausejistl "(select count(*) from pnsignals s where s.recorddb id = r.recorddb-id) >= $signal-num1" append searchlist "<p><li>Number of Signals greater than or equal $signal-numl\n" } elseif {![empty-string-p $signalnum2]} { lappend whereclauselistl "(select count(*) from pn-signals s where s.recorddb id = r.recorddbjid) <= $signalnum2" append search-list "<p><li>Number of Signals less than or equal $signal-num2\n" } append searchlist "<ul>\n" foreach signal-type $signal-types { regsub -all " " $signaltype "_" type ad-page-variables [list [list "signal_${type}_num" "1"]] lappend whereclauselistl "(select count(*) from pnrsignals s where s.record dbid = r.recorddbid and signal-type = '[DoubleApos $signal-type]') >= [set signal_${type}_num]" append searchlist "<li>At least [set signal_${type}bnum] $signal-type\n" } foreach signal-name $signal-names { set nametype [split $signal-name "-"] set name [lindex $namejtype 0] set type [lindex $name_type 1] lappend whereclauselistl "exists (select 1 from pn-signals s where s.recorddbid = r.recorddbid and signal-type = '[DoubleApos $type]' and signal name ='[DoubleApos $name]')" append searchlist "<li>Must contain $name (Type: $type)\n" } append searchlist "</ul>\n" I if {$medications.visible p == "t"} if {![empty-string-p $medications]} { set medication-list [split $medications ";"] set medicationsearch list [list] foreach medication $medicationlist set medication [string trim $medication] if { ![empty-string-p $medication] I{ lappend medicationsearchlist "(select count(*) from pn-record medication-map m map where m_map.recorddb_id = r.recorddbid and upper(m-map.medication) like upper('%[DoubleApos $medication]%')) > 0" 92 lappend where clausej1istl "([join $medicationsearchlist " $medications-option "])" append search-list "<p><li>medications: [join $medicationjlist ", $medicationsoption "] \n" if {$symptoms-visible-p == "t"} { if {![empty-string-p $symptoms]} { set symptom-list [split $symptoms ";"] set symptom searchlist [list] foreach symptom $symptomjlist { set symptom [string trim $symptom] if {! [empty-string-p $symptom]} { lappend symptomsearchlist "(select count(*) from pn-record-symptom-map s-map where s_map.recorddbid = r.recorddb_id and upper(s-map.symptom) like upper('%[DoubleApos $symptom]%')) > 0" I lappend whereclausejlistl "([join $symptom-searchlist " $symptoms-option "])" append search-list "<p><li>symptoms: [join $symptomjlist ", $symptoms-option "] \n" I if { $annotations visible-p # # == "t"} { if {$annotation numi == "0"1 { set annotationnum " # } # # if {$annotation num2 set annotationnum2 # } == "99"} { foreach category-id $annotationcategory-ids { set category-name [database totcl-string $db "select category-name from pnannotationcategories where category-id = $category-id"] ad-page variables [list [list "annotation_${categoryidlnum1" "1"] [list "annotation-${ categoryjid }_num2" "999999"]] lappend whereclauselistl "exists (select 1 from pn-annotations a where a.recorddbid r.recorddbid and category-id = $categoryjid and number between [set annotation_${ categoryid }_numl] and [set annotation_${ category-id _num2])" append searchlist "<li>Between [set annotation_${categoryid}_numl] and [set annotation_${ category-id Inum2] $category-name annotations\n" append searchlist "<ul>\n" 93 = #group the description-id by categories foreach descriptionid $annotation description-ids { #check if this description is in this current category set type [database tojtcl string-or-null $db "select type from pn-annotation-descriptions where category-id = $categoryjid and description-id = $descriptionjid"] if { ![emptystring-p $type]} ad-page-variables [list [list "annotation-d_${description id}numl" "I"] [list 'annotation_d_${description id} num2" "999999"]] lappend whereclauselistl "exists (select 1 from pn-annotations-type-map a-t-map, pn-annotations a where a.recorddbid = r.recorddbid and a-t-map.annotationid = a.annotationid and a.category-id = $categoryjid and a_ tmap.type ='[DoubleApos $type]' and (a t map.number between [set annotationd ${descriptionid }numl] and [set annotation_d_${descriptionid }_num2] or a t map.episodes between [set annotation-d_${descriptionjid}lnuml] and [set annotation_ d${ description-id }num2]))" append searchlist "<li>Between [set annotation_d_${descriptionj d}_numl] and [set annotation-d_${descriptionid}_num2] of type $type\n" } } append searchlist "</ul>\n" } } set whereclauses Uoin $whereclauselistl " and \n"] if {![empty-string-p $recorddbidlist]} { if {$record-db_id_listboolean == "or" { set recordmodifier "in addition to" elseif {$recorddbidlistboolean == "and"} I set recordmodifier "within" else { set recordmodifier "complement of' append search_.list "<p><li>located $record-modifier records: [join [databasetotcllist $db "select recordid from pnrecords where recorddbid in ([join $record_db_id_list ","])"J ", "] \n" if { [empty-string-p $whereclauses]} { set whereclauses "recorddbid in ([join $recorddbidjlist ","])" } else { if {$recorddb_id_listboolean == "not" { set whereclauses "($where clauses) \n and recorddb-id not in ([join $recorddbidlist ",")" } else { set whereclauses "($where-clauses) \n $recorddb_id_list boolean recorddbjid in ([join $recorddb_id_list ","])" } } 94 } append searchlist "</ul>" #search in database set sql-query "select record-db-id, record-id from pn-records r where $where-clauses set resultlist "<ul>" set resultcount 0 set recorddbidlist [list] set selection [ns db select $db $sql-query] while { [ns_db getrow $db $selection]} setvariables-after-query incr resultcount append result-list "<li><a href=\"record-view.tcl?[exporturl_vars record-dbid]\" target=record>$recordid</a>\n" lappend recorddbidlist $record-dbid } nsdb releasehandle $db if {$resultcount == 0} { append resultlist "<li><i>no records match</i>\n" } elseif {$resultcount == 1} { append result-list "<p><li>1 record found\n" } else { append result-list "<p><li>$result-count records found\n" } append resultlist" <p> <li><a href=\"record-search.tcl?[exporturl_vars recorddbidlist databasesvisible-p database ids record-types recorder-types sex age<1 age2 age-option duration1 duration2 durationoption diagnoses diagnoses-option medications medications-option symptoms symptoms-option category-id category-num1 category-num2 category-types categorytypes-option recordtype-visible-p recorder-type-visible-p sex visible-p subject-age-visible-p durationvisible-p diagnoses-visible-p medications_visiblep symptoms-visible-p signal-num1 signal-num2 signal-types signal-names signals visiblep annotationscategory-ids annotation-descriptionids annotations_visible-p]\">refine or broaden search criteria</a> </ul> set contextbar [pn-searchcontextbar [list "record-search.tcl" "Record Search"] "Results"] 95 ## record-search-2.adp <pn-header-plain title="Record Search Results"></pnheaderplain> <b>Record Search Criteria</b> <br> You are searching for physiologic records with these criteria: <%= $search-list %> <p> <b>Search Results</b> <br> <%= $result-list %> <p> <br> <b>SQL Query</b> <br> <form method=post action="record-sql-query.tcl"> <textarea name="sql-query" wrap=soft rows=20 cols=100> <%= $sql-query %> </textarea> <p><center><input type=submit value="Update SQL Query"></center> </form> <pn-plainfooter></pn-plainfooter> 96