Creating, manipulating and deleting MMBase objects with the Transaction Handler updated 11 july 2001 John Balder Rob Vermeulen Introduction The basic concept behind MMBase is the idea of a Persistent Cloud of Objects (PCO). These objects come in different flavors, often referred to as types. (The concept builders in MMBase terminology is used for the tools that maintain objects of a certain type. Also is it used for the xml definition files that define a certain type.) Standard types for a variety of (multi media) objects are provided as well as the possibility to define customized object types. Furthermore, binary relations between objects can be defined. Most interaction with an actual MMBase system is web (i.e. browser) based. The authoring is done with a default web interface that is hierarchically structured. Objects can be created with custom parameter values and parameterized queries allow for search. This interface is sufficient but not always efficient. Short cuts and bulk input are not possible. Also there is a need within the community for customized authoring interfaces. The main problem with such interfaces is to get a bulk of information from a web page (filled in by a human author) to the MMBase engine. It's essential to do this efficiently because handling this per item would be too costly. Things started because Rico Jansen of VPRO made an interesting building block that could be used. This is based on the idea to create a temporary MMBase structure consisting of one of more objects with relations. We will call this a Temporary Cloud of Objects (TCO). A TCO can be "committed", which means that all objects from the TCO are copied to the persistent MMBase cloud (PCO). The TCO is created and manipulated by a user via input created through HTML forms. This approach allows for easier user interaction that also may span several HTML pages. It is imaginable to carry out all kinds of checks before committing this temporary structure. This may allow for semantic checking of the TCO. 2/17/16 pag. 1 van 12 The process In order to use this mechanism the developer needs to design at least two (S)HTML pages. One or more pages contain HTML FORM elements to collect the necessary information from the user and store it in variables. (There are several mechanism one can use here, for example: MMbase SESSION-variables, posting variables, javascript etc.) Another page contains the "transaction code" that describes the creation of the TCO with its instantiated objects and relations. This code contains the variables that have been bound previously. After substitution of these variables a special module evaluates the code: the TransactionHandler. The design So far, we have introduced a static metaphor of clouds of (typed) objects. One cloud is persistent and generally accessible, other clouds are temporary and private (i.e. only accessible by one user). Furthermore temporary clouds can be "committed" to the permanent cloud, i.e. there content is copied into the persistent cloud. It will be clear that we will need a more precise model in order to be able to understand (and implement) this. First of all we need to define what operations we need to create and manipulate temporary clouds. Furthermore we need to specify what committing means formally. Borrowing from database theory and guided by the structure of the current MMBase system we came up with the following definition. All interaction with the permanent MMBase cloud is done through transactions. A transaction is a process context that allows for creation, manipulation and committing a temporary cloud of objects. More specific: one transaction is related to exactly one TCO. Below we define a minimal set of operations. Note that having done so we are able to define more application-oriented operations on top of theses, when needed. We will need the following basic operations to handle transactions: a create transaction operator a destroy transaction operator a commit transaction operator Once we have a transaction, and thus a temporary cloud, we want to be able to manipulate objects (nodes in MMBase terminology). Thus we need the following basic operations: create object in the TCO 2/17/16 pag. 2 van 12 delete object from the TCO Note that we choose not to define the equivalent of the commit operator. The create operator implicitly adds the object to the TCO. Finally we will need an operator to manipulate (data)fields in these objects. We can do this with one operation: set field of object to value This basic set of operations allows us to use all functionality. So far, we did not yet discuss the lifetime of transactions. A transaction exists from the moment of its creation until it is committed (or destroyed). From practical experience we decided that one should be able to leave a transaction and enter it again at a later time, in order to do other things. The same should be possible for objects. Thus we will also need the following operators: open transaction open object The proposed mechanism allows the developer to handle more then one transaction and to access transactions on different .shtml pages. For reasons of convenience we defined an operator to access objects from the persistent cloud. It copies an object from the permanent cloud to the current temporary cloud: access object Note, that this operator does not add any functionality as it can be defined in terms of the create object and set object field operators. Next we have to define which (minimal set of) parameters we need with each operator. This is largely a pragmatic process where we take into consideration the functional possibilities of the existing system and the requirements that follow from the chosen metaphor. We came up with the following (initial) set of parameters. transaction operators and their parameters parameters meaning 2/17/16 comment pag. 3 van 12 create id ID of the transaction for later reference commit if true, commit at end of context time in sec. after which transaction is destroyed ID of previously defined transaction ID of previously defined transaction ID of previously defined transaction if true, commit at end of context timeOut delete id commit id open id commit object operators and their parameters parameters meaning createObject id ID of the object for later reference type type of the object to be created ID of previously defined object ID of previously defined object deleteObject id openObject id accessObject id ID of the object for later reference mmbaseId mmbaseId (number) of the object to be copied mmbaseId (number) of the object to be permanently deleted if true: remove object with attached relations type of the relation to be created ID of an object ID of an object markObject Delete createRelation mmbaseId remove Relations type source destination may be omitted when no further reference is needed default: true default: 60 sec required required required default: true comment may be omitted when no further reference is needed required required required may be omitted when no further reference is needed required required required required required When the id of a transaction or object is omitted at creation (access), the system will create one. This id is not available outside this operation. Note that we added two operations for objects not yet defined. 1. As all operations work on the temporary cloud, the delete operator thus deletes temporary objects. If we want an operator for permanent deletion, we need another name for that operator. As this is an operation that will be effectuated later in time, which is after the commit of the transaction, we called it markForDelete. If the removeRelations parameter 2/17/16 pag. 4 van 12 is not true an object that has relations attached to it will not be deleted. If the parameter is set to true the object and its attached relations will be deleted. 2. As the position of relations within MMBase is being reconsidered we have not yet decided on the form in which the transaction manager will handle them. For the time being we defined one operation to easily create relations. Note that the designer is responsible that the objects to be related are of the right type. That means consistent with the definitions within MMbase. This operation is for testing only and subject to change depending on the development of relations. Integrity of Persistent Cloud of Objects With the introduction of the accessObject and markObjectDelete operators we introduced a situation that needs consideration. One problem arises when we access (and thus copy) an object and later on commit the object to the PCO. Somebody else might have changed the object and then it is no longer clear what the next state of the object should be. One solution is to ignore this and to commit anyway. This may lead to losing information. Another way to solve this is locking an object whenever someone accesses it. This lock is removed when the object is committed again. Clearly we will want a timeout on this lock to avoid deadlock situations when no commit is done. Currently we implemented a timeout of 60 seconds. Note that this means that the designer will need to handle more cases where transactions fail. Syntax We will now present the complete syntax for our language. We choose an XML-compliant "dialect". Also, because this fits in nicely with our idea of contexts. Remember that our code is embedded in some other code, be it SCAN or some other language. So first we need some tag to indicate to we are going to work with transactions. <transactions [ exceptionPage="ex-page-def" ] [ key = "password"> </transactions> Note, all symbols are part of the definition except "[" and "]" which denote optional elements. The parameter exceptionPage specifies a (s)html page that is shown whenever an error occurs handling the transactions. This can either be a syntax error or it can be an error resulting from an erroneous operation. Also several variables are set to 2/17/16 pag. 5 van 12 provide information about the error. See Appendix C for a list of these variables. The key parameter is used to access servers that require a password for transactions. This facility gives extra security. See appendix D for an explanation of the parameters that need to be set in the TransactionManager module definition. Within this transactions context one can specify zero or more transaction contexts. A transaction context is one of the following. <create [ id ="id"] [ commit="trueFalse" ] [ timeout="number"] > </create> <open id="id" [ commit="trueFalse" ] > </open> In these contexts one can define objects. Furthermore we have to other operators that define an empty context, i.e. where we cannot manipulate objects, which only affect the transaction proper. <commit id="id"/> <delete id="id"> The object contexts have the following syntax. <createObject type="MMbase-type" [ id ="id"] > </creatObject> <accessObject mmbaseId="mmbaseId"[ id ="id"] > </accessObject> <openObject id ="id" > </openObject> Within the above object contexts one can set the fields of this objects. This has the following syntax. <setField name="name-of-field"> field-value </setField> Finally we have two object contexts that are empty and only used for there effect. <deleteObject id="id"\> <markObjectDelete mmbaseId="mmbaseId"\> 2/17/16 pag. 6 van 12 This is the complete syntax of the transaction language. In the next section we also provide the dtd definition. 2/17/16 pag. 7 van 12 Appendix A. dtd definition of the language <?xml encoding="UTF-8"?> <!ELEMENT transactions (create | open | commit | delete)* > <!ATTLIST transactions exceptionPage CDATA #IMPLIED> <!ATTLIST transactions key CDATA #IMPLIED> <!ELEMENT create (createObject | createRelation | openObject | accessObject | deleteObject | markObjectDelete)* > <!ATTLIST create id CDATA #IMPLIED> <!ATTLIST create commit (true | false) "true"> <!ATTLIST create timeOut CDATA "60"> <!ELEMENT open deleteObject | <!ATTLIST open <!ATTLIST open (createObject | createRelation | openObject | accessObject | markObjectDelete)* > id CDATA #REQUIRED> commit (true | false) "true"> <!ELEMENT commit (EMPTY) > <!ATTLIST commit id CDATA #REQUIRED> <!ELEMENT delete (EMPTY) > <!ATTLIST delete id CDATA #REQUIRED> <!ELEMENT createObject (setField*)> <!ATTLIST createObject id CDATA #IMPLIED> <!ATTLIST createObject type CDATA #REQUIRED> <!ELEMENT <!ATTLIST <!ATTLIST <!ATTLIST <!ATTLIST createRelation createRelation createRelation createRelation createRelation (setField*)> id CDATA #IMPLIED> type CDATA #REQUIRED> source CDATA #REQUIRED> destination CDATA #REQUIRED> <!ELEMENT openObject (setField*)> <!ATTLIST openObject id CDATA #REQUIRED> <!ELEMENT deleteObject (EMPTY) > <!ATTLIST deleteObject id CDATA #REQUIRED> <!ELEMENT accessObject (setField*)> <!ATTLIST accessObject mmbaseId CDATA #REQUIRED> <!ATTLIST accessObject id CDATA #IMPLIED> <!ELEMENT markObjectDelete (EMPTY)> <!ATTLIST markObjectDelete mmbaseId CDATA #REQUIRED> <!ATTLIST markObjectDelete deleteRelations (true | false) "false"> <!ELEMENT setField #PCDATA > <!ATTLIST setField name CDATA #REQUIRED> <!ATTLIST setField url CDATA #IMPLIED> 2/17/16 pag. 8 van 12 Appendix B: an Example We describe a simple example here. It adds a person object to the MMBase (persistent) cloud of objects (PCO). First we need a page that collects information from the user. Aside from layout information this page contains a form that collects the input. <FORM ACTION="execute1.shtml" METHOD="POST" ENCTYPE="x-www-formurlencoded"> <INPUT NAME="SESSION-FIRSTNAME" TYPE="text" SIZE="20"> <INPUT NAME="SESSION-NAME" TYPE="text" SIZE="20"> <INPUT NAME="SESSION-EMAIL" TYPE="text" SIZE="20"> <INPUT NAME="name" TYPE="submit" VALUE="Submit"> <INPUT NAME="name" TYPE="reset" VALUE="Reset"> </FORM> Note that the target of the post operation is another shtml page. Also note that the names of the input fields all begin with SESSION-, thus marking them as special MMBase variables. The second page contains the following code. <transactions> <create> <createObject type="people"> <setField name="firstname">$SESSION-FIRSTNAME^</setField> <setField name="lastname">$SESSION-NAME^</setField> <setField name="email">$SESSION-EMAIL^</setField> </createObject> </create> </transactions> The different contexts are clearly visible here. Note that, as a default, the transaction is committed when closed. Also note that no id was specified for the transaction or for the object. This was not necessary, as we don't refer to them later on. 2/17/16 pag. 9 van 12 Appendix C. Variables set by error handling SESSION-TRANSACTION-OPERATOR Value is: one of create | open | commit | delete SESSION-TRANSACTION-ID Value is: transaction id SESSION-OBJECT-OPERATOR Value is: one of createObject | openObject | accessObject| deleteObject | markObjectDelete | createRelation SESSION-OBJECT-ID Value is: object id SESSION-FIELD-OPERATOR Value is: setField SESSION-FIELD-NAME Value is: specified name of field SESSION-TRANSACTION-ERROR Value is: description of actual error as defined in the following table. Transaction operator create delete commit open Object operator createObject deleteObject accessObject markObjectDelete 2/17/16 SESSION-TRANSACTION-ERROR id already exists timeout parameter not numeric no id or unknown id no id or unknown id no id or unknown id SESSION-TRANSACTION-ERROR id already exists type does not exist no id or unknown id id already exists no mmbaseId or unknown mmbaseId object can not be accessed because of locking object can not be committed accessed because of locking no mmbaseId or unknown mmbaseId object can not be deleted because of connected relations pag. 10 van 12 createRelation Field operator setField 2/17/16 object can not be deleted because of locking type does not exist no, or unknown source id no, or unknown destination id SESSION-TRANSACTION-ERROR wrong field name wrong field value, eg. wrong type pag. 11 van 12 Appendix D. Security Issues One potential danger of transactions is that malicious users can input their own transactions into MMBase. For instance when they input transaction code into a FORM parameter that is later displayed in an HTML page. To circumvent this problem the installer of MMBase can define a transaction key for a particular server. Every transaction then also needs this key to be executed. If no server key is defined all transactions will be handled. Note that, although the secret key thus appears in many shmtl pages, it will not be visible. This is because all transaction code is "eaten" by the server. To facilitate the transition from free to keyed transactions the security mechanism has two modes. In "signal" mode all invalid transactions are reported in the log-file. In "secure" mode an invalid transaction is aborted and a transaction error is generated. Below is part of a transactionmanager.xml file. It sets a key and sets the mode to "signal". <properties> <property name="keycode">3658s2P7</property> <property name="security">signal</property> </properties> 2/17/16 pag. 12 van 12