Microsoft Dynamics AX 2012 ® History and versioning pattern, including PO uptake for change orders Concept Paper This paper describes a pattern for document versioning and change management introduced in Microsoft Dynamics AX 2012. It also describes the pattern’s application to a number of documents – the purchase order, product receipt, and packing slip. September 2011 CCAX2012IC0049 Table of Contents Overview..................................................................................................... 3 Functionality enabled by the versioning pattern ......................................... 3 Purchase orders .................................................................................................................... 3 Earlier versions of Microsoft Dynamics AX ............................................................................. 3 Microsoft Dynamics AX 2012 PO-specific conditions ................................................................ 3 Product receipts and packing slips ........................................................................................... 4 Earlier versions of Microsoft Dynamics AX ............................................................................. 4 Microsoft Dynamics AX 2012 specific conditions ..................................................................... 4 Versioning pattern data model .................................................................... 5 Assumptions ..................................................................................................................... 5 Abstract document versioning data model ............................................................................. 5 Physical data model considerations ...................................................................................... 6 Purchase order data model..................................................................................................... 7 Packing slip data model ......................................................................................................... 8 Inventory integration and accounting implications.................................................................. 9 Version comparison ............................................................................................................... 9 Document state management ................................................................... 10 Purchase order state management .........................................................................................11 Product receipt and packing slip state management .................................................................12 2 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS Overview Many of the documents stored by an ERP system in a relational database are not static. They can be changed or corrected by users based on changes in the environment they describe or when errors are found. Microsoft Dynamics® AX 2012 introduces a pattern for how modifications to such documents can be effectively managed and efficiently represented in the database. This document explains how the pattern can be applied to any new or existing document where versioning and change management functionality is desired, and also describes how the pattern was applied to a number of standard Microsoft Dynamics AX 2012 documents. Functionality enabled by the versioning pattern The following are two main functional benefits of applying the versioning pattern: Full traceability (history) of changes made to a document. At any time, a new version of the document can be recorded. Any two versions of the document can be compared by using a generic version comparison tool. A standardized document state model enabling change management with workflow approvals. The document is editable only in the draft, in review, and rejected states. A new version is recorded upon approval. Purchase orders A purchase order (PO) is the most complex use of the versioning pattern in Microsoft Dynamics AX 2012. Besides the traceability and change management with workflow integration, the pattern also enables other procurement functionality extensions, such as encumbrance accounting for confirmed purchase orders. Earlier versions of Microsoft Dynamics AX In Microsoft Dynamics AX 2009 and earlier versions, users were able to change purchase orders inplace, without any trace of what they changed and when. It was not possible to set up the standard Microsoft Dynamics AX application to require workflow approval for new or changed purchase orders. The purchase order confirmation journal provided limited traceability, because its use was optional, and because it contained only a subset of all order data. Every confirmation required a full copy of all confirmation journal records, which, when used extensively, caused a lot of data duplication. Microsoft Dynamics AX 2012 PO-specific conditions The change management functionality for purchase orders is optional in Microsoft Dynamics AX 2012 and is turned off by default. It can be configured at the level of the legal entity, vendor, or specific order. When it is activated, any change to order data requires workflow approval. When it is not activated, the PO is always considered pre-approved. Intercompany, direct delivery, and production/MRP-derived orders never have change management enabled. 3 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS Any change that is identified as part of the contract (that is, material to the vendor) requires a new order confirmation before an invoice or product receipt can be processed against it. This also applies when change management is not activated for an order, and thus the change tracking functionality is always available (that is, every confirmation creates a new version of the document). Figure 1 Purchase order versions form Product receipts and packing slips Use of the pattern for product receipts and packing slips is limited to versioning, and no workflow integration is enabled. Earlier versions of Microsoft Dynamics AX Packing slips were actually static documents in Microsoft Dynamics AX 2009 and earlier versions. Users could not directly modify those documents. However, the documents could be corrected (offset) by another packing slip document if the Receive Now/Deliver Now field was set with a sign opposite to the sign of the order line quantity. There was no control over which packing slip was corrected if there were multiple available. Microsoft Dynamics AX 2012 specific conditions Product receipt and packing slip documents can be explicitly corrected in Microsoft Dynamics AX 2012. However, only the same types of corrections that were possible in Microsoft Dynamics AX 2009 are supported. In other words, only the received or delivered quantity can be corrected, and only downward. Intercompany direct delivery packing slip corrections are also supported. 4 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS Versioning pattern data model The versioning pattern defines how multiple versions of a document can be stored in a relational database to minimize data repetition and provide efficient access to any of the versions. Assumptions The document to be versioned has a tree structure, in which nodes are represented by records in database tables, and parent/child relationships between nodes are represented by foreign keys between corresponding database tables. There is no limit to the depth of the tree. Figure 2 illustrates an abstract document model. Typically, nodes are stored in various database tables, where fields correspond to document node–type attributes. DocumentNode ID DocumentRootNode DocumentChildNode ID (FK) ID (FK) ParentNode (FK) Figure 2 Abstract document data model Abstract document versioning data model DocumentVersion DocumentNodeHistory ID RootNode (FK,AK1) VersionDateTime (AK1) IsArchived ID DocumentNode Node (FK,AK1) ValidFrom (AK1) ValidTo ID IsDeleted DocumentRootNode DocumentChildNode ID (FK) ID (FK) State ParentNode (FK) Figure 3 Abstract document versioning data model 5 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS The abstract document model that has the versioning pattern applied is extended with the following entities: DocumentVersion, which represents the recorded versions of the document. DocumentNodeHistory (the history table), which represents historical versions of a specific node that it references. Most documents have multiple node types and multiple DocumentNode entities, and therefore they require multiple DocumentNodeHistory entities – one per node type. Note that each document version is identified by a timestamp (VersionDateTime). In order to access historical values of any of the document nodes, the DocumentNodeHistory table should be queried with filters on ValidFrom and ValidTo, in such a way that the following condition is true: NodeHistory.ValidFrom ≥ DocumentVersion.VersionDateTime ≥ NodeHistory.ValidTo The date effectiveness feature introduced in Microsoft Dynamics AX 2012 makes querying the history tables very convenient. The DocumentNodeHistory entity should contain all the attributes of DocumentNode that can change values throughout the life cycle of the node. It is not necessary to copy immutable attributes. One of the implications of this model is that the document nodes present in any version cannot be physically deleted. When a document needs to support node deletion, an IsDeleted attribute is used (soft deletion). Logically deleted nodes have this attribute set to Yes, whereas those that still exist in the current document version have it set to No. If there were no changes to a specific node of a document between versions n and n+1, there is no need for multiple records in the history table, because a single record can have the ValidFrom to ValidTo range spanning the timestamps of both versions. New nodes have ValidFrom set to the timestamp of the first version they are a part of and ValidTo set to the maximum DateTime value. Nodes modified in version V require two records in the history table: one for the old values, where ValidTo = V.VersionDateTime–1, and one for the new values, where ValidFrom = V.VersionDateTime. Nodes deleted in version V have ValidTo on the latest history record set to V.VersionDateTime–1. The DocumentRootNode entity has a new State attribute of type VersioningDocumentState that is used for change management and workflow integration purposes. When a new version is created, the current nodes can be archived to the history tables either immediately or upon the first change to any node of the document. In the former case, the IsArchived attributes would be Yes for all document versions. In the latter case, the latest version of a document can have it set to No if no changes have been made to the document up to the time that the version was created. This mode saves database space, and makes the initial version creation very quick and simple. Physical data model considerations For complex document structures, it may be valuable to have all node attributes repeated in the history tables, including those that are immutable. In that way, the history tables can be queried independently and joined directly without the use of the node tables. Additionally, queries against the current document version can easily be converted to queries against a specific document version. In order to simplify the process of updating the history tables (archiving), it is useful to have an IsModified attribute added to all the node tables and set to Yes upon record update. In that way, the archiving process needs to create a new history table record for all node records that have the IsModified flag set to Yes but can skip all the others. 6 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS Purchase order data model PurchaseOrder PurchaseOrderVersion PurchID PurchaseOrderHistory ID InvoiceAccount (FK) OrderAccount (FK) Currency (FK) DeliveryAddress (FK) DlvTerm (O) (FK) Payment (O) (FK) State ChangeRequestRequired PurchaseOrder (FK,AK1) VersionDate (AK1) IsArchived ID PurchID (FK,AK1) ValidFrom (AK1) ValidTo (AK1) InvoiceAccount OrderAccount Currency DeliveryAddress DlvTerm (O) Payment (O) MiscCharge ID MiscChargeHistory SourceDocumentLine (FK,AK1) MarkupCode (FK) CurrencyCode (FK) Value MarkupCategory IsDeleted ID MiscCharge (FK,AK1) ValidFrom (AK1) ValidTo (AK1) MarkupCode CurrencyCode Value MarkupCategory MiscChargeReference ID PurchaseOrderLine InventTransId PurchId (FK) ItemId (O) (FK) ProcuementCategory (O) (FK) PurchQty QtyOrdered PurchPrice LineAmount InventDimId (FK) IsDeleted MiscCharge (FK,AK1) RelationType PurchaseOrderLineHistory ID InventTransId (FK,AK1) ValidFrom (AK1) ValidTo (AK1) PurchQty QtyOrdered PurchPrice LineAmount InventDimId (FK) PurchaseLineMiscCharge ID (FK) InventTransId (FK,AK1) LineNum (AK1) PurchaseOrderMiscCharge ID (FK) PurchId (FK,AK1) LineNum (AK1) Figure 4 Simplified purchase order logical data model The simplified PO logical data model in Figure 4 shows how the versioning pattern was applied to a PO document. White entities were present in earlier versions of Microsoft Dynamics AX: PurchaseOrder (PurchTable), PurchaseOrderLine (PurchLine), and the *MiscCharge* entities (MarkupTrans). These define the basic PO tree structure: PurchaseOrder is the root node, PurchaseOrderLine is its child node type, and the *MiscCharge* entities represent nodes that can be children of both PurchaseOrder and PurchaseOrderLine. The highlighted entities were added to support versioning. Physical table names of the history tables are based on the physical table names of the corresponding node tables. For example, the physical table name for the PurchaseOrderLineHistory entity is PurchLineHistory. Adjustments to the physical model include the use of the IsModified flag and duplication of most of the immutable attributes to the history tables (for example, PurchLineHistory.ItemId) to simplify queries against the archived versions (in other words, there is no need to join PurchTable, PurchLine, and MarkupTrans in such queries). 7 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS Due to the application of the versioning pattern, the PO confirmation journal (VendPurchOrderJour) was reduced so that it includes only confirmation-specific fields and a reference to PurchaseOrderVersion (PurchTableVersion). The configuration journal line table (VendPurchOrderTrans) was completely removed. Packing slip data model CustPackingSlipVersion RecId bigint CustPackingSlipJour (FK,AK1) VersionDateTime (AK1) AccountingDate InternalPackingSlipId LedgerVoucher (O) ParmId Qty Volume Weight bigint datetime datetime nvarchar(20) nvarchar(20) nvarchar(20) numeric(28,12) numeric(28,12) numeric(28,12) CustPackingSlipJour SalesTable SalesId dataAreaId nvarchar(20) nvarchar(4) SourceDocumentHeader (FK) bigint RecId bigint SourceDocumentHeader (FK) dataAreaId (FK,AK1) PackingSlipId (AK1) DeliveryDate (AK1) SalesId (FK,AK1) LedgerVoucher (O) ParmId Qty Volume Weight InternalPackingSlipId bigint nvarchar(4) nvarchar(20) datetime nvarchar(20) nvarchar(20) nvarchar(20) numeric(28,12) numeric(28,12) numeric(28,12) nvarchar(20) CustPackingSlipTrans SalesLine InventTransId dataAreaId (FK) nvarchar(20) nvarchar(4) SalesId (FK) SourceDocumentLine (FK) RecId (AK1) nvarchar(20) bigint char(10) RecId bigint SourceDocumentLine (FK,AK1) dataAreaId (FK,IE1,IE2) PackingSlipId (FK,IE1) DeliveryDate (FK,IE1) SalesId (FK,IE1) LineNum (IE1) InventTransId (FK,IE2) inventQty Qty Remain RemainInvent StatValueMST ValueMST Weight bigint nvarchar(4) nvarchar(20) datetime nvarchar(20) numeric(28,12) nvarchar(20) numeric(28,12) numeric(28,12) numeric(28,12) numeric(28,12) numeric(28,12) numeric(28,12) numeric(28,12) CustPackingSlipTransHistory RecId bigint CustPackingSlipTrans (FK,AK1) ValidFrom (AK1) ValidTo (AK1) dataAreaId InventQty Qty Remain RemainInvent StatValueMST ValueMST Weight bigint datetime datetime nvarchar(4) numeric(28,12) numeric(28,12) numeric(28,12) numeric(28,12) numeric(28,12) numeric(28,12) numeric(28,12) Figure 5 Packing slip physical data model Product receipt and packing slip documents have limited correction capabilities. Therefore, only linelevel history tables were created: CustPackingSlipTransHistory and VendPackingSlipTransHistory. These contain only the quantity and attributes that are functionally dependent on the quantity. Because no header attributes can be corrected, no CustPackingSlipJourHistory or VendPackingSlipJourHistory table was introduced. However, there existed a few CustPackingSlipJour and VendPackingSlipJour fields that were dependent on line quantity changes: Volume, Weight, and Qty. These were added to CustPackingSlipVersion and VendPackingSlipVersion. Because SalesParmTable, PurchParmTable, and related tables are used as worksheet tables for editing new packing slip versions, the latest version is never archived to the history tables. In particular, for packing slips that have just one version, the history table is not used at all. 8 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS Inventory integration and accounting implications Product receipt and packing slip documents that include stocked items have an effect on the inventory subsystem. The relation between a packing slip and the inventory transactions used to be based on the PackingSlipId, DeliveryDate, LedgerVoucher, and InternalPackingSlipId fields. Now that the documents can change, the LedgerVoucher and InternalPackingSlipId fields need to be version-specific. Therefore, they were added to the Version entities. When the correction is processed, the original accounting date, Delivery Date, may already be in a closed fiscal or inventory period. If that happens, the correction is posted on the next open date. The AccountingDate field on the Version entity tracks that. Version comparison The VersioningCompare form can be used to visualize differences between versions. It displays a document tree on the left side, with icons that indicate changed, deleted, and added nodes. For a selected node, there is a list of changed attributes shown on the right side of the form, with Old value and New value columns. The form uses the VersioningCompare abstract class as a data provider interface. Documents for which the form is to be used need to provide a subclass that implements all the abstract methods. Figure 6 Generic comparison tool 9 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS Document state management Besides the data model the versioning pattern also defines how the new data elements should be managed. Central to that is the document state concept. All actions that are relevant to the versioning data model happen as part of document state transitions. VersioningDocument is an abstract class that is meant to be a common API for document state management. Each document that supports versioning should have a corresponding VersioningDocument subclass and implement its abstract methods. The document state attribute (VersioningDocumentState EDT) should never be manipulated directly. Instead, VersioningDocument methods should be used to trigger state transitions. / Delete / Submit / Approve Draft In review / Confirm Approved / Finalize Confirmed Finalized / Cancel / ReSubmit / Reject / Cancel / Delete Rejected / New Change Request / New Change Request Figure 7 Document state model The following are the main VersioningDocument abstract methods that the document implementing the versioning pattern needs to provide: isChangeRequestRequired – When the document is in the draft, in review, or rejected state, its current version is called a change request. When change approval is not needed, this method should return false. In that case, the document will never be in any of the change request states. archiveCurrentVersion – This method should perform the archiving of the current document version to the history tables. All the new and modified records (that is, those that have the IsModified flag set to Yes and the IsDeleted flag set to No) should be copied to corresponding history tables that have ValidFrom set to the latest VersionDateTime and ValidTo set to the maximum value. Due to the date effectiveness of the history tables, any existing records that have an overlapping validity period will automatically have their ValidTo value set to the latest VersionDateTime–1. For soft-deleted records (that is, those that have both the IsModified and IsDeleted flags set to Yes), the corresponding history records should have ValidTo set to the VersionDataTime of the previous version of the document. createNewVersion – This method should create a new document version record. 10 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS The following are the main VersioningDocument class public methods that should be used to trigger a state transition: initiate – This method sets the initial document state to draft or approved, depending on whether approval is required or not. submitChangeRequest – This method should be called as part of submission of the change request to workflow. It handles the transition from the draft state to the in review state. approveChangeRequest – This method handles the transition from the in review state to the approved state. It calls createNewVersion to create a new DocumentVersion instance that has VersionDateTime set to current system date. createChangeRequest – This method handles the creation of a change request – that is, a new draft version of the document. This is only possible if the document is in the approved or confirmed state. All UIs related to the document should display the document in read-only mode when its state is approved or confirmed, and should provide a “Request change” function that calls the createChangeRequest method. At a minimum, this method sets the state to draft and, in that way, makes the document editable in the UI. If the previous version was not archived to the history tables, the archiving happens as part of change request creation. confirm – This method handles the transition from the approved state to the confirmed state. If a change request is not required, this is the time that a new DocumentVersion instance is created. change – When a change request is not required, this method is used to trigger archiving of the latest version to the history tables. It should be called when any of the document’s records change in any of its tables. Purchase order state management When change management is enabled for a PO, and changes require workflow approval, the PO follows the full versioning state model. There is also additional auto-confirmation logic for cases where the previous version was confirmed, and none of the changed values is considered relevant for vendor confirmation. The inventory subsystem is updated on approval, so change requests are invisible from an inventory perspective. Therefore, product receipts or invoices for the order cannot be processed when there is a change request for it. Item arrival registration is still possible up to the previously approved quantity. 11 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS When change management is not enabled for a PO, the validation of whether the changes require vendor confirmation happens as part of the change transaction. All PO tables have the insert, update, and delete methods extended to call VersioningDocument.change for that purpose. In addition to a transition from the confirmed state to the approved state, that method also triggers archiving of the previous version to the history tables, if the previous version is not yet archived. «table» : PurchTable «class» : PurchTableType «class» : VersioningPurchaseOrder «table» : PurchLine «class» : PurchTableTemplateEventHandler «class» : PurchPurchOrderJournalCreate : Approver : Requestor insert() Draft insert() initiate() insert() submitChangeRequest() In review completed() approveChangeRequest() createNewVersion() Approved run() initJournalHeader() Confirmed confirm() createChangeRequest() archiveCurrentVersion() Draft update() submitChangeRequest() In review completed() approveChangeRequest() {No changes relevant to the vendor detected} confirm() Confirmed Figure 8 Purchase order state management sequence diagram Product receipt and packing slip state management Product receipt and packing slip documents do not support the change approval process. Instead, the FormLetter framework drives the processing of those documents. Because the framework uses its own worksheet tables (the *Parm* tables) for document editing, those documents that are stored in the journal are always in the confirmed or finalized state, and the latest version is never archived to the history tables. 12 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS Existing class hierarchies for packing slip processing were used for correction processing. To accommodate the variations in the processing logic that are needed for corrections vs. initial versions, new versioning strategy class hierarchies were introduced: FormLetterParmDataVersioning and FormLetterJournalCreateVersioning. When an existing processing method requires different behavior for a correction, that method calls the versioning strategy object. The object is instantiated based on whether an initial version or a correction is involved. If the behavior for the correction or initial version requires document-specific data access, the versioning object then calls back the existing FormLetter object by using new interfaces: FormLetterParmDataCorrectable or FormLetterVersionableJournalCreate. In that way, the same logic for variations on initial versions vs. corrections is used for sales and purchase packing slips. 1 1 FormLetterParmDataVersioning formletterParmData -VersioningStrategy PurchFormletterParmData SalesFormletterParmData PurchFormletterParmDataPackingSlip FormLetterParmDataInitialVersion SalesFormletterParmDataPackingslip «interface» FormLetterParmDataCorrectable FormLetterParmDataCorrection FormLetterParmDataCancel Figure 9 FormLetterParmDataVersioning class hierarchy FormletterJournalCreate -VersioningStrategy FormLetterJournalCreateVersioning FormLetterVersionableJournalCreate 1 FormLetterJournalCreateCorrection 1 PurchPackingSlipJournalCreate SalesPackingSlipJournalCreate FormLetterJournalCreateInitialVersion Figure 10 FormLetterJournalCreateVersioning class hierarchy 13 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS Microsoft Dynamics is a line of integrated, adaptable business management solutions that enables you and your people to make business decisions with greater confidence. Microsoft Dynamics works like and with familiar Microsoft software, automating and streamlining financial, customer relationship and supply chain processes in a way that helps you drive business success. U.S. and Canada Toll Free 1-888-477-7989 Worldwide +1-701-281-6500 www.microsoft.com/dynamics This document is provided “as-is.” Information and views expressed in this document, including URL and other Internet Web site references, may change without notice. You bear the risk of using it. Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes. You may modify this document for your internal, reference purposes. © 2012 Microsoft Corporation. All rights reserved. Microsoft, Microsoft Dynamics, and the Microsoft Dynamics logo are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners. 14 HISTORY AND VERSIONING PATTERN, INCLUDING PO UPTAKE FOR CHANGE ORDERS