HSEHA-UM020A-EN-E 8/8/07 3:02 PM Page 1 Historian SE HIGH AVAILABILITY ADVANCED USER’S GUIDE PUBLICATION HSEHA-UM020A-EN-E–September 2007 Contact Rockwell Customer Support Telephone — 1.440.646.3434 Online Support — http://support.rockwellautomation.com Copyright Notice © 2007 Rockwell Automation Technologies, Inc. All rights reserved. Printed in USA. This document and any accompanying Rockwell Software products are copyrighted by Rockwell Automation Technologies, Inc. Any reproduction and/or distribution without prior written consent from Rockwell Automation Technologies, Inc. is strictly prohibited. Please refer to the license agreement for details. Trademark Notices FactoryTalk, Rockwell Automation, Rockwell Software, the Rockwell Software logo are registered trademarks of Rockwell Automation, Inc. The following logos and products are trademarks of Rockwell Automation, Inc.: FactoryTalk Historian Site Edition (SE), RSView, FactoryTalk View, RSView Studio, FactoryTalk View Studio, RSView Machine Edition, RSView ME Station, RSLinx Enterprise, FactoryTalk Services Platform, and FactoryTalk Live Data. The following logos and products are trademarks of OSIsoft, Inc.: PI System, Sequencia, Sigmafine, gRecipe, sRecipe, and RLINK. Other Trademarks ActiveX, Microsoft, Microsoft Access, SQL Server, Visual Basic, Visual C++, Visual SourceSafe, Windows, Windows ME, Windows NT, Windows 2000, Windows Server 2003, and Windows XP are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Adobe, Acrobat, and Reader are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. ControlNet is a registered trademark of ControlNet International. DeviceNet is a trademark of the Open DeviceNet Vendor Association, Inc. (ODVA). Ethernet is a registered trademark of Digital Equipment Corporation, Intel, and Xerox Corporation. OLE for Process Control (OPC) is a registered trademark of the OPC Foundation. Oracle, SQL*Net, and SQL*Plus are registered trademarks of Oracle Corporation. All other trademarks are the property of their respective holders and are hereby acknowledged. Restricted Rights Legend Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.227-7013 Warranty This product is warranted in accordance with the product license. The product’s performance may be affected by system configuration, the application being performed, operator control, maintenance, and other related factors. Rockwell Automation is not responsible for these intervening factors. The instructions in this document do not cover all the details or variations in the equipment, procedure, or process described, nor do they provide directions for meeting every possible contingency during installation, operation, or maintenance. This product’s implementation may vary among users. This document is current as of the time of release of the product; however, the accompanying software may have changed since the release. Rockwell Automation, Inc. reserves the right to change any information contained in this document or the software at anytime without prior notice. It is your responsibility to obtain the most current information available from Rockwell when installing or using this product. Version: 9.00.05 Table of Contents Chapter 1: About this Manual....................................................................................................1 HA for Advanced Users .................................................................................................................3 Chapter 2: HA Client/Server Background ................................................................................5 Preferred Upgrade Path.............................................................................................5 Connection Preferences ............................................................................................5 Configuration of Collectives in Client Applications ....................................................6 Member Server Priority Settings................................................................................7 Availability ..................................................................................................................8 Server Selection Algorithm ........................................................................................9 Failover ......................................................................................................................9 Chapter 3: Levels of HA Implementation ...............................................................................11 Level 1 Implementation............................................................................................11 Level 2 Implementation............................................................................................12 Level 3 Implementation............................................................................................13 Application Behavior with HA .....................................................................................................17 Chapter 4: DataSheet Control .................................................................................................19 Connection Preferences ..........................................................................................19 Initial Connection .....................................................................................................19 Transition to a New Collective Member...................................................................19 Non-Standard HA Behavior .....................................................................................20 Chapter 5: Interface Configuration Utility ..............................................................................21 Initial Connection .....................................................................................................21 Transition to a New Collective Member...................................................................22 Connecting to an HA PI Server ...............................................................................23 Chapter 6: MDB Builder ...........................................................................................................25 Connection Preferences ..........................................................................................25 High Availability Advanced User Guide Page iii Contents Chapter 7: OSI OPC Server......................................................................................................27 Connection Preferences ..........................................................................................27 Initial Connection .....................................................................................................27 Transition to a New Collective Member...................................................................28 Chapter 8: PI ActiveView .........................................................................................................31 Chapter 9: PI Advanced Computing Engine (ACE) ...............................................................33 Connection Preferences ..........................................................................................33 Initial Connection .....................................................................................................33 Transition to a New Collective Member...................................................................34 Configuring Redundant Scheduler Service .............................................................34 Chapter 10: PI Analysis Framework .........................................................................................37 Connection Preferences ..........................................................................................37 Chapter 11: PI Autopoint Synchronization (PI APS) ...............................................................39 Initial Connection .....................................................................................................39 Transition to a New Collective Member...................................................................40 Upgrading an Existing PI Server to HA ...................................................................42 Chapter 12: PI BatchView ..........................................................................................................43 Initial Connection .....................................................................................................43 Transition to a New Collective Member...................................................................44 Chapter 13: FactoryTalk Historian DataLink............................................................................47 Connection Preferences ..........................................................................................47 Transition to a New Collective Member...................................................................47 Chapter 14: FactoryTalk Historian ProcessBook....................................................................49 Connection Preferences ..........................................................................................49 Initial Connection .....................................................................................................50 Transition to a New Collective Member...................................................................50 FactoryTalk Historian ProcessBook Displays..........................................................51 VBA..........................................................................................................................52 Chapter 15: PI OLEDB ................................................................................................................55 Connection Preferences ..........................................................................................55 Initial Connection .....................................................................................................55 Transition to a New Collective Member...................................................................56 Recommendations...................................................................................................56 Page iv Chapter 16: PI Server Applications...........................................................................................59 Initial Connection .....................................................................................................59 Transition to a New Collective Member...................................................................59 Configuration Data...................................................................................................59 Chapter 17: PI SQC.....................................................................................................................61 Ad hoc SQC Charts .................................................................................................61 Server side SQC Alarms..........................................................................................61 Chapter 18: RtBaseline Services Administration....................................................................63 Connection Preferences ..........................................................................................63 Version Information..................................................................................................63 Collectives as PI Data Sources ...............................................................................64 Secondary Server ....................................................................................................64 Chapter 19: RtWebParts ............................................................................................................67 Connection Preferences ..........................................................................................67 Initial Connection .....................................................................................................67 Transition to a New Collective Member...................................................................68 RtWebParts Advanced Configuration......................................................................70 Appendix A: Technical Support and Resources ......................................................................73 Before You Call or Write for Help ............................................................................73 Index ..............................................................................................................................................75 High Availability Advanced User Guide Page v Chapter 1: About this Manual This guide provides advanced-level information about application behavior in a High Availability (HA) PI System. Serving as a companion to the High Availability User Guide, it is geared toward users and administrators who write data to the PI Server, configure their applications' connection settings, and administer the PI System. Chapter 2 gives high-level detail about how HA is configured and explains the importance of connection preferences. Chapter 3 provides detail about the three levels of HA PI System Implementation and gives an overview of how Rockwell Automation applications behave at each of these levels. Subsequent chapters offer a product-by-product description of HA-related behavior exhibited within Rockwell Automation applications. Use these remaining chapters as a reference guide to finding out more specific information on how HA affects the applications you use. For an introduction to Rockwell Automation's implementation of HA, see the High Availability User Guide. High Availability Advanced User Guide Page 1 Part I HA for Advanced Users The chapters in the following section provide both information on how Rockwell Automation applications connect to an HA PI System, and an overview of how applications behave at each level of HA implementation: HA Client/Server Background Levels of HA Implementation High Availability Advanced User Guide Page 3 Chapter 2: HA Client/Server Background Preferred Upgrade Path In order to retain compatibility with existing displays and spreadsheets, we recommend you follow these guidelines when upgrading a server to an HA version. Following these recommendations also allows clients operating without an upgraded PI SDK to continue running without problems. Make the existing server the primary Keep the machine name of the existing server the same Adopt the existing ServerID as the collective ID The existing, non replicated PI Server becomes an HA primary server and maintains the same ServerID. Note: If the preferred upgrade path is not followed, information about servers persisted in displays, spreadsheets, module aliases, .ini files, and so on may no longer be valid or may now point to a secondary server that may not be able to perform all the needs of a particular client application. Connection Preferences An HA PI System is comprised of two types of member servers with different capabilities—a primary and a secondary. Secondary members have restrictions that make them unsuitable for certain types of operations. Applications that interact with a collective are subject to these limitations and should behave in a consistent, useful manner. The HA PI SDK gives applications control over the member selection when connecting to a collective. This control is provided in the form of a connection preference that can be one of the following values: RequirePrimary—Only the primary member server is acceptable. If it is unavailable, return an error when trying to connect. High Availability Advanced User Guide Page 5 HA Client/Server Background PreferPrimary—If the primary member server is available, connect to it. If not, any secondary member server is acceptable. Any—Any member server is fine. Use the locally configured priority to choose the member for connection. Configuration of Collectives in Client Applications Once an HA-aware PI SDK is installed applications have use of a PI Connection Manager dialog box that is HA-aware. The configuration information for a collective is more extensive than for a single server, but the process is simple because every member of a collective can provide details of the collective structure. By simply opening a connection to an existing server that has become a member of a collective (either through the connection manager or through any PI SDK application), the PI SDK learns of the new configuration and updates the local Known Servers Table to reflect it. If the server is new to the client workstation, it is added with the PI Connection Manager dialog box. To configure the Known Servers Table with settings for all members of the collective, specify a default user name and a path to any one of the member servers, and allow the dialog box to validate the server. PI Connection Manager As demonstrated in this screen capture, the PI Connection Manager displays collectives differently than non-replicated PI Servers. A collective is denoted with an icon showing multiple PI Servers. Non replicated PI Servers are denoted with an icon showing a single PI Server. Page 6 Member Server Priority Settings You can select a collective and examine attributes of the member servers to determine which member server is being addressed when connected. The dialog also offers a menu item that allows you to force a failover (Server > Force Failover). You may want to select this option if the current connection is responding poorly, or you anticipate an outage. Details of the PI Connection Manager interface are available in the online help available from the help menu in the dialog box. Member Server Priority Settings Within a collective made up of a primary and multiple secondary servers, priorities are assigned to each member server of the collective. These are initially set when you add a server or the collective is initially detected during a connection. You can modify the priority for member servers on each workstation in the Collective Member Information dialog box in the PI Connection Manager (Server > Member Information). The highest priority member is given a priority of 1 and the other servers are given incremental values (2,3,4,...). The values of this property are set when the server is first added or when the SDK first detects that it has become a collective. You can then modify these values to establish your preferred connection order. If two servers share the same priority setting, the servers are selected in alphabetical order. A priority value of -1 indicates connections to that member are not allowed. Setting the server priority in the PI Connection Manager High Availability Advanced User Guide Page 7 HA Client/Server Background If the current application's connection preference allows, these priorities are consulted to choose the member server for the connection when a connection is attempted (either initially or during failover). Applications that support a connection preference of Any—such as FactoryTalk Historian ProcessBook and RtWebParts—always consider priorities when connecting. Applications that use a connection preference of PreferPrimary consult these priorities when failing from the primary server or when failing between secondary servers (when the primary is unavailable and the current secondary connection is lost). By carefully choosing the member server priorities among a user community, a certain amount of static load balancing is automatically implemented. Secondary Server Limitations Writing configuration data is not supported. This limitation includes: • PI Point attributes • Digital State Sets • User and Group information • Module Database modules, PI Aliases, PI Properties, and values Reading and writing batch data is not supported by default Writing time-series data using the PI SDK is not supported by default. This limitation refers to time-series data written with the PI SDK. Availability Collective member servers are in constant communication to synchronize configuration changes and status information. When the PI SDK establishes an initial connection with a collective member it retrieves status information for the entire collective. While connected, the member server alerts each client connection of any changes in status of the member servers. Knowing the status of each member server allows the PI SDK to avoid trying to connect to servers that are marked unavailable except as a last resort. For example, in a collective consisting of one primary and three secondary members, the application initially establishes a connection with the primary. During the connection it learns that two of the secondary servers with local priority settings of 1 and 2 are down. When the primary goes down, the PI SDK attempts a failover to the third secondary member server first even though the priority would suggest one of the other secondary members because it is aware that those servers are unavailable. If this attempt fails, the PI SDK still attempts to connect to each member in case its status information is out of date. Page 8 Server Selection Algorithm Server Selection Algorithm The following rules are effective with the initial release of HA (PR1). As the platform evolves and servers supply more information (CPU usage, connection count, and so on), the PI System will provide more efficient resource allocation through dynamic load balancing. The connection preference (page 5) governs the connection logic. A setting of RequirePrimary explicitly sets the server to the single primary. A PreferPrimary setting also returns the primary server unless the primary is unavailable. In that case the condition is equivalent to a connection preference of Any and the following selection rules apply. The member server with the smallest priority setting (page 7) (number 1 is first, 2 is second) is chosen unless the collective has indicated it is unavailable. A connection attempt is made to the chosen server. If it succeeds the process ends. If it fails, the next available member server with the next smallest priority is chosen. If all available servers have been tried without success, the servers marked unavailable are considered and tried in order of priority. The one exception is if you have marked a member with a priority of -1, which indicates that connections to that member should not be made. This process continues until a connection to every member of the collective has been attempted. If no connection is established an error is returned. The next PI SDK call that requires server access begins the process again, starting with the connection preference. Failover When an application using the HA PI SDK connected to a collective loses its connection (for example, network failure, software failure, hardware failure), you initiate a member switch in the PI Connection Manager, or the server is shutdown, the PI SDK automatically attempts a reconnection to another member. In a typical case, the application is attached to the primary and upon losing its connection it tries a connection to each secondary member server of the collective that is available until a new connection is achieved or all are tried. With the primary down, the secondary member servers are tried in order of their configured priority and availability. During the failover process, when a server connection is lost, a Disconnect event is fired and when reconnected, an Open event is fired. Note that because replication is not instantaneous and because data delivery is fanned from input sources and data streams between the initial connection and the new connection, after a failover the data may not be perfectly synchronized. High Availability Advanced User Guide Page 9 Chapter 3: Levels of HA Implementation The following levels of HA PI System implementation are referred to throughout this guide. Level 1: HA PI Server, pre-HA PI SDK, pre-HA release client application Level 2: HA PI Server, HA PI SDK, pre-HA release client application Level 3: HA PI Server, HA PI SDK, HA-aware client application HA PI Server HA PI SDK Level 1 x Level 2 x x Level 3 x x HA Client x Note: If you upgrade to an HA PI SDK or client application without upgrading to an HA Server, you do not receive any High Availability benefits and your applications behave as they did prior to your upgrade. Level 1 Implementation In this configuration, a server is upgraded to an HA version, but you have not installed an HA version of the client application. Initial Connection Applications using the PI SDK are configured to talk to single servers. When the installation has followed the preferred upgrade path, these applications connect to the primary member server of the collective. High Availability Advanced User Guide Page 11 Levels of HA Implementation Transition to a New Collective Member At Level 1 there is no notion of a collective or failover. If the server is not available initially or goes down while the client is connected, you see the same errors as with a non-replicated PI Server. Note: If the preferred upgrade path is not followed, information about servers persisted in displays, spreadsheets, module aliases, .ini files, and so on may no longer be valid or may now point to a secondary server that may not be able to perform all the needs of a particular client application. Level 2 Implementation At Level 2, the PI Server and PI SDK are upgraded to an HA version, but you are not running an HA version of your client application. This situation occurs when you have installed at least one HA client application (resulting in the HA PI SDK installation), but are running a pre-HA version of another application. Initial Connection The HA PI SDK provides a default connection preference of PreferPrimary. Pre-HA applications, being unaware of the ability to control their connection, pick up this default and always attempt to connect to the primary server. When the primary is available, these applications are not subject to restrictions on the primary. When the primary is unavailable these applications connect to a secondary and face the restrictions of that server. Because the application is not HA-aware it may attempt operations that are not supported on the secondary that fail and return error messages. An HA-aware client eliminates the possibility for user error by disabling these operations, thus making it apparent that they are not supported. Transition to a New Collective Member Even though the application is not HA-aware, failover capabilities are still provided by the HA PI SDK. There are several cases when an application may make a transition to a new collective member: It is automatically connected to a different collective member when a connection is lost (for example, network failure, software failure, hardware failure) You attempt to perform a function not supported on the secondary server (automatic failover) Page 12 Level 3 Implementation You initiate a member switch with the PI Connections Manager dialog In a typical Level 2 transition, an application is attached to the primary when it loses its connection. The application tries a connection to each secondary member server of the collective that is available until a new connection is achieved or all are tried. With the primary down, the secondary member servers are tried in order of their configured priority and availability (see Server Selection Algorithm (page 9)). The second possible case for transition arises when you are connected to a secondary server and attempt an operation not supported on that server. Examples include attempting to write time-series data to the PI SDK or using the PI BatchView add-in in FactoryTalk Historian ProcessBook while connected to a secondary. In this scenario, applications reconnect to the primary unless the primary is unavailable, in which case they fail and return error messages. The transition consists of two parts, an initial disconnection and a subsequent reconnection. During the period when an application is disconnected the application behaves as it did prior to HA when a connection was lost; however, after a brief period the connection is restored to another member. The length of this period is dependent on how the connection was lost. An orderly server shutdown results in a short transition period as the application is aware it needs a new connection before the server actually completes the shutdown process. In the case of an unanticipated connection loss, a call to the server fails after a timeout period and the PI SDK checks that the server is unreachable before attempting a new connection. See the High Availability and PI Server Replication Manual for further details. A loss of a server connection during the failover process triggers a Disconnect event. Once reconnected, the PI SDK initiates an Open event. Pre-HA applications that define behavior for these events or detect disconnection and reconnection through other means continue to exhibit this behavior during the failover process. For example, a ProcessBook trend draws an X at the current time when disconnected. Upon reconnection it draws another X at the corresponding time, and then continues plotting incoming events. Level 3 Implementation In this configuration your PI Server and PI SDK are upgraded to HA versions, and you are running a client application that is HA-aware. This is the preferred mode of operation. As an organization incorporates HA PI into their system it will likely progress through Level 1 and 2 until all clients achieve Level 3 behavior. Initial Connection HA-aware applications set their own connection preference as appropriate for their task. The types of applications that use a particular preference and their behavior are described below. High Availability Advanced User Guide Page 13 Levels of HA Implementation Require Primary An application that is used exclusively to write configuration data that is not supported on secondary servers uses a connection preference of RequirePrimary. With this setting errors return upon initial connection if the primary is unavailable and failover is disabled. This configuration does not make the application highly available because it prevents the application from working when it attempts to fail over to a secondary server. Prefer Primary Applications that have some features restricted on secondary servers (for example, allow modification of configuration data, write values using the PI SDK, or work with batch data), but that also can be used to view this data or work with other replicated data, typically use a connection preference of PreferPrimary. With this preference, as long as the primary is available, the application remains fully functional. If for some reason the primary is unavailable, the application connects to a secondary member server and can only perform functions supported on that server. HA-aware client applications disable user interface features that are not supported with the current connection. Any Applications that do not need the functions that are restricted on secondary servers can use the most flexible connection preference, Any. As described in the section on Server Selection Algorithm (page 9), with this setting the PI SDK chooses the server based on priorities that you set for your workstation. This allows an organization to spread the load on the collective across the available servers through configuration. When this preference is set, the application is saying it does not care which server it connects to because all servers provide the needed functionality. Fortunately, the Rockwell Automation applications that make use of this preference are among the most widely used products—FactoryTalk Historian ProcessBook and RtWebParts. Transition to a New Collective Member The transition behavior in Level 3 is similar to Level 2. The only differences are in the selection of the new connection target and application specific behavior to smooth the transition experience. As explained in the section Server Selection Algorithm (page 9), the choice of the target server for the new connection is dependent on the application's stated connection preference. At Level 3, applications are HA-aware and provide a specific connection preference, thus influencing the choice of the next target. Page 14 Level 3 Implementation HA-aware applications can determine that they are connected to a collective, and when a Disconnect event is initiated they can indicate that a transition is in progress. This is optional behavior and may not be present in all applications. High Availability Advanced User Guide Page 15 Part II Application Behavior with HA The following chapters provide details on how Rockwell Automation client products and application servers are affected when used within an HA environment: DataSheet Control Interface Configuration Utility MDB Builder OSI OPC Server PI ActiveView PI Advanced Computing Engine PI Autopoint Synchronization PI BatchView FactoryTalk Historian DataLink FactoryTalk Historian ProcessBook PI OLEDB PI Server Applications PI SQC RtBaseline Services (Administration Pages) RtWebParts High Availability Advanced User Guide Page 17 Chapter 4: DataSheet Control Connection Preferences The DataSheet control always connects with the RequirePrimary connection preference. Initial Connection When the DataSheet Control initially connects to a PI Server, it attempts to share a PI SDK connection that already exists. If a connection does not exist, it creates its own. When in ProcessBook, it uses the ProcessBook connection. If there is an existing connection to a secondary server, the PI SDK attempts to connect to the primary server. If connection to the primary fails (because it has faulted) the following error appears in the DataSheet control: [-16030] Batch database access disabled on secondary server of collective Transition to a New Collective Member Because the DataSheet Control columns are generated from the Batch Database, a transition away from the primary server in the collective results in an error returned from the server. If batches are enabled on secondary servers, an error is not returned. See the section on Non-Standard HA Behavior (page 20) for a description of this behavior. When the DataSheet control is in Run mode (as opposed to Build mode) and the primary server is lost, no new batches update in the grid. If a Refresh is initiated from the right click menu, the following error appears in the DataSheet space: [-16030] Batch database access disabled on secondary server of collective The same error occurs when moving from Build mode to Run mode. High Availability Advanced User Guide Page 19 DataSheet Control Non-Standard HA Behavior If you enable batches and PI SDK writes on a secondary server and your primary is unavailable, the DataSheet Control will write data to the secondary server it is connected to. When the primary server is available again, the connection will switch to the primary server and the data that was written to the secondary will not display in the grid. Data written when connected to a secondary server is not replicated to the primary or any other secondary server. Moreover, if you enable the batch database on a secondary server and you lose connection to your primary server, you will see no indication from the DataSheet Control that the data is being written to a secondary server. The default settings for a PIServer prohibits batch and PI SDK writes on secondary servers. With these default settings the DataSheet Control can only write data to the primary server which also contains the Batch context information. This keeps all manually entered information together with the context information. This configuration is recommended to ensure the integrity of your data; however, it does not receive any benefit from an upgrade to an HA PI System. Note: For the PR1 release we highly recommend that you do put batch data on a secondary server. See the High Availability and PI Server Replication guide for more details. Page 20 Chapter 5: Interface Configuration Utility The Interface Connection Utility (ICU) uses the PI SDK to communicate with the PI System to manage settings stored in the Module Database, PI Point attributes, and Digital State Sets. Note: It is strongly recommended that only an HA version of the ICU be used when connecting to an HA PI Server. Initial Connection If there are any existing interfaces configured with the ICU before the upgrade, the information is already stored in the current PI Server Module Database. After an upgrade to an HA PI Server the information is the same on primary and secondary PI Servers. At Level 3, the ICU has been modified to use the PI SDK Connection Manager dialog to define which non-replicated servers and collectives for which it manages PI Interfaces. The ICU requests a primary server when connecting, but allows read-only access to secondary servers. At Level 2, ICU behavior depends on the collective name. If the collective name is the same as the primary PI Server name, the ICU displays all existing interfaces. If the collective name is different then the primary PI Server name, the ICU can connect to a collective, but does not show any existing interfaces. However if a new interface is registered with ICU after upgrading a PI Server to an HA version, and you use the same name for the collective that was given to the non replicated PI Server (this can be different then the primary server name), the ICU correctly displays this newly added interface. At Level 1, the ICU works with individual servers. This is the same as pre-HA behavior. High Availability Advanced User Guide Page 21 Interface Configuration Utility Transition to a New Collective Member At Level 3, when a failover occurs from the primary to a secondary server, the ICU writeable/read-only status changes to read-only, and the settings are displayed in read-only mode. When a failover occurs from a secondary server to the primary server, the ICU writeable/read-only status changes to writeable, and the settings are displayed as editable, allowing changes to be saved. During a failover, any unsaved changes made to the currently loaded interface remain listed in the ICU so that they can be saved to the primary server when it becomes available again. Unsaved changes are not stored if the ICU is closed before the primary server becomes available again or if you select and load a different PI Interface. At Level 2, PI ICU connects to a collective, but all interface information is only stored on the primary PI Server. The module database on a secondary PI Server is not replicated back to the primary. If the primary PI Server is disconnected and only the secondary PI Server is available, then you are not able to add a new interface. The Configure New Interface dialog box does not display any error messages. You can close this dialog box with the Close button, but this action does not add the interface. At Level 1, PI ICU connects to individual servers, not the collective. You can only create a new interface on the primary server. Separate batch files are created (modified) on the client machine and modules are created (modified) on the Module Database of each server. The Module Database is only replicated from the primary to the secondary. Page 22 Connecting to an HA PI Server At Level 3, the PI SDK PI Connection Manager dialog is available on the ICU menu bar for configuring SDK connections. The ICU allows you to select the servers and collectives on which it will load PI Interfaces. This can be done by selecting the Loading tab of the Options dialog box (Tools > Options). The list of servers on this dialog is HA-aware, and is populated by the server list in the PI SDK Known Servers Table. Options dialog box in HA ICU At Level 2, the Connections dialog box displays only the collective name, but you may not always be able to connect to the collective because the ICU connection dialog uses PI API, and not the HA-aware PI SDK. You can only make the connection if the collective name is the same as your primary PI Server. If it differs, then the connection fails. However the PI API-based Connections dialog in ICU does not allow you to add a new collective. If you want to add a collective, we recommend you use the PI SDK PI Connection Manager dialog box. This dialog allows you to add a collective into the Known Servers Table, for example AboutPI SDK utility. As soon as the collective is added to the Known Servers Table, the ICU displays it in the Loading tab of the Options dialog box (Tools > Options) where you can select it. At Level 1, the Connections dialog box displays individual server names, and not the collective name. You are able to connect to each server and perform ICU tasks. High Availability Advanced User Guide Page 23 Chapter 6: MDB Builder There is no PR1 version of Module Database Builder (MDB), therefore there is no Level 3 HA implementation. At Level 2, module writing(edit/delete/create) is restricted while you are connected to a secondary PI Server. If you fail over to a secondary member, you are still able to read from the MDB, however, you may receive an error message when you try to write data to the PI System. If the primary is available when you attempt to write on a secondary the PI SDK attempts to automatically reconnect you to the primary. If the primary is not available, your write operation will generate an error and fail. If you are connected to a primary, all MDB functionality works normally. Connection Preferences In PR1, MDB Builder connects to an HA PI Server with a PreferPrimary connection preference. High Availability Advanced User Guide Page 25 Chapter 7: OSI OPC Server OSI OPC Server is deployed with two executables. Both of the executables support OPC Data Access (DA) and Historical Data Access (HDA) functionalities. However, when discovering installed OPC Servers, you see one of them as an OPC DA Server and the other as OPC HDA Server. It is important to have OPC clients connected to appropriate OPC servers. Although DA and HDA functionalities are different, the OPC Server uses similar ways to communicate with a PI Server. These functionalities are supported by using appropriate PI SDK and PI API methods. Since PI API currently does not support HA implementation, some DA and HDA functionalities of the OPC Server are not available during transition to a new member of a collective. Connection Preferences By default, OSI OPC Server uses a connection preference of Any when opening connections. However, it is important that the name of the collective be the same as the name of the primary PI Server. This affects the OPC Server's supported functionalities. Initial Connection When OSI OPC Server is initially launched, it does not attempt to connect to the default PI Server. It gets the list of known PI Servers from the PI SDK's Known Servers Table. An initial connection occurs in the following cases: When an OPC client tries to browse OPC Server's address space by selecting a specific PI Server in the browse operation When an OPC client requests to create a browser with specified filter attributes When an OPC client attempts to make a read or write call, or subscribe for advise Depending on how the collective is set up, the behavior of the OPC Server can be different during the initial connection. It is important to distinguish between when the collective name is the same as the primary, and the collective name is not the same as the primary. High Availability Advanced User Guide Page 27 OSI OPC Server During initial connection, if a collective name is the same as the primary name, you will not see any difference in behavior when the OPC Server is connected to a member of the collective. However, if a collective name is different from the primary and the primary is down, some functions do not work when you connect to another member of the collective. It this case, the advises in DA Server only get an initial update and stop updating from then on. All point edits or deletes do not work. After the primary comes back up again point edits and deletes start working. However, the advises that were started while the primary server was down do not resume. The OPC clients should redo the advise operation. Transition to a New Collective Member When an active connection within a collective of PI Servers makes a transition to a new member, OPC Server loses connection to a collective for a short time. The response from the OPC Server can be delayed while the transition is completed and a connection channel is restored. After OPC Server is connected to a secondary, some functionalities of the OPC Server stop working. Since OPC Server is designed to use both PI SDK and PI API to communicate with PI Server, all OPC functions supported with PI API calls are lost. Following a server transition, some OPC functionality is preserved, however the PI API connection to PI Server is lost. The following table summarizes the supported and unsupported DA and HDA functions after a transition when a collective name is the same as the primary PI Server or when a collective name is different from the primary PI Server: OPC Functionalit y Page 28 Collective is the same as primary Collective is the same as primary Collective is not the same as primary Collective is not the same as primary Connected to primary Connected to secondary Connected to primary Connected to secondary DA Read Yes Yes Yes Yes DA Write Yes No Yes No DA Advise Yes No No No DA Browse Yes Yes Yes Yes HDA Read Yes Yes Yes Yes HDA Write Yes No Yes No HDA Advise Yes No No No HDA Delete Yes No No No HDA Browse Yes Yes Yes Yes Transition to a New Collective Member OPC Functionalit y Collective is the same as primary Collective is the same as primary Collective is not the same as primary Collective is not the same as primary Point Updates Yes No No No Planned Shutdown The preferred way of setting up a collective is to have the same name for both the collective and primary PI Server. When this is true, a planned shutdown of the primary PI Server impacts the OPC Clients (DA and HDA) that are subscribed for advise. Tag edits and deletes also do not work on a secondary. See the table above for more details. OPC clients receive a notification that the PI Server has become unreachable and no further updates are sent. The remaining functionalities are restored once the OPC Server establishes connection to another member of the collective. When the primary PI Server returns, the OPC Server starts sending updates for OPC clients that signed up for advise, and the functionalities previously lost become available again. High Availability Advanced User Guide Page 29 Chapter 8: PI ActiveView HA behavior in PI ActiveView resembles that for FactoryTalk Historian ProcessBook. See the chapter on FactoryTalk Historian ProcessBook (page 49) for details. High Availability Advanced User Guide Page 31 Chapter 9: PI Advanced Computing Engine (ACE) PI Advanced Computing Engine (ACE) consists of three components: Wizard Manager Scheduler Each ACE component can actually connect to multiple PI Servers. An ACE component always connects to the ACE Data Server. The ACE Data Server is the PI Server where the ACE configuration is stored. It may also connect to other PI Servers where time-series data are retrieved or results are stored. Connection Preferences At Level 3, when connecting to the ACE Data Server, the Wizard and Scheduler have connection preferences of RequirePrimary, since they are only useful when they are connected to the primary server and able to modify the Module Database. The Manager has a connection preference of PreferPrimary. When connecting to other PI Servers, the connection preference for all three components is Any. Initial Connection ACE connects to a collective member as follows. ACE Data Server Other PI Servers Level 3 RequirePrimary for Wizard and Scheduler. PreferPrimary for Manager Any Level 2 PreferPrimary Any Level 1 N/A N/A High Availability Advanced User Guide Page 33 PI Advanced Computing Engine (ACE) ACE Wizard and Scheduler are only useful when connected to the primary server where they are able to modify the Module Database, therefore at Level 3 HA implementation they are restricted to operate only on the primary. ACE Manager is fully functional when connected to a primary server, and useful for performing read operations when connected to a secondary. By default, ACE Manager uses the connection preference PreferPrimary. Transition to a New Collective Member The behavior for both planned shutdown and unexpected loss of connection is identical and is summarized in the following table. ACE Data Server Other PI Servers Level 3 No transition for Wizard and Scheduler. Automatic transition for Manager. Automatic transition Level 2 Automatic transition Automatic transition Level 1 No transition No transition At Level 3, ACE Wizard and Scheduler must connect to the primary server, therefore there is no failover capability. At Level 2, following a transition to a secondary, ACE Wizard displays an error message indicating that the application does not have write access to the ACE Data Server, and you are not able to proceed. ACE Scheduler continues receiving the changes to ACE calculations in the Module Database. However, it would not be able to update the Module Database with correct statuses in some cases. For example, suppose you stop a calculation. The ACE Scheduler receives this change and stops the calculation. But the calculation status is still Stopping since the ACE Scheduler won't be able to update the calculation status to Off. This scenario assumes that the ACE Manager can connect to the primary while the ACE Scheduler is unable to connect to it, and it may present a rare case. At Level 3 and 2, following a transition to a secondary, ACE Manager becomes read-only and you are not able to modify any ACE configuration or stop or start a calculation. See the High Availability User Guide for more detail. Configuring Redundant Scheduler Service It is possible to configure redundant ACE Scheduler services at all three levels of HA implementation. In each case, you can start the Scheduler service on two machines executing Page 34 Configuring Redundant Scheduler Service the calculations configured on the same ACE Data Server. Only one of the services would execute the calculations while the other one is in standby mode. When the ACE Scheduler executing the calculations is stopped gracefully, the other ACE Scheduler automatically starts the calculations. At Level 3, if the ACE Scheduler running calculations is stopped due to an unexpected loss of connectivity, the second ACE Scheduler automatically begins calculating. At Level 2 and 1, if the ACE Scheduler running calculations is stopped due to an unexpected loss of connectivity, the second ACE Scheduler does not start calculating. High Availability Advanced User Guide Page 35 Chapter 10: PI Analysis Framework Connection Preferences By default the PI Analysis Framework Server(PI AF Server) uses a connection preference of PreferPrimary when connecting to its associated PI Server. This setting is appropriate for most uses. The preference setting can be modified in the PIAFServer.exe.config file located in the <PIPC>\PIAF directory. The PIServerRole key, illustrated below, controls how PI AF Server connects. <appSettings> <add key="PISystemName" value="localhost" /> <add key="PIUserName" value="" /> <add key="PIUserPassword" value="" /> <add key="PIServerRole" value="PreferPrimary" /> </appSettings> The PI Analysis Framework client uses the connection preferences of the hosting application. If no connection is open it uses the default connection preference of PreferPrimary. Since the AFModeler resides within the ProcessBook application, and ProcessBook defaults to a preference of Any, the connection preferences of ProcessBook should typically be overridden to use either RequirePrimary or PreferPrimary. See the chapter on FactoryTalk Historian ProcessBook (page 49) for how to specify its connection preferences. High Availability Advanced User Guide Page 37 Chapter 11: PI Autopoint Synchronization (PI APS) At Level 3, PI Autopoint Synchronization (PI APS) can connect to a collective and perform all APS tasks correctly. If only a secondary server is available, the APS configuration Utility displays settings for this interface in Read-only mode. You can review existing settings, but cannot add, delete, or modify an interface, and cannot perform synchronization. These options are disabled in read-only mode. At Level 3, PI APS always connects to a collective with the connection preference of PreferPrimary. At Level 2, PI APS can connect to a collective. If you are connected to a collective, you are able to register a new interface. All interface information is stored on a primary PI Server and replicated to secondaries (If it is connected. If not, it is replicated after connection is restored). At Level 1, the PI APS configuration utility connects to individual servers, not the collective. You are only able to register a new interface on the primary. Modules are created (modified) on the module database of the primary server and can be replicated to the secondaries in your collective. If you select Create points automatically, Edit points automatically, or Delete points, points are created, modified, and deleted on the primary PI Server. Modifications to the point database on the primary PI Server are replicated on the secondary PI Server. When synchronizing with the secondary PI Server, if you select the options to create, edit, or delete points automatically on the PI Server, the SyncEngine generates the following fatal error: Failed to create point on server [-10420] However, the corresponding piconfig or excel files are created for point deletes, edits, and available points. Initial Connection At Level 3, you can use the built-in PI SDK connection dialog to add or connect to a collective. If the primary server is available you can perform all APS tasks. If only a High Availability Advanced User Guide Page 39 PI Autopoint Synchronization (PI APS) secondary server is available the APS Configuration Utility displays settings for this interface in Read-only mode. At Level 2, the Connection dialog displays only the collective name, but you are not always able to connect to a collective because the PI APS connection dialog uses the PI API, not the PI SDK. The connection can only be established if the collective name is the same as the primary PI Server name. If the names are different, the connection fails. We recommend that you use the PI SDK-based PI Connection Manager dialog to add collectives into the Known Servers Table, for example AboutPI SDK utility. As soon as the collective is added to the Known Servers Table, it appears in the Loading tab of the Options dialog box (Tools > Options). You can register the new interface as usual, but modules can only be created on the primary PI Server module database. The information will be replicated on secondary PI Servers. At Level 1, the Connection dialog displays individual server names, not the collective name. You can connect to each server, but you can perform APS tasks ( register new, modify, delete interface) only on the primary PI Server. If you try to delete, create, or modify an existing interface on a secondary server you will see different dialog boxes with PI SDK errors because the PI SDK cannot create, modify, or delete modules on a secondary server. You can do synchronization on a secondary server, but points are not created on the PI Server, only point files are created. Transition to a New Collective Member At Level 3, the PI APS connects to a collective with a connection preference of PreferPrimary. If the primary is not available, the APS works in read-only mode. When connection to the primary is restored, the APS returns back to normal read-write mode. If the APS Configuration Utility does not detect that the primary is available again you can select Connect to Primary from the Interface menu, as illustrated below. Page 40 Transition to a New Collective Member Connect to Primary menu item in PI APS At Level 2, APS connects to a collective. If connection to the primary PI Server is lost the PI APS is still able to connect to the collective and it will display any existing interfaces, but you are not able to register any new interface, nor delete or modify an existing one. Various error messages appear indicating that the Module Database cannot be modified on a secondary PI Server. For example: Error message in PI APS Moreover, point synchronization between the DCS and PI Server fails because SyncEngine cannot write information in Module DataBAse about current sync status. Error message in PI APS When the primary PI Server is available again, you can perfom all APS task as usual. However the APS configuration utility does not show the current collective status. Use the PI SDK PI Connection Manager dialog to check if the primary server is available. At Level 1, the PI APS working with individual servers, not collective. High Availability Advanced User Guide Page 41 PI Autopoint Synchronization (PI APS) Upgrading an Existing PI Server to HA If there are any existing interfaces registered with PI APS before the upgrade, the information is already stored in the current PI Server Module Database. After an upgrade to an HA PI Server, the information will be the same on both primary and secondary PI Servers. Page 42 Chapter 12: PI BatchView Initial Connection BatchView has several components: a symbol in ProcessBook called the BatchGroup, an add-in to Excel, a stand-alone application called QuickSearch, and ActiveX controls and dlls to use for programming. BatchGroup Symbol in FactoryTalk Historian ProcessBook When this symbol is loaded it attempts to retrieve batch data. If the collective member that you are connected to does not support batch data access, the PI SDK automatically attempts to switch the connection to a collective member that does support batch data access. This will likely change ProcessBook's connection so that it connects to the primary collective member. If many of your displays use BatchGroup symbols, we recommend setting ProcessBook's connection preference to PreferPrimary for those collectives from which you retrieve batch data. When the parent application (ProcessBook) has a connection to a collective member that does not support batch data access, the connection may take longer due to the automatic reconnection to a collective member supporting batch data access. Excel Add-in When a query for batch data is executed (for example, when you create a new BatchView formula array, you open a spreadsheet with an existing formula array, or you refresh an existing formula array) a connection is made with the default preference, PreferPrimary. If the collective member connected to does not support batch data access, the PI SDK automatically attempts to switch the connection to a collective member that does support batch data access. This changes Excel's connection to the primary collective member. Excel maintains a single connection to each PI Server or collective regardless of how many PI add-ins are running. All add-ins get the same connection. High Availability Advanced User Guide Page 43 PI BatchView When the parent application (Microsoft Excel) has a connection to a collective member that does not support batch data access, the connection may take longer due to the automatic reconnection to a collective member supporting batch data access. QuickSearch When you click the Search button, QuickSearch connects to the collective with a connection preference of PreferPrimary. If you connect to a collective member that does not support batch data access, the PI SDK automatically attempts to switch the connection to a collective member that does support batch data access. If you cannot connect to a collective member that supports batch data access, you see the following error: One or more searches could not be completed ([servername]) Once a collective member that supports batch data access comes online, click the Search button again to reconnect and retrieve your batch results. Programmatic Access to PI BatchView ActiveX Controls When you program against BatchView ActiveX controls, the pibatchsearch.dll requires a connection to a PI Server in order to search. This connection either happens automatically or the parent application establishes the connection. If the dll connects automatically, it connects with the PreferPrimary connection preference. If in either connection case the collective member connected does not support batch data access, the PI SDK automatically attempts to switch the connection to a collective member that does support batch data access. Transition to a New Collective Member In general, batch data access is only supported on the primary collective member. That means there is not any batch data access on secondary collective members and a transition to a secondary collective member is not desirable for viewing batch data. The PI SDK assists in maintaining connection to a collective member supporting batch data access. If the connection to the collective transitions to a collective member that does not support batch data access, the next attempt to retrieve batch data causes the PI SDK to automatically attempt to switch the connection to a collective member that does support batch data access. Note: For the PR1 release we highly recommend that you do put batch data on a secondary server. See the High Availability and PI Server Replication guide for more details. Page 44 Transition to a New Collective Member BatchGroup Symbol in FactoryTalk Historian ProcessBook If a collective member supporting batch data access is not available, the status dialog reports the following error: Reading batch data on this collective member is not permitted [servername] Moreover, no data—batch, tag or otherwise—are shown in any of the symbols of the batch group. Attempts are made at regular intervals to transition to a collective member that has batch data access. Once successfully reconnected, the batch symbol is reverted and the correct batch data is retrieved. Run-time changes made to the batch group are not kept. Behavior is somewhat different if the data is designed such that the batch data is on one collective and the tag data in a batch trend is on another collective. If the batch data collective switches to a collective member not supporting batch data access then you see the behavior in the above paragraph. But if the trend data collective switches members, you continue to see your batch data and the batch trend data behaves the same as the ProcessBook trend. If an anchored batch in the batch group cannot be restored because the connection is to a collective member that does not support batch data access, that batch is not shown. This also happens in the Batches Not Found dialog box, which sometimes appears when you open a display that has an anchored batch that cannot be found on the server. These anchored batches are not deleted from the display and they are shown once ProcessBook connects to a collective member supporting batch data access. Excel Add-in If a collective member supporting batch data access is not available, you get the following error: One or more searches could not be completed ([servername]) QuickSearch If a collective member supporting batch data access is not available, you get the following error: One or more searches could not be completed ([servername]) Programmatic access to PI BatchView ActiveX controls There are two cases here: one where the parent application maintains the connection and one where the code relies on the pibatchsearch.dll to connect. High Availability Advanced User Guide Page 45 PI BatchView When the parent maintains the connection to the collective (this includes the cases where you write VBA code in ProcessBook and Microsoft Excel), the parent application determines the behavior. When you allow the pibatchsearch.dll to automatically connect, if a collective member supporting batch data access is not available, the dll returns the error: One or more searches could not be completed ([servername]) Page 46 Chapter 13: FactoryTalk Historian DataLink Connection Preferences By default, DataLink uses the PreferPrimary connection setting. Transition to a New Collective Member At Level 3 and Level 2, for all DataLink write calls (for example, piputval macros) to the collective, the PI SDK first attempts to connect to the primary. If the primary is not available, then DataLink connects to a secondary. However, DataLink only writes to the secondary if the Replication_EnableSDKWriteValues parameter is set to 1 in the pitimeout table on the PI Server. If this value is not set to 1, or if a secondary is not available, then the server issues an error. High Availability Advanced User Guide Page 47 Chapter 14: FactoryTalk Historian ProcessBook Connection Preferences By default ProcessBook uses a connection preference of Any when opening connections. This setting is appropriate for most basic uses, and allows client configuration to support load balancing. When using more advanced features of ProcessBook, such as custom displays with VBA, and add-ins (SQC, BatchView, AF Modeler) this setting frequently results in a connection to a secondary server and may present problems. Configuration Settings On workstations where ProcessBook is used to write data or access data that is not available on secondary members of a collective, a configuration setting has been provided that modifies the default connection preference. By editing the procbook.ini file you can configure the preference for a particular collective. Entries of the form <collective name>=<preference> in the [DATA MANAGER] section of the procbook.ini file control how ProcessBook establishes a connection to a server in a particular collective. The value represented above by <collective name> represents the name listed for the collective in the PI Connection Manager. The value represented above by <preference> must be one of the following values: PreferPrimary—the primary server in the collective is preferred but not required RequirePrimary—the primary server is required Any—any server in the collective is acceptable Note: For PI ActiveView, connection preferences are configured in acview.ini. In general, if the advanced feature requires a primary server to operate correctly, specifying PreferPrimary connects you to the primary member whenever it is available. When the primary is unavailable, a secondary server is used and all features of ProcessBook that don't High Availability Advanced User Guide Page 49 FactoryTalk Historian ProcessBook require the primary server continue to function. If the use of ProcessBook on a workstation requires access to the primary, then specifying RequirePrimary forces the application to only connect to a primary member and fail when such a connection cannot be completed. See Connection Preference (page 5) and Server Selection Algorithm (page 9) for more details. Settings Files The settings file used to configure ProcessBook defaults, procbook.ini, is created for individual users when they make a change to the Preferences dialog box in: C:\Program Files\Rockwell Software\FactoryTalk Historian\Server\PIPC\dat If an .ini file has not been created for the user or if possible entries are not present, the general machine copy stored in the \PIPC\Dat\ folder is used. Initial Connection When ProcessBook is first opened, it attempts to connect to the default PI Server. Subsequently, connections to PI Servers are only attempted when a display requests data from them or you explicitly connect using the PI Connection Manager. Whether the collective in question is the default or just one of the sources configured to provide data for a symbol on a display, the first attempt to connect to a collective during a ProcessBook session may result in a connection to a secondary PI Server if the connection preference is configured to be Any or PreferPrimary. If the connection preference is configured to be RequirePrimary and the primary is unavailable, ProcessBook does not connect to the collective and any symbols requesting data from that source report NO DATA, as they previously would have when a non-replicated PI Server is not available. Transition to a New Collective Member When a collective server has been opened either with the connection preference PreferPrimary or Any and the current connection to the server is lost, ProcessBook attempts to connect to another member of the collective. In some cases, the reconnection attempt occurs on a subsequent call to the server. This is similar to the automatic reconnection available in earlier versions of ProcessBook, but instead of calls failing until the single server comes back online, another member in a collective responds to the call, resulting in higher availability. Page 50 FactoryTalk Historian ProcessBook Displays Failover respects the connection preference set during the Open/Login call or set in the .ini file entry for that collective. If the collective is configured in procbook.ini to use PreferPrimary, but is connected to a secondary that becomes unavailable, ProcessBook first attempts to connect to the primary PI Server. If unsuccessful, ProcessBook attempts to connect to a secondary server. FactoryTalk Historian ProcessBook Displays Recommended Connection Preference Setting Most ProcessBook displays do not write data to the PI Server and therefore work equally well against primary and secondary member servers. A setting of Any is sufficient and supports load balancing. A setting of PreferPrimary also works well but does not allow the workstation to participate in load balancing when running ProcessBook. If you use RequirePrimary, and the primary is unavailable, the application behaves as it did in prior versions when the server is unreachable. When this occurs during the opening of a display, the error returned by the connection attempt is displayed in the Status Report dialog box for each tag on the server. Typically, the message reads: The requested server in not currently available. Primary. Dynamic symbols on the display are presented in the same manner as any other symbol attached to a disconnected server, though there is no recovery unless the primary server comes back online. High Availability Advanced User Guide Page 51 FactoryTalk Historian ProcessBook Status Report Dialog If this occurs after the display is already open, ProcessBook detects the loss of connection, as described previously, and tries to reconnect to the server. This call fails, generates the same error as in the previous case (The requested server in not currently available, Primary), and updates the Status Report dialog. VBA The ability to interact with ProcessBook displays using Visual Basic code allows customization of display behavior. However the code within a display is not restricted to manipulating the ProcessBook object model; it provides an entry point into just about any kind of processing desired. The connection preference and member server capabilities become important at the point where this custom code interacts with an HA PI system. If the VBA code is PI API-based it does not gain any HA-related benefits because the PI API has no notion of collectives. For ProcessBook displays in ActiveView, any embedded VBA script is subject to all of the limitations of secondary servers. Recommended Connection Preference Setting Any—If you use ProcessBook displays with VBA code that primarily work with the ProcessBook object model, or make calls unrelated to the PI System, the default setting of Any is appropriate. If the VBA code is PI SDK-based and only reading information from a PI Server, then you do not need to make any changes. PreferPrimary—Some custom displays may have VBA code that interacts with the PI System in a manner not supported on secondary servers, such as writing real-time data, changing configuration information, or creating batches. If such custom VBA code is not critical to the function of the display, then it is advisable to set the connection preference to PreferPrimary. With this setting, the display connects to the primary server when possible, and in the event of a failover, the display remains connected. However, in the event of failover, the custom VBA code that interacts with a PI secondary in an unsupported fashion is not functional. Therefore, it is advisable that administrators test their displays with secondary servers to determine if the limited functionality after a failover to secondary is acceptable. RequirePrimary—If you use ProcessBook on a workstation almost exclusively to run custom displays that perform functions not supported on a secondary (for example a lab data entry screen), then we advise you set the connection preference to RequirePrimary as the application is basically inoperative without this connection. Page 52 VBA Behavior at Different Levels Level 3: VBA programmers can use the new HA PI SDK features to provide an enhanced user experience. When connected to a secondary and features are not available, User Interface controls can be disabled or the code can switch to a different member to complete actions. In order to take advantage of HA features it is necessary that the VBA programmer alter their code if their applications write data, change configuration data, or create batches. It is the responsibility of the custom display developer to deliver HA versions of their displays if their code is affected and the changes add value to the behavior. Level 2: When connected to a secondary server, calls will return an error if they write data, change configuration data, or create batches. Error handling in a display's VBA is specific to that piece of code. The PI SDK returns different errors for these new failures, so typically only the code's generic error handling is activated in this circumstance. See the High Availability and PI Server Replication manual for more details. Level 1: If the recommended upgrade path is followed to convert a PI Server to HA, your old displays point to what has become the primary server. ProcessBook behaves as it did before HA. High Availability Advanced User Guide Page 53 Chapter 15: PI OLEDB Connection Preferences At Level 3 and 2, PI OLEDB connects to a collective with the connection type PreferPrimary. Note: In PI OLEDB a connection preference is referred to as connection type. Before accessing data through PI OLEDB you need to provide information necessary to establish a connection to the PI Server. In OLEDB, this information is typically persisted as a string referred to as a connection string. It consists of a series of keyword-value pairs separated by semicolons. The equal sign (=) connects each keyword and its value. Example: Keyword1 = Value1; Keyword2 = Value2 The number of available keywords is extended by "Connection Type." Example: Connection Type=RequirePrimary Note: The location of the connection string varies by application. See your database application's user manual for more details. Initial Connection If the connection preference is set to RequirePrimary then PI OLEDB either directly connects to the primary PI Server of a collective or returns an error after the connection timeout. It does not fail over to a secondary PI Server. If the connection preference is set to PreferPrimary or Any, the connection may be made to a secondary PI Server. In this case there is a difference for DML and DDL type queries: High Availability Advanced User Guide Page 55 PI OLEDB PreferPrimary—Statements that are not supported on a secondary return error messages. The error messages are basically pass-through PI SDK error messages. Any—The provider starts in read-only mode. All tables and views are read-only and attempts to call DDL statements (create database, drop table, etc.) return Not supported messages. Transition to a New Collective Member If your PI System makes an unexpected transition to a new collective member, there is no guarantee that your SQL statements will successfully execute. The OLEDB consumer application therefore needs to implement its own proper error handling and retry attempts. If there is no SQL statement in progress during a server transition, you do not receive an error upon transition. If your connection preference is set to PreferPrimary, following a server transition unsupported DML and DDL statements fail and you receive errors generated by the PI SDK. Recommendations Improve User Experience Application developers can inform users about limitations in several ways: Connection type Any exposes PI OLEDB tables as read-only. Third party OLE DB consumers (typically table oriented tools) automatically disable write functionality if tables are read-only. This functionality is ideal for scenarios where your users are not allowed to update data or write to the PI System. This is an especially important safeguard when users have the ability to directly type and execute any SQL statement. The HA-aware version of PI OLEDB exposes a picollective table in the pisystem catalog. The table offers information about collectives and the current connected collective member. This information can be used to trigger a modification of the front end interface, for example to gray out dialog options or disable write functionality. Page 56 Recommendations PI OLEDB Tester dialog box Administration In a future PI OLEDB release we will support an additional connection preference: connection type = collective name This allows you to connect to a specific collective member, for example for administration purposes. Write operations are enabled but depend on server side support. High Availability Advanced User Guide Page 57 Chapter 16: PI Server Applications The server applications consists of PIPE Scheduler PIPE Recalculator PI Alarm and RtSQC PI Totalizer PI Batch Since the server applications are always installed as part of the HA PI Server, only Level 3 is applicable. Initial Connection Each server application connects to the PI Server where the server application is installed. For example, if the server application is installed on the primary server, it connects to the primary server; if the server application is installed on a secondary server, it connects to the secondary server. Transition to a New Collective Member Server applications can not switch to another collective server member. Configuration Data Configuration data can only be changed on the primary server. Each server application running on a server generates its own results. Thus, it is very likely that the results for the same configuration data would differ on each collective member. Some of the reasons are: High Availability Advanced User Guide Page 59 PI Server Applications Time-series data may not arrive at each collective member at the exact same time. Configuration changes made on the primary server may not propagate to secondary members instantaneously. The same calculation may be executed at different times because the latency associated with the hardware and software differs for each collective member. Restarting server applications affects the exception reporting results while the server restart may insert the Shutdown event affecting compression results. Page 60 Chapter 17: PI SQC The SQC symbol is used to create SQC charts in ProcessBook displays. There are two types of SQC symbol configuration - client-side/ad hoc, and server side. Ad hoc SQC Charts Data for these charts are typically evaluated on the client and an ad hoc chart is drawn based on the results of the evaluation. Recommended Connection Preference Setting Ad hoc alarms require no special consideration. The default setting of Any works well and supports load balancing. Server side SQC Alarms If SQC Alarms are properly configured and your data is fanned from interface nodes, then you will not see any difference in your alarms when you make a transition to a secondary PI Server. However, there are two scenarios where the use of SQC Alarms requires special consideration: Alarm Creation Manual Entry of Source Point SQC Alarm Creation Creating SQC Alarms currently involves a specific sequence of operations: High Availability Advanced User Guide Page 61 PI SQC 1. Creation of the PI points that hold the control limits and execution state of the SQC Alarm 2. Setting the initial value of the limits and execution state 3. Creation of the SQC Alarm itself The SQC Alarm Manager plug-in for Rockwell Automation's System Management Tools performs the sequence of operations for the user via a wizard-like user interface. In PR1, the configuration databases are replicated—including the PI Point database. Changes to the point database made on the primary member of a collective are replicated to each of the secondary nodes, therefore allowing for the replication of SQC Alarms. In this context, steps 1 and 3 work very well. Step 2 is easily accomplished on a primary member of a collective. On a secondary member of a collective the ability to write time-series data using the PI SDK is disabled by default. This means that it may not be possible to write the initial values of control limits and execution state of an SQC Alarm before the SQC Alarm is created on a secondary member of a collective (via replication). It is entirely possible that a new SQC Alarm on a secondary member of a collective could be placed in the Error state. If you know that an SQC Alarm is in the Error state, the best way to get it back into operation is to: check the PI Message Log, rectify the cause of the 'Error' state (for example, use the piconfig utility to write the initial values to the control limit and execution state points), and finally stop and restart the PI Alarm subsystem on the server SQC Alarms that Require Manual Entry If an SQC Alarm uses a source point that is manually entered, then the manual entry application must write to all members of a collective. If data is written to some members and not to others, the history of the SQC Alarm will differ on each server. If one or more members of a collective are offline when data is written to the online members, the SQC Alarm's history will be different once the offline servers are back online. Page 62 Chapter 18: RtBaseline Services Administration Connection Preferences The chief aim of the RtBaseline Services (RtBLS) Administration pages is to configure Data Sources and Datasets. This configuration data is stored in a specified PI Server's Module Database (MDB). At Level 3 and Level 2 implementation, the MDB for a PI Server collective can serve as the configuration store for RtBLS. In this case use a connection type of PreferPrimary for the RtBLS Administration pages. The primary server in a collective is preferred, because any changes to RtBLS configuration must be written to the MDB, and only the MDB of the primary server in a collective allows writes (the MDBs for the secondary servers in a collective are read-only). However, if the primary server in a collective is unavailable, it is still useful to connect to a secondary server so that you can review the RtBLS configuration, but not change it. At Level 1, RtBLS Administration is unchanged. Version Information The RtBLS Version Information page includes the display of information about the PI Server or collective that holds the configuration information for both RtBaseline Services and RtWebParts. When configuration information is being stored in a PI Server collective, the information displayed consists of the collective identity and its status, separated by a hyphen. The identity information includes the name of the actively-connected collective member, enclosed in single quotes, followed by the name of the collective enclosed in parentheses. The status information consists of the role of the actively-connected collective member within the collective (primary or secondary), followed by a comma, followed by the connection status (connected or disconnected). For example, if a PI Server collective named MyCollective is storing the configuration information, and RtBLS is currently connected to a secondary server within a collective named Member2, an item on the Version Information page displays as follows: High Availability Advanced User Guide Page 63 RtBaseline Services Administration Configuration PI Server: 'Member2' (MyCollective) - Secondary, Connected When all members of a collective are unavailable, the identity of the last actively-connected collective member is retained, but its role within the collective is no longer displayed, and the connection status is Disconnected, as in the following example: Configuration PI Server: 'LastConnectedMember' (MyCollective) Disconnected Collectives as PI Data Sources The RtBLS PI Data Sources administration page reflects the Known Servers Table for the computer that hosts RtBLS. Collectives appear on this page as entries in the list of Configured PI Servers, and are distinguished from individual PI Servers by an asterisk ("*"). Example of PI Server Collectives Appearing in Configured PI Servers List Adding a server name to the Configured PI Servers list on this page edits the Known Servers Table. To add a PI Server collective to the list, provide the name of any server in the collective, and click the Save button. The collective appears in the list using the name of the collective rather than the name of the member server used when adding the collective. Secondary Server Assuming that RtBLS configuration data is being stored in the Module Database of a PI Server collective, configuration changes can not be saved if the primary server in that Page 64 Secondary Server collective is not available. The following RtBLS administration pages display a warning message if the primary server in the collective is not available: Relational Data Sources Web Service Data Sources PI Calculation Datasets Relational Datasets Web Service Datasets TagList Configuration User interface elements that attempt to write configuration changes, such as the Save button, are not disabled because there is no automatic means to re-enable them should the primary server become available again. Even when the warning message is not displayed, it is possible that the primary server has become unavailable since the page was first displayed, and that an attempt to save changes will fail. Any failed attempt to save configuration changes results in an error dialog with an appropriate message. The changes that you attempted to save are preserved in the page, but not written to the configuration store. Example of Warning Message When Primary PI Server is Unavailable Any time that one of these administration pages is displayed or refreshed, the availability of the primary server is reflected by the presence or absence of the warning message. Many actions can cause a page to be displayed or refreshed, such as the first attempt to view a page, an attempt to save configuration changes, or clicking the refresh button in the browser. If the primary server becomes available again, a warning message does not appear when the page is refreshed, and attempts to save configuration changes should succeed. High Availability Advanced User Guide Page 65 RtBaseline Services Administration Page 66 Chapter 19: RtWebParts Connection Preferences When RtBaseline Services(RtBLS) establishes a connection to a PI Server collective to retrieve data for display in RtWebParts, it uses a connection type of Any. This same connection type is used for connections for event pipes to receive updates to data. However, since all connections that support RtWebParts are made through a single instance of RtBLS, they all use the same member server priority data in the Known Servers Table for the computer that hosts RtBLS, and therefore all connect to the same server by default. Initial Connection At Level 3 and Level 2 implementation, when RtBLS receives its first request from RtWebParts for data from a PI Server collective, it attempts to connect to the collective member given highest priority in its Known Servers Table using the Any connection preference. Should this result in a connection to a secondary server in the collective there may be resulting limitations for web parts as described later in this chapter. Typically, the primary server in the collective has the highest member server priority in the Known Servers Table on the computer that hosts RtBLS, which results in a connection to the primary server every time, as long as the primary server is available. At Level 1, there is no change in RtWebParts behavior. Annotation and Value Attribute Data At Level 3 and Level 2, when connected to a secondary server, annotation and value attribute data is unlikely to be available. Such data is not replicated from primary to secondary, and by default the PI SDK returns an error when attempts are made to write directly to a secondary collective member. The PI SDK is the most common mechanism for writing annotation and value attribute data, however using it to write to a secondary server will likely result in an error. Therefore, High Availability Advanced User Guide Page 67 RtWebParts indicators that normally appear in web parts (such as RtTrend and RtXYPlot) to denote that annotations are available, or that a value is questionable or substituted, are unlikely to appear. No error is encountered; the data is simply unlikely to be present on a secondary collective member. However, it is possible to configure the PI SDK to allow annotation and value attribute data to be written to secondary collective members. In this rare case, such data can then be retrieved from secondary collective members. Also, some interfaces that write time-series data to the PI Servers in a collective may also write value attribute data. Since these interfaces do not use the PI SDK to write to PI Servers, and since they write to all servers in the collective, this limited value attribute data is also available from all servers in the collective. RtActiveView When you initially connect to a secondary server, PDIs shown in RtActiveView that have embedded VBA scripts are subject to the following limitations of secondary servers: Annotation and value attribute data may not be available; no error is returned, the data simply is not present. Attempts to write time-series data via the PI SDK may experience errors. Attempts to read or write batch data experience errors. Errors occur as a result of attempts to write any of the following types of configuration data via the PI SDK: • PI Point Attributes • User or Group Information • Digital State Sets • Any Module Database Data Transition to a New Collective Member At Level 3 and Level 2, when your RtBLS connection makes a transition from one collective member to another, you may notice some differences in behavior. Connection Timeout If the transition from one collective member to another takes longer than the configured Connection Timeout for the collective, you see the same behavior as in pre-HA releases when a PI Server becomes unavailable and then comes back online. Page 68 Transition to a New Collective Member Updating with Secondary Data When your RtBLS connection makes a transition from one collective member to another, you may notice a momentary increase in CPU utilization on client computers. This is due to the need to fully refresh displays with data from the server to which the connection has made a transition. In addition, the refreshed data may not exactly match what was previously seen, particularly when recent data is being displayed. Since all servers in a collective process new data individually, it is unlikely that all servers return the same time-series data for the same request at the same time, particularly for recent data. Any such differences become evident when a server fails and time-series displays are refreshed using data from a different server in the collective. Annotations and Value Attribute Data As previously mentioned, annotation and value attribute data is not replicated from primary to secondary PI Servers. The most likely way for this data to be written is via the PI SDK, and by default, the PI SDK returns an error when an attempt is made to write directly to a secondary collective member. Though this PI SDK error can be avoided, it is most likely that annotation and value attribute data will not be available from secondary collective members. RtTrend, for example, can show this data in its display, by displaying an icon indicating that a value has been annotated. On failover to a secondary, since it is unlikely for this data to be present, the icon disappears when the RtTrend is refreshed with data from the new server. Module Database Changes to a PI Server collective's Module Database (MDB) are always made to the primary PI Server in the collective, and then propagated to the other members. Since there is a delay between the time in which the changes are written to the primary, and when they are propagated to the other members, it is possible that MDB data retrieved from one PI Server in a collective may not exactly match that retrieved from another PI Server in the same collective. Consequently, when an RtBaseline Services connection makes a transition from one collective member to another, any RtWebParts that display data from the MDB may be refreshed with slightly different data. The frequency with which MDB changes are propagated from the primary server in a collective to the secondary servers in that collective is configurable. To avoid unexpected behavior in RtWebParts, we recommend propagating MDB changes as they occur. High Availability Advanced User Guide Page 69 RtWebParts RtTreeView The RtTreeView web part displays Modules in the MDB. If all MDB changes have not been propagated to all servers in a collective, making a transition from one server to another could result in a change in the modules displayed in RtTreeView. However, since it does not automatically update itself, such changes would only become evident if the web page were explicitly refreshed. Data Sources and Datasets The Data Sources and Datasets configured in RtBaseline Services are stored in a Module Database. If this store resides in the MDB of a collective, there may be consequences apparent in RtWebParts. For example, if a Dataset is added (stored in the MDB), and a web part is modified to display that Dataset, when you make a transition to a different server in the collective to which the new Dataset information in the MDB has not been propagated, the web part is unable to display the Dataset data and reports an error indicating that the Dataset is not available. RtWebParts is equally susceptible to Data Source configuration information that has not been propagated throughout the collective prior to making a transition from one collective member to another. Consequently, we recommend propagating MDB changes from the primary server in a collective to the secondary servers in that collective as frequently as possible, preferably as they occur. PI OLEDB Relational Dataset Any web part that displays data from the MDB that is retrieved via a relational Dataset based on the PI OLEDB provider is susceptible to the latency in propagation of MDB changes from the primary server to the secondary servers. In short, after making a transition from one server to another, such a web part may not display the same data. RtWebParts Advanced Configuration At Level 3 and Level 2, the web pages for performing advanced configuration of RtWebParts (RtWP) are also affected by PI Server Collectives when used as the advanced configuration data store. These advanced configuration pages use a connection type of PreferPrimary when connecting to a collective. The primary server in a collective is preferred, because any changes to web part configurations must be written to the Module Database (MDB), and only the MDB of the primary server in a collective allows writes. However, if the primary server in a collective is unavailable, it is still useful to connect to a secondary server, so that you can review (but not change) the configuration. Page 70 RtWebParts Advanced Configuration Secondary Server Assuming that RtWP advanced configuration data is being stored in the Module Database of a PI Server collective, configuration changes cannot be saved if the primary server in that collective is not available. At Level 3, the following RtWP advanced configuration pages show a warning message if the primary server in the collective is not available: RtTable Configuration RtTrend Configuration RtMessenger Configuration These pages display the following warning message, in red font and at the top of the page, if the primary server in the collective is not available at the time the page is displayed: At this time, the Primary PI Server for the Configuration Database is not available. Therefore, any attempt to modify the configuration may fail. After a page is displayed, should the primary server become available, there is no mechanism to automatically remove the warning message. User interface elements that would attempt to write configuration changes, such as the Save button, are not disabled, because there is no automatic means to re-enable them should the primary server become available again. Even when the warning message is not displayed, it is possible that the primary server has become unavailable since the page was first displayed, and that an attempt to save changes will fail. Any failed attempt to save configuration changes results in an error dialog with an appropriate message. The changes that you attempted to save are preserved in the page, but not written to the configuration store. Any time that one of these administration pages is displayed or refreshed, the availability of the primary server is reflected by the presence or absence of the warning message. Many actions can cause a page to be displayed or refreshed, such as the first attempt to view a page, an attempt to save configuration changes, or clicking the refresh button in the browser. If the primary server becomes available again, the warning message does not appear when the page is refreshed, and attempts to save configuration changes should succeed. At Level 2 you are susceptible to the same problems, but no warning message is displayed. High Availability Advanced User Guide Page 71 Appendix A: Technical Support and Resources Contact Rockwell Automation Technical Support at the following: • Customer Support Telephone — 1-440-646-3434 • Online Support — http://support.rockwellautomation.com Knowledge Center The Knowledge Center provides a searchable library of documentation and technical data, as well as a special collection of resources for system managers. For these options, click Knowledge Center in the Technical Support Web site. The Search feature allows you to search Support Solutions, Bulletins, Support Pages, Known Issues, Enhancements, and Documentation (including user manuals, release notes, and white papers). System Manager Resources include tools and instructions that help you manage: archive sizing, backup scripts, daily health checks, daylight savings time configuration, PI Server security, PI system sizing and configuration, PI trusts for interface nodes, and more. Before You Call or Write for Help When you contact Rockwell Automation Technical Support, please provide: Product name, version, and/or build numbers Computer platform (CPU type, operating system, and version number) The time that the difficulty started The message log(s) at that time High Availability Advanced User Guide Page 73 RtWebParts Find the Version and Build Numbers To find version and build numbers for each PI System subsystem (which vary depending on installed upgrades, updates or patches) use either of the following methods: If you have PI System Management Tools (PI SMT) installed, choose Start > Programs > PI System > PI System Management Tools. In PI SMT, select the server name, then under System Management Plug-Ins, open Operation > PI Version. The PI Version tree lists all versions. If you do not have PI SMT installed, open a command prompt, change to the pi\adm directory, and enter piversion -v. To see individual version numbers for each subsystem, change to the pi\bin directory and type the subsystem name followed by the option -v (for example, piarchss.exe –v). View Computer Platform Information To view platform specifications: In Windows, right-click My Computer and choose Properties. For more detailed information, choose Start > Run, and enter msinfo32.exe In UNIX, use the command, uname -a Page 74 Index A any • 14 availability • 8 C client collective configuration • 6 connection preferences • 5, 14 DataLink • 47 DataSheet Control • 19 OSI OPC Server • 27 PI ACE • 33 PI Advanced Computing Engine (ACE) • 33 PI AnalysisFramework • 37 PI OLEDB • 55 ProcessBook • 49 RtBaselineServices • 63 RtWebParts • 67 D DataSheet Control • 19 connection preferences • 19 initial connection • 19 non-standard HA behavior • 20 transition to a new collective member • 19 F FactoryTalk Historian DataLink • 47 connection preferences • 47 transition to a new collective member • 47 FactoryTalk Historian ProcessBook • 49 BatchGroup symbol • 43, 45 configuration settings • 49 connection preferences • 49 High Availability Advanced User Guide displays • 51 initial connection • 50 transition to a new collective member • 50 VBA • 52 failover • 9 H HA client/server background • 5 I initial connection • 11, 12, 13 DataSheet Control • 19 FactoryTalk Historian ProcessBook • 50 Interface Configuration Utility (ICU) • 21 Level 1 • 11 Level 2 • 12 Level 3 • 13 OSI OPC Server • 27 PI Advanced Computing Engine (ACE) • 33 PI Autopoint Synchronization (APS) • 39 PI BatchView • 43 PI OLEDB • 55 PI Server Applications • 59 RtWebParts • 67 Interface Connection Utility • 23 initial connection • 21 transition to a new collective member • 22 L levels of HA implementation • 11 M MDB Builder • 25 connection preferences • 25 Page 75 Index member server priority settings • 7 initial connection • 67 module database • 69 transition to a new collective member • 68 O OSI OPC Server • 27 connection preferences • 27 initial connection • 27 transition to a new collective member • 28 overview of product behavior • 1 S secondary server limitations • 8 server selection algorithm • 9 T P PI ActiveView • 31, 68 PI Advanced Computing Engine • 33 connection preferences • 33 intial connection • 33 transition to a new collective member • 34 PI Analysis Framework • 37 connection preferences • 37 PI Autopoint Synchronization (APS) • 39 initial connection • 39 transition to a new collective member • 40 PI BatchView • 20, 43 Excel Add-in • 43, 45 initial connection • 43 QuickSearch • 44, 45 transition to a new collective member • 44 PI OLEDB • 55, 70 connection preferences • 55 initial connection • 55 recommendations • 56 transition to a new collective member • 56 PI Server Applications • 59 PI SQC • 61 prefer primary • 14 preferred upgrade path • 5 R RtBaseline Services • 63 collectives as PI data sources • 64 connection preferences • 63 secondary server • 65 RtWebParts • 67 advanced configuration • 70 annotations • 67 connection preferences • 67 Page 76 transition to a new collective member • 12, 14 DataSheet Control • 19 FactoryTalk Historian DataLink • 47 FactoryTalk Historian ProcessBook • 50 Interface Control Utility (ICU) • 22 Level 1 • 12 Level 2 • 12 Level 3 • 14 OPC Server • 28 PI Advanced Computing Engine (ACE) • 34 PI Autopoint Synchronization (PI APS) • 40 PI BatchView • 44 PI OLEDB • 56 PI Server Applications • 59 RtWebParts • 68 V VBA • 52 behavior at different levels • 53