Configuring Replication with highavailability RAC clusters. David Breinholt Kerry Howell Outline • • • • • • • • Our internal requirements The challenges of 99.9% uptime Our architecture How we setup SharePlex How we configured SharePlex CRS Integration Useful commands Final Thoughts Service Level Agreement • • • • • Must maintain 99.9% availability Maximum allowed outage of 43.8 minutes/month Not allowed to combine monthly allocations Financial penalties will be paid for overages Zero scheduled downtime allowed This product is Highly visible to Executive team, so it is in our best interest to proactively meet the requirements of the Service Level Agreement. The Challenge • • • • • • • • Upgrade Oracle – upgrade inactive silo, failover Patch Oracle – patch inactive silo, failover Quarterly software releases – inactive silo, failover Hardware failure/updates - failover to inactive silo Software failure – failover to inactive silo User error – use flashback table/database Disaster recovery – storage replication to NJ site OS Patches/upgrades … all with near zero downtime! Architecture UT - Silo A - RAC Cluster UT - Silo B - RAC Cluster Logical Replication inst_a1 inst_a2 Bi-directional Block Level Storage Replication NJ - Silo A - RAC Cluster inst_a1 inst_a2 inst_b1 inst_b2 Block Level Storage Replication Logical Replication NJ - Silo B - RAC Cluster inst_b1 inst_b2 Research • RAC – – Upgrading with no downtime is questionable Does not facilitate our quarterly releases methodology • Data Guard – – – – Upgrading with no downtime is questionable Target site is down (or read-only) when propagating changes Active data guard is additional cost Switchover takes minutes+ to complete • Golden Gate – – Ability to compare/repair data is an add-on (expensive) Could not justify the cost of the product missing functionality • SharePlex – Seemed to satisfy all our requirements Decision During our discovery effort it was determined that SharePlex was the best solution for the following reasons: • • • • • Intuitive setup and configuration Cost effective compared to other solutions Detects when table get out of sync Includes the ability to compare and repair tables Allows bi-directional replication Setup In the source environment: Create the required tablespaces CREATE TABLESPACE SPLEXDAT DATAFILE SIZE 50M AUTOEXTEND ON NEXT 100M MAXSIZE 5G LOGGING EXTENT MANAGEMENT LOCAL AUTOALLOCATE BLOCKSIZE 8K SEGMENT SPACE MANAGEMENT AUTO FLASHBACK ON; CREATE TABLESPACE SPLEXIDX DATAFILE SIZE 50M AUTOEXTEND ON NEXT 100M MAXSIZE 5G LOGGING EXTENT MANAGEMENT LOCAL AUTOALLOCATE BLOCKSIZE 8K SEGMENT SPACE MANAGEMENT AUTO FLASHBACK ON; Setup - Continued Obtain new IP addresses in the same network as the SCAN interface IP addresses (for each cluster): Utah: Silo A: 192.168.1.10 Silo B: 192.168.1.20 New Jersey: Silo A: 192.168.2.10 Silo B: 192.168.2.20 Setup - Continued Obtain DNS entries for the new IP addresses: Utah: 192.168.1.10 - inst_a-splex-vip.xactware.com 192.168.1.20 - inst_b-splex-vip.xactware.com New Jersey: 192.168.2.10 - njinst_a-splex-vip.xactware.com 192.168.2.20 - njinst_b-splex-vip.xactware.com Setup - Continued Create the CRS VIP to allow SharePlex to failover between RAC nodes: Utah Silo A (only on one of the nodes) as root: ${CRS_HOME}/bin/appvipcfg create -network=1 -ip=192.168.1.10 -vipname=shareplex.vip -user=oracle -group=dba as oracle: crsctl start resource shareplex.vip Utah Silo B (only on one of the nodes): as root: ${CRS_HOME}/bin/appvipcfg create -network=1 -ip=192.168.1.20 -vipname=shareplex.vip -user=oracle -group=dba as oracle: crsctl start resource shareplex.vip Setup - Continued Add SharePlex environment variables to .bashrc.* Utah Silo A: SP_SYS_HOST_NAME="inst_a-splex-vip.xactware.com" ; export SP_SYS_HOST_NAME SP_INSTALL_DIR="/u02/app/quest"; export SP_INSTALL_DIR SP_SYS_VARDIR="${SP_INSTALL_DIR}/vardir" ; export SP_SYS_VARDIR SP_SYS_PRODDIR="${SP_INSTALL_DIR}/sp" ; export SP_SYS_PRODDIR SP_OCT_ASM_SID="+ASM1" ; export SP_OCT_ASM_SID - (different for each host in the cluster) Utah Silo B: SP_SYS_HOST_NAME="inst_b-splex-vip.xactware.com" ; export SP_SYS_HOST_NAME SP_INSTALL_DIR="/u02/app/quest"; export SP_INSTALL_DIR SP_SYS_VARDIR="${SP_INSTALL_DIR}/vardir" ; export SP_SYS_VARDIR SP_SYS_PRODDIR="${SP_INSTALL_DIR}/sp" ; export SP_SYS_PRODDIR SP_OCT_ASM_SID="+ASM1" ; export SP_OCT_ASM_SID - (different for each host in the cluster) Setup - Continued Create Silo A TNS Entries: Utah Silo A – Node 1: INST_A_SP = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = utdb01a)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = utdb02a)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = inst_a) (INSTANCE_NAME = inst_a1) ) ) Utah Silo A – Node 2: INST_A_SP = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = utdb01a)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = utdb02a)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = inst_a) (INSTANCE_NAME = inst_a2) ) ) Setup - Continued Create Silo B TNS Entries: Utah Silo B – Node 1: INST_B_SP = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = utdb01b)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = utdb02b)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = inst_b) (INSTANCE_NAME = inst_b1) ) ) Utah Silo B – Node 2: INST_B_SP = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = utdb01b)(PORT = 1521)) (ADDRESS = (PROTOCOL = TCP)(HOST = utdb02b)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = inst_b) (INSTANCE_NAME = inst_b2) ) ) Setup - Continued Create the required oratab entries: For Silo A (all nodes): inst_a:/u01/app/oracle/product/11.2.0/dbhome_1:N inst_a_sp:/u01/app/oracle/product/11.2.0/dbhome_1:N For Silo B (all nodes): inst_b:/u01/app/oracle/product/11.2.0/dbhome_1:N inst_b_sp:/u01/app/oracle/product/11.2.0/dbhome_1:N Make sure the entries look like this: inst_b_sp:/u01/app/oracle/product/11.2.0/dbhome_1:N Not this: inst_b_sp:/u01/app/oracle/product/11.2.0/dbhome_1:N # Line added by agent Setup - Continued Create and mount an NFS share for each RAC cluster. This mount point will contain the SharePlex installation and replication configuration files. You must use the following mount options: rw,hard,proto=tcp,rsize=32768,wsize=32768,intr,timeo=600,retrans=2,vers=3,llock,suid See: https://support.software.dell.com/shareplex-for-oracle/kb/30450 Setup - Continued Install SharePlex (on one node of each cluster): cd /dba/software/quest/splex tar -xf ./SharePlex-8.0.1-b40-oracle110-sun-11-sparc-m64.tar ./SharePlex-8.0.1-b40-oracle110-sun-11-sparc-m64.tpm Please enter the product directory location? /u02/app/quest/sp (/u02 is the NFS mount created earlier) Please enter the variable data directory location? /u02/app/quest/vardir Please specify the SharePlex Admin group (select a number): 1. [oinstall] 2. dba ? 1 Please enter the ORACLE_SID that corresponds to this installation? [inst_a1] inst_a (Needs to be the RAC TNS entry) Please enter the ORACLE_HOME directory that corresponds to this ORACLE_SID? /u01/app/oracle/product/11.2.0/dbhome_1 Please enter the TCP/IP port number for SharePlex communications? [2100] <enter> Proceed with installation? [yes] <enter> Do you have a valid SharePlex for Oracle v. 8.0.1 license? [yes] <enter> Please enter the License key? SORRYCANTSHARETHISWITHYOUPEOPLE Please enter the customer name associated with this license key? XACTWARE INC Setup - Continued Configure SharePlex on Silo A ONLY (one node): cd /u02/app/quest/sp/bin ./ora_setup Enter the Oracle SID for which SharePlex should be installed [inst_a1] : inst_a_sp (this is the new sp TNS alias) Enter a DBA user name : <sysdba user> Enter password for the DBA account, which will not echo : <password>@inst_a_sp Would you like to create a new SharePlex user ? [y] : y Enter username for new user [SPLEX/SPLEX] : splex Enter password for new user : splexpwd@inst_a_sp Re-enter password for new user : splexpwd@inst_a_sp Do you want to enable replication of tables with TDE? [n] : <enter> Enter the default tablespace for use by SharePlex [USERS] : SPLEXDAT (The data tablespace created earlier) Enter the temporary tablespace for use by SharePlex [TEMP] : <enter> Enter the index tablespace for use by SharePlex [ ] : SPLEXIDX (The index tablespace created earlier) Oracle ASM detected. Enable SharePlex ASM support? [y] : <enter> Setup - Continued For RAC configurations add License for other nodes Run the following on the Silo A host to be added: /u02/app/quest/sp/util/splex_uname|grep “Host ID” Host ID = 2230858275 On the host where the SharePlex VIP is running: /u02/app/quest/sp/install/splex_add_key 3) Add license key for alternate host Enter option: 3 Adding license for machine ID : 2230858275 (Host ID from splex_uname above) 2) Enter license key manually Enter option: 2 Enter Key: SORRYCANTSHARETHISWITHYOUPEOPLE Enter Customer Name: XACTWARE INC The SharePlex license has been successfully added for host 2230858275. q) Quit License Utility Enter option: q Setup – Continued Verify the paramdb file on Silo A: cd /u02/app/quest/vardir view paramdb SP_COP_TPORT 2100 SP_COP_UPORT 2100 SP_OCT_TDE_SHARED_SECRET "" # SPO TDE Shared Secret SP_OCT_ASM_SUPPORT "1" # ASM support on or off SP_ORD_LOGIN_O.inst_a_SP "77bff9a39a0c2bb430c6c013e35ba8617cc4521279ff" SP_ORD_OWNER_O.inst_a_SP "SPLEX" SP_SYS_LIC_2230925271 "SORRYCANTSHARETHISWITHYOUPEOPLE:XACTWARE INC" SP_SYS_LIC_2231029195 "SORRYCANTSHARETHISWITHYOUPEOPLE:XACTWARE INC" SP_SHS_SHMSIZE "33554432" # Statistics shared-memory Size SP_OCT_USE_DST "0" # If 1, factor in DST when computing REDOLOG_ENSURE SP_OCT_REPLICATE_SEQUENCES 0 SP_OCT_REPLICATE_DDL 0 SP_OCT_AUTOADD_ENABLE 0 SP_DEQ_THREADS "1" # Number of compare threads to run Make sure the following entry does not exist: SP_OCT_ASM_SID="+ASM1" Setup – Continued Verify the paramdb file on Silo B: cd /u02/app/quest/vardir view paramdb SP_COP_TPORT 2100 SP_COP_UPORT 2100 SP_OCT_TDE_SHARED_SECRET "" # SPO TDE Shared Secret SP_OCT_ASM_SUPPORT "1" # ASM support on or off SP_ORD_LOGIN_O.inst_b_SP "77bff9a39a0c2bb430c6e89c013ba8617cc4511279ff" SP_ORD_OWNER_O.inst_b_SP "SPLEX" SP_SYS_LIC_2231091337 "SORRYCANTSHARETHISWITHYOUPEOPLE:XACTWARE INC" SP_SYS_LIC_2231027673 "SORRYCANTSHARETHISWITHYOUPEOPLE:XACTWARE INC" SP_SHS_SHMSIZE "33554432" # Statistics shared-memory Size SP_OCT_USE_DST "0" # If 1, factor in DST when computing REDOLOG_ENSURE SP_OCT_REPLICATE_DDL 0 SP_OCT_AUTOADD_ENABLE 0 SP_DEQ_THREADS "1" # Number of compare threads to run Make sure the following entry does not exist: SP_OCT_ASM_SID="+ASM1" Setup – Continued SharePlex Configuration Settings: SP_OCT_USE_DST Note: If 1, factor in DST when computing REDOLOG_ENSURE (default 1) SP_OCT_REPLICATE_SEQUENCES Note: If 1, replicate sequences (default 1) SP_OCT_REPLICATE_DDL Note: If 1, replicate alter/truncate statements (default 1) SP_OCT_AUTOADD_ENABLE Note: If 1, auto add new tables that match wildcard configuration (default 1) SP_DEQ_THREADS Note: This defines the number of compare threads to run (dynamic) See: http://documents.software.dell.com/DOC136923 Configuration Create the Silo A SharePlex configuration file: cd /u02/app/quest/vardir/config vi schema_name_all datasource:o.inst_a_SP #source tables target tables routing map expand schema_name.% schema_name.% inst_b-splex-vip@o.inst_b_SP Note: This will replicate the schema_name schema from silo a to silo b. Configuration - Continued Create the Silo B SharePlex configuration file: cd /u02/app/quest/vardir/config vi schema_name_all datasource:o.inst_b_SP #source tables target tables routing map expand schema_name.% schema_name.% inst_b-splex-vip@o.inst_a_SP Note: This will replicate the schema_name schema from silo b to silo a. Configuration - Continued Start SharePlex: On silo A and silo B (on the host with SP VIP): sp_cop & Stop Target Post: On silo B ONLY (on the host with SP VIP): sp_ctrl stop post Activate Config: On the Silo A (on the host with the SP VIP): sp_ctrl activate config schema_name_all Note: the “activate config” begins tracking changes to the schema schema_name. Configuration - Continued Verify Process State: On both silo A and silo B sp_ctrl status Brief Status for inst_a-splex-vip Process State --------------- -----------------------------Cop Running Cmd & Ctrl Running Capture Running Read Running Post Stopped by user PID -------4207 4272 5236 5310 5336 Running Since -------------------01-Nov-13 17:10:24 01-Nov-13 17:12:55 01-Nov-13 17:15:41 01-Nov-13 17:15:46 01-Nov-13 17:15:51 Note: On silo B the status of post should be “Stopped by user” Configuration - Continued The next step is to duplicate the Silo A database to the Silo B database. It does not matter which method you use to get the source copied to the target (impdp, rman, etc.) as long as you have a consistent SCN to pass to SharePlex. When using rman make sure to record the following from the duplication log file: scn 322033901803 Setup Configure SharePlex on Silo B ONLY (Do NOT create a new SP user): cd /u02/app/quest/sp/bin ./ora_setup Enter the Oracle SID for which SharePlex should be installed [inst_a1] : inst_b_sp (this is the new sp TNS alias) Enter a DBA user name : <sysdba user> Enter password for the DBA account, which will not echo : <password>@inst_b_sp Would you like to create a new SharePlex user ? [y] : n Enter username of an existing user : splex Enter password for splex : splexpwd@inst_b_sp Do you want to enable replication of tables with TDE? [n] : <enter> Enter the default tablespace for use by SharePlex [USERS] : SPLEXDAT (The data tablespace created earlier) Enter the temporary tablespace for use by SharePlex [TEMP] : <enter> Enter the index tablespace for use by SharePlex [ ] : SPLEXIDX (The index tablespace created earlier) Oracle ASM detected. Enable SharePlex ASM support? [y] : <enter> Setup - Continued For RAC configurations add License for other nodes Run the following on the Silo B host to be added: /u02/app/quest/sp/util/splex_uname|grep “Host ID” Host ID = 2231027673 On the host where the SharePlex VIP is running: /u02/app/quest/sp/install/splex_add_key 3) Add license key for alternate host Enter option: 3 Adding license for machine ID : 2231027673 (Host ID from splex_uname above) 2) Enter license key manually Enter option: 2 Enter Key: SORRYCANTSHARETHISWITHYOUPEOPLE Enter Customer Name: XACTWARE INC The SharePlex license has been successfully added for host 2231027673. q) Quit License Utility Enter option: q Setup – Continued Verify the paramdb file on Silo B: cd /u02/app/quest/vardir view paramdb SP_COP_TPORT 2100 SP_COP_UPORT 2100 SP_OCT_TDE_SHARED_SECRET "" # SPO TDE Shared Secret SP_OCT_ASM_SUPPORT "1" # ASM support on or off SP_ORD_LOGIN_O.inst_b_SP "77bff9a39a0c2bbeab430c6e89c013e35ba8617cc4511279ff" SP_ORD_OWNER_O.inst_b_SP "SPLEX" SP_SYS_LIC_2231091337 "SORRYCANTSHARETHISWITHYOUPEOPLE:XACTWARE INC" SP_SYS_LIC_2231027673 "SORRYCANTSHARETHISWITHYOUPEOPLE:XACTWARE INC" SP_SHS_SHMSIZE "33554432" # Statistics shared-memory Size SP_OCT_USE_DST "0" # If 1, factor in DST when computing REDOLOG_ENSURE SP_OCT_REPLICATE_DDL 0 SP_OCT_AUTOADD_ENABLE 0 SP_DEQ_THREADS "1" # Number of compare threads to run Make sure the following entry does not exist: SP_OCT_ASM_SID="+ASM1" Setup - Continued Lookup Datasource Name (Silo B): Queues Statistics for inst_b-splex-vip Name: inst_a-splex-vip (o.INST_A_SP-o.INST_B_SP) (Post queue) Number of messages: 1029659 (Age 24 min; Size 485 mb) Backlog (messages): 0 (Age 0 min)) Reconcile Queues: On Silo B - issue the reconcile command. sp_ctrl reconcile queue queuename for datasource-datadest scn scn_number Example: reconcile queue inst_a-splex-vip for o.INST_A_SP-o.INST_B_SP scn 322033901804 If this Process hangs it is likely related to no activity on the source. Log into the source database and create at least one transaction, then force a log switch. NOTE: This will not apply transactions yet... Setup - Continued Cleanup Target Tables: Run the cleanup.sql script to truncate all of the SharePlex internal tables. Only run this on Silo B: cd /u02/app/quest/sp/bin sqlplus splex/<pass>@inst_b_sp @cleanup.sql Setup - Continued Modify Triggers: On Silo B (or both silos if you are bi-directional), disable triggers on the replicated tables, or run the sp_add_trigger.sql utility script so that the triggers ignore the SharePlex user. This will prevent circular replication. cd /u02/app/quest/sp/util sqlplus “/as sysdba” @sp_add_trigger.sql Enter SharePlex username --> splex Verify all the triggers in the replicated schema have the following: WHEN ( USER != 'SPLEX' ) Setup - Continued Before moving on with the next steps, make sure that there are no replicated tables with constraints that are “on delete cascade”. This will cause issues with replication and must be disabled. ALTER TABLE table_name ADD CONSTRAINT constraint_name FOREIGN KEY (column) REFERENCES table_name (column_name) ON DELETE CASCADE ENABLE VALIDATE NOTE: If you are planning on bi-directional replication, neither silo can have this set. Setup - Continued Sequences: If you are planning on active-active in your configuration then you have 2 options for sequences. (1) - Alter one cluster to odd and the other to even and increment by 2. Example: select sequence.nextval from dual; (until odd or even), then alter sequence sequence_name increment by 2; (2) - Use sequence ranges for each silo in the cluster. The recommended method is to use odd/even to avoid conflicts. Determine the best method for your setup and make the sequence changes before moving on with the next step. Setup - Continued Start SharePlex Post Process: On silo B, start the Post process. sp_ctrl start post NOTE: Silo B will now post the changes from the beginning of the duplicate database until current. May take some time to catch up. Monitor Queue Status: On Silo A and Silo B: sp_ctrl qstatus CRS Configuration Create Start / Stop Script: logfile="/u02/app/quest/scripts/log/spctl.log" ; export logfile sp_env="/export/home/oracle/.bashrc.inst_b" ; export sp_env sp_start_cmd='/u02/app/quest/sp/bin/sp_cop -uinst_b &' spctrl='/u02/app/quest/sp/bin/sp_ctrl' case "$1" in 'start') echo "Starting Shareplex" | tee -a ${logfile} su - oracle <<EOF source ${sp_env} set|grep "SP_" >> ${logfile} ${sp_start_cmd} >> ${logfile} sleep 30 exit EOF echo | tee -a ${logfile} ;; 'stop') echo "Stopping Shareplex" | tee -a ${logfile} su - oracle <<EOF source ${sp_env} set|grep "SP_" >> ${logfile} CRS Configuration echo "shutdown" | ${spctrl} | tee -a ${logfile} exit EOF echo | tee -a ${logfile} ;; ‘check') echo "Checking Shareplex" | tee -a ${logfile} sp_check=$(ps -ef|grep "sp_cop"|grep -v grep|wc -l) if [ ${sp_check} -gt 0 ] ; then echo "Shareplex is running" | tee -a ${logfile} echo | tee -a ${logfile} exit 0 else echo "Shareplex is not running " | tee -a ${logfile} echo | tee -a ${logfile} exit 1 fi ;; 'clean') echo "Cleaning Shareplex" | tee -a ${logfile} ;; esac CRS Configuration Create the CRS Resource: On Silo A and Silo B (as root): ${CRS_HOME}/bin/crsctl add resource shareplex -type cluster_resource -attr "AGENT_FILENAME="%"CRS_HOME"%"/bin/scriptagent, ACTION_SCRIPT=/u02/app/quest/scripts/spctl.sh, DESCRIPTION=, DEGREE=1, ENABLED=1, AUTO_START=always, START_TIMEOUT=0, UPTIME_THRESHOLD=1h, CHECK_INTERVAL=10, STOP_TIMEOUT=0, SCRIPT_TIMEOUT=60, RESTART_ATTEMPTS=2, OFFLINE_CHECK_INTERVAL=0, START_DEPENDENCIES=hard(shareplex.vip) pullup(shareplex.vip), STOP_DEPENDENCIES=hard(intermediate: shareplex.vip),CARDINALITY=1, FAILURE_INTERVAL=0, FAILURE_THRESHOLD=0, SERVER_POOLS=*, PLACEMENT=balanced, LOAD=1, ACTIVE_PLACEMENT=1” Set CRS Resource Permissions: On Silo A and Silo B (as root): ${CRS_HOME}/bin/crs_setperm shareplex -u user:oracle:r-x Administration Useful CRS Commands: Start: ${CRS_HOME}/bin/crsctl start resource shareplex VIP will start automatically # ${CRS_HOME}/bin/crsctl start resource shareplex.vip Stop: ${CRS_HOME}/bin/crsctl stop resource shareplex ${CRS_HOME}/bin/crsctl stop resource shareplex.vip Relocate: ${CRS_HOME}/bin/crsctl relocate resource shareplex -s <existing node> -n <new node> -f Administration - Continued sp_ctrl commands: status – checks the status of shareplex processes qstatus – shows the current status of the shareplex queues shutdown – shutdown the shareplex sp_cop process show log reverse – shows the event_log in reverse order show post detail – show a detail view of the post process show capture detail – show a detail view of the capture process show sync – shows if there are known out of sync tables compare – compares a table or wildcard list of tables repair – repairs a table or wildcard list of tables compare status – shows the current status of a compare process repair status – shows the current status of a repair process Administration - Continued Compare/Repair Table: sp_ctrl > compare sp_demo.test ID Tablename Status Time ------ ------------------------------------ ---------- ---------1 ”SP_DEMO"."TEST" Out Sync N/A Total Rows ---------2 %Comp ----100 Total Time ---------0:07 Total Rows ---------0 %Comp ----100 Total Time ---------0:07 sp_ctrl > repair sp_demo.test ID Tablename Status Time ------ ------------------------------------ ---------- ---------1 ”SP_DEMO"."TEST" Repaired N/A NOTE: After running compare and finding “out of sync” it will not show in “show sync” Administration - Continued Adding an object to existing configuration: (1) Create the object on the source (and target if not using DDL replication). (2) sp_ctrl > activate config schema_name_all NOTE: Do not deactivate configuration as you will lose transactions and be forced to re-synchronize the source and target environments. Final Thoughts We are pleased with the overall value of the SharePlex product. The cost is reasonable for the functionality it provides, and we are satisfied with the stability and performance as well. If you combine that with the ease of setup and administration it becomes clear that SharePlex is a good fit for Xactware.