KB 150078 HowTo cleanup the IDENTIKEY Authentication Server audit database on Windows. Creation date: 13/04/2012 Last Review: 02/04/2013 Document type: How To Revision number: 5 Security status: EXTERNAL Summary Starting from IDENTIKEY Authentication server 3.4, by default auditing and reporting are to/from a PostgreSQL ODBC database. When the IDENTIKEY Authentication Server is used intensively or when LDAP Sync is used, the database can grow big. This article describes how to clean up the database. Problem details. IDENTIKEY Authentication Server (IK) does not contain tools to remove auditing data from an ODBC database. For archiving we need to be able to remove older records from the database. When using the LDAP Sync tool every query on user attributes is logged. When the LDAP Sync is used on a large number of users and/or with a high frequency, the number of audit records will grow fast. To reduce the size of the logged audit data when LDAP Sync is used, we can remove all records related to the query of a user attribute, as these are the majority of the records produced by the LDAP Sync, and can be seen as less important. We also need to be able to schedule this task so that it can be done on a regular basis. The audit data is stored in two tables in the Database. There is CONSTRAINT between the two tables that prevents deleting records in the main table when there are still records in the second table that are connected to it. In this KB we will: 1. Provide instructions to change the CONSTRAINT to be able to remove records in both tables. 2. Provide instructions for removing older audit data 3. Provide instruction for removing less important LDAP sync audit data 4. Provide instruction for scheduling 5. Provide instruction for freeing disk space using vacuum Before removing audit data from the database it is advised to make a backup of it using pg_dump. (See the IK Administrator guide for more details) Please see the following KB for LINUX installation instructions: Applies to: IK3.4 KB 150078 – 02/04/2013 2012 VASCO Data Security. All rights reserved. Page 1 of 7 http://www.vasco.com/Images/KB_160012.pdf Problem Solution. 1. Change the CONSTRAINT to be able to remove records in both tables. • Open pgAdmin III (Start – All Programs – PostgreSQL 8.3 – pgAdmin III) • Open the postgres database. ( The default password is: digipassword ) • Browse to the vdsauditmsgfield table constraints vdsauditmsgfield0 • Change the constraint from: ALTER TABLE vdsauditmsgfield ADD CONSTRAINT vdsauditmsgfield0 FOREIGN KEY (vdstimestamp, vdsamid) REFERENCES vdsauditmsg (vdstimestamp, vdsamid) MATCH SIMPLE ON UPDATE RESTRICT ON DELETE RESTRICT; To: ALTER TABLE vdsauditmsgfield ADD CONSTRAINT vdsauditmsgfield0 FOREIGN KEY (vdstimestamp, vdsamid) Applies to: IK3.4 KB 150078 – 02/04/2013 2012 VASCO Data Security. All rights reserved. Page 2 of 7 REFERENCES vdsauditmsg (vdstimestamp, vdsamid) MATCH SIMPLE ON UPDATE RESTRICT ON DELETE CASCADE; o Click on SQL SQL field. and copy the Alter Table vdsauditmsgfield drop.. to the o Click on the green arrow constraint. o Copy the text : to run the query. This will remove the ALTER TABLE vdsauditmsgfield ADD CONSTRAINT vdsauditmsgfield0 FOREIGN KEY (vdstimestamp, vdsamid) REFERENCES vdsauditmsg (vdstimestamp, vdsamid) MATCH SIMPLE ON UPDATE RESTRICT ON DELETE CASCADE; to the SQL field and run the query clicking on the green arrow. This will create the modified constraint. Applies to: IK3.4 KB 150078 – 02/04/2013 2012 VASCO Data Security. All rights reserved. Page 3 of 7 o Click a few times on the refresh button to see the new constraint. 2. instructions for removing older audit data You can now use following SQL query to remove older data (adapt date to your needs): Delete from vdsauditmsg where vdstimestamp < '2012-03-17'; Ps: If you first want to have an idea on the number of records you will delete, you can run: Select count(*) from vdsauditmsg where vdstimestamp < '2012-03-17'; Applies to: IK3.4 KB 150078 – 02/04/2013 2012 VASCO Data Security. All rights reserved. Page 4 of 7 PS: See further how to use vacuum to free disk space. 3. instruction for removing less important LDAP sync audit data You can use following SQL query for deleting all audit records recording the query of a user attribute: Delete from vdsauditmsg where vdsdesc = ‘A query for [User Attributes] records was successful.’; If you first want to have an idea on the number of records you will delete, you can run: Select count(*) from vdsauditmsg where vdsdesc = ‘A query for [User Attributes] records was successful.’; PS: See further how to use vacuum to free disk space. 4. instruction for scheduling To run the query from a bat file we use the psql command with some options. • Problem here is that this command asks for a password, therefore the same instructions as in KB120110 apply. Create a textfile with the name pgpass.conf. This file can be located where ever you want. Enter the following line in the pgpass.conf file: localhost:5432:postgres:[user]:[password] [user] = the username to be used is: digipass [password] = the password for the user digipass is by default: digipassword • Create a batch file with the following contents: set PGPASSFILE=[full path]\pgpass.conf “C:\Program Files\VASCO\IDENTIKEY 3.4\PostgreSQL\bin\psql” -d postgres -U digipass -c "delete from vdsauditmsg where vdsdesc='A query for [User Attributes] Applies to: IK3.4 KB 150078 – 02/04/2013 2012 VASCO Data Security. All rights reserved. Page 5 of 7 records was successful.';" Eventually you can also redirect the output to a file in order to see how many records where deleted and add the vacuum operation (see further). For more information on the psql command, use psql --help. 5. instruction for freeing Disk space using Vacuum When records are deleted in postgreSQL they are not physically removed from their table; they remain present until a VACUUM is done. See: http://www.postgresql.org/docs/8.3/static/sql-vacuum.html Using pgadminIII: When the free disk space is so small that a vacuum is not possible any more, see KB160020 or use the procedure below: 1. 2. 3. 4. 5. 6. 7. 8. Backup all of the databases on your postgresql server Backup your configuration files Stop Postgresql Erase the contents of the data directory Run initdb to reinitalize your data directory Restore configuration files Start Postgresql Restore the dump of all the databases you made Details on this procedure can be found on www.postgresql.org. Applies to: IK3.4 KB 150078 – 02/04/2013 2012 VASCO Data Security. All rights reserved. Page 6 of 7 Applies to: IK3.4 KB 150078 – 02/04/2013 2012 VASCO Data Security. All rights reserved. Page 7 of 7