NESUG 16
Pharmaceuticals
PH003
Date/Time Stamped Files and Audit Trails: What Part 11 Compliant SAS®
Systems are Made of.
Carolyn Dougherty, ViroPharma Incorporated, Exton, PA
ABSTRACT
Clinical data reporting systems are considered high risk with respect to 21 CFR Part 11 compliance. The FDA’s
recent change in scope for Part 11 is simply a narrower interpretation; Part 11 is still a requirement and therefore,
clinical data reporting systems are still subject to Part 11. Analysis of clinical trial data is usually conducted using
SAS® software.
Datasets extracted from a clinical data management database (raw datasets) and
submission/analysis datasets are created with SAS®. In addition, data are summarized using SAS®. How can
processes followed to produce these datasets and analyses be made compliant? Date/time stamped files and
audit trails are two key components of Part 11 that, if maintained in the process of reporting clinical trial data, will
greatly improve compliance. Maintaining an audit trail for dataset and table generation, and tracking date/time
stamps of these files and the SAS® programs used to create them are not small tasks. This tracking, including
who created what and when, is often a manual process or a non-existent process. This paper describes an
automated system that tracks generation of all SAS® programs, datasets and analyses; a capability not inherently
available from SAS®. This system improves Part 11 compliance in a clinical reporting environment.
INTRODUCTION
Reporting of clinical trial data for a clinical study report requires a large amount of programming to produce
analyses of the data. Beginning with raw datasets, programs are created to generate submission or analysis
datasets followed by programs created to produce analyses as required in the statistical analysis plan. This
means, for a single clinical study, there could be one hundred or more programs used to produce multiple
datasets and sometimes hundreds of summary tables. 21 CFR Part 11 requires data controls be implemented
within systems that “create, modify, maintain, or transmit electronic records”. Therefore, reporting of clinical trial
data clearly falls within Part 11 requirements. The requirements further state:
“Such procedures and controls should include use of secure, computer generated, time-stamped
audit trails to independently record the date and time of operator entries and actions that create,
modify, or delete electronic records. Record changes shall not obscure previously recorded
information. Such audit trail documentation shall be retained for a period at least as long as that
required for the subject electronic records and shall be available for agency review and copying.”
Therefore, it is important per Part 11 to maintain an audit trail of programs, datasets and output included in the
process of reporting clinical trial data. Per the FDA Guidance on Computerized Systems Used in Clinical Trials,
an audit trail is a secure, time stamped record that allows reconstruction of the course of events relating to the
creation, modification and deletion of an electronic study record.
This paper will describe three macros created to produce an audit trail containing information about summary
tables in a clinical study report, including information about the datasets used to generate the summary tables.
We took the approach described below to generate an audit trail because SAS® and PDF (Portable Document
Format; the file format of summary tables) did not have an easy mechanism for storing specific metadata about
submission datasets and tables.
CLINICAL REPORTING DELIVERABLES INCLUDED IN THE AUDIT TRAIL
First, we began with identifying what information and physical files could be ‘tracked’ as part of the audit trail from
our existing process for reporting clinical trial data. As mentioned, analysis programming for a clinical trial begins
with raw datasets and continues through production of summary tables. The process is depicted in the following
basic figure and identifies the main deliverables for tracking in an audit trail:
1
NESUG 16
Pharmaceuticals
Clinical Data
Management
Database
Raw
Dataset
s
SubmissionDataset.sas7bdat
DataSetProgram.sas
SummaryTable.pdf
TableProgram.sas
Dataset Program
Audit Trail File
(DataSetProgram.sas7bdat)
Table Program
Audit Trail File
(TableProgram.sas7bdat)
The above process is easier to understand by focusing on only one specific CRF panel. For example, the figure
below describes the reporting process for medical history data.
Medical
History
Database
Mhx_r.sas7bdat
Medhis.sas7bdat
u_medhis.sas
smhx.sas
Audit trail file:
u_medhis.sas7bdat
smhx.pdf
Table 10.1.5
Summary of Medical History
Audit trail file:
Smhx.sas7bdat
Table 10.1.5, Summary of Medical History, is only one summary table from a complete table of contents for a
clinical study report. The remaining examples in this paper, will use medical history to identify audit trail
information. However, the same information collected for the audit trail of medical history is collected for all
submission datasets and all summary tables for a study.
SUBMISSION OR ANALYSIS DATASETS
When producing the submission dataset medhis.sas7bdat, the macro %trackraw is executed in the program
u_medhis.sas. %trackraw produces the audit trail file, u_medhis.sas7bdat, with the following information:
2
NESUG 16
Pharmaceuticals
Audit trail file: u_medhis.sas7bdat:
• Name of raw dataset(s) used to generate the submission dataset
• Raw dataset(s) date/time stamp
• Name of program (DataSetProgram.sas) used to generate the submission dataset
• Userid of programmer who generated the submission dataset (i.e. executed the program)
• Location/server on which the submission dataset was generated
The audit trail file for each submission dataset includes one record per raw dataset used to generate the
submission dataset. Below is an example of an audit trail file for a submission dataset. In this example, only one
raw dataset was required to generate the submission dataset medhis.sas7bdat.
Contents of the dataset program audit
Raw
Raw Dataset Date/Time
Dataset
mhx_r
20DEC2002:10:39:00
trail file, u_medhis.sas7bdat:
Submission
Dataset Program
User ID
u_medhis
cdougherty
Location ID
statsrv01
In addition, when a submission dataset is generated, the name of the program used to generate the submission
dataset is stored as the label of the submission dataset. This allows us to easily identify what program generated
the dataset; an important piece of the audit trail information noted later in the paper when producing the audit trail
reports.
SUMMARY TABLES
The macro %mtltrack is executed within the table program, smhx.sas, to generate the summary table, smhx.pdf.
%mtltrack produces the audit trail file smhx.sas7bdat with the following information:
Audit trail file: smhx.sas7bdat
• Table Number, Title and Subset (extracted from an electronic table of contents)
• Name of submission dataset(s) used to generate the table
• Name of program (DataSetProgram.sas) used to generate the submission dataset (taken from the label
on the submission dataset)
• Submission dataset date/time stamp
• Name of program (TableProgram.sas) used to generate the table (SummaryTable.pdf)
• TableProgram.sas date/time stamp
• Date/time the table (SummaryTable.pdf) was generated
• Total number of pages in the table
• Userid of programmer who generated the table (SummaryTable.pdf)
• Location/server on which the table (SummaryTable.pdf) was generated
The audit trail file for the each summary table includes one record per submission dataset used to generate the
summary table. Below is an example of an audit trail file for a summary table. In this example, two submission
datasets were required to generate the summary table smhx.pdf.
3
NESUG 16
Pharmaceuticals
Contents of the table program audit trail file, smhx.sas7bdat:
Table #
Table Title
Table Subset
10.1.5
Summary
of Medical
History
All
Randomized
Subjects
10.1.5
Summary
of Medical
History
All
Randomized
Subjects
Submission
Dataset
medhis
Dataset
Program.
sas
u_medhis
demo
u_demo
Submission
Dataset
Date/Time
Stamp
11MAR2003:
12:44:11
13FEB2003:
10:28:10
Table
Program.
sas
smhx
Date/Time Stamp
TableProgram.sas
Date/Time Stamp
Table Generated
Total
Pages
User ID
Location
ID
22JAN2003:
11:32:00
11MAR2003:
12:47:00
1
cdougherty
statsrv01
smhx
22JAN2003:
11:32:00
11MAR2003:
12:47:00
1
cdougherty
statsrv01
4
NESUG 16
Pharmaceuticals
AUDIT REPORT
Although some of the above information which is stored in audit trail files may seem redundant, the files are
human readable and therefore can be referenced easily at any time in the process. Storing these files makes the
following information readily available when the generation of summary tables for a clinical study report is
completed. Execution of the macro %audittrl combines all audit trail data and ‘real-time’ date/time stamps into
one dataset and audit_trail.pdf is created.
Dataset Audit Trail File:
u_medhis.sas7bdat
Table Audit Trail File:
Real-Time Data:
smhx.sas7bdat
• Current date/time of submission dataset
• Current date/time of SummaryTable.sas
• Current date/time of raw dataset
Audit_Trail.pdf
The three macros executed in this audit trail process can be validated to confirm that expected results are
provided. However, it is important to note that a piece of the system relies on programming practices. If the
submission dataset program macro (%trackraw) or the summary table program macro (%mtltrack) is not called in
the appropriate programs, the Audit_Trail.pdf would be a failing audit and appropriate submission dataset
programs and summary table programs would have to be re-executed to produce a passing audit.
The following reports are included in Audit_Trail.pdf. Examples of each report are included on subsequent pages.
AUDIT TRAIL REPORT #1:
This report identifies whether the audit trail is a passing or failing audit trail. It confirms that the datasets and
tables are acceptable (date/time stamps are in the appropriate order and the sequence of events can be traced)
or will identify where problems exist. Note that the parenthetical letters for each bullet item below correspond to
the letters in the example on the next page.
•
•
•
•
(A) The report confirms the submission dataset(s) has not changed since the SummaryTable.pdf was
generated (medhis.sas7bdat is dated before the smhx.pdf and date/time stamp of medhis.sas7bdat used
to create smhx.sas is equal to current date/time stamp of medhis.sas7bdat).
(B) The report confirms the TableProgram.sas has not changed since the SummaryTable.pdf was
generated (smhx.sas is dated before smhx.pdf and date/time stamp of smhx.sas used to create smhx.pdf
is equal to current date/time stamp of smhx.sas). Notice the example audit fails this test because the
table program (SDISP) is currently dated (04JUN2003:14:48) after the table was generated
(04JUN2003:11:59).
(C) The report confirms the DatasetProgram.sas has not been changed since the submission dataset was
generated (u_medhis.sas is dated before medhis.sas7bdat). This is why attaching the name of the
submission dataset program as the label for the submission dataset was necessary.
(D) The report confirms the raw data has not changed since submission datasets and tables were
generated (current mhx_r.sas7bdat must be dated before current medhis.sas7bdat and smhx.pdf).
AUDIT TRAIL REPORT #2:
• The report identifies the order in which submission datasets were generated as well as the user ID of the
programmer who generated each dataset and the server name on which each dataset was generated.
AUDIT TRAIL REPORT #3:
• The report provides a complete list of tables and the date/time of each table for medical writers as well as
identification of who generated each table and the server name on which the table was generated.
AUDIT TRAIL REPORT #4:
• The report identifies the submission dataset(s) used when generating each summary table. This report
easily identifies all TableProgram.sas programs that must be re-run if a specific submission dataset is
changed.
5
NESUG 16
Pharmaceuticals
CONCLUSION
Three macros incorporated into a process from CRF data to final summary tables for a clinical study report can
help maintain a thorough audit trail for SAS® deliverables and improve Part 11 compliance. First, the macro
%trackraw is executed during the generation of each submission dataset to store metadata of the raw datasets
and submission dataset. Second, the macro %mtltrack is executed in each table program to store metadata for
the summary table when the summary table is generated. Last, %auditrtl is executed when tables are complete,
or at an interim timepoint, to combine all of the audit trail data into audit trail reports. Integration of the audit trail
data for the audit trail reports enables any programmer or manager to identify who generated what and when. In
addition, it allows individuals to verify that the sequence of editing programs, creating datasets and generating
tables is appropriate and that the sequence of these events can be easily traced. All of this information gathered
from the audit trail can be summarized in only a few minutes; much less time than if checked manually. Most
importantly, if audited by the FDA to confirm compliance with 21 CFR Part 11, the audit trail is readily available to
the FDA auditors.
6
NESUG 16
Pharmaceuticals
SAS Conference
Protocol DUMMY123
Jun 6, 2003 8:36
Audit Trail
AUDIT_TRAIL
Audit Trail Table #1 - All Tracking Information from Raw Datasets to Tables
Table
#
10.1.1
Title
Summary of
Subject
Disposition –
All Enrolled
Subjects
10.1.2
Summary of
Subject
Enrollment –
All Enrolled
Subjects
10.2.1
10.2.2
Table
Prog
Name
SDISP
SENRL
Table
Prog
Date
04JUN2003
11:59:00
Table
Prog
Date
NOW
04JUN2003
14:48:00
Table
Run
Date
04JUN2003
12:00:00
Submission
Dataset
Name
Submission
Dataset
Prog
Name
Submission
Dataset
Date
Submission
Dataset
Date
NOW
Sub-mission
Dataset
Prog Date
NOW
(A)
(B)
Submission
Dataset
Date=
Submission
Dataset
Date
NOW
Table
Prog
Date=
Table
Prog
Date
NOW
A_AE
U_A_AE
10MAR2003
16:26:40
10MAR2003
16:26:40
24FEB2003
14:06:00
ERROR
DISP
U_DISP
11MAR2003
12:22:44
11MAR2003
12:22:44
11MAR2003
12:21:00
ERROR
DEMO
U_DEMO
13FEB2003
10:28:10
13FEB2003
10:28:10
13FEB2003
10:24:00
ERROR
04JUN2003
12:08:00
04JUN2003
12:08:00
04JUN2003
12:08:00
DEMO
U_DEMO
13FEB2003
10:28:10
13FEB2003
10:28:10
13FEB2003
10:24:00
Summary of
SDEM
Subject
Demographics
– All Enrolled
Subjects
04JUN2003
11:58:00
04JUN2003
11:58:00
04JUN2003
11:59:00
DEMO
U_DEMO
13FEB2003
10:28:10
13FEB2003
10:28:10
13FEB2003
10:24:00
Summary of
SMHX
Medical History
– All Enrolled
Subjects
04JUN2003
12:13:00
04JUN2003
12:13:00
04JUN2003
12:13:00
MEDHIS U_MEDHIS
11MAR2003
12:44:11
11MAR2003
12:44:11
11MAR2003
12:43:00
DEMO
13FEB2003
10:28:10
13FEB2003
10:28:10
13FEB2003
10:24:00
U_DEMO
[Note: cells for the four columns on the right are green if the criteria for acceptance are met, or red if the criteria for acceptance are not met]
7
(C)
Submission
Dataset
Prog
Date
NOW<
Submission
Dataset
Date
(D)
Raw
Dataset
Date
NOW
= Raw
Dataset
Date
for
ADS
NESUG 16
Pharmaceuticals
SAS Conference
Protocol DUMMY123
Jun 6, 2003 8:36
Audit Trail
AUDIT_TRAIL
Audit Trail Table #2 - Order Submission Datasets Generated
Submission Dataset Name Submission Dataset Prog Name Submission Dataset Date NOW
User ID
Location ID
DEMO
U_DEMO
13FEB2003:10:28:10
cdougherty
statsrv01
EXCL
U_EXCLUS
13FEB2003:10:58:37
cdougherty
statsrv01
COMMENTS
U_COMM
13FEB2003:17:01:31
cdougherty
statsrv01
VAS
U_VAS
14FEB2003:11:29:41
cdougherty
statsrv01
A_VITAL
U_AVITAL
14FEB2003:14:22:09
cdougherty
statsrv01
VITALS
U_VITALS
20FEB2003:12:09:59
cdougherty
statsrv01
PE
U_PHYSEX
24FEB2003:16:54:02
cdougherty
statsrv01
A_AE
U_A_AE
10MAR2003:16:26:40
cdougherty
statsrv01
ECG
U_ECG
11MAR2003:10:13:20
cdougherty
statsrv01
A_ECG
U_A_ECG
11MAR2003:10:17:56
cdougherty
statsrv01
LAB
U_LAB
11MAR2003:11:31:15
cdougherty
statsrv01
A_LAB
U_A_LAB
11MAR2003:11:36:37
cdougherty
statsrv01
LABRANGE
U_LABRNG
11MAR2003:11:40:41
cdougherty
statsrv01
CONMEDS
U_CONMED
11MAR2003:12:08:14
cdougherty
statsrv01
DISP
U_DISP
11MAR2003:12:22:44
cdougherty
statsrv01
INCL
U_INCLUS
11MAR2003:12:37:19
cdougherty
statsrv01
MEDHIS
U_MEDHIS
11MAR2003:12:44:11
cdougherty
statsrv01
PKSAMP
U_PKSAMP
11MAR2003:12:51:35
cdougherty
statsrv01
8
NESUG 16
Pharmaceuticals
SAS Conference
Protocol DUMMY123
Jun 6, 2003 8:36
Audit Trail
AUDIT_TRAIL
Audit Trail Table #3 - Final Table Date/Time Stamps
Table
#
Title
Table Subset
Table
Prog
Name
Table Run Date
Programmer
Location
ID
10.1.1
Summary of Subject Disposition
All Enrolled Subjects
SDISP
04JUN2003:12:00:00
cdougherty
statsrv01
10.1.2
Summary of Subject Enrollment
All Enrolled Subjects
SENRL
04JUN2003:12:08:00
cdougherty
statsrv01
10.2.1
Summary of Subject Demographics
All Enrolled Subjects
SDEM
04JUN2003:11:59:00
cdougherty
statsrv01
10.2.2
Summary of Medical History
All Enrolled Subjects
SMHX
04JUN2003:12:13:00
cdougherty
statsrv01
10.2.3
Summary of Abnormal Physical Examination Findings at Baseline
All Treated Subjects
SPE
04JUN2003:12:15:00
cdougherty
statsrv01
12.1.1
Summary of Study Drug Exposure
All Treated Subjects
SSMED
04JUN2003:12:16:00
cdougherty
statsrv01
12.2.1.1
Overall Summary of Treatment Emergent Adverse Events
All Treated Subjects
SAE
04JUN2003:11:41:00
cdougherty
statsrv01
12.2.1.2
Overall Summary of Treatment Emergent Study Drug Related
Adverse Events
All Treated Subjects
SAE
04JUN2003:11:41:00
cdougherty
12.2.2
Summary of Treatment Emergent Adverse Events and Their
Relationship to Study Drug: No. Observed and Severity
All Treated Subjects
SAEPID
04JUN2003:11:42:00
cdougherty
12.2.3.1
Summary of Treatment Emergent Adverse Events By System
Organ Class, Preferred Term and Severity
All Treated Subjects
SAES
04JUN2003:11:44:00
cdougherty
12.2.3.2
Summary of Treatment Emergent Study Drug Related Adverse
Events By System Organ Class, Preferred Term
All Treated Subjects
SAES
04JUN2003:11:44:00
cdougherty
9
statsrv01
statsrv01
statsrv01
statsrv01
NESUG 16
Pharmaceuticals
SAS Conference
Protocol DUMMY123
Jun 6, 2003 8:36
Audit Trail
AUDIT_TRAIL
Audit Trail Table #4 - Identify Which Table Programs Use Which Submission Datasets
Submission Dataset
Name
A_AE
A_ECG
A_LAB
MEDHIS
Title
Table Pr og Name
Adverse Events Causing Death, and Other Serious or Significant Adverse Events
LAE
Overall Summary of Treatment Emergent Adverse Events
SAE
Summary of Treatment Emergent Adverse Events and Their Relationship to Study Drug: No. Observed and
SAEPID
Summary of Treatment Emergent Adverse Events By System Organ Class, Preferred Term and Severity
SAES
Summary of Treatment Emergent Adverse Events by Frequency
SAESF
Summary of Treatment Emergent Adverse Events of Interest
SAESH
Summary of Treatment Emergent Laboratory Adverse Events
SAESL
Summary of Subject Disposition
SDISP
12-Lead ECG: Change from Baseline QTc Intervals
LECGC
Summary of All Borderline and Prolonged QT Intervals
SECG
Summary of Abnormal 12-Lead ECG Intervals at End of Study
SECGA
Summary of Changes in 12-Lead ECG Intervals
SECGC
Summary of Clinically Significant 12-Lead ECG Intervals per Investigator
SECGCS
Listing of Clinical Laboratory Tests for Subjects with Treatment Emergent Clinically Significant Abnormal Laboratory Values per
Investigator
LLABS
Summary of Treatment Emergent Clinically Significant Abnormal Laboratory Values per Investigator
SLABC
Summary of Clinical Laboratory Evaluations: Change from Baseline
SLABS
Summary of Laboratory Values >= 3 Times the Upper Limit of Reference Range
SLABT
General Medical History
LMEDH
Summary of Medical History
SMHX
10
NESUG 16
Pharmaceuticals
ACKNOWLEDGMENTS
SAS is a Registered Trademark of the SAS Institute, Inc. of Cary, North Carolina.
Thanks to Spencer Hudson, whose idea it was to generate this audit trail system. Additional thanks to those who
helped review this paper so that it could be possible to present at NESUG: Spencer Hudson, ViroPharma
Incorporated; Heidi Shea, ViroPharma Incorporated; Dan Wang, ViroPharma Incorporated; Louisa Feeley,
ViroPharma Incorporated; Souma Chattopadhyay, ViroPharma Incorporated; and Daphne Ewing, Synteract, Inc.
CONTACT INFORMATION
Your comments and questions are greatly valued and appreciated. Please contact the author at:
Carolyn Dougherty
ViroPharma Incorporated
405 Eagleview Boulevard
Exton, PA 19341
Work Phone: (610) 321-6285
Fax: (610) 458-7380
Email: Carolyn.Dougherty@ViroPharma.com
Web: www.ViroPharma.com
11