Sizing of SDTM Variables and Datasets Two interrelated issues involving sizing considerations that have been identified for further clarification from the FDA are: Sizing of Variables/ Data Elements/Columns Overall File/Dataset Size Because of how these two issues are interrelated, a single discussion document addresses both of them. The following references have contributed to this document: 1. 2. 3. 4. 5. 6. SDTMIG 3.1.3: SDTM IG (V3.1.3/2012-07-16) SDTM-MSG: SDTM Metadata Submission Guideline (V1.0/2011-12-31) SDS: Study Data Specifications (V2.0/July 18, 2012) CDER-CDS: CDER Common Data Standards Issues Document (V1.1/December 2011) PBO: 2013PhUSEBreakoutCABVP-CBER-CDER, FDA/PhUSE CSC (March 2013) PWG1: Workgroup1-OpenCDISC_Updates, FDA/PhUSE CSC (March 2013?) These documents discuss these issues in various ways, sometimes in more than one location in a given document. To provide full context for the issues, a review of each document was performed, and relevant text has been extracted into appendices of this document. To provide a more concise discussion, however, details have been summarized, with differences highlighted. 1. Sizing of Variables/Data Elements/Columns See Appendix A for a full accounting of the text from various documents related to this issue. There are three main categories related to variable length to consider. For ease of review, each of these categories is summarized individually: 1. Column size constraints due to general CDISC and SAS Transport v5 requirements Various columns are identified as having a maximum length. This is mostly due to SAS Transport V5 requirements or, in the case of vertical data structures, values that could have constraints if the dataset were transformed to a horizontal structure (e.g., -TESTCD, --TEST). For values that could become variable names and labels, there are some implications that setting these to the maximum length is appropriate (e.g., SDTMIG states “--TESTCD and IDVAR will never be more than 8, so length can always be set to 8”). For domains split into multiple datasets, sizing decisions should be made prior to implementing the split, and variables should be consistently sized in each dataset. This holds true for SUPPQUAL datasets within a domain (e.g., SUPPLBHE and SUPPLBCH). This holds true for all FA-related datasets (i.e., all FA-related datasets, regardless of parent domain, must have consistent variable lengths, FA being it’s own domain). NB: My (C. Radovsky) recollection is that PhUSE minutes from the meeting in March indicated that FDA was OK with resizing per dataset, but I am not able to find these minutes. Possible Discussion Topics: Should any of these variables be subject to length evaluations? All or some (e.g., --TESTCD length of 8 OK, but not --TEST length of 40)? ? 2. Column size for variables associated with a Codelist The references related to Codelist length are recent developments, few and consistent: Codelist variable length should not be greater than the longest term in the Codelist. FDA material doesn’t call out this issue explicitly. It seems bundled under the general statement that variable length should not exceed value length. Possible Discussion Topics: Approach used to represent a Codelist for submission can impact decisions on length. Consolidating to a minimum set of codelists may reduce development overhead within sponsor organizations, but increases the size of the codelist variables. (Both consolidated and not are technically compliant approaches). ? 3. Column size issues not otherwise constrained by the prior two categories The references related to variable length previously constrained only the 200 character limit are also recent developments, few, and consistent: Length should be based on data values and not set to 200 characters by default. A small amount of padding is acceptable. Working Group 1 discussions addressed the issue as follows: Severity based on “excessive” chars in var length o 2-5 Chars – Information o 6-20 Chars – Warning o >20 Chars – Error Possible Discussion Topics: While the WG1 discussions came up with a reasonable interpretation of FDA language, it was “arbitrarily” reasonable, and met with resistance in the industry. Is this more a communication challenge than issue with the logic rule? Given that sponsors need to address all rule violations with a severity of Warning and Error, is there benefit in distinguishing the severity between the two? Consider phased implementation (e.g., warning/error when above 20 for the first year, warning/error when above 5 after that)? ? 2. Overall File/Dataset Size See Appendix B for a full accounting of the text from various documents related to this issue. References to file/dataset size and splitting decisions can be summarized as a whole: One GB is the target for maximum file size, with some flexibility. As noted under the variable length section, variable sizing issues need to be addressed prior to decisions on splitting datasets, and variable length should be consistent across split datasets. As with variable padding, a scale of severity is identified in some material, and is likely subject to the same considerations. CDISC references address splitting based on categorical variables (e.g., --CAT/--SCAT). FDA references note this, as well as other mechanisms such as grouping by subject number, but these other mechanisms are identified as inferior. Possible Discussion Topics: Is there a rationale/benefit to maintain references to splitting by less desirable methods? Per above, a scale of severity is identified in some material, and is likely subject to the same considerations as variable length severity. The FDA CDER Common Data Standards Issues Document mentions the use of a /SPLIT subdirectory, which is not identified in other material. That said, it is a critical piece of information, not only in providing a clear mechanism for submitting both split and not split datasets at the same time, but addresses procedural challenges with some software (e.g., to my recollection, WebSDM doesn’t handle having both versions of the data in a single folder). ? Appendix A Following the main document, details related to variable length are addressed in three main categories: 1. Column size constraints due to general CDISC and SAS Transport v5 requirements a. SDTMIG 3.1.3 (p. 28, section 188.8.131.52) i. Values of --TESTCD must be limited to 8 characters . . . ii. QNAM serves the same purpose as --TESTCD within supplemental qualifier datasets, and so values of QNAM are subject to the same restrictions as values of --TESTCD. iii. ETCD (the companion to ELEMENT) and TSPARMCD (the companion to TSPARM) are limited to 8 characters. . . . iv. ARMCD is limited to 20 characters and does not have special character restrictions. b. SDTMIG 3.1.3 (p. 35, linked section 184.108.40.206) i. The maximum SAS Version 5 character variable length of 200 characters should not be used unless necessary. ii. Sponsors should consider the nature of the data, and apply reasonable, appropriate lengths to variables. For example: The length of flags will always be 1 --TESTCD and IDVAR will never be more than 8, so length can always be set to 8 The length for variables which use controlled terminology can be set to the length of the longest term. c. SDTMIG 3.1.3 (p. 50, section 220.127.116.11.1) i. . . . the length of --TEST is normally limited to 40 characters to conform to the limitations of the SAS V5 Transport format currently used for submission datasets. ii. The convention above should also be applied to the Qualifier Value Label (QLABEL) in Supplemental Qualifiers (SUPP--) datasets. IETEST values in IE and TI are exceptions to the above 40-character rule and are limited to 200 characters since they are not expected to be transformed to a column labels. d. SDTM-MSG (p. 20, section 5.1.2) i. ARMCD can be up to 20 characters in length with no special character restrictions. The arm name (ARM) variable length must not exceed 200 characters due to SAS version 5 transport requirements. e. SDTM-MSG (p. 20, section 5.1.3) i. The full inclusion/exclusion criteria text should be stored in IETEST and corresponding metadata [up to 200 characters]. f. SDTM IG 3.1.3 (p. 22, section 18.104.22.168) i. Variables of the same name in separate datasets should have the same SAS Length attribute to avoid any difficulties if the sponsor or FDA should decide to append datasets together. g. SDTM-MSG (p. 31, section 6.3) i. Each of the value-level QNAMs (defined by the “Name” attribute) has a different length, a label, and associated controlled terminology (CodeLists). h. SDS (p. 5, section 2.3) i. Variable Name has a Maximum Length of 8. ii. Variable Descriptive Label has a Maximum Length of 40. iii. Dataset Label has a Maximum Length of 40. i. SDS (p. 5, section 2.4) i. For all datasets, in order to significantly reduce dataset file sizes, the allotted character column length/size for each column should be the maximum length used. Lengths/sizes of columns should not arbitrarily be set to 200. ii. An inclusion of a small amount of padding to column width may be acceptable as long as this doesn’t result in significant increases in file size. j. CDER-CDS (p. 12) i. For both CDISC and non-CDISC datasets, in order to significantly reduce dataset file sizes, the allotted character variable length/size for each column in a dataset should be the maximum length used. Lengths/Sizes of columns should not arbitrarily be set to 200. ii. Alternative solutions to this problem that involve some inclusion of a small amount of padding to column width may be acceptable as long as they don’t result in significant increases in file size due to the padding. 2. Column size for variables associated with a Codelist a. SDTMIG 3.1.3 (p. 35, linked section 22.214.171.124) i. The length for variables which use controlled terminology can be set to the length of the longest term. 3. Column size issues not otherwise constrained by the prior two categories a. SDTMIG 3.1.3 (p. 95, section 6.4.2) i. [For split FA datasets]. . . Variables of the same name in multiple datasets should have the same SAS Length attribute. b. SDS (p.5) . . . the allotted character column length/size for each column should be the maximum length used. Lengths/sizes of columns should not arbitrarily be set to 200. c. d. e. f. . . . An inclusion of a small amount of padding to column width may be acceptable as long as this doesn’t result in significant increases in file size. CDER-CDS (p. 12) For both CDISC and non-CDISC datasets, in order to significantly reduce dataset file sizes, the allotted character variable length/size for each column in a dataset should be the maximum length used. . . . Alternative solutions to this problem that involve some inclusion of a small amount of padding to column width may be acceptable as long as they don’t result in significant increases in file size due to the padding. PBO (slide 2) Ideally, column widths should only be the maximum length used in that column across records Solution: Should we use extra characters used or %? Exceptions? PBO (slide 32) SD0017(others): Variable lengths Description: o The value of xxxx should be no more than yy characters in length --TEST, --TESTCD, TSPARM, ARMCD, o Are there extenuating circumstances to these rules where character limits may not be enough? Adjust OpenCDISC checks to accommodate? o What’s the difference between SD0017 and SD1058? Duplicate rules? PWG1 (slide 9) Variable length is too long for actual data i. Severity based on “excessive” chars in var length ii. 2-5 Chars – Information iii. 6-20 Chars – Warning iv. >20 Chars – Error Appendix B Following the main document, details related to Overall File/Dataset Size and resulting split domains are addressed as a whole: 1. SDTM IG 3.1.3 (p. 22, section 126.96.36.199) a. Sponsors may choose to split a domain of topically related information into physically separate datasets. In such cases, one of two approaches should be implemented: 1) For a domain based on a general observation class, splitting should be according to values in --CAT (which must not be null). 2) The Findings About (FA) domain (Section 6.4) can be split either by --CAT values (per the bullet above) or relative to the parent domain of the value in --OBJ. For example, FACM would store Findings About CM records. See Section 6.4.2 for more details. 2. SDTM MSG (p. 24, section 188.8.131.52) a. If the decision had been made to split a submission dataset, it is recommended that the sponsor communicate with their review division regarding exactly what needs to be included in the submission, i.e. the split datasets or both the split datasets and the unsplit datasets. 3. SDS (p. 5, section 2.2) a. Each dataset is provided in a single transport file. The maximum size of an individual dataset is dependent on many factors. In general, datasets greater than 1 GB in size should be split into smaller datasets, each no larger than 1GB in size. Datasets divided to meet the maximum size restrictions should contain the same variable presentation so they can be easily concatenated. b. Datasets which are divided should be clearly named to aid the reviewer in reconstructing the original dataset, e.g., xxx1, xxx2, xxx3, etc. The files that have been divided and need to be concatenated should be noted in the data definition document. This documentation should identify the range of subject numbers (or other criteria used for division) in the label for each of the divided datasets. . . . 4. CDER-CDS (p. 9) a. LB Domain (Laboratory): The size of the LB domain is often quite large and can exceed the reviewers’ ability to open the file using standard-issue computers. This size issue can be addressed by splitting the large LB dataset into smaller data sets according to LBCAT and LBSCAT, using LBCAT for initial splitting. If the size is still too large, then use LBSCAT for further splitting. For example: use the dataset name lbc.xpt for chemistry lbh.xpt for hematology and lbu.xpt for urinalysis. Splitting it other ways (by subject or file size, etc) makes the data less useable. Sponsors should submit these smaller files in addition to the larger non-split standard LB domain file. The smaller split files should be submitted in a separate Sub-directory /SPLIT which is clearly documented in addition to the larger non-split standard LB domain file in the CRT directory. Please see File Size section for information about file size limits. 5. CDER-CDS (p. 12) a. Dataset Splitting If datasets are greater than 1 gb in size, please split the datasets into smaller datasets no larger than 1 gb in size. Datasets should be resized to the maximum length used prior to splitting. This will ensure split datasets have matching variable lengths for future merges. Split data should be noted in the data definition document, clearly identifying the method used for the dataset splitting. For additional information or clarification on file size limitations or splitting of datasets submitted to CDER, contact [email protected] 6. PBO (slide 3) Dataset Sizes Description: o Based on FDA’s SDS and CID Solution: o Limit set to 1 gb o Warning acceptable for this? o Should warning/error be based on certain limits? Warning 1 gb to 1.25 gb Error > 1.5?