2. FURTHER. FORWARD. FASTER. Implementing Row- and Cell-Level Security in Classified Databases Writer: Art Rask Contributors : Don Rubin, Bill Neumann, Scott Palmateer Technical Reviewers: Lara Rubbelke, Darmadi Komo, Jack Richins Published: April 2005 Updated: January 2012 Applies to: SQL Server 2012, SQL Server 2008 R2, SQL Server 2008, SQL Server 2005 Summary: This paper describes how SQL Server can be used to support row-level and cell-level security (RLS/CLS) based on security labels. The examples provided in this white paper show how RLS and/or CLS are used to meet classified database security requirements. Copyright This document is provided “as is.” Information and views expressed in this document, including URL and other Internet website references, may change without notice. You bear the risk of using it. Some examples depicted herein are provided for illustration only and are fictitious. No real association or connection is intended or should be inferred. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes. © 2011 Microsoft Corporation. All rights reserved. Implementing Row and Cell Level Security in Classified Databases Contents Introduction ........................................................................................................... 1 Assumptions and Design Principles .................................................................... 1 Fine-Grained Access Control ................................................................................ 1 A Brief Review of Security Labeling .................................................................... 3 Terminology .............................................................................................................................................................. 5 Security labels for row- and cell-level access control ............................................................................... 7 Overview of the Database Solution..................................................................... 7 Row-Level Security ................................................................................................ 8 Defining the labeling policy ................................................................................................................................ 8 Roles.......................................................................................................................................................................... 10 What can I see? ..................................................................................................................................................... 11 Changes to the base table ................................................................................................................................ 14 Defining the RLS view......................................................................................................................................... 14 Insert, update, and delete ................................................................................................................................. 15 Working with Labels............................................................................................................................................ 20 Foreign keys ........................................................................................................................................................... 27 Enabling Discreet Error Messaging ............................................................................................................... 27 Pulling it All Together (Part 1) ........................................................................... 28 Cell-Level Security ............................................................................................... 29 Cell-Level Encryption in SQL Server .............................................................................................................. 29 Key access control................................................................................................................................................ 32 Changes to the base table ................................................................................................................................ 33 Defining the RLS/CLS view................................................................................................................................ 34 Encrypting cell data on insert/update .......................................................................................................... 34 Pulling it All Together (Part 2) ........................................................................... 35 Implementing Row and Cell Level Security in Classified Databases Strengthening Security with Audit and TDE .................................................... 35 Additional Considerations for SQL Server 2012 Databases ........................... 36 Performance ......................................................................................................... 36 Row-Level Disclosure .......................................................................................... 37 Predicate evaluation order ............................................................................................................................... 37 Mitigation................................................................................................................................................................ 38 Conclusion ............................................................................................................ 42 Appendix A – Example XML Schema for Labels .............................................. 42 For More Information ......................................................................................... 43 Implementing Row and Cell Level Security in Classified Databases Introduction This paper reviews the requirements for row- and cell-level security based on security labels. The paper presents a Microsoft® SQL Server™-based design that provides rowlevel security based on security labels and views. Additional design elements, which allow cell-level security, are introduced. Cell-level security is also controlled by security labels. This white paper is accompanied by a toolkit, hosted on Codeplex, containing implementation samples and a robust code generation tool. This can be found by searching Codeplex for “Label Security Toolkit.” Assumptions and Design Principles The approach described in this paper makes the architectural assumption that applications using the database will connect using a specific identity for each end user. This could be either Integrated Windows authentication or a SQL Server login. This precludes the use of middle-tier connection pooling with a single identity, which is commonly used for scalability and management benefits. However, systems which have auditing and other security requirements that are significant enough to justify row-level security labeling usually have associated auditing requirements which require that the user identity be carried all the way to the database connection. Therefore, connection pooling is often not an option anyway. This design is also guided by these principles: 1. Rely on the SQL Server and Microsoft Windows® security model. Avoid handrolled authentication mechanisms. 2. Keep it simple. In some common scenarios, the number of possible security label combinations can grow very large. Avoid proliferation of fine-grained security groups, while still allowing very fine-grained control of data. 3. The design must scale to very large datasets, containing tens of millions of rows. In general, the objective is to constrain access to data for user accounts, which are administered by privileged administrators. This design does not impose row-level or cell-level security on administrators, in particular members of the sysadmin or db_owner fixed roles. Users with these rights will always be able to gain access to all data stored in the database. Fine-Grained Access Control Access control of information based on user permissions is a fundamental part of most computer software. Microsoft Windows, for example, uses access control lists (ACLs) to control which users can access files and folders in the NTFS file system. Microsoft SQL Server enforces access controls on the server login, databases, and on objects within a database (such as tables). In both cases, the level at which information is controlled extends only to a certain level of granularity. Windows controls access to user files, but not to portions of user files. SQL Server, like most other RDBMSs, controls access to tables, and even columns, but does not provide row-level or cell-level security within Implementing Row and Cell Level Security in Classified Databases 1 tables. In some scenarios, however, there is a requirement to control access at a more granular level. A list of patients and diagnoses, for example, may be stored in a single file or table. Any one doctor, however, may only be permitted to review information related to their own patients. In such a case, merely setting an ACL on a file or issuing a GRANT/DENY SELECT on a table will not meet the business requirements. Similar scenarios exist in many environments, including finance, law, government, and military applications. Consumer privacy requirements are yet another emerging driver for finer control of data. The typical approach to meeting such requirements in database applications has been to implement the necessary logic in application code. The business logic layer of an ntier application might apply the filtering, for example. Or, in a two-tier client-server application, the client might do it. This approach may be effective for the application, but the data is not actually secured. A user connecting to the back-end database with Microsoft Access or a SQL query tool will have unrestricted access to all rows in tables on which they have SELECT permission. Another common approach, which mitigates the last issue mentioned, is to wrap all data access in stored procedures. Users are denied all permissions on the underlying tables, and are instead given execute permissions on the stored procedures that implement the filtering logic. This approach has its own drawbacks. For example, ad hoc user reporting against such a database is difficult or impossible. What is needed is a way to present the actual tables (or views) to user accounts with the filtering logic applied automatically, based on the user context. In this case, all users might have access to the Patient table but, for each user, SELECT * FROM Patient returns only the data that user should see. SQL Server can be used for a secure implementation of fine-grained access control along these lines. In fact, techniques have been developed by Microsoft Consulting Services, and others, for doing exactly this using the various features of the SQL Server relational engine. In particular, after the release of SQL Server 2005, the fundamental building blocks were present for implementing row and cell level security. Enhancements in SQL Server 2008 provided additional methods for hardening such a database (i.e., stronger auditing and full-database encryption). This paper discusses a specific design framework for row- and cell- level security based on security labels, which emerged from work done by Microsoft Consulting Services for Microsoft customers and partners. Before exploring this solution, we will introduce security labels, a common paradigm for defining fine-grained access control on data. Implementing Row and Cell Level Security in Classified Databases 2 A Brief Review of Security Labeling A security label is a piece of information which describes the sensitivity of a data item (an object). It is a string containing markings from one or more categories. Users (subjects) have permissions described with the same markings. Each subject’s permissions can be thought of as a label of their own. The subject’s label is compared against the label on the object to determine access to that object. For example, the following table fragment has rows annotated with security labels in the Classification column. Table 1 ID Name Classification 1 John Doe SECRET 2 Frank Jones TOP SECRET 3 Sam Barnes UNCLASSIFIED A system containing this data might have user accounts as shown in Table 2. Table 2 User Clearance Alice SECRET Bob UNCLASSIFIED (no clearance) Each user’s clearance (expressed as a security label) determines which rows in the table they can access. If Alice issues a SELECT * FROM <tablename> against this table, she should get the following results. Table 3 ID Name Classification 1 John Doe SECRET 3 Sam Barnes UNCLASSIFIED If Bob issues SELECT * FROM <tablename>, he should see different results as shown in Table 4. Table 4 ID Name Classification 3 Sam Barnes UNCLASSIFIED Access controls can get more complex than this. There may be more than one access criterion expressed in a security label. For example, in addition to a classification level, a piece of data may only be visible to members of a certain project team. Assume this 3 Implementing Row and Cell Level Security in Classified Databases group is called PROJECT Q, and consider the following example. Table 5 ID Name Classification 1 John Doe SECRET, PROJECT Q 2 Frank Jones TOP SECRET 3 Sam Barnes UNCLASSIFIED Let’s modify our user permissions as well. Table 6 User Clearance Alice SECRET, PROJECT Q Bob UNCLASSIFIED (no clearance) Charlie TOP SECRET We’ve added Charlie, a user with TOP SECRET clearance. We’ve also augmented Alice’s label with the PROJECT Q marking. Now, if Alice issues SELECT * FROM <tablename>, she should see the results in Table 7. Table 7 ID Name Classification 1 John Doe SECRET, PROJECT Q 3 Sam Barnes UNCLASSIFIED If Charlie issues SELECT * FROM <tablename>, he will see the following results. Table 8 ID Name Classification 2 Frank Jones TOP SECRET 3 Sam Barnes UNCLASSIFIED Although Charlie has a TOP SECRET clearance, he does not have the PROJECT Q marking, so he cannot see row 1. Alice, however, satisfies both the SECRET marking and the PROJECT Q marking, so she can see row 1. Row 2, requiring a TOP SECRET clearance, is visible to Charlie only. This basic approach can be extended to additional markings. In some real-world scenarios, security labels can include several markings from different categories, and Implementing Row and Cell Level Security in Classified Databases 4 the number of possible label combinations can be quite large. Terminology This section introduces terminology used to accurately describe labels, and the comparison of labels. A label is a string that describes either the sensitivity of an object, or the permissions of a subject. The label is a collection of markings. Each marking describes a particular permission. The markings in a label come from one or more categories. Table 9 shows a breakdown of the previous example. Table 9 Label SECRET, PROJECT Q Markings Category SECRET Classification PROJECT Q Project Team A subject can access an object if the subject label dominates the object label. Given two labels, A and B, label A is said to dominate label B if every category present in label B is satisfied by markings on label A. Determining whether the markings are satisfied depends on attributes of each category. For our purpose, each category can be characterized by the following attributes. Table 10 Attribute Description Domain The possible markings in the category. Hierarchical (yes or no) Whether or not the category is hierarchical. Hierarchical categories have an ordering among values. This order determines access. A marking can satisfy any marking at or below its level in the hierarchy. Nonhierarchical categories have no ordering among values. A marking is either present or not present. Cardinality How many values from the domain can be applied to the object. Comparison Rule Whether the subject must have any or all of the markings applied to the object from this category (referred to as the Any and All comparison rules, respectively). Here are a few examples to illustrate this. Assume we have a security labeling policy with two categories as in Table 11. Implementing Row and Cell Level Security in Classified Databases 5 Table 11 Category Domain Hierarchical Cardinality Comparison Rule Classification TOP SECRET SECRET CONFIDENTIAL UNCLASSIFIED Yes 1..1 Any Q BN G K No Compartment (exactly one) 0..* (0, 1, or many) All Now let’s look at some example labels. In each case, the question is: does label A dominate label B? Example 1 Label A SECRET,Q Label B SECRET,Q,G To compare these labels, we must compare the markings in each category. Classification: The SECRET marking in label A satisfies the SECRET marking in label B. Compartments: The Q compartment on label A does not satisfy the Q,G compartments on label B, since ALL compartments on B must be present in A. So, label A does not dominate label B. Example 2 Label A TOP SECRET,Q,G,BN Label B CONFIDENTIAL,Q,G Classification: The TOP SECRET marking in label A satisfies the CONFIDENTIAL marking in label B. Compartments: The Q,G,BN compartments on label A do satisfy the Q,G Implementing Row and Cell Level Security in Classified Databases 6 compartments on label B, since ALL compartments on B are present on A. So, label A dominates label B. Example 3 Label A SECRET,Q,K Label B CONFIDENTIAL Classification: The SECRET marking in label A satisfies the CONFIDENTIAL marking in label B. Compartments: Label B has no compartments listed, which means there are no compartment requirements. So, label A dominates label B. Security labels for row- and cell-level access control The use of security labels is prevalent in many environments, in both the public and private sectors. The multilevel secure (MLS) information management field (in which a system securely manages information with multiple sensitivity levels) has grown up since the 1970s. So, the design presented here will be based on security labels as the paradigm for defining permissions on data and users. Overview of the Database Solution The design discussed in the following pages describes in detail an approach for adding label-based row- and cell-level security to a SQL Server database. This design is aimed at defining any arbitrary labeling policy and enforcing it with SQL Server 2008 R21. The design does the following: Adds metadata tables for defining arbitrary labeling categories and markings. Allows the user’s permissions to be defined intuitively through role memberships for basic markings (for example, Top Secret, Confidential; USA, UK, Taskforce Z). Provides for write-up, write-down, or other control models for writing data. (Optionally) Makes selective use of encryption within the database to provide cell-level security, exploiting the fully internal, self-managing certificate store in In general, the design framework in this paper works for SQL Server 2005 through 2012. Exceptions to this are the recommendations to use Audit and TDE, which are both features available in SQL Server 2008 and later. 1 Implementing Row and Cell Level Security in Classified Databases 7 SQL Server 2005 and later. Provides formulaic guidelines, such that tools can be used to automate much of the implementation given basic input choices by developers or administrators. Provides strategies for protecting against certain narrow vulnerabilities in rowlevel security—vulnerabilities which are shared by multiple vendors’ row-level security solutions. Row-Level Security The mechanism we will use to enforce row-level security is SQL Server views. Views allow a predefined query to be presented to a user or application as if it were a table. Also, users can be granted access to a view, but denied access to underlying tables. This prevents the user from bypassing the view and going straight to the base table. We will use a specially constructed view which applies all the necessary logic to enforce row-level security based on labels. In some scenarios, views are used to pre-cook a complex query—perhaps for a report—into a single database object that can be easily queried. We will not be doing this. Our intent is to simply wrap the base table in a view with a nearly identical definition. Users (or applications) will then query or update the view, and even join it to other tables, as if it were the table itself. Access to the base table will be denied. Creating this view will require us to do the following four things. 1. 2. 3. 4. Create some tables that define the label categories and markings, and which assign properties to each unique security label combination. This only needs to be done once for each database. Create roles for the marking values. Membership in these roles will be used to express the label that is assigned to a user. This also only needs to be done once for each database. Add a single integer column to the base table. Define the view. Defining the labeling policy We start by creating a few tables that define the label metadata. These are shown in the ER diagram in Figure 1. The tblCategory table defines the categories that labels can include. For each category, there is a certain set of possible markings. These are defined in the tblMarking table. If the category is hierarchical, the parent-child relationships are established with related records in the tblMarkingHierarchy table. The columns in these tables should be mostly self-explanatory based on the earlier discussion of security label categories. Implementing Row and Cell Level Security in Classified Databases 8 Figure 1 The following SQL statements show (roughly) how we would set up an example labeling policy. --Categories INSERT tblCategory (ID, Name, Hierarchical, CompareRule, Cardinality, Required) VALUES (1, 'Classification', 1, 'Any', 'One', 1); INSERT tblCategory (ID, Name, Hierarchical, CompareRule, Cardinality, Required) VALUES (2, 'Compartment', 0, 'All', 'Many', 0); INSERT tblCategory (ID, Name, Hierarchical, CompareRule, Cardinality, Required) VALUES (3, 'Nationality', 0, 'Any', 'Many', 0); INSERT tblCategory (ID, Name, Hierarchical, CompareRule, Cardinality, Required) VALUES (4, 'Need-to-Know', 0, 'Any', 'Many', 0); GO Implementing Row and Cell Level Security in Classified Databases 9 --Classification markings INSERT tblMarking (CategoryID, MarkingRoleName, MarkingString) VALUES (1, 'clTOPSECRET', 'TOPSECRET'); INSERT tblMarking (CategoryID, MarkingRoleName, MarkingString) VALUES (1, 'clSECRET', 'SECRET'); INSERT tblMarking (CategoryID, MarkingRoleName, MarkingString) VALUES (1, 'clCONFIDENTIAL', 'CONFIDENTIAL'); INSERT tblMarking (CategoryID, MarkingRoleName, MarkingString) VALUES (1, 'clUNCLASSIFIED', 'UNCLASSIFIED'); --Classification hierarchy INSERT tblMarkingHierarchy (ParentCategoryID, ParentMarkingRoleName, ChildCategoryID, ChildMarkingRoleName) VALUES (1, 'clTOPSECRET', 1, 'clSECRET') INSERT tblMarkingHierarchy (ParentCategoryID, ParentMarkingRoleName, ChildCategoryID, ChildMarkingRoleName) VALUES (1, 'clSECRET', 1, 'clCONFIDENTIAL') INSERT tblMarkingHierarchy (ParentCategoryID, ParentMarkingRoleName, ChildCategoryID, ChildMarkingRoleName) VALUES (1, 'clCONFIDENTIAL', 1, 'clUNCLASSIFIED') GO --Compartment markings INSERT tblMarking (CategoryID, RoleName, MarkingString) VALUES (2, 'cpQ', 'Q') INSERT tblMarking (CategoryID, RoleName, MarkingString) VALUES (2, 'cpG', 'G') Etc.… Next, let’s turn our attention to the tblUniqueLabel table. This table is used to map a particular combination of markings—a specific security label instance—to a unique ID. For the sake of efficiency, this table is populated on demand. Whenever a piece of data with a given security label is added to the database, a stored procedure is called to get an ID that represents that unique label. If there is no corresponding row in tblUniqueLabel, a new row is added and the new ID returned. We want to have just as many rows in this table as we need—no more and no less. Finally, the tblUniqueLabelMarking table associates individual marking values and the security label instances. Roles Having defined the structure of labels in our policy, we need to do something to tie in to the SQL Server security model. For each marking in every category we defined, we Implementing Row and Cell Level Security in Classified Databases 10 create a corresponding database role2. Database roles must adhere to the following guidelines. For nonhierarchical categories that use an Any or All comparison rule, create a role for each possible value. The name of the role must match the value in the tblMarking.MarkingRoleName column for that marking. For hierarchical categories, do the same thing. In addition, however, the roles must be nested to model the hierarchy. For each role you create, add the parent role that is defined in the tblMarkingHierarchy table as a member. This nesting ensures, for example, that users with Secret clearance can access data at that level and below. These roles allow users to be granted permissions that precisely express the security labels to which they should have access. It is the job of the application or database administrator (DBA) to correctly maintain role memberships for the users of the system. In the following sections, we’ll see how the role memberships are used to identify which data users can access. What can I see? Before we turn our attention to the application table(s), we will create a helper view that encapsulates the actual row-level security logic. We’ll call it vwVisibleLabels. The starting point of the view definition is as in the following code example. SELECT ID, Label FROM tblUniqueLabel WITH (NOLOCK) WHERE …. The WHERE clause definition is based on the category attributes in our labeling policy. The important attribute is Comparison Rule. For each category that has a comparison rule of Any, add the following predicate to the WHERE clause. ID IN (SELECT ID FROM tblUniqueLabelMarking WITH (NOLOCK) WHERE CategoryID = <HardCodedCatID> AND IS_MEMBER(MarkingRoleName) = 1) This clause gets all the IDs of unique security labels which contain markings from the given category and of which the current user is a member. Let’s look at that more closely. The subquery scans the tblUniqueLabelMarking table for rows from a specific 2 If using Integrated Windows authentication with the SQL login mapped to a Windows user account, a Windows group can be used as well. This alternative applies to all discussions of database roles throughout this white paper. Implementing Row and Cell Level Security in Classified Databases 11 category. From among these rows, it chooses the ones for which the current user is a member of the database role named in the MarkingRoleName column. This check is accomplished with the SQL Server intrinsic function IS_MEMBER. For each one of these rows, the ID of the corresponding record in tblUniqueLabel is returned to the outer query. And what about categories with a Comparison Rule of All? For each category that has a comparison rule of All, add the following predicate to the WHERE clause. 1 = ALL( SELECT IS_MEMBER(MarkingRoleName) FROM tblUniqueLabelMarking (NOLOCK) WHERE CategoryID = <HardCodedCatID> AND UniqueLabelID = tblUniqueLabel.ID) This predicate results in comparisons that require the user to have all of the applied markings in order to have access. The subquery gets a list of values returned by IS_MEMBER for every related marking in the category, for a given record in tblUniqueLabel. If all of these return values are 1, the predicate is satisfied. Pull all this together for our earlier example case and you get a definition for the vwVisibleLabels helper view as follows: CREATE VIEW vwVisibleLabels AS SELECT ID, Label FROM tblUniqueLabel WITH (NOLOCK) WHERE ID IN --Classification (SELECT UniqueLabelID FROM tblUniqueLabelMarking WITH (NOLOCK) WHERE CategoryID = 1 AND IS_MEMBER(MarkingRoleName) = 1) AND --Compartments 1 = ALL(SELECT IS_MEMBER(MarkingRoleName) FROM tblUniqueLabelMarking WHERE CategoryID = 2 AND UniqueLabelID = tblUniqueLabel.ID) GO This view is called vwVisibleLabels, but you could just as well think of it as “the list of all the security labels present in the database to which I (the current user) have access.” 12 Implementing Row and Cell Level Security in Classified Databases Table 12 summarizes the design rules for setting up the labeling policy, depending on the attributes of each category. Table 12 Category Attribute Design Implication Domain Define all markings in the tblMarking, and tblMarkingHierarchy tables. Create a database role to represent each marking, using role nesting for hierarchical categories. Membership in the role means the user has that access. Hierarchical Cardinality Yes Nest roles to model hierarchy No No action 1 Constrain the number of markings on one label using a trigger on tblUniqueLabelMarking. 1..* No action 0..1 or 0..* Use trigger to limit markings-per-label to 1. Also, handle the zeromarkings-on-a-label case as described in NoMarkingBehavior later in this table. Any Use the following WHERE-clause predicate in the vwVisibleLabels view. ID IN (SELECT UniqueLabelID FROM tblUniqueLabelMarking WITH (NOLOCK) WHERE CategoryID = n AND IS_MEMBER(MarkingRoleName) Comparison Rule = 1) All Use the following WHERE-clause predicate in the vwVisibleLabels view. 1 = ALL(SELECT IS_MEMBER(MarkingRoleName) FROM tblUniqueLabelMarking (NOLOCK) WHERE CategoryID = n AND UniqueLabelID = tblUniqueLabel.ID) No Restriction NoMarking Behavior NoAccess For each row in tblUniqueLabel that has zero markings in the category, add a row in tblUniqueLabelMarking assigning the public role to the label. This makes (only) this category nonrestrictive for that label. If Comparison Rule is All, add an additional condition to the WHEREclause predicate in the vwVisibleLabels view: AND EXISTS(SELECT MarkingRoleName FROM tblUniqueLabelMarking (NOLOCK) WHERE CategoryID = n AND UniqueLabelID = tblUniqueLabel.ID) Implementing Row and Cell Level Security in Classified Databases 13 Note that, to handle some scenarios, there is a category attribute called NoMarkingBehavior that has not been mentioned yet. This controls how access is evaluated for a label with no markings from a given category. In the great majority of scenarios, the absence of markings means that category can be ignored—it imposes no restriction. In some scenarios, however, the absence of markings in the category can mean no one is able to view the data (except an account explicitly granted permission to bypass RLS). Changes to the base table Now let’s look at the change that must be made to the base table to which we will add row-level security. The change is very minor. (There will be more when we get to the topic of cell-level security). The only change required on the base table is the addition of an integer column to hold the ID for the security label assigned to the row. This column – call it SecLabelID, although any name will do – references an ID in the tblUniqueLabel table from the label policy metadata. When a row is inserted into the base table with a security label, the ID corresponding to that label in tblUniqueLabel must be retrieved (or generated) and placed in the SecLabelID column of the new row. Similarly, if a row is updated so that its security label changes, the SecLabelID should change accordingly. (The updateability of the security label is dependent on the security requirements in each situation; this may be expressly forbidden). For performance reasons, create a non-clustered index on the SecLabelID column. Do not skip this or performance will suffer! Finally, create a foreign key relationship between SecLabelID on the base table and the ID column in tblUniqueLabel. As an aside, it should be noted that it is not absolutely necessary to change the base table by adding the SecLabelID column. If there is a compelling reason in your application to avoid any change to the existing table structure, a “junction” table can be used to relate base table rows to IDs in tblUniqueLabel. This junction table would simply contain the primary key columns from the base table, plus a SecLabelID column, with foreign keys defined to the base table and tblUniqueLabel respectively. The definition of the RLS view described in the next section would then use a slightly different join condition. For the rest of this paper, we will assume the simpler case of adding a SecLabelID column directly to the base table. Defining the RLS view Now we are ready for the last step. We will create a view over the base table that will, in essence, replace the table in the eyes of the end user or application. Here is what the view definition should look like. Implementing Row and Cell Level Security in Classified Databases 14 CREATE VIEW UserTable AS SELECT <base table column list which does not include any columns from vwVisibleLabels> FROM tblBaseTable WITH (READCOMMITTED) INNER JOIN vwVisibleLabels ON tblBaseTable.SecLabelID = vwVisibleLabels.ID GO GRANT SELECT ON UserTable TO <app_users> DENY SELECT, INSERT, UPDATE, DELETE ON tblBaseTable TO <app_users> GO There are two points of interest in this view definition, as follows: By joining the security label identifier in the base table against the Visible Labels view, the label dominance logic—which depends on user membership in security groups—controls access to rows in the base table. The READCOMMITTED locking hint applied to the base table prevents dirty reads against the base table. This prevents a reader from viewing rows within another user’s pending transaction which may have been updated with sensitive data, but whose security label identifier(s) has not yet changed. The locking hint in the view overrides the reader’s transaction isolation level or any explicit locking hints they use when querying the view. REPEATABLE READ and SERIALIZABLE are acceptable here as well. Note that this assumes that the other user (the updater) has a transaction isolation level other than READ UNCOMMITTED. We now have a view that transparently enforces label-based row-level security without any application logic, and in a way that remains in effect even if the application layer is bypassed. Of course, we have only discussed using this view for selecting rows from the table. Next, we will address how an application can support INSERT, UPDATE and DELETE against the data in the base table. Insert, update, and delete Everything discussed so far pertains to SELECTing rows from the underlying table. Most applications need to write information to the table as well. Inserting, updating, or deleting rows in a table with label-based row-level security raises some questions. Which rows can the user update? Are there any restrictions on the level of security label the user can apply to newly inserted rows? Or, does the user even have a say in the matter? The answers to these questions depend on the requirements for the project at hand. In practice, the desired behavior for writes may be very simple and straightforward with Implementing Row and Cell Level Security in Classified Databases 15 minimal restrictions (except that user’s scope of action be limited to rows they can “see”). Some projects, however, have more nuanced or complex requirements. This paper will not present a one-size-fits-all approach for writes, since there is too much variety seen in practice. For example, some of the things that vary across projects include: Are write operations always brokered via stored procedures, rather than being issued as INSERT/UPDATE/DELETE statements against the RLS view? If statements against the view are used, is the security label allowed to be written as part of an INSERT or UPDATE? Some systems only allow this through a workflow separate from ‘normal’ data changes. If the security label column can be written in an INSERT or UPDATE, is the statement allowed to use a label ID, or does a label string need to be accepted as a column value and parsed to resolve the identifier? Are there any other special rules or logic that must be applied? For example, are there limits on how the security label can be changed (e.g., only up, only down, no higher or lower than the current user’s subject label)? This paper will present some general cases to illustrate the tools provided by this design and by the native SQL Server security model for addressing these questions. These are intended to make clear the tools available, but leave the implementation details open to be shaped by your project requirements. The Simple Case: Updateable Views The most straightforward case of supporting writes against an RLS-protected table is simply to rely on SQL Server support for updateable views. If the view definition only includes columns from the base table (this is the general recommendation), then the view can be used to update the base table. The label-based access control provided by the view will inherently limit the write-able rows to those the user can access. To illustrate this in practice, we will use two general example cases: 1. Users cannot specify security labels. They interact with the view as if RLS was not even there, and the classification is assigned by some other workflow. 2. Users can specify security labels, either in INSERT or UPDATE statements against the view. Case #1: Users cannot specify security labels Let’s assume a very simple table, for the sake of illustration: Implementing Row and Cell Level Security in Classified Databases 16 CREATE TABLE tblCustomer (ID int IDENTITY NOT NULL PRIMARY KEY, Name nvarchar(50) NOT NULL, Phone nvarchar(20) NOT NULL, Email nvarchar(256) NOT NULL, SecLabelID int NOT NULL) GO CREATE VIEW Customer AS SELECT tblCustomer.ID, Name, Phone, Email FROM tblCustomer WITH (READCOMMITTED) INNER JOIN vwVisibleLabels ON tblCustomer.SecLabelID = vwVisibleLabels.ID GO ALTER TABLE tblCustomer ADD CONSTRAINT DF_tblCustomer_SecLabelID DEFAULT dbo.LookupVisibleLabelID('<Label><Cl>SECRET</Cl></Label>') FOR SecLabelID; GO DENY SELECT, INSERT, UPDATE, DELETE ON tblCustomer TO MyAppUsers GRANT SELECT, INSERT, UPDATE, DELETE ON Customer TO MyAppUsers GO In the above T-SQL code, we create a very simple table for customer data and wrap it in an RLS view called Customer. The Customer view does not expose the SecLabelID from the base table. We then add a default constraint for the SecLabelID column so that all newly inserted rows will be labeled SECRET. (For discussion on the XML formatting of labels, see the section on “Working with Labels” later in this paper). After the appropriate GRANT statements, users can now SELECT, INSERT, and UPDATE rows in the Customer view. By default new rows are labeled SECRET. Users can update the business data in the Customer view, but not the classification. Other classifications may be assigned to rows through a system process or a specialized workflow. One more step is required to allow DELETE against the Customer view. Since the definition of the customer view includes a join, SQL Server can’t pass a delete through to the underlying tables. We deal with this by creating a simple instead-of trigger for DELETEs on the view: Implementing Row and Cell Level Security in Classified Databases 17 CREATE TRIGGER Customer_IODelete ON Customer INSTEAD OF DELETE AS BEGIN SET NOCOUNT ON; DELETE tblCustomer FROM tblCustomer INNER JOIN deleted ON (tblCustomer.ID = deleted.ID) END GO Now a user-issued DELETE statement against the Customer view will behave just like a DELETE against a base table. The possibly affected rows are limited, however, based on their security labels. Even for bigger tables with many more columns than the one shown here, the techniques for write-ability are the same. Case #2: Users can specify security labels (actually label IDs) Starting with the same customer table as in Case #1, we will use a slightly different view definition. Now the SecLabelID is included in the view columns. CREATE VIEW Customer AS SELECT tblCustomer.ID, Name, Phone, Email, SecLabelID FROM tblCustomer WITH (READCOMMITTED) INNER JOIN vwVisibleLabels ON tblCustomer.SecLabelID = vwVisibleLabels.ID GO DENY SELECT, INSERT, UPDATE, DELETE ON tblCustomer TO MyAppUsers GRANT SELECT, INSERT, UPDATE, DELETE ON Customer TO MyAppUsers GO Now user-issued INSERT and UPDATE statements can include a security label ID. This ID may be obtained with the LookupVisibleLabelID function. Or, it might be transparently provided through the application UI (e.g., a dropdown list of labels populated from vwVisibleLabels with ID in the value member). Implementing Row and Cell Level Security in Classified Databases 18 DELETE against this Customer view is enabled by creating the same instead-of trigger shown for Case #1. Finally, note that certain combinations of the above cases can be supported using SQL Server column-level security. For example, assume the view needs to expose SecLabelID for display, but not allow the label on existing records to be updated by users. A column-level DENY can prevent updates, as in this example: DENY UPDATE ON Customer(SecLabelID) TO MyAppUsers Writes Via Stored Procedures For some systems, it is not necessary to allow write operations against the RLS view. In these cases, the flexibility to read and report against RLS views is enough, and INSERT, UPDATE, and DELETE operations are brokered through stored procedures called by an application. For this case, there is not much more to say in this paper. It is useful, however, to show a quick example of what such stored procedures would look like. The following example performs a basic insert into our example customer table: CREATE PROCEDURE usp_InsertCustomer @Name nvarchar(50), @Phone nvarchar(20), @Email nvarchar(256), @Label xml(dbo.SecurityLabel), @CustomerID int OUTPUT AS DECLARE @sec_label_id int; exec usp_GetSecLabelID @Label, @sec_label_id OUTPUT; INSERT tblCustomer (Name, Phone, Email, SecLabelID) VALUES (@Name, @Phone, @Email, @sec_label_id); SET @CustomerID = SCOPE_IDENTITY(); GO The notable thing in this example is the method for passing in a security label and for resolving it to an ID. The security label is expressed using a typed XML variable, the schema for which is directly derived from categories defined in the labeling policy. With the label in hand, a helper stored procedure called usp_GetSecLabelID is used to get a corresponding ID. This procedure either retrieves the existing ID for this label or, if the label has not been previously recorded in tblUniqueLabel, inserts new rows in tblUniqueLabel and tblUniqueLabelMarking and returns the new ID for the label. These 19 Implementing Row and Cell Level Security in Classified Databases topics are discussed in more detail in the section on “Working with Labels”. The following stored procedure example illustrates using the vwVisibleLabels view to restrict delete operations based on the user’s permissions: CREATE PROCEDURE usp_DeleteCustomer @CustomerID int AS DELETE tblCustomer FROM tblCustomer INNER JOIN vwVisibleLabels ON (tblCustomer.SecLabelID = vwVisibleLabels.ID) WHERE tblCustomer.ID = @CustomerID; GO For Complex Scenarios: Instead-Of Triggers If the project requirements demand that write operations be allowed directly against the RLS view, and that the behavior of such writes falls outside of what is supported by SQL Server updateable views, it may be necessary to use instead-of triggers. Instead-of triggers allow for arbitrary T-SQL to run in response to an INSERT, UPDATE, or DELETE against a view. Examples of the requirements that may force you to use instead-of triggers include: Including the label as a string in the INSERT or UPDATE, requiring some T-SQL code to intercept this and resolve it to an ID. Enforcing arbitrary rules about what labels a user can explicitly apply to a row. For example, labels can only be updated to a ‘higher’ level, or an inserted/updated label must be no higher than what the user themselves can see. Using some special logic to decide how to automatically label a row. For example, label inserted rows with the user’s permissions. Including logic to prevent foreign key values being assigned which reference rows in another RLS-protected table if the user has no access to those rows. Examples of these and other scenarios are included in the toolkit samples on Codeplex. Working with Labels A challenge that comes up when working with security labels is that they are generally represented as strings. They might be entered as strings by users, or be imported from other systems. This introduces several potential problems. Strings are often Implementing Row and Cell Level Security in Classified Databases 20 inconsistently formatted, despite the best of intentions. There may be more than one string commonly used to mean the same thing (e.g., “TOP SECRET” and “TS”). And, externally sourced strings are always a potential security threat until they are thoroughly scrubbed for malicious input. Besides working with string labels in an error-free and secure fashion, we also eventually run into the problem of comparing labels for equivalence. How can we work with labels in a way that, if we have two in hand, we can reliably and unambiguously decide whether or not they mean the same thing? SQL Server has features that can help with this, and the design framework presented here uses them to provide a common, robust approach to working with labels within the scope of an RLS-protected database. Keep in mind that how we work with labels is technically distinct from controlling access to rows. Row level access control is about the label policy metadata, visible labels view, and RLS views described earlier in this paper. Providing some conventions for easily working with labels is complementary to those things – it helps get the job done, but it is not what actually provides access control. Representing Labels SQL Server’s support for typed XML offers a way to enforce inherent structure and validation on string-based input. With typed XML you can declare variables, column data types, and stored procedure or function arguments which must be well-formed XML and must comply with a specific XSD schema. In keeping with the design principle of being formulaically driven by the metadata that describes the label policy, the design framework presented here uses an XSD schema built up from the categories in the label policy. As shown in the ER diagram in Figure 1, the tblCategory table includes a column named Abbreviation. This is a short-hand tag for referencing the category, and it can be used as element names in an XML representation of a label. For example, assume a label policy had the following categories: ID Name Abbreviation Cardinality Required 1 Classification Cl One Yes 2 Compartments Cp Many No 3 Releasability RelTo Many No 4 Need To Know N Many No Then a sample instance of the typed XML label might be as follows: <Label> <Cl>SECRET</Cl> <RelTo>USA</RelTo> <RelTo>JPN</RelTo> Implementing Row and Cell Level Security in Classified Databases 21 <RelTo>EGY</RelTo> <N>LIGHTHOUSE</N> </Label> The basic rules are: 1. The root element is named “Label”. 2. Child elements are named with the abbreviation of the respective category. 3. There is one child element for each marking, and the marking is text content within the element. No lower level elements are allowed. 4. When present, the category markings must appear in the same order as the category IDs. So, in the example above the “Cl” could not be the last child element. 5. Markings are validated with XSD restriction attributes as follows a. Must be a non-empty string b. Max length is 50 c. Only letters, numbers, underscore, dash or space are allowed. The first character must be a letter. This is certainly not the only possible XML label format. This particular format was chosen because it is easily derived from the label metadata and because it is terse – the total number of characters is less than some other formats might require. The actual XSD schema for this example is included as Appendix A. More extensive treatment of the label format can be found in the Label Security Toolkit on Codeplex. It is not required that labels be displayed to users or exchanged with external systems in this format. But, it is expected that application code will normalize labels into this format before calling RLS-related stored procedures or functions. Where necessary, it is a simple matter to define a formatting function which converts this internal XML representation to a specific display format preferred by users. Manipulating Labels Another challenge with strings arises when manipulating their content. Doing so generally requires parsing and concatenation code that can be error prone and difficult to write. A benefit of the typed XML approach is that all the techniques that exist for manipulating XML become available for working with labels. This applies both in the application layer (XML DOM libraries, XML serializable classes, etc.) and in the database (XML data type methods, SELECT…FOR XML, etc.). In addition, the design framework presented here includes a handful of T-SQL helper functions for manipulating the content of labels. These helper functions make it easy to perform basic operations on a label instance (e.g., adding/removing a marking, listing markings in a category) using pure T-SQL. Implementing Row and Cell Level Security in Classified Databases 22 These helper functions and their purpose are listed in the table below: Name Purpose fn_NormalizeLabel Ensures that a label is canonicalized to a consistent and comparable form. The result has no duplicate markings and all markings are ordered alphabetically by category. fn_AddMarking Given a label, adds the specified marking to the specified category fn_SetMarking Given a label, directly sets the marking for the specified category, overwriting any existing markings. Multiple, comma-separated markings can be specified. fn_RemoveMarking Given a label, removes the specified marking from the specified category. tvf_GetMarkings Given a label, lists all the markings in the specified category. This is a table-valued function that returns the markings as a table. fn_ValidateLabel Checks whether the label has markings for categories with Required=true and checks that all the markings are valid entries from the tblMarking table. Common Tasks with Labels In the implementation of a database which uses row (and/or cell) level access control, several basic label-related tasks are necessary. The RLS design framework provides a handful of helper stored procedures and functions which encapsulate these tasks. Perhaps the most common such task is getting an ID to represent a given label. Assume, for example, that you want to insert a new row in an RLS protected table. You have the security label in hand that describes the intended access restrictions on the row. To perform the INSERT, you need an integer ID for the label. The usp_GetSecLabelID stored procedure will handle this for you. Here is an example: DECLARE @label xml(dbo.SecurityLabel) = '<Label><Cl>TOPSECRET</Cl><Cp>A</Cp></Label>'; DECLARE @sec_label_id int; exec usp_GetSecLabelID @label, @sec_label_id OUTPUT; INSERT tblCustomer (Name, Phone, Email, SecLabelID) VALUES (@Name, @Phone, @Email, @sec_label_id); The usp_GetSecLabelID procedure validates the label and the checks the tblUniqueLabel table to see if an ID already exists. If an ID does exist already, it is returned. If not, the procedure decomposes the label markings and inserts the appropriate rows in tblUniqueLabel and tblUniqueLabelMarking, finally returning the newly created ID. Implementing Row and Cell Level Security in Classified Databases 23 A variant of this task is looking up an existing label id, but not generating a new one if none exists already. This may be what your application specifically needs to do (perhaps user-provided labels must be among classifications already supported in your database; new labels require approval by a security officer). It also may be necessary if your code needs to call a T-SQL function instead of a stored procedure. One example where this is necessary is in a table constraint – functions can be used here but not stored procedures. A wrinkle in the function vs. stored procedure choice is that SQL Server prohibits functions from having ‘side effects’. So, a T-SQL function cannot create new label IDs; it can only lookup existing ones. The following functions are provided for this purpose: Function Description LookupLabelID Returns the ID for the label, if it exists in the label policy. Otherwise returns NULL. (@label xml (dbo.SecurityLabel)) RETURNS int By default, EXECUTE permission is NOT granted to general users. This function is intended for use by privileged user contexts. LookupVisibleLabelID Returns the ID for the label, if it exists in the label policy AND is visible to the current user. Otherwise returns NULL. (@label xml (dbo.SecurityLabel)) RETURNS int By default, EXECUTE permission is granted to general users. The reverse analogs to these functions, which return a known label given an ID, are: Function Description LookupLabelFromID Returns a label instance for the ID, if it exists in the label policy. Otherwise returns NULL. (@label_id int) RETURNS xml (dbo.SecurityLabel) By default, EXECUTE permission is NOT granted to general users. This function is intended for use by privileged user contexts. LookupVisibleLabelFromID Returns a label instance for the ID, if it exists in the label policy AND is visible to the current user. Otherwise returns NULL. (@label_id int) RETURNS xml (dbo.SecurityLabel) By default, EXECUTE permission is granted to general users. A more esoteric task, perhaps necessary in more specialized scenarios, is getting the “subject label” for a user. This operation produces a label which describes the permissions of the user with respect to the label policy. Implementing Row and Cell Level Security in Classified Databases 24 For example, let’s say there is a user named Alice. Alice has a Top Secret clearance, and is granted access to compartments B and C. In addition, Alice has been granted the LIGHTHOUSE “need to know” permission. (These permissions are for illustration only, and generally follow from the examples in the preceding pages; keep in mind that label structure can be defined as needed for your system). A description of Alice’s permissions with respect to the label policy can be stated as follows: Alice’s Subject Label: <Label> <Cl>TOPSECRET</Cl> <Cp>B</Cp> <Cp>C</Cp> <N>LIGHTHOUSE</N> </Label> Two stored procedures are provided for getting user permissions in this form: Stored Procedure Description usp_GetUserLabel Returns a label instance for the named user, if the user exists. Otherwise returns NULL. This works if the user is a SQL Server principal, an explicit Windows principal, or a Windows principal implicitly granted access through membership in a Windows group. @username nvarchar(256), @label xml (dbo.SecurityLabel) OUTPUT By default, EXECUTE permission is NOT granted to general users. This function is intended for use by privileged users. usp_GetCurrentUserLabel @label xml (dbo.SecurityLabel) OUTPUT Returns a label instance for the current user context. By default, EXECUTE permission is granted to general users. Another task that may be necessary in specialized scenarios is explicitly comparing two labels to see if one implies access to the other. Recall from the introductory discussion of security labels that one label is said to “dominate” another if the first label implies access to the second. A T-SQL function called fn_Dominates is provided that allows two security label instances to be compared in this way. Most straightforward RLS scenarios will not require the use of this function. But, for certain requirements, it may be necessary to programmatically check one label against another. For example, suppose a user is not allowed to label data at a level higher than what Implementing Row and Cell Level Security in Classified Databases 25 they themselves can access. A stored procedure or instead-of trigger could enforce this as follows: --@data_label is passed in from application DECLARE @user_label xml (dbo.SecurityLabel); exec usp_GetCurrentUserLabel @user_label OUTPUT; IF dbo.fn_Dominates(@user_label , @data_label) = 0 RAISERROR('User rights not sufficient to write this data', 12, 1); As another example, suppose no data can be inserted to a certain table with a classification below Top Secret. We could define a function to check this, using the fn_Dominates function, and declare a table constraint which calls the function: CREATE FUNCTION dbo.fn_CheckMin (@sec_label_id int) RETURNS bit AS BEGIN DECLARE @data_label xml (dbo.SecurityLabel); SET @data_label = dbo.LookupLabelFromID(@sec_label_id); DECLARE @classification_marking nvarchar(50); SELECT @classification_marking = Marking FROM dbo.tvf_GetMarkings(@data_label, 'Cl'); --create a label with only the classification marking DECLARE @data_label_class xml (dbo.SecurityLabel) = '<Label></Label>'; SET @data_label_class = dbo.fn_SetMarking(@data_label_class, 'Cl',@classification_marking); --create a label with minimum classification, for comparison DECLARE @minimum_label xml (dbo.SecurityLabel); SET @minimum_label = '<Label><Cl>TOPSECRET</Cl></Label>'; RETURN dbo.fn_Dominates(@data_label_class , @minimum_label); END GO CREATE TABLE HighValueData (ID int NOT NULL PRIMARY KEY, SomeData varchar(20) NOT NULL, SecLabelID int NOT NULL CONSTRAINT CK_MinClass CHECK (dbo.fn_CheckMin(SecLabelID) = 1) ) A similar check in the opposite sense could be used to ensure no data above a certain classification enters a table. (Be very careful of the potential performance impacts of Implementing Row and Cell Level Security in Classified Databases 26 using a user-defined function in this manner, especially if the target table must support a high volume of inserts). Foreign keys Before leaving the topic of row-level security, we should address the question of tables that reference an RLS-protected table via foreign keys. Allowing users to see rows in a table that reference restricted rows in another table could be considered a leak of information. This can be addressed when modifying the base table. When adding the SecLabelID, this column can be added to the primary key. Whenever the primary key is referenced in another table, the row-level label will go with it, and can be used in the definition of a view over that table. Enabling Discreet Error Messaging One of the “ease of use” features in SQL Server is error messages that provide assistance in tracking down the cause of a problem. For example, assume you issue a SELECT against a 100,000 row table and get a data type conversion error. It can be exceptionally difficult to determine which row(s) is the source of the problem. SQL Server helps in this situation by including the data value which caused the error in the error message – this can be a huge time-saver in tracking down the problem. Here is an example: Msg 245, Level 16, State 1, Line 1 Conversion failed when converting the nvarchar value 'K4B 1S2' to data type int. This helpful feature can be a problem, however, when using views for row level security. It is possible – accidentally or deliberately – to cause a SQL statement to throw an error that includes data from a row the user is not supposed to see! Clearly, this is not acceptable. Luckily, SQL Server 2005 and later supports a trace flag to alter this behavior. Trace flag 3625 is a global trace flag that must be included in the startup parameters of the SQL Server service. When it is enabled, any error message which could normally include data in the message is altered to mask the information. For example, the error message above becomes the following: Msg 245, Level 16, State 1, Line 1 Conversion failed when converting the ****** value '******' to data type ******. The server only masks data in error messages for users who are NOT members of the sysadmin role. Implementing Row and Cell Level Security in Classified Databases 27 Pulling it All Together (Part 1) The aspects of the design presented above make up a design framework for adding supporting objects to a database and outfitting new or existing tables with label-based row-level security. For convenience, the set of supporting objects is referred to as a “label policy”. Figure 2 depicts this graphically. The bottom layer is the base table itself. Over the base table we create the RLS view, which enforces row-level security for reads against the table. This makes use of the supporting meta-information shown in the label policy on the side of the diagram. For write operations, automatic support for updateable views may be sufficient, depending on the project requirements. Optionally, INSTEAD OF triggers for insert and update may be added to handle more complex requirements. To support deletes on the table, a simple INSTEAD OF trigger is required. Select Insert Update Delete I.O. Insert I.O. Update I.O. Delete User Accessible View (Join of base table on vwVisibleLabels) Label Policy vwVisibleLabels SecurityLabel XML Schema Metadata Tables Base Table Helper Procedures and Functions Database Roles and/or AD Groups Figure 2 Only the RLS view and selected objects in the label policy (e.g., the vwVisibleLabels view, certain helper functions, and the XML schema definition for security labels) are exposed to user logins. To make understanding and implementing this design framework easy, a toolkit has been assembled which includes a code generation tool, how-to documents, and a set of samples. The code generation tool allows developers or administrators to graphically describe the structure of security labels required by the system, and uses that to produce scripted output for all the elements of the label policy. This content can be found by searching the Codeplex code-sharing site for “Label Security Toolkit.” In the following sections, we build on this framework to include cell-level label security. Implementing Row and Cell Level Security in Classified Databases 28 Cell-Level Security It is possible that data may need control at a finer level of detail than the entire row. Most of a row might need to be visible to one set of users, while certain more sensitive cells might require additional permissions to view. This is shown notionally in Figure 3. Different patterns in Figure 3 represent the different labels that are applied to the data. While the next-to-last row is controlled with row-level security, there are several cells scattered through the table that have their own distinct labels. Figure 3 We would like to be able to control the visibility of the data based on user permissions, and do so as close to the data itself as possible. Ideally, the database engine could simply show data from labeled cells, or not, based on the identity of the connected user. SQL Server 2005 introduced encryption capabilities in the database engine itself. These can be used to encrypt and decrypt any arbitrary data, using a certificate and key infrastructure that is internally managed by the database engine. There is no need to pass in certificates or keys from an external source. The following pages present a design which can accomplish this level of cell-level control, in a way that is relatively easy to implement and administer. The basic objectives of the design are: Support for the arbitrary labeling of cells of data. Dynamic evaluation of the user’s label, to show only the appropriate cells. Acceptable performance impact at high volume. Cell-Level Encryption in SQL Server SQL Server provides internal functions to easily encrypt and decrypt data using a certificate, asymmetric key, or symmetric key. It manages all of these in an internal certificate store. The store uses an encryption hierarchy that secures certificates and keys at one level with the layer above it in the hierarchy. Optionally, the Extensible Key Management feature (EKM), added in SQL Server 2008, allows parts of the encryption hierarchy to reside in external key management devices. Implementing Row and Cell Level Security in Classified Databases 29 Figure 43 The fastest mode of encryption supported by the internal API is symmetric key encryption. This mode is suitable for handling large volumes of data. The symmetric keys themselves can be stored encrypted by X509.v3 certificates. SQL Server supports several symmetric key encryption algorithms. The algorithms are implemented in the Windows Crypto API. Refer to SQL Server Books Online for the supported algorithms and the key size for each algorithm. Within the scope of a database connection, SQL Server can maintain multiple open 3 Source: SQL Server 2008 Books Online Implementing Row and Cell Level Security in Classified Databases 30 symmetric keys. By “open” we mean they are retrieved from the store and ready to be used for decrypting data. When a piece of data is decrypted, there is no need to specify the symmetric key to use. Instead, the engine matches the encrypted byte stream to an open symmetric key, if the correct key has been decrypted and is open. This key is then used to perform decryption and return the data. If the correct key is not open, the engine returns NULL. The ability of a key to be “open” depends directly on the permission GRANTs on the key. Given these mechanics of SQL Server encryption support, suppose the following approach. 1. Create a symmetric key for each unique label that is used to mark data in the database. 2. Encrypt data in labeled cells with the corresponding key. 3. Control access to keys in such a way that exactly the keys that map to labels to which the user has access can be opened. Provide a simple way to have all these keys opened when the connection is established. 4. Use a view over the base table to include calls to the decryption API in the SELECT statement that defines the view. The view definition shown below gives a simple example of what such a view might look like. CREATE VIEW MyTable AS SELECT ID, DecryptByKey(SensitiveData), DecryptByKey(OtherSensitiveData), NonSensitiveData FROM BaseTable GO Given this approach, let’s look at what happens when a user selects against the view. All the keys which map to labels that the user can access will have been opened. Therefore, all the cells with labels the user can access will be decrypted when the SELECT statement is executed. Conversely, all the keys that map to labels the user cannot access will not be opened. When the SELECT statement is executed, cells with these labels come back NULL, providing the user with no information on whether there is even data present in the cell. This approach accomplishes the granular, dynamic control over data in a relational table that we are seeking. Implementing Row and Cell Level Security in Classified Databases 31 Key access control Of course, the suitability of the design hinges on control of the keys. SQL Server defines permissions on keys in terms of a single SQL Server principal. The permission to open a key, for example, can be granted to a user named Bob, or to a group named AppUsers. In our scenario, however, we want to control rights to the key based on arbitrary combinations of principals – to be specific, based on arbitrary combinations of role memberships which define the user’s subject label. We want to avoid defining a principal for every possible label combination. As it happens, the semantics for deciding which keys a user can access are identical to those for controlling row access. This type of ACL’ing requires a more subtle approach. Instead of granting key permissions to users, or to roles, we will grant permissions to a system-defined broker user account. We’ll call it the KeyBroker account. KeyBroker can open keys. Users and user roles are denied all permissions on keys. We get a list of the labels to which the user should have access, and ask KeyBroker to open the corresponding keys. Using impersonation features of SQL Server, we can define a stored procedure which can be called by users, but which will ‘EXECUTE AS’ KeyBroker. This procedure is shown below. A cursor is defined on the vwVisibleLabels view. Note two columns in that view which we have not mentioned yet: KeyName and CertName. The cursor selects these two columns from the view. Using the EXECUTE AS CALLER impersonation feature, the stored procedure temporarily reverts to the identity of the calling user and opens the cursor. This ensures that the rows returned from vwVisibleLabels are based on the calling user’s role memberships. We then immediately revert back to the KeyBroker identity. The rest of the procedure loops through the rows in the cursor, opening each key by using the dedicated certificate used to encrypt it. CREATE PROCEDURE usp_EnableCellVisibility WITH EXECUTE AS 'KeyBroker' AS DECLARE @KeyName nvarchar(256) DECLARE @CertName nvarchar(256) DECLARE Key_Cursor CURSOR LOCAL FORWARD_ONLY STATIC FOR SELECT KeyName, CertName FROM vwVisibleLabels EXECUTE AS CALLER --Since the cursor is STATIC, it is fully --populated here based on the caller’s identity OPEN Key_Cursor REVERT FETCH NEXT FROM Key_Cursor INTO @KeyName, @CertName WHILE @@FETCH_STATUS = 0 Implementing Row and Cell Level Security in Classified Databases 32 BEGIN open symmetric key @KeyName using certificate @CertName FETCH NEXT FROM Key_Cursor INTO @KeyName, @CertName END CLOSE Key_Cursor DEALLOCATE Key_Cursor GO Exit from the stored procedure automatically reverts the user context back to the calling user. This approach opens exactly those keys associated with labels that are dominated by the user’s label. The user never has any access to keys, and so cannot open any others. And, there is no input to this procedure which can be spoofed. Evaluation of rights is based only on membership in SQL Server database roles. With the correct symmetric keys open, selecting from the view causes labeled cells to be visible if the user’s label dominates. All other labeled cells appear as NULL. The expectation is that this stored procedure (usp_EnableCellVisibility) is called once by an application or end user immediately after establishing the database connection. The keys opened remain open for the life of the connection. A corresponding procedure (usp_DisableCellVisibility) is provided to close the keys if needed. This is not strictly necessary, as closing the connection (or releasing it to the connection pool) does the necessary cleanup. Changes to the base table Changes to the base table to support cell-level security are fairly minor. Most important, the data type of the column to be protected must be compatible with the encrypted data values. The intrinsic function EncryptByKey returns varbinary. This can be stored in a character or binary field (for example, varchar, nvarchar, or varbinary). If the original data type is not compatible with varbinary content—a numeric type such as int, for example—the column data type must be changed. In the next section, we’ll see how to make the data type appear unchanged to the user. Ensuring the correct data type is the only required change. Another change that may be desirable in some scenarios is the addition of a companion column to hold the label that applies to the cell. The main purpose of this would be to comply with policies requiring that the label live with the cell data. This could be stored as the ID from the tblUniqueLabel table, an instance of the SecurityLabel typed XML variable, or the raw label string from source data. Including this information also makes it easy to display the label alongside the data. Implementing Row and Cell Level Security in Classified Databases 33 Defining the RLS/CLS view Finally, we need to redefine the user-accessible view to include logic for decrypting data in protected cells. The next code example shows what the view definition should look like. It is almost identical to the view definition we showed previously when discussing row-level security. The only difference is that some columns are wrapped in a SQL Server intrinsic function called DecryptByKey. To keep the example simple, we assume just a few columns in the base table. CREATE VIEW UserTable AS SELECT ID, DecryptByKey(SensitiveData), CONVERT(money, CONVERT(varchar(50), DecryptByKey(SensitiveMoneyData))) AS SensitiveMoneyData, NonSensitiveData FROM tblBaseTable WITH (READCOMMITTED) INNER JOIN vwVisibleLabels ON tblBaseTable.SecLabelID = vwVisibleLabels.ID GO GRANT SELECT ON UserTable TO <app_users> DENY SELECT, INSERT, UPDATE, DELETE ON tblBaseTable TO <app_users> GO As mentioned in the previous section, the encryption functions use character or binary data as input and output. If the original data type of a protected column is numeric, for example, the view definition should include a conversion of the varbinary decryption output to the original data type. An example of this is shown in the third column of the view above. Encrypting cell data on insert/update The encryption of cell data can be handled in much the same way we handled complex rules for row-level security. INSTEAD OF triggers defined on the view handle any necessary security rule checks and also encrypt the cell(s) based on their label. Implementing Row and Cell Level Security in Classified Databases 34 Pulling it All Together (Part 2) Combining cell-level security with the previously presented row-level security model, we have the design shown in Figure 5 as a pattern for outfitting a table with both types of fine-grained access control. Select Insert Update Delete I.O. Insert I.O. Update I.O. Delete Label Policy vwVisibleLabels User Accessible View (Join of base table on vwVisibleLabels with DecryptByKey for CLS-protected columns) SecurityLabel XML Schema Base Table Metadata Tables Helper Procedures and Functions Database Roles and/or AD Groups Figure 5 Strengthening Security with Audit and TDE Although they are not strictly part of the design for row- or cell-level access control, there are two features of SQL Server 2008 and later that are strongly recommended for enhancing the security of an RLS and/or CLS implementation. The first of these is SQL Server Audit. The Audit feature allows fine-grained, secure auditing of any access to objects in a database. In particular, it is an excellent tool for rigorously tracking changes to the metadata tables and role memberships in the label policy. A second feature that is excellent for enhancing security of an RLS/CLS protected database is Transparent Database Encryption (TDE). TDE causes the data and log files (and full-text catalogs, if present) to be encrypted on disk. The encryption occurs transparently as data moves through SQL Server’s IO buffers, so no complicated setup is required and the encryption is all-encompassing for the encrypted database. In addition, backups of a TDE-encrypted database are encrypted as well. This can mitigate on of the perennial concerns with any database protected by fine-grained access control: if someone gets their hands on the backup file they can read the raw data, without regard for any access control the DBMS would normally enforce. With an encrypted backup this concern is significantly reduced. Implementing Row and Cell Level Security in Classified Databases 35 Additional Considerations for SQL Server 2012 Databases SQL Server 2012 is compatible with all aspects of the design framework discussed in this whitepaper. A few new features of the SQL Server 2012 release deserve to be mentioned, however. User-defined server roles, added in SQL Server 2012, allow server roles to be created with a custom subset of server-level permissions. These add flexibility to the product's fixed server roles, and can be useful for separating and limiting administrative permissions for those people given operational access to the server. SQL Server 2012 also introduces partially contained databases (CDB), which provides a level of independence between databases and server instances. Using contained databases, you can, for example, create and manage users within database scope so that moving to another server does not require re-creating users and/or associating database users with logins on the new instance. In the language of CDB, the RLS design presented here is fully contained and does not cross the application model boundary. The CLS design includes uncontained entities, since it uses encryption features (i.e., symmetric keys) which are outside the application model. If you use CDB in application databases along with RLS or CLS, be sure that you are familiar with the security implications of CDB, especially with respect to users and logins. These are discussed in SQL Server Books Online in the section called "Threats Against Partially Contained Databases." Performance A key question in assessing the encryption-based CLS design is its impact on performance. An acceptable solution must implement the required security, yet have an acceptable performance impact with tables containing millions of rows. A reference implementation of this design has been tested with encryption-based CLS on a dataset containing one million rows. The test server specs were: CPU 2 x 550 MHz RAM 512 MB Disk 4 x 18 GB @ 10,000 RPM 1 x 20 GB on Fiber channel SAN Operating System Windows 2003 Standard Database SQL Server 2005 Beta 2 .NET Framework 1.1 for application code 2.0 for SQL Server code A detailed description of the test is beyond the scope of this document. In summary, the following results were obtained for the impact of cell-level encryption on response times. Implementing Row and Cell Level Security in Classified Databases 36 Impact on insert performance: ~40 percent degradation Impact on selective queries: < 10 percent degradation Impact on aggregate queries: ~10-50 percent degradation The performance profile of every application is unique, and these results should not be taken as a fixed guideline. However, the testing does indicate that this design impacts performance to an acceptable degree. Note that the measured impact is on a low end (550 MHz) server with limited RAM. Assuming a similar performance impact for production applications, additional hardware resources could make up for the degradation. Row-Level Disclosure The view-based row-level security design described previously has some specific, narrow vulnerabilities. The design prevents the return of rows to which the user should not have access, whether the data is queried from an application, a reporting tool, or a direct connection with a SQL query tool. However, given the right conditions, data may be exposed by inference if an error message is thrown. This section describes the conditions, the potential disclosure, and presents three approaches to mitigation. Predicate evaluation order If a specifically formed query is issued against the user-accessible view that implements row-level security, an error may be returned which reveals the existence of information in the table to which the user is not supposed to have access. The query might look like the following: SELECT * FROM UserTable --actually a view WHERE LastName = ‘Smith’ AND LEN(LastName)/0 > 10 Suppose that there are no rows where LastName equals Smith to which the current user has access, but that there is at least one to which the user does not have access. Internally, the database server’s query optimizer will decompose any view in the statement into its parts. It will then build a query plan that is based on all of the WHERE clause conditions in the view(s) and in the user-supplied statement’s WHERE clause. If the statement executes successfully, no information will leak. However, if the WHERE clause includes invalid predicates (such as division by zero), there is a risk of leakage. Depending on the query plan generated by the query optimizer, the invalid predicate may be evaluated before the predicate which restricts row visibility. If this happens, an error will be thrown that implies the existence of a row that should not be ‘seen’. Implementing Row and Cell Level Security in Classified Databases 37 Here is how it might work with the example statement above. Suppose UserTable is actually a view defined per the design presented earlier. The actual query as seen by the optimizer will be something like: SELECT * FROM BaseTable (READCOMMITTED) WHERE (LastName = ‘Smith’ AND LEN(LastName)/0 > 10) AND BaseTable.SecLabelID IN (SELECT ID FROM tblUniqueLabel WHERE ID IN .. /* category predicates */) Suppose the optimizer chooses to first use an index on LastName to narrow the result set before the other conditions are evaluated. It will find the row(s) where LastName =’Smith’, then evaluate LEN(LastName)/0 > 10. This will immediately throw an exception, prior to the evaluation of the row security predicates. The result may be that the presence of at least one row with LastName = ‘Smith’ is revealed, even if the user should not see any of those rows. This issue is not unique to view-based row-level security with SQL Server. Other vendors’ row-level security solutions also are susceptible to various forms of inferencebased exploitation. It is important to observe the conditions which must be met for this vulnerability to be an issue: 1. A user must be able to directly submit a maliciously formed SQL query, either by using a SQL query tool, or SQL injection against a poorly written front-end application. 2. The user must know enough about the table definition to write the query. 3. The query optimizer must choose a plan which allows the invalid predicate to evaluate before the row access predicate(s), and to behave inconsistently based on whether matching data is present or not present. This is a function of several variables, including what indexes exist, the distribution of data in the table, and the nature of the other predicates in the statement. 4. The user must be able to directly see the error that is generated. This is a given with a SQL query tool, but a well-written application will not return raw error information. 5. The user must have malicious intent and the technical understanding to draw inferences from error messages. Mitigation In many cases, the chances of exploiting the vulnerability previously described will be deemed small enough that no action is needed. If, however, it is absolutely necessary to ensure that this vulnerability is mitigated, three options are recommended. All use Implementing Row and Cell Level Security in Classified Databases 38 the same basic design explained in this paper, but add an additional piece. Table-valued function One way to eliminate the vulnerability is to ensure that the row-access predicate is fully applied to the underlying data before any user-supplied predicates. This can be achieved using a table-valued function (TVF). The role of this TVF can easily be seen in a modified version of Figure 5. Insert Update Delete I.O. Insert I.O. Update I.O. Delete Select User Accessible View (Passthrough) Label Policy Passthrough Table Valued Function vwVisibleLabels SecurityLabel XML Schema RLS / CLS View Metadata Tables Base Table Helper Procedures and Functions Database Roles and/or AD Groups Figure 6 Figure 6 is much like Figure 5, except that the view which implements row and/or cell level security is wrapped in a TVF. The TVF is then referenced by a user-accessible view. The definition on the TVF would be as shown in the following code. CREATE FUNCTION [dbo].[fn_MyTable_tvf]() RETURNS @ret TABLE ( <column specs from underlying RLS view> ) AS BEGIN INSERT @ret SELECT <columns> FROM vwRLSView RETURN END GO Implementing Row and Cell Level Security in Classified Databases 39 Note that the TVF must be a multi-statement table-valued function. An inline tablevalued function will not have the intended effect. Wrapping the row-access view will force it to be fully materialized as a table variable before any user-supplied predicates are evaluated. This eliminates the vulnerability. The drawbacks to this approach are: 1. Additional complexity. 2. Performance. This requires the entire underlying table, with row-level security applied, to be buffered in a table variable which is then queried by the useraccessible view. For small tables, the impact is minor. For large datasets (millions of rows), the performance impact is severe. Encryption The second mitigation involves using the same encryption strategy as that described for cell-level security. Again, assume that all other aspects of the design are as shown in Figure 5. However, we will add an additional level of encryption to the data in each row. In addition to the cell-level encryption, every user-accessible column will be encrypted with the symmetric key that is associated with the row label. This ensures that, regardless of the query plan, the user will only read data values if their label allows access. For example, assume a simple table with four columns. The table is protected with rowand cell-level security as depicted in Figure 5. The contents of one row might be as shown in the following table. Column 1 Column 2 Column 3 Column 4 SecLabelID Data1 E(Data2, K1) Data3 E(Data4, K2) 19 This row is labeled for row-level access control. The label is mapped to the record in tblUniqueLabel with an ID of 19. Columns 2 and 4 are protected with cell-level security. In the example row shown, column 2 is encrypted with a key for label 1 (K1) and column 4 is encrypted with a key for label 2 (K2). What the labels are doesn’t matter for this example. This mitigation approach would apply additional encryption, so that the data stored in the underlying table looks like this: Column 1 Column 2 Column 3 Column 4 SecLabelID E(Data1, K19) E(E(Data2, K1), K19) E(Data3, K19) E(E(Data2, K2), K19) 19 Implementing Row and Cell Level Security in Classified Databases 40 Every user-accessible column is encrypted with the symmetric key mapped to label 19 (K19). This work is done in the INSTEAD OF triggers. The user-accessible view in Figure 5, which includes decrypt statements, would be modified to include additional decrypts as shown in the following code. CREATE VIEW UserTable AS SELECT DecryptByKey(Column1), DecryptByKey(DecryptByKey(Column2)), DecryptByKey(Column3), DecryptByKey(DecryptByKey(Column4)) FROM vwRLSView WHERE ………… --omitted for clarity Note that, if the table in question only uses row level security to start with (i.e., no cell level security), there need only be a single encryption pass and things are somewhat simpler. The same principles described for managing open keys for cell-level security apply here. Only if the user’s label (role memberships) were appropriate would K19 be open. Therefore, no potential query plan could access impermissible data and disclose information. This approach has drawbacks as well: 1. Additional complexity 2. Performance. More CPU time will be spent on encryption/decryption. Also, since all searchable columns are encrypted, SQL indexes cannot be used to speed up queries. Despite these issues, it is likely that this approach would scale to large datasets better than the TVF approach. 3. Loss of DBMS integrity enforcement. The database cannot enforce integrity constraints such as unique indexes or foreign key constraints on columns that contain encrypted data. Application error management The first two mitigation approaches address the issue solely at the database layer. A third option is to address it in other parts of the overall application. In this approach, some small risk is accepted that errors may occur which could provide inference channels. At an intervening layer of the solution (for example, the object-oriented data access layer) careful error trapping and logging is implemented. By trapping specific errors from the database, and propagating only a general failure message to the user, the amount of detail that is available to support inference is reduced. And, logging of errors provides an audit trail, which can be monitored for suspicious patterns that would indicate a user is attempting to exploit the system. Implementing Row and Cell Level Security in Classified Databases 41 Mitigation summary Whether or not either mitigation approach should be used is a judgment call based on the situation. It is possible that the threat scenario for a given application makes the chances of exploitation so insignificant that the penalty in complexity and performance is not worthwhile. These approaches are available, however, to eliminate the vulnerability, if necessary. Conclusion The design presented in this paper enables a SQL Server database to support row- and cell-level security based on an arbitrary security label policy. Access restriction on rows and cells is enforced inside the database by using intrinsic structures such as views and SQL Server data encryption. The evaluation of user access is based on the intrinsic SQL Server security model. This assembly of thoroughly tested, secure elements is superior to approaches that use ad hoc, custom code to do the primary authorization and filtering steps. Appendix A – Example XML Schema for Labels The following example shows an XML schema definition for a label policy, assuming these categories. ID Name Abbreviation Cardinality Required 1 Classification Cl One Yes 2 Compartments Cp Many No 3 Releasability RelTo Many No 4 Need To Know N Many No The schema definition for labels is as follows. Note that most of the text is boilerplate. Only the items highlighted in yellow are altered based on the category definitions for the label policy. CREATE XML SCHEMA COLLECTION [dbo].[SecurityLabel] AS N'<?xml version="1.0" encoding="utf-16"?> <xs:schema elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema" > <xs:element name="Label" type="thisLabel" /> <xs:complexType name="thisLabel"> <xs:sequence> <xs:element name="Cl" minOccurs="0" maxOccurs="1" nillable="false"> <xs:simpleType> <xs:restriction base="xs:string"> <xs:maxLength value="50" /> <xs:pattern value="\p{L}([\w_\- ])*" /> Implementing Row and Cell Level Security in Classified Databases 42 </xs:restriction> </xs:simpleType> </xs:element> <xs:element name="Cp" minOccurs="0" maxOccurs="unbounded" nillable="false"> <xs:simpleType> <xs:restriction base="xs:string"> <xs:maxLength value="50" /> <xs:pattern value="\p{L}([\w_\- ])*" /> </xs:restriction> </xs:simpleType> </xs:element> <xs:element name="RelTo" minOccurs="0" maxOccurs="unbounded" nillable="false"> <xs:simpleType> <xs:restriction base="xs:string"> <xs:maxLength value="50" /> <xs:pattern value="\p{L}([\w_\- ])*" /> </xs:restriction> </xs:simpleType> </xs:element> <xs:element name="N" minOccurs="0" maxOccurs="unbounded" nillable="false"> <xs:simpleType> <xs:restriction base="xs:string"> <xs:maxLength value="50" /> <xs:pattern value="\p{L}([\w_\- ])*" /> </xs:restriction> </xs:simpleType> </xs:element> </xs:sequence> </xs:complexType> </xs:schema>' GO For More Information SQL Server Website http://www.microsoft.com/sqlserver/ SQL Server TechCenter http://technet.microsoft.com/en-us/sqlserver/ SQL Server DevCenter http://msdn.microsoft.com/en-us/sqlserver/ Did this paper help you? Please give us your feedback (add mail to link). Tell us on a scale of 1 (poor) to 5 (excellent), how would you rate this paper and why have you given it this rating? For example: Implementing Row and Cell Level Security in Classified Databases 43 Are you rating it high because of having good examples, excellent screenshots, clear writing, or another reason? Are you rating it low because of poor examples, fuzzy screenshots, or unclear writing? This feedback will help us improve the quality of white papers we release. Send feedback. Implementing Row and Cell Level Security in Classified Databases 44