Multi-tenant Table
Administration
Flexibility without the complications.
Richard Banville
Fellow, Progress Software
June 7, 2011
Agenda
 Administration architecture
 Multi-tenant Administration: “How to”
 Utility usage examples
 Multi-tenant Configuration Tool demo
2
© 2011 Progress Software Corporation. All rights reserved.
Multi-tenant Identity Definitions
 Tenant
• “Named” group of users
• Share same application and data
 Multi-tenancy
• Deployment supporting multiple tenants
 Default tenant
• Users who do not assert tenant identity
 Super tenant
• Tenant with ability to access/manage data of any tenant
 Multi-tenant group
• Multiple tenants sharing data for a specific multi-tenant table
3
© 2011 Progress Software Corporation. All rights reserved.
Multi-tenant Allocation Definitions
 Table instance
Customer
• Defined by a single table definition (schema)
Record Data
• Each table instance contains
Name Index
Cust-num Index
Blob Data
– A data segment for the record data
– A separate data segment for each index/lob of the table
• Each tenant/group has a different table instance
• A tenant need not instantiate every multi-tenant table
 Data segment
Record Data
Name index
• Tenant/group specific chain of data for each table/index/lob
• Stored/encapsulated in its own physical data partition
 Physical data partition
Record Data
Name index
• Managed storage for each object of a multi-tenant table instance
• Do not span storage areas
4
© 2011 Progress Software Corporation. All rights reserved.
Multi-tenant Administration
What needs administration?
 Tenant Administration
• Tenants
• Table definition
• Groups
• Allocation
• Domains
• Location
• Users
• Maintenance
• Security/Data Access
 Performance
• Monitoring
• Configuration
5
 Data Administration
© 2011 Progress Software Corporation. All rights reserved.
Multi-tenant Model
3
Suzi@HD_Domain
Allen@HD_Domain
Cat@HD_Domain
Rich@HD_Domain
Rich@Lowes1
John@Lowes1
Claudio@Lowes2
Louie@Lowes2
Domains/Tenants
2
1
Name
Tenant
Data
0 thru 4
HomeDepot
Customers
Orders
HD_Domain HomeDepot
Name
Tenant
Lowes1
Lowes
Lowes2
Lowes
Name
Tenant
blank
Default
Edward
Tenancy Layer
Users
© 2011 Progress Software Corporation. All rights reserved.
…
Lowes
Customers
Orders
Items
…
Default
de-allocated or migrated
data
Shared
_file
state
6
Items
_field
…
_tenant
…
7
© 2011 Progress Software Corporation. All rights reserved.
Physical Model By Data and Datatype
Data
*Type II Storage Areas
HomeDepot
Customers
Orders
Tenancy Layer
Items
Lowes
Customers
CustOrderIdxs
Area7
Area8
Area9
Area10
Orders
Items
…
Default
de-allocated or migrated
data
Shared
_file
state
8
…
CustOrder Data
_field
_tenant
ItemData
ItemIdxs
…
…
© 2011 Progress Software Corporation. All rights reserved.
Shared Data
A11
A12
Shared Indexes
Physical Model By Tenant
Data
Type II Storage Areas
HomeDepot
HD CustOrder Data
Customers
Orders
Tenancy Layer
Items
HD CustOrderIdxs
…
HD ItemData
Area 7
HD ItemIdxs
Lowes
Lowes CustOrder Data
Customers
Orders
Items
…
Lowes CustOrderIdxs
Default
Default Data Area
de-allocated or migrated
data
Lowes ItemData
Area 8
Lowes ItemIdxs
Area 9
Default Index
Shared
_file
state
9
_field
…
_tenant
…
© 2011 Progress Software Corporation. All rights reserved.
Shared Data
A10
A11
Shared Indexes
Physical Model By Tenant By Data And Datatype
Data
Type II Storage Areas
HomeDepot
HD CustOrder Data A7
Customers
HD CustOrderIdxs
Orders
Tenancy Layer
Items
…
Lowes
A9
Lowes CustOrder Data A11
Customers
Orders
Items
…
Lowes CustOrderIdxs
Default
A13
A8
A10
HD ItemIdxs
A12 Lowes ItemData
A14
Lowes ItemIdxs
Default Index Area
Default Data Area
de-allocated or migrated
data
HD ItemData
A15
A16
A17
A18
Shared
_file
state
10
_field
…
_tenant
…
© 2011 Progress Software Corporation. All rights reserved.
Shared Data
Shared Indexes
Example: Adding A New Tenant “AceHardware”
Data
Type II Storage Areas
AceHardware
Ace CustOrder Data A23
Customers
Ace CustOrderIdxs
Orders
Tenancy Layer
Items
…
Shared
11
© 2011 Progress Software Corporation. All rights reserved.
A25
A24 Ace ItemData
A26
Ace ItemIdxs
•
•
Determine physical layout 1st
•
Table instance allocation:
delayed, immediate, none
•
Tenant Access: Allowed, disallowed
Create tenant & assign partition locations next
New Inventory Table With Existing Tenants
Data
Type II Storage Areas
HomeDepot
HD CustOrder Data A7
Customers
HD CustOrderIdxs
Tenancy Layer
Orders
Items
Lowes
Lowes CustOrder Data A11
Customers
Lowes CustOrderIdxs
Orders
Items
•
12
…
A9
…
Example: Adding an Inventory table
© 2011 Progress Software Corporation. All rights reserved.
A13
A8
A10
HD ItemData
HD ItemIdxs
A12 Lowes ItemData
A14
Lowes ItemIdxs
New Inventory Table With Existing Tenants
Data
Type II Storage Areas
HomeDepot
HD CustOrder Data A7
Customers
HD CustOrderIdxs
Tenancy Layer
Orders
Items
A19
Lowes
Lowes CustOrder Data A11
Customers
Lowes CustOrderIdxs
Orders
Inventory
13
A10
HD Inventory Data
Inventory
Items
•
…
A9
A8
…
Lowes Inventory Data
A13
A21
HD ItemIdxs
HD InventoryIdxs
A20
A12 Lowes ItemData
A14
A22
Tenant specific space allocation immediate or delayed
© 2011 Progress Software Corporation. All rights reserved.
HD ItemData
Lowes ItemIdxs
Lowes InvIdxs
Groups
 Tenants have their own table instance if:
Data
AceHardware
Customers
• Not part of a group
…
Tenancy Layer
TaxCode
State
14
HomeDepot
Customers
…
TaxCode
State
© 2011 Progress Software Corporation. All rights reserved.
Groups
 Tenants have their own table instance if:
Data
AceHardware
Customers
• Not part of a group
…
Tenancy Layer
TaxCode
State
HomeDepot
Customers
 Group data
• Shared amongst tenants in the group
• Can exists with no tenants assigned
…
TaxCode
State
• Follow allocation rules already defined
 Each group has only one table instance
• Cannot delete table if it has groups
TaxCodeGroup
TaxCodes
StateGroup
Tax Code Data
State Data
A30
A31
Tax Code Indexes
State Indexes
States
 Can be spread across storage areas.
15
© 2011 Progress Software Corporation. All rights reserved.
Groups
 Tenants have their own table instance if:
Data
AceHardware
Customers
• Not part of a group
…
Tenancy Layer
TaxCodeGroup
StateGroup
HomeDepot
Customers
…
TaxCodeGroup
StateGroup
 Group data
• Shared amongst tenants in the group
• Can exists with no tenants assigned
• Follow allocation rules already defined
 Each group has only one table instance
• Cannot delete table if it has groups
TaxCodeGroup
TaxCodes
StateGroup
Tax Code Data
State Data
A30
A31
Tax Code Indexes
State Indexes
States
 Can be spread across storage areas.
16
© 2011 Progress Software Corporation. All rights reserved.
17
© 2011 Progress Software Corporation. All rights reserved.
Multi-tenant Administration
Appearance of database isolation where possible
 Administration mechanisms
• Data Dictionary
• SQL DDL
• Browser based config tool
• Command line tools
• ABL administrative APIs
 Database enablement:
proutil <db> -C enableMultitenancy
dbutil describe: 14 Multi-tenancy
Yes
 Create storage areas
dbutil prostrct add[online] <db> <addmt>.st
• No new syntax to database structure files (.st)
• Coordinate with database designers/developers
• Have a plan based on physical model
• Naming conventions can ease your pain
18
© 2011 Progress Software Corporation. All rights reserved.
Identify Multi-tenant Tables
3
Suzi@HD_Domain
Allen@HD_Domain
Cat@HD_Domain
Rich@HD_Domain
Rich@Lowes1
John@Lowes1
Claudio@Lowes2
Louie@Lowes2
Domains/Tenants
2
1
Name
Tenant
Data
0 thru 4
HomeDepot
Customers
Orders
HD_Domain HomeDepot
Name
Tenant
Lowes1
Lowes
Lowes2
Lowes
Name
Tenant
blank
Default
Edward
Tenancy Layer
Users
© 2011 Progress Software Corporation. All rights reserved.
…
Lowes
Customers
Orders
Items
…
Default
de-allocated or migrated
data
Shared
_file
state
19
Items
_field
…
_tenant
…
Object Creation Considerations w/out Tenants
Schema manipulation tools such as data dictionary, SQL DDL, etc
 Convert existing table to multi-tenant table
• All objects of table instance MUST reside in TII storage area
• One way conversion
• Default partition contains existing data
 Create new multi-tenant tables
• No need for default partition (optional)
• New multi-tenant “default index” goes in same area as table
 Adding new indexes with new tables
• Activation state at index level, NOT tenant level
– Can be changed after the fact
20
© 2011 Progress Software Corporation. All rights reserved.
Sequence Creation Considerations
 Sequences
• Shared or tenant specific
• Definition database wide (max/min/cycle values)
• Current value tenant specific
• Not group specific
– Avoid multi-tenant sequences for group data
– Group data is shared amongst specific tenants
21
© 2011 Progress Software Corporation. All rights reserved.
Tenant Creation
3
Suzi@HD_Domain
Allen@HD_Domain
Cat@HD_Domain
Rich@HD_Domain
Rich@Lowes1
John@Lowes1
Claudio@Lowes2
Louie@Lowes2
Domains/Tenants
2
1
Name
Tenant
Data
0 thru 4
HomeDepot
Customers
Orders
HD_Domain HomeDepot
Name
Tenant
Lowes1
Lowes
Lowes2
Lowes
Name
Tenant
blank
Default
Edward
Tenancy Layer
Users
© 2011 Progress Software Corporation. All rights reserved.
…
Lowes
Customers
Orders
Items
…
Default
de-allocated or migrated
data
Shared
_file
state
22
Items
_field
…
_tenant
…
Tenant Creation Using ABL Multi-tenant APIs
using OpenEdge.DataAdmin.*.
define variable service as DataAdminService no-undo.
service = new DataAdminService("demo").
define variable tenant as ITenant no-undo.
define variable partition as IPartition no-undo.
tenant = new Tenant("HomeDepot").
assign
tenant:Type
tenant:IsOnline
tenant:Description
tenant:DefaultDataArea
tenant:DefaultIndexArea
tenant:DefaultLobArea
tenant:DefaultAllocation
= "Regular" /* or super */
= yes
= "Describe HomeDepot"
= service:GetArea("Data Area")
= service:GetArea("Index Area")
= service:GetArea("Lob Area")
= "Delayed". /* Immediate or None */
/* Create the tenant */
service:CreateTenant(tenant).
…
23
/* continued on next page */
© 2011 Progress Software Corporation. All rights reserved.
Tenant Creation Using ABL Multi-tenant APIs
Reference
Interfaces,
Services
using OpenEdge.DataAdmin.*.
define variable service as DataAdminService no-undo.
service = new DataAdminService("demo").
define variable tenant as ITenant no-undo.
define variable partition as IPartition no-undo.
tenant = new Tenant("HomeDepot").
Set Area
Defaults
assign
tenant:Type
tenant:IsOnline
tenant:Description
tenant:DefaultDataArea
tenant:DefaultIndexArea
tenant:DefaultLobArea
tenant:DefaultAllocation
= "Regular" /* or super */
= yes
= "Describe HomeDepot"
= service:GetArea("Data Area")
= service:GetArea("Index Area")
= service:GetArea("Lob Area")
= "Delayed". /* Immediate or None */
/* Create the tenant */
service:CreateTenant(tenant).
…
24
/* continued on next page */
© 2011 Progress Software Corporation. All rights reserved.
Partition Assignment
Reassign partition location information prior to allocation.
Allocation State: Delayed or None
…
/* continued from previous page */
assign
partition
= tenant:Partitions:Get(service:GetTable("Customer"))
partition:Area = service:GetArea("HD CustomerArea“)
partition
= tenant:Partitions:Get(service:GetTable("Order"))
partition:Area = service:GetArea("HD OrderArea")
partition
= tenant:Partitions:Get(service:GetTable("Item"))
partition:Area = service:GetArea("HD ItemArea")
partition:AllocationState = “None”.
Allocate
/* Allocate Space */
tenant:Allocate(). /* indicate allocation of everything in delayed state */
Service:UpdateTenant(tenant).
delete object service.
25
© 2011 Progress Software Corporation. All rights reserved.
Partition Location Re-assignment
 Reassign partition location after allocation
• Table/index move OR Dump, de-allocate, reassign via APIs/tools, reload
• Using table/index move utilities
proutil <db> -C tablemove <table> <area> tenant <tenant>
• New de-allocate utility (bye-bye data)
proutil <db> -C deallocate <table> tenant <tenant>
• Binary Dump: data is tenant indifferent
– Dump file: <table>_<ID>{T|G}_<tenant | group>.bd[n]
– Example: customer_2T_HomeDepot.bd1
proutil <db> -C dump <table> tenant <tenant>
• Utilities also support “group” based operations
26
© 2011 Progress Software Corporation. All rights reserved.
Using Domains
3
Suzi@HD_Domain
Allen@HD_Domain
Cat@HD_Domain
Rich@HD_Domain
Rich@Lowes1
John@Lowes1
Claudio@Lowes2
Louie@Lowes2
Domains/Tenants
2
1
Name
Tenant
Data
0 thru 4
HomeDepot
Customers
Orders
HD_Domain HomeDepot
Name
Tenant
Lowes1
Lowes
Lowes2
Lowes
Name
Tenant
blank
Default
Edward
Tenancy Layer
Users
© 2011 Progress Software Corporation. All rights reserved.
…
Lowes
Customers
Orders
Items
…
Default
de-allocated or migrated
data
Shared
_file
state
27
Items
_field
…
_tenant
…
Domain Creation
 Domain  Tenant: Domains ALWAYS associated w/tenants
•
(default, regular named tenant, super tenant)
define variable domain as IDomain no-undo.
define variable cAccessCode as character init “secret-code" no-undo.
domain = new Domain("HD_Domain")
assign
domain:DomainType
domain:AccessCode
domain:Tenant
domain:IsEnabled
domain:Description
= service:GetDomainType("_oeusertable")
= cAccessCode
= service:GetTenant("HomeDepot")
= yes
= "HomeDepot security domain".
service:CreateDomain(domain).
 Renaming a tenant “cascades” the rename to the domains
 Renaming a domain “cascades” the rename to the users
 NOTE: Multiple domains can map to the same tenant
28
© 2011 Progress Software Corporation. All rights reserved.
Adding Users
3
Suzi@HD_Domain
Allen@HD_Domain
Cat@HD_Domain
Rich@HD_Domain
Rich@Lowes1
John@Lowes1
Claudio@Lowes2
Louie@Lowes2
Domains/Tenants
2
1
Name
Tenant
Data
0 thru 4
HomeDepot
Customers
Orders
HD_Domain HomeDepot
Name
Tenant
Lowes1
Lowes
Lowes2
Lowes
Name
Tenant
blank
Default
Edward
Tenancy Layer
Users
© 2011 Progress Software Corporation. All rights reserved.
…
Lowes
Customers
Orders
Items
…
Default
de-allocated or migrated
data
Shared
_file
state
29
Items
_field
…
_tenant
…
User Creation
 Users  Domain  Tenant
(Adding users is optional)
• Setting user establishes tenant identity determining data access
define variable domain as IDomain no-undo.
define variable auser as IUser no-undo.
auser = new User(“fblake").
Nothing
new here
assign
auser:GivenName = “Frank“
auser:SurName = “Blake"
auser:Password = “HD_CEO".
domain = service:GetDomain("HD_Domain").
domain:Users:Add(auser).
service:UpdateDomain(domain).
User Id:
fblake
Password: HD_CEO
30
© 2011 Progress Software Corporation. All rights reserved.
Userid/Password is incorrect.
User Creation
 Users  Domain  Tenant
(Adding users is optional)
• Setting user establishes tenant identity determining data access
define variable domain as IDomain no-undo.
define variable auser as IUser no-undo.
auser = new User(“fblake").
Nothing
new here
assign
auser:GivenName = “Frank“
auser:SurName = “Blake"
auser:Password = “HD_CEO".
domain = service:GetDomain("HD_Domain").
domain:Users:Add(auser).
service:UpdateDomain(domain).
User Id:
fblake@HD_Domain
Password: HD_CEO
 NOTE: “Can” permissions extended to tenant qualified userid
31
© 2011 Progress Software Corporation. All rights reserved.
Group Creation
Type II Storage Areas
Data
HomeDepot_NH1
Tenancy Layer
Customers
32
…
HomeDepot_NH2
Customers
…
HD #1 Data Area
HD #2 Data Area
A100 A101
A200 A201
HD #1 index Area
HD #2 index Area
CreditCheckGroup
Credit
© 2011 Progress Software Corporation. All rights reserved.
Credit Data
A300 A301
Credit Indexes
Group Creation
Type II Storage Areas
Data
HomeDepot_NH1
Customers
…
HD #1 Data Area
A100 A101
HD #1 index Area
Tenancy Layer
CreditCheckGroup
33
HomeDepot_NH2
Customers
…
HD #2 Data Area
A200 A201
HD #2 index Area
CreditCheckGroup
CreditCheckGroup
Credit
© 2011 Progress Software Corporation. All rights reserved.
Credit Data
A300 A301
Credit Indexes
Group Creation
Type II Storage Areas
Data
HomeDepot_NH1
Customers
…
HD #1 Data Area
A100 A101
HD #1 index Area
Tenancy Layer
CreditCheckGroup
HD_ItemsGroup
HomeDepot_NH2
Customers
…
HD #2 Data Area
A200 A201
HD #2 index Area
CreditCheckGroup
HD_ItemsGroup
CreditCheckGroup
Credit Data
Credit
HD_ItemsGroup
HD Item Data
A300 A301
Credit Indexes
HD Item Indexes
Items
•
34
© 2011 Progress Software Corporation. All rights reserved.
Multiple Home Depot tenants sharing
same item list, same credit check data.
Group Creation
 Create the group
define variable partitiongroup as IPartitionGroup no-undo.
partitiongroup = new PartitionGroup("HD_ItemsGroup“).
assign
partitiongroup:Table
= service:GetTable("Item")
partitiongroup:DefaultDataArea = service:GetArea(“A300")
Set Area
partitiongroup:DefaultIndexArea = service:GetArea(“A301")
Defaults
partitiongroup:DefaultLobArea = service:GetArea(“A300")
partitiongroup:DefaultAllocation = "Immediate“ /* or delayed */
partitiongroup:Description
= "Home Depot Items".
service:CreatePartitionGroup(partitiongroup).
35
© 2011 Progress Software Corporation. All rights reserved.
Adding Tenants To Groups
 Table allocation state: None
• Simply add tenant as member of the group for that table
define variable partitiongroup as IPartitionGroup no-undo.
assign
partitiongroup = service:GetPartitionGroup("HD_ItemGroup").
partitiongroup:Tenants:Add(service:GetTenant("HomeDepot_NH1")).
partitiongroup:Tenants:Add(service:GetTenant("HomeDepot_NH2")).
service:UpdatePartitionGroup(partitiongroup).
 NOTE: Tenant allocation state must be “None” to join a group
36
© 2011 Progress Software Corporation. All rights reserved.
Adding Tenants To Groups
 Tenant’s table instance allocated?
• It’s a bit more difficult.
• Move or remove data from tenant’s table instance
– Dump/load, buffer copy/delete via super tenant, etc
• De-allocate tenant’s table instance (bye-bye data)
proutil <db> -C deallocate <table>
[ tenant <name> | group <name> ]
• Add tenant as member of the group for that table
– See previous slide
37
© 2011 Progress Software Corporation. All rights reserved.
Adding A New Multi-tenant Object
Once tenants exist…
 Use favorite tool
• DB admin tool, navigator, load .df
 Default allocation state (per tenant/group):
•
Immediate
– Creates new storage partition for each object of the table instance
– Allocation performed for each tenant/group
• Delayed or None
– Will not allocate any storage
– Allows non-default area assignment
38
© 2011 Progress Software Corporation. All rights reserved.
Adding indexes to existing tables
 Activate at creation
• Index wide, not tenant specific
• Can be slow to activate immediately (off-line)
 Index activate online
proutil <db> -C idxactivate <index-name>
[ tenant <name> | group <name> ]
• Concurrent for different tables of same tenant
• Concurrent for same or different table between tenants
 R-code dependencies
• Existing static queries maintain but don’t use newly activated index
• Tenant “mixed” activation with updated r-code
– Index name is inactive and cannot be referenced. (995)
39
© 2011 Progress Software Corporation. All rights reserved.
40
© 2011 Progress Software Corporation. All rights reserved.
Various Data Definition Files
Flexibility with compatibility
 Data dump files (.d)
• Output directory change only: <tenant>/<table>.d
 Binary dump files (.bd)
• File name format change only • <table>_<#>T_<tenant>.bd[n]
<table>.bd[n] becomes:
OR <table>_<#>G_<group>.bd[n]
• customer_5T_HomeDepot.bd1 OR state_2G_itemGroup.bd1
 Bulkload file definition (.fd)
• No change
 Data definition files (.df)
ADD TABLE “Customer”
MULTITENANT yes
AREA "Customer/Order Area”
41
© 2011 Progress Software Corporation. All rights reserved.
Maintenance Utilities
 Area and object utilities now support tenant/group keyword
proutil <db> -C dbanalys [ area <area> ]
[ shared | tenant <name> | group <name> ]
• Set/display create/toss limits
• Index rebuild
• Index move/table move
• Index fix
• Binary dump/load
• Index compact
• Index check
• Index activate
 Database wide utilities remain unchanged
42
• Roll forward
• Auditing
• Truncate bi
• Increase startup parameters to
• Backup/restore
• Move schema
© 2011 Progress Software Corporation. All rights reserved.
Analysis Tools Report By Tenant/Group
dbutil <db> -C dbanalys area <area> [ tenant <name> | group <name> ]
RECORD BLOCK SUMMARY FOR SHARED OBJECTS:
-Record Size (B)Table
Records
Size
Min
Max Mean
PUB.state
51
1.8K
31
42
36
---------------------------------Subtotals:
51
1.8K
31
42
36
RECORD BLOCK SUMMARY FOR GROUP HD_ItemGroup: -14
-Record Size (B)Table
Records
Size
Min
Max Mean
PUB.item
51
2.2K
37
49
44
----------------------------------Subtotals:
51
2.2K
37
49
44
RECORD BLOCK SUMMARY FOR TENANT HomeDepot_NH1: 1
-Record Size (B)Table
Records
Size
Min
Max Mean
PUB.customer
4 280.0B
64
73
70
PUB.order
9 312.0B
34
35
34
PUB.order-line
31
1.0K
33
34
33
--------------------------------Subtotals:
44 593.0K
33
73
46
43
© 2011 Progress Software Corporation. All rights reserved.
Promon: Tenant Identity, Sorting And Filtering
 M. Modify Defaults
• S. Sort user lists
: by user Id OR tenant Id
• T. Tenant filter for user lists : one OR range
User Control: by user number
Usr:Ten
0:0
5:2
6:1
7:0
Name
richb
u2@t2
u1@t1
richb
User Control: by tenant Id
Usr:Ten
0:0
7:0
6:1
5:2
Type
BROK
SELF/ABL
SELF/ABL
SELF/ABL
Name
richb
richb
u1@t1
u2@t2
Currently Connected Tenants
Tenant Id
0
1
2
44
© 2011 Progress Software Corporation. All rights reserved.
Name
User Count
Default
t1
t2
3
1
1
Type
BROK
SELF/ABL
SELF/ABL
SELF/ABL
45
© 2011 Progress Software Corporation. All rights reserved.
Summary
 Administration Architecture
 Create tenant related stuff
• Self provisioning API
 Various utility usage
 Multi-tenant Configuration Tool demo
46
© 2011 Progress Software Corporation. All rights reserved.
?
Questions
47
© 2011 Progress Software Corporation. All rights reserved.
48
© 2011 Progress Software Corporation. All rights reserved.
Sept. 19 – 22, 2011
Boston Westin Waterfront Hotel
and Boston Convention & Exhibition Center