Key-Value
Databases
Key-value databases
Definition
◦ Associative array stored on a disk; it is a single key
lookup, a dictionary
Pros/Cons
◦ They can be read very quickly
◦ Not so good for reverse lookups or additional
analytics.
Examples
◦ Redis
◦ Memurai (Windows version of Redis)
◦ DynamoDB
How do they work?
Basic data structure
◦ <key, value> pairs
Characteristics
◦ Keys are unique
Basic operations
◦ Insert pairs
◦ Delete pairs
◦ Update values
◦ Find a value associated to a key
When to use them
◦ When working with huge amounts of data that
does not need relational constraints and integrity
DynamoDB (AWS)
Features
◦ SSD storage
◦ Automatic replication
◦ Encryption at rest
◦ Point-in-time recovery
Pros/Cons
◦ Flexible scalability
◦ High availability
◦ Transaction support
◦ AWS
DynamoDB
BASIC CONCEPTS
Tables, items, attributes
Tables
◦ Collection of data
◦ Schemaless
◦ Example: People, Cars
Items
◦ Group of attributes uniquely identifiable among all others
◦ Needs unique primary key
◦ No limit
◦ Examplie: John Smith, The Batmobile
Attributes
◦ Fundamental data element
◦ Scalar or nested (up to 32 levels)
◦ Example: PersonID, FirstName, Year, Model
Primary Keys
Partition key
◦ Simple primary key
◦ Used internally with hash function to determine the partition to store item
◦ Hash attribute
◦ Unique among all items
Partition + Sort key
◦ Composite primary key
◦ Composed of two attributes
◦ Partition key determines partition
◦ Not necessarily unique
◦ Sort key determines order inside partition
◦ Must be unique within partition
◦ Range attribute
Must specify type (string, number or binary) at creation
Secondary Indexes
Alternative means of retrieval
Allows querying data in a table using alternate key
Two types supported:
◦ Global:
◦ Partition + sort key
◦ Max. 20 per table
◦ Local:
◦ Same partition, but different sort
◦ Max. 5 per table
Optional projection attributes
Streams
Optional feature
Produces a “stream” containing all data modification events in a table
◦ Item added: full item description
◦ Item updated: before and after item description
◦ Item deleted: full deleted item description
Composed of stream records
◦ Name of table, event timestamp, metadata, etc.
◦ Deleted 24 hours after creation
Combined with AWS Lambda, allow similar behavior to triggers in SQL
Data Types
Scalar: represents one single value
◦ Number
◦ String
◦ Binary
◦ Boolean
◦ Null
Document: complex structure with nested
attributes
◦ List
◦ Encolsed by [ ]
◦ Map
◦ Collection of pairs enclosed by { }
◦ Set: multiple scalar values
◦ String set
◦ Number set
◦ Binary set
◦ Enclosed by [ ]
◦ Must all be of same type
General functions
Control Plane
◦ CREATETABLE
◦ Creates a new table
◦ Optionally create secondary index and enable streams
◦ DESCRIBETABLE
◦ Returns info about table
◦ LISTTABLES
◦ Returns names of all tables
◦ UPDATETABLE
◦ Modifies settings of table
◦ DELETETABLE
Data Plane
◦ Classic API
◦ PUTITEM: writes single item to table
◦ BATCHWRITEITEM: writes up to 25 items to a table
◦ GETITEM: retrieves single item from table
◦ BATCHGETITEM: retrieves up to 100 items from one or more
tables
◦ QUERY: retrieves all items with a specific partition key
◦ Optional condition on sort key
◦ SCAN: retrieves all items in the specified table or index
◦ Optional filter condition
◦ UPDATEITEM: modifies one or more attributes in an item
◦ DELETEITEM: Deletes single item
◦ PartiQL API
◦ EXECUTESTATEMENT, BATCHEXECUTESTATEMENT
Query vs. Scan
Read Consistency
DynamoDB supports two levels of read consistency
Eventually Consistent Reads
◦ Might not reflect the results of the most recently completed write operations
◦ High availability
◦ Fastest response time
Strongly Consistent Reads
◦
◦
◦
◦
Returns the most up to date data
Might fail in case of delays or outage
Higher latency
Use more throughput capacity (approx. 2xECR)
Deploying DynamoDB locally
Make sure you have JRE already installed in your computer
Download DynamoDBLocal:
◦ https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.DownloadingAndRu
nning.html#DynamoDBLocal.DownloadingAndRunning.title
Extract folder
From a command line, navigate to extracted folder and run:
◦ java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar –sharedDb
A local version of DynamoDB should now be running (don’t close the command window)
Download and install NoSQL WorkBench:
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/workbench.settingup.html
DynamoDB Shell
Try the following:
◦ Create a new table named Person
◦ Primary key: PersonID (number)
◦ Insert a new item to Person
◦ PersonID: 101
◦ FirstName: ‘John’
◦ LastName: ‘Smith’
◦ Age: 29
◦ Email: ‘john.smith@gmail.com’
◦ Retrieve person with PersonID 101
◦ Update age of person with PersonID 101 to 30
PartiQL
SQL dialect designed to work with non-relational (semi-structured and nested) data
Open source
Data storage independent
Supported PartiQL CRUD operations in DynamoDB:
◦ SELECT
◦ UPDATE
◦ INSERT
◦ DELETE
PartiQL + DynamoDB
SELECT
◦ Used to retrieve data from a table
◦ Equivalent to a GetItem, Query or Scan operation
(depending on WHERE condition)
◦ SELECT expression [, ...]
FROM table[.index]
[ WHERE condition ] [ [ORDER BY key [DESC|ASC]
, ...]
◦ WHERE condition should use partition key (with
equality or IN operator) or else operation will
result in full table scan
◦ Examples:
◦ SELECT OrderID, Total
FROM "Orders“
WHERE OrderID = 1 OR OrderID = 2
◦ SELECT OrderID, Total
FROM "Orders“
WHERE OrderID IN [1, 2, 3] ORDER BY OrderID DESC
◦ SELECT OrderID, Total
FROM "Orders"
WHERE Total BETWEEN 500 AND 600
◦ SELECT *
FROM "Orders"
WHERE OrderID > 1
PartiQL + DynamoDB
UPDATE
◦ Modifies the value of one or more attributes
within an item in a table
◦ If item does not have attribute, the attribute is added
◦ UPDATE table
[SET | REMOVE] path [= data] […]
WHERE condition
[RETURNING returnvalues]
<returnvalues> ::= [ALL OLD | MODIFIED OLD |
ALL NEW | MODIFIED NEW] *
◦ WHERE condition should evaluate to ONE item, if
no item is found, an error will be produced
◦ Examples:
◦ UPDATE "Music"
SET AwardsWon=1
SET AwardDetail={'Grammys':[2020, 2018]} WHERE
Artist='Acme Band' AND SongTitle='PartiQL Rocks’
RETURNING ALL OLD *
◦ UPDATE "Music"
SET AwardDetail.Grammys
=list_append(AwardDetail.Grammys,[2016])
WHERE Artist='Acme Band' AND SongTitle='PartiQL Rocks’
◦ UPDATE "Music"
REMOVE AwardDetail.Grammys[2]
WHERE Artist='Acme Band' AND SongTitle='PartiQL Rocks’
◦ UPDATE "Music"
SET AwardDetail.BillBoard=[2020]
WHERE Artist='Acme Band' AND SongTitle='PartiQL Rocks'
PartiQL + DynamoDB
DELETE
◦ Removes an existing item from a table
◦ DELETE FROM table
WHERE condition [RETURNING returnvalues]
<returnvalues> ::= ALL OLD *
◦ Same as UPDATE, can only delete one single item
◦ Examples:
◦ DELETE FROM "Music" WHERE "Artist" = 'Acme Band' AND "SongTitle" = 'PartiQL Rocks’
◦ DELETE FROM "Music" WHERE "Artist" = 'Acme Band' AND "SongTitle" = 'PartiQL Rocks' RETURNING ALL OLD *
PartiQL + DynamoDB
INSERT
◦ Adds an item to a table
◦ INSERT INTO table VALUE item
◦ Example:
◦ INSERT INTO "Music" value {'Artist' : 'Acme Band','SongTitle' : 'PartiQL Rocks'}