Module 4: DynamoDB Query
Learning Objectives
- Implement functionality that queries a provided DynamoDB table by partition key
- Implement functionality that queries a provided DynamoDB table by partition key and sort key
- Discuss how to query a provided DynamoDB table using a secondary index
Introduction to DynamoDB Query Operations
The Query operation is one of the most powerful features of Amazon DynamoDB. It allows you to retrieve multiple items from a table or secondary index using the primary key attributes. Query provides fast, efficient access to data by leveraging the table's structure, making it essential for applications that need to access data in specific patterns.
Unlike the Scan operation, which examines every item in a table, Query only looks at items with the specified partition key value, making it much more efficient for large tables. Query operations can be further refined using sort key conditions, allowing for precise data retrieval based on your application's needs.
It's important to note that only tables with a composite primary key (partition key + sort key) can be queried effectively. This is one of the benefits of using a composite key structure in your database design.
DynamoDBMapper Query Methods
In previous lessons, you've learned about the DynamoDBMapper methods load(), save(), and delete(). Up to this point, you've been able to use the load() method to get one item from a table based on the key(s) for the table. We've mentioned throughout these lessons, however, that there's a way to query a table to retrieve multiple items. In this lesson, you're going to learn how to retrieve multiple items from a table using DynamoDBMapper methods query() and queryPage().
Querying the Songs Table
In previous lessons, we've used the Songs table as an example with a composite primary key. Let's review our Songs table, but this time with a few more songs!
| artist | song_title | genre | year | favorited |
|---|---|---|---|---|
| Black Eyed Peas | I Gotta Feeling | pop | 2009 | false |
| Black Eyed Peas | Pump It | pop | 2005 | true |
| Black Eyed Peas | Where is the Love? | pop | 2006 | true |
| Black Eyed Peas | Let's Get it Started | pop | 2003 | false |
| Linkin Park | Numb | rock | 2003 | true |
| Eminem | Not Afraid | rap | 2010 | false |
| Daddy Yankee | Gasolina | latin pop | 2004 | true |
| Linkin Park | In the End | rock | 2000 | true |
Previously, we were only able to get one item from the Songs table using the DynamoDBMapper load() method, but what if we want to be able to get all of the Black Eyed Peas songs in our table? As we've briefly mentioned in previous lessons, DynamoDBMapper supports a query method. It's important to note that the DynamoDBMapper query() method only works for tables that use a composite primary key (partition + sort key). You cannot query on a table that uses only a partition key, which is one of the benefits of using a composite primary key.
A query is a question---it's asking for specific information. Querying a database table means requesting data from the table---it's like asking the table a question and receiving specific information in reply. The DynamoDBMapper query() method accepts an annotated class and a DynamoDBQueryExpression, which includes a condition and a value.
The code for querying the Songs table for all items of a specific partition key (artist) looks like the following:
public List<Song> getSongsByArtist(String artist) {
Song song = new Song();
song.setArtist(artist);
DynamoDBQueryExpression<Song> queryExpression = new DynamoDBQueryExpression<Song>()
.withHashKeyValues(song);
DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
PaginatedQueryList<Song> songList = mapper.query(Song.class, queryExpression);
return songList;
}Let's break this code snippet down piece by piece. We first declare a method getSongsByArtist() that takes in artist and returns a List of Songs. Then the next two lines create a new Song instance and then set the partition key, artist, to the artist passed into the method.
public List<Song> getSongsByArtist(String artist) {
Song song = new Song();
song.setArtist("Black Eyed Peas");The next section of code is what probably looks unfamiliar to you; the DynamoDBQueryExpression.
DynamoDBQueryExpression<Song> queryExpression = new DynamoDBQueryExpression<Song>()
.withHashKeyValues(song);We first create a new instance of DynamoDBExpression, queryExpression, which is of type Song, since that's the annotated class that maps out our table items to Java. There are lots of methods we can use to filter our expression, but we're going for the bare minimum here, so we're only specifying our hash key value. If you would like a full list of all the accepted methods, you can look here. As a reminder, in DynamoDB a hash key is the same thing as a partition key, and hash key is used in Java when referencing the partition key. The method withHashKeyValues() accepts an object, which in this case is song. The object that's passed in must have at least the partition key value set, just like we learned with the DynamoDBMapper delete() method.
The next few lines of code set-up the instance of DynamoDBMapper, which you should be familiar with, and then we call the query() method, passing in the Song class and the newly created queryExpression.
DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
PaginatedQueryList<Song> songList = mapper.query(Song.class, queryExpression);
return songList;Note that the query() method returns a PaginatedQueryList, which in this example is of type Song. PaginatedQueryList implements the List interface, so we can return this list directly in our method. We'll talk more about what paginated means in the next section of this reading.
Since the only condition we specified in our DynamoDBExpression was the partition key, this query will get all the values that have the specified partition key. Remember that even though we gave the withHashKeyValues() method an object, the method is only looking for the value we set as the partition key, which is why 'artist' is the only value we set for song.
If we call getSongsByArtist("Black Eyed Peas") the method will return a PaginatedQueryList of the following bolded values:
| artist | song_title | genre | year | favorited |
|---|---|---|---|---|
| Black Eyed Peas | I Gotta Feeling | pop | 2009 | false |
| Black Eyed Peas | Pump It | pop | 2005 | true |
| Black Eyed Peas | Where is the Love? | pop | 2006 | true |
| Black Eyed Peas | Let's Get it Started | pop | 2003 | false |
| Linkin Park | Numb | rock | 2003 | true |
| Eminem | Not Afraid | rap | 2010 | false |
| Daddy Yankee | Gasolina | latin pop | 2004 | true |
| Linkin Park | In the End | rock | 2000 | true |
Pagination
Our Songs table may be very small, but most tables that you're working with will include hundreds, thousands, or even millions of items! When dealing with data samples that large, your query request could return a ton of data, which is difficult to deal with all at once. To help deal with large data output, most databases use some form of pagination.
Pagination splits database output into manageable chunks, often referred to as pages. In DynamoDB, query results are divided into pages of data that are at most 1 MB in size. The query operations are paginated by default, which is why the query() method returns a PaginatedQueryList. The method processes the first pages of result and then additional pages (if there are any) as necessary. With DynamoDB's auto-pagination, you never have to deal with more than 1 MB of data at a time, which helps ensure you never run out of memory or have to deal with slow query requests.
The query() method's auto-pagination is very useful, but sometimes we want a bit more control over the number of items we're returning from a query request. We can gain greater control over pagination by narrowing the scope of our query results using the withLimit() method in our DynamoDBExpression.
The withLimit() method accepts an integer and with each query request made, returns the first 'x' number of items that matches the query. Since the query() method's auto-pagination makes multiple requests to retrieve the entire result set, the withLimit() method doesn't work as expected. For example, if you pass in the limit as 3, but the query() method makes three requests to the table, then it will end up returning 9 items. This is because the limit is 3 for each request made to the table. Since we want to specifically limit the number of items that are returned, we need to use the queryPage() method instead of query().
The queryPage() method works very similarly to the query() method, but it only retrieves one page of data. The method defaults to returning the first page of data, but if there is more than one page, there is a way to get the next page(s), which we'll discuss in the next section of the reading.
The following code snippet retrieves the first 3 items for the given artist:
public List<Song> getSongsByArtist(String artist) {
Song song = new Song();
song.setArtist(artist);
DynamoDBQueryExpression<Song> queryExpression = new DynamoDBQueryExpression<Song>()
.withHashKeyValues(song)
.withLimit(3);
DynamoDBMapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
QueryResultPage<Song> songQueryResults = mapper.queryPage(Song.class, queryExpression);
List<Song> songList = songQueryResults.getResults();
return songList;
}This code snippet is very similar to the code snippet up above, with two exceptions: the first is that we've added the withLimit() method to our DynamoDBQueryExpression with a value of 3. The second is that we're using queryPage() instead of query(), which returns a QueryResultPage, instead of a PaginatedQueryList. A QueryResultPage is a container for a page of query results. If you're interested, you can learn more about it here. We want to return a list of Songs, which we can retrieve by calling QueryResultPage's getResults() method.
Without the use of the withLimit() method, calling getSongsByArtist("Black Eyed Peas") would return all four Black Eyed Peas songs, which we saw up above. With the withLimit() method, however, this query returns only the first three Black Eyed Peas songs, which are bolded in the table below.
| artist | song_title | genre | year | favorited |
|---|---|---|---|---|
| Black Eyed Peas | I Gotta Feeling | pop | 2009 | false |
| Black Eyed Peas | Pump It | pop | 2005 | true |
| Black Eyed Peas | Where is the Love? | pop | 2006 | true |
| Black Eyed Peas | Let's Get it Started | pop | 2003 | false |
| Linkin Park | Numb | rock | 2003 | true |
| Eminem | Not Afraid | rap | 2010 | false |
| Daddy Yankee | Gasolina | latin pop | 2004 | true |
| Linkin Park | In the End | rock | 2000 | true |
Getting the next items
In our previous query, we got the first three out of four Black Eyed Peas songs, but what if we want to make a request for the next item, as well?
You can do so by using DynamoDBQueryExpression's withExclusiveStartKey() method. withExclusiveStartKey() provides a way to pass in the key that refers to an item in a DynamoDB table, and DynamoDB will retrieve the results of the query that come after that item. We call this key the exclusiveStartKey because it excludes the item with the provided key from the next page of results.
For our previous query, we retrieved the first three Black Eyed Peas songs. To retrieve the next set of Black Eyed Peas songs, we can set the exclusive start key in our DynamoDBQueryExpression to refer to the item with the partition key 'Black Eyed Peas' and the sort key 'Where is the Love?'
The following code will set-up a query request using the withExclusiveStartKey() method.
public List<Song> getSongsByArtist(String artist, String exclusiveStartSongTitle) {
Song song = new Song();
song.setArtist(artist);
Map<String, AttributeValue> exclusiveStartKey = null;
if (exclusiveStartSongTitle != null) {
exclusiveStartKey = new HashMap<>();
exclusiveStartKey.put("artist", new AttributeValue().withS(artist));
exclusiveStartKey.put("song_title", new AttributeValue().withS(exclusiveStartSongTitle));
}
DynamoDBQueryExpression<Song> queryExpression = new DynamoDBQueryExpression<Song>()
.withHashKeyValues(song)
.withExclusiveStartKey(exclusiveStartKey)
.withLimit(3);
DynamoDBMapper mapper = new DynamoDBMapper(DynamoDBClientPovider.getDynamoDBClient());
QueryResultPage<Song> songQueryResults = mapper.queryPage(Song.class, queryExpression);
return songQueryResults.getResults();
}Let's walk through this code snippet. The method signature for getSongsByArtist looks similar to our previous query methods, except it's now also taking in an exclusiveStartSongTitle, which we'll use to build our exclusive start key.
The next two lines are the same as the previous methods, where we create a new Song instance and then set the partition key, artist, to the artist passed into the method.
Then the next section is where we build the exclusiveStartKey. The exclusiveStartKey is a Map<String,AttributeValue>. The keys of this Map refer to names of the keys used in the DynamoDB table. For our Songs table, the keys are the Strings "artist" and "song_title", the names of our partition and sort keys. The values of this Map are of type AttributeValue, which is a DynamoDB type that represents data for an attribute and includes both the data type and the data itself. The withS method specifies that the attribute value is of type String. (You can review AttributeValue's Javadoc for more details.) You can see that we're setting the AttributeValue to the artist and exclusiveStartSongTitle passed into the method.
In the next section, we're building our DynamoDBQueryExpression, which looks similar to our previous query method, except we're also calling withExclusiveStartKey() and passing in the exclusive start key we created above.
Finally, we're making a call to queryPage() and returning the results as a list of Songs.
To retrieve the next set of Black Eyed Peas songs after "Where is the Love?", we would call getSongsByArtist("Black Eyed Peas", "Where is the Love?"). Since there is only one Black Eyes Peas song after "Where is the Love?" in our table, this method call would return a list of Songs with one item: Black Eyed Peas' "Let's Get it Started".
Note that in our code snippet above, we check if exclusiveStartSongTitle is null before setting the exclusive start key. What happens when exclusiveStartKey is null when we call queryPage()? This is the default behavior, so the query would return the first set of items in our table that match our QueryExpression.
If a client wanted to retrieve the first two pages of Black Eyed Peas songs, they could call our getSongsByArtist method in our SongDao class with the following code:
//this call will return the songs "I Gotta Feeling", "Pump It",
//and "Where is the Love?"
List<Song> firstPageOfSongs = songDao.getSongsByArtist("Black Eyed Peas", null);
// retrieve title of last song returned on the first page using a helper method (method implementation not shown),
// which would return "Where is the Love?" in this case and returns null if the provided list is empty
String lastSongTitle = SongHelper.getLastSongTitle(firstPageOfSongs);
//this call will return the song "Let's Get it Started"
List<Song> secondPageOfSongs = songDao.getSongsByArtist("Black Eyed Peas", lastSongTitle);The first call that's made returns the first three Black Eyed Peas songs (because we set our limit to 3), which are "I Gotta Feeling", "Pump It", and "Where is the Love?". When we made this call, we passed in a null value, because we want to get the first items that match our query. To retrieve the next set of items, we now pass in the key value of the last song returned by our first request, "Where is the Love?". This tells the query to retrieve the items found after the key value given. This query request retrieves the song "Let's Get it Started", which is the only remaining song that meets our query criteria.
There are a few things to point out: the first is that our limit is 3, but the second time we call getSongsByArtist(), there is only 1 more item that matches our query. When we use the withLimit() method we're setting an upper limit, which means that the query will return no more than that number. Since there is only 1 more item that meets our query condition, (artist = "Black Eyed Peas"), only 1 song (songTitle = "Let's Get it Started") is returned from that query request.
Summary
In this reading, we learned about using the DynamoDBMapper query() and queryPage() methods to retrieve all the items that have the same partition key. We also learned about pagination and how it splits the data retrieved from a query request into manageable chunks to help deal with large returns. The withLimit() method gives us even more control over the number of items we retrieve from a query result and the exclusiveStartKey allows us to retrieve the next page of results. In the next lesson, we'll learn how to narrow the scope of our query even further by adding additional query conditions to the sort key!
Narrowing Your Query
In the previous reading, you learned how to use the DynamoDBMapper's query() method to return all the items in a table that have a given partition key. We also discussed how to use the withLimit() method in conjunction with DynamoDBMapper's queryPage() method to return at most a certain number of items. By using the withExclusiveStartKey() method, we could then get the next 'x' amount of items from the table. In this lesson, we're going to learn how to query with comparison operators to narrow the scope of our query even further and retrieve more specific output than just querying on the partition key.
Querying with Comparison Operators
You're working on a database for an online ordering system. One of the tables, OnlineOrders, keeps track of each users purchases and the cost of the purchase.
| user_id | order_id | cost |
|---|---|---|
| 62da86ba | 20121207-54297 | 24.89 |
| 62da86ba | 20140425-03576 | 45.87 |
| 62da86ba | 20140425-54879 | 32.01 |
| 62da86ba | 20170923-54129 | 103.87 |
| 62da86ba | 20190305-78159 | 209.13 |
| 62da86ba | 20200105-42158 | 89.65 |
| d1a93bea | 20091204-78216 | 34.23 |
| d1a93bea | 20130817-45789 | 17.03 |
The OnlineOrders table has three attributes: a user_id, which is the partition key and unique for each user, an order_id, which is the sort key and made up of the date the product was ordered combined with a unique product number, and the cost of each order.
The fully annotated Java class, Order:
@DynamoDBTable(tableName = "OnlineOrders")
public class Order {
private String userId;
private String orderId;
private Double cost;
@DynamoDBHashKey(attributeName = "user_id")
public String getUserId() {
return userId;
}
public void setUserId(String userId) {
this.userId = userId;
}
@DynamoDBRangeKey(attributeName = "order_id")
public String getOrderId() {
return orderId;
}
public void setOrderId(String orderId) {
this.orderId = orderId;
}
@DynamoDBAttribute (attributeName = "cost")
public Double getCost() {
return cost;
}
public void setCost(Double cost) {
this.cost = cost;
}
}Using this class, we could query on userId to get all the orders from a specific user like we learned in the last lesson. What if, however, we want to get all the orders from a user between two dates? We can do this using the withKeyConditionExpression() method, which accepts a condition that specifies the key values for the items to be retrieved by the query request. The condition must specify what the partition key is equal to. The condition can optionally perform a comparison test on a sort key value to retrieve the items that meet the given comparison condition for the sort key.
In the following code snippet, we're using the withKeyConditionExpression() to retrieve all of the orders made by userId = '62da86ba' between the first day of 2014 ('20140101') and the last day of 2017 ('20171231').
Map<String, AttributeValue> valueMap = new HashMap<>();
valueMap.put(":userId", new AttributeValue().withS("62da86ba"));
valueMap.put(":startDate", new AttributeValue().withS("20140101"));
valueMap.put(":endDate", new AttributeValue().withS("20171231"));
DynamoDBQueryExpression<Order> queryExpression = new DynamoDBQueryExpression<Order>()
.withKeyConditionExpression("user_id = :userId and order_id between :startDate and :endDate")
.withExpressionAttributeValues(valueMap);
DynamoDBMapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
PaginatedQueryList<Order> orderList = mapper.query(Order.class, queryExpression);This code snippet may look a bit intimidating, so let's break it down piece by piece. The first thing we're doing is creating a new HashMap, valueMap, to map the attribute values, which is what we did in the previous lesson with DynamoDBDeleteExpression and in the previous reading with the exclusiveStartKey.
Map<String, AttributeValue> valueMap = new HashMap<>();
valueMap.put(":userId", new AttributeValue().withS("62da86ba"));
valueMap.put(":startDate", new AttributeValue().withS("20140101"));
valueMap.put(":endDate", new AttributeValue().withS("20171231"));Notice that the key values we add to valueMap start with a ":" --- this is required. The values that we're defining are Strings, so we use the method withS(). This method changes depending on what your value type is. If we were using a Boolean value instead of a String, for example, we would use the method withBOOL(). The values we add to valueMap are the values we are going to use in our withKeyConditionExpression() method. The user id is our partition key value, ('62da86ba'), the start date for our condition is the first day of 2014, ('20140101') and the end date for our condition is the last day of 2017 ('20171231').
The next piece of code sets up our DynamoDBQueryExpression, which we used in the previous reading when querying.
DynamoDBQueryExpression<Order> queryExpression = new DynamoDBQueryExpression<Order>()
.withKeyConditionExpression("user_id = :userId and order_id between :startDate and :endDate")
.withExpressionAttributeValues(valueMap);The first line should look familiar, but the methods we call on queryExpression are new. The first one is the withKeyConditionExpression(), which specifies the condition we're querying on. To make sure you fully understand this condition, we're going to break it down even further.
("user_id = :userId ...
This first part of the condition is what our partition key is equal to. This part of the condition is required. In the previous reading, we used the method withHashKeyValues() to specify the value of the partition key, but in this example we're using the method withKeyConditionExpression(). You must use one of these methods in your DynamoDBQueryExpression, but you can't use both. Since we can't specify the partition key value using the withHashKeyValues() method, we have to make sure we're specifying the partition key value in the withKeyConditionExpression() method, which is why the above code is required.
Notice that the partition key attribute name is 'user_id', which is the name of the attribute in the table. The withKeyConditionExpression() is looking at the table, not the Order class you set-up, so the attribute names for the keys must match the ones given in the table. We then use the equals sign (=) to set the user_id equal to the key variables we declared in valueMap, (':userId'). Notice that once again we're using the ':' when we call this value, just as we did when we added it to the valueMap.
As we said, specifying the partition key value for our query is required, but everything else in the withKeyConditionExpression() method is optional. So technically we could use the method as an alternative way of querying on the partition key value like we did in the previous reading. In this example, however, we want to additionally query on the sort key, so we need to add in an extra part to our condition.
... and order_id between :startDate and :endDate")
The and keyword tells the withKeyConditionExpression() method that there's another part to our query. The last part of the condition sets up the values for our sort key, which is order_id. Notice that once again we're using the attribute name from the table, not the class. We then use the comparison operator between, which tells the method which operation to perform on the following values (there are several possible comparison operators to use, which we'll discuss more in the next section). The between operator requires two values so that it can retrieve the items that are found between those values. In this case, those two values are :startDate and :endDate. Once again, we must use the ":"" when referencing these values from the HashMap. We use the and keyword to specify that there are two values, the start date and the end date.
Looking back at the DynamoDBQueryExpression, we can see how the condition fits all together.
DynamoDBQueryExpression<Order> queryExpression = new DynamoDBQueryExpression<Order>()
.withKeyConditionExpression("user_id = :userId and order_id between :startDate and :endDate")
.withExpressionAttributeValues(valueMap);The last line of the above code snippet calls the withExpressionAttributeValues() method. This method passes in valueMap, which is how the withKeyConditionExpression() method knows where to get the values :userId, :startDate, and :endDate from.
The final piece of our query request sets up our mapper and uses DynamoDBMapper query() to query the OnlineOrders table just as we saw in the previous lesson.
DynamoDBMapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
PaginatedQueryList<Order> orderList = mapper.query(Order.class, queryExpression);Now that you have a better understanding of the separate pieces of the query request, we'll show you the entire code snippet again:
Map<String, AttributeValue> valueMap = new HashMap<>();
valueMap.put(":userId", new AttributeValue().withS("62da86ba"));
valueMap.put(":startDate", new AttributeValue().withS("20140101"));
valueMap.put(":endDate", new AttributeValue().withS("20171231"));
DynamoDBQueryExpression<Order> queryExpression = new DynamoDBQueryExpression<Order>()
.withKeyConditionExpression("user_id = :userId and order_id between :startDate and :endDate")
.withExpressionAttributeValues(valueMap);
DynamoDBMapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
PaginatedQueryList<Order> orderList = mapper.query(Order.class, queryExpression);This code snippet will return a list of the following italicized, bolded orders, all of which have a user_id of '62da86ba' and an order_id between the dates '20140101' and '20171231':
| user_id | order_id | cost |
|---|---|---|
| 62da86ba | 20121207-54297 | 24.89 |
| 62da86ba | 20140425-03576 | 45.87 |
| 62da86ba | 20140425-54879 | 32.01 |
| 62da86ba | 20170923-54129 | 103.87 |
| 62da86ba | 20190305-78159 | 209.13 |
| 62da86ba | 20200105-42158 | 89.65 |
| d1a93bea | 20091204-78216 | 34.23 |
| d1a93bea | 20130817-45789 | 17.03 |
Additional Comparison Operators
Our previous example used the between comparison operator, but as we mentioned, there are other operators that can be used to query the sort key. If you're interested, you can learn more about conditions and comparison operators here.
The following list shows all the available comparison operators that can be used with the sort key and an example of how they're set-up:
- = , < , <=,> , >=
order_id >= :startDate - begins_with
begins_with(order_id, :startDate) - between
order_id between :startDate and :endDate
Remember, the partition key can only use the equals (=) operator.
Adding a Limit
In the last reading, we learned about using the withLimit() method to limit our query even farther, and we can do the same thing with condition expressions!
Just as before, we'll use the queryPage() method instead of the query() method, since query() makes multiple requests to the table and therefore returns more items than our specified limit. We'll use the same OrderLimits and Order class as before, but this time we'll make a query request when user_id = '62da86ba' and order_id > '20140101' with a limit of 4 items.
Map<String, AttributeValue> valueMap = new HashMap<>();
valueMap.put(":userId", new AttributeValue().withS("62da86ba"));
valueMap.put(":startDate", new AttributeValue().withS("20140101"));
DynamoDBQueryExpression<Order> queryExpression = new DynamoDBQueryExpression<Order>()
.withKeyConditionExpression("user_id = :userId and order_id > :startDate")
.withExpressionAttributeValues(valueMap)
.withLimit(4);
DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
QueryResultPage<Order> orderList = mapper.queryPage(Order.class, queryExpression);This code snippet has the same format as the previous example, we've just added the withLimit() method to queryExpression, and we're using queryPage() instead of query().
Without setting a limit, this query would retrieve 5 items, but since we set the limit to 4, the query only retrieves 4 items! You could also use exclusiveStartKey to get the next values through the same set-up as shown in the previous reading.
The above query retrieves the following italicized, bolded items:
| user_id | order_id | cost |
|---|---|---|
| 62da86ba | 20121207-54297 | 24.89 |
| 62da86ba | 20140425-03576 | 45.87 |
| 62da86ba | 20140425-54879 | 32.01 |
| 62da86ba | 20170923-54129 | 103.87 |
| 62da86ba | 20190305-78159 | 209.13 |
| 62da86ba | 20200105-42158 | 89.65 |
| d1a93bea | 20091204-78216 | 34.23 |
| d1a93bea | 20130817-45789 | 17.03 |
Database Consistency Models
You've now learned a few methods for manipulating and retrieving data in DynamoDB and can see the power and usefulness of databases. It's important to understand, however, that databases aren't perfect. Sometimes, there may be inconsistency with the data you're retrieving from a database due to an update lag across the servers.
A database consistency model refers to the model the database uses when dealing with consistency in the data. These models define the rules databases will follow to provide consistent data for the user.
Eventual Consistency
Eventual consistency is DynamoDB's default consistency model and the only model you need to know right now. As you learned in your introductory lesson on databases, DynamoDB is a distributed database, which means that the database is spread out across multiple servers. When you write items to a table (either saving or deleting), that item needs to be updated on multiple servers. Sometimes, however, you may read data (either loading or querying) before all the servers have a chance to update the data you recently wrote. This can lead to returning stale (or incorrect) data.
Why does this happen in the eventual consistency model? When you read data using the eventual consistency model, it chooses a random server to read from, which may be one that hasn't been updated yet. You don't need to wait long for the server to update, however. Consistency across all copies of data (on different servers) is usually reached within a second. If you repeat the read request after this time, it will return the updated data.
As we said previously, eventual consistency is the default consistency model in DynamoDB. There is another consistency model in DynamoDB that you don't need to know about now, but if you're interested, you can read more about it here. You should, however, understand how the eventual consistency model works in case you ever retrieve stale data from DynamoDB or in case you have to implement a feature that could involve reading an item in DynamoDB soon after updating it. Eventual consistency is something you may have to consider when building features. For example, have you ever updated your account information on a website and seen messaging like "your changes may take a few minutes to take effect"? The website is addressing its eventual consistency model by letting its users know why they may still see stale data on their accounts.
Summary
In this reading, we learned how to further narrow the scope of your queries using comparison operators. Comparison operators allow you to retrieve items that equal a given partition key value and meet the given condition of the sort key. We used the example of finding orders between and then after a certain date, but there are lots of possibilities of how you could use comparison operators in your queries!
We also learned about the eventual consistency database model, which is the default consistency model for DynamoDB. It's important to understand the limitations of the eventual consistency model so that you can make informed decisions about your building features and avoid retrieving stale data from your database.