Module 4: DynamoDB Query

Introduction to DynamoDB Query Operations

The Query operation is one of the most powerful features of Amazon DynamoDB. It allows you to retrieve multiple items from a table or secondary index using the primary key attributes. Query provides fast, efficient access to data by leveraging the table's structure, making it essential for applications that need to access data in specific patterns.

Unlike the Scan operation, which examines every item in a table, Query only looks at items with the specified partition key value, making it much more efficient for large tables. Query operations can be further refined using sort key conditions, allowing for precise data retrieval based on your application's needs.

It's important to note that only tables with a composite primary key (partition key + sort key) can be queried effectively. This is one of the benefits of using a composite key structure in your database design.

Key Topics

DynamoDB Query Implementation

Basic Query for Items with the Same Partition Key

Here's how to query a DynamoDB table to retrieve all items with a specific partition key value:

public List<Song> getSongsByArtist(String artist) {
    Song song = new Song();
    song.setArtist(artist);

    DynamoDBQueryExpression<Song> queryExpression = new DynamoDBQueryExpression<Song>()
        .withHashKeyValues(song);
    
    DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
    PaginatedQueryList<Song> songList = mapper.query(Song.class, queryExpression);
    
    return songList;
}

In this example, if we call getSongsByArtist("Black Eyed Peas"), it would return all songs by that artist from our table.

Paginated Queries

For large datasets, DynamoDB uses pagination to divide query results into manageable chunks of up to 1MB each. The query() method handles pagination automatically, but sometimes we need more control.

Using queryPage() with limits:

public QueryResultPage<Song> getLimitedSongsByArtist(String artist, int limit) {
    Song song = new Song();
    song.setArtist(artist);

    DynamoDBQueryExpression<Song> queryExpression = new DynamoDBQueryExpression<Song>()
        .withHashKeyValues(song)
        .withLimit(limit);
    
    DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
    QueryResultPage<Song> queryResultPage = mapper.queryPage(Song.class, queryExpression);
    
    return queryResultPage;
}

This method returns only the specified number of items, which is useful for implementing features like "Load More" in user interfaces.

Using Comparison Operators for Sort Keys

We can use comparison operators to create more specific queries based on the sort key. For example, to find songs released after a certain year:

public List<Song> getSongsByArtistAfterYear(String artist, int year) {
    Song song = new Song();
    song.setArtist(artist);

    Map<String, AttributeValue> expressionAttributeValues = new HashMap<>();
    expressionAttributeValues.put(":year", new AttributeValue().withN(String.valueOf(year)));

    DynamoDBQueryExpression<Song> queryExpression = new DynamoDBQueryExpression<Song>()
        .withHashKeyValues(song)
        .withRangeKeyCondition("year", new Condition()
            .withComparisonOperator(ComparisonOperator.GT)
            .withAttributeValueList(new AttributeValue().withN(String.valueOf(year))))
        .withConsistentRead(false);
    
    DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
    PaginatedQueryList<Song> songList = mapper.query(Song.class, queryExpression);
    
    return songList;
}

Handling Pagination with ExclusiveStartKey

For implementing pagination across multiple requests, we use the ExclusiveStartKey parameter:

public QueryResultPage<Song> getNextPageOfSongsByArtist(
        String artist, 
        int limit, 
        Map<String, AttributeValue> lastEvaluatedKey) {
    
    Song song = new Song();
    song.setArtist(artist);

    DynamoDBQueryExpression<Song> queryExpression = new DynamoDBQueryExpression<Song>()
        .withHashKeyValues(song)
        .withLimit(limit);
    
    if (lastEvaluatedKey != null) {
        queryExpression.setExclusiveStartKey(lastEvaluatedKey);
    }
    
    DynamoDBMapper mapper = new DynamoDBMapper(DynamoDbClientProvider.getDynamoDBClient());
    QueryResultPage<Song> queryResultPage = mapper.queryPage(Song.class, queryExpression);
    
    // The LastEvaluatedKey can be used for the next page request
    Map<String, AttributeValue> lastKey = queryResultPage.getLastEvaluatedKey();
    
    return queryResultPage;
}

This approach allows for efficient navigation through large result sets by providing a continuation point for subsequent queries.

Eventual Consistency

DynamoDB provides two consistency options for reads: eventually consistent and strongly consistent. By default, queries use eventually consistent reads, which means you might get slightly outdated data but with higher throughput and lower latency.

For cases where you need the most up-to-date data, you can specify consistent reads:

queryExpression.setConsistentRead(true);

However, consistent reads consume twice the read capacity units and may have higher latency.

Best Practices for DynamoDB Queries

• Design for Query Efficiency: Structure your data model to support common query patterns
• Use Key Condition Expressions: Limit queries to specific partition keys to reduce costs
• Be Careful with Filter Expressions: Filters are applied after the query and consume read capacity
• Implement Pagination: Use LastEvaluatedKey for handling large result sets
• Choose Projections Wisely: Only request the attributes you need to reduce data transfer
• Consider Consistent Reads: Use consistent reads when data freshness is critical
• Use Indexes Strategically: Create and use secondary indexes for alternate access patterns

Resources