Friday, February 3, 2017

AWS DynamoDB

How to use DynamoDB from AWS CLI. No extra SDK or API. Examples.

These examples assumes you've followed AWS tutorial on setting up your own table. But so we're on the same page, here's my table details:
  • Primary partition key: thisDate (String)
  • That's it...
Why would you ever want to use DynamoDB from command line when you can use PHP, Java, .Net, etc? For me, I wanted to use DynamoDB for storage of metrics, logs, and flags that my PowerShell scripts may use to do (or not do) some activity for my servers. Hey, it's Tuesday, run this script unless there is an entry in the database that tells you otherwise. Until now, I've been using Oracle DB for this purpose which is way overkill. I'm doing this on Powershell, but I'm not using AWS CLI for Powershell. I could have ran these from Command Prompt or Bash. Be sure to format properly for JSON whatever environment you may try these on. Also these JSONs were created using AWS Web Console's DynamoDB Create Item button.

Putting an entry into the table
1
2
3
4
5
$item = '{"thisDate":{"S":"20170131"},"Enabled":{"S":"Yes"}}'
$jsonItem = $item | convertTo-json
aws dynamodb put-item --table-name myTest 
                      --item $jsonItem 
                      --condition-expression "attribute_not_exist(thisDate)"

The --condition-expression used here makes sure the thisDate column is unique and not overwrite existing value

Retrieving an entry from the table

1
2
3
4
$item = '{"Date":{"S":"20170131"}}'
$jsonItem = $item | convertTo-json
aws dynamodb get-item --table-name myTest 
                      --key $jsonItem

This will return the content in JSON format.

Retrieving content that matches certain criteria from the table

1
2
3
4
5
6
$attrib_names = '{"#d":"thisDate"}' | convertTo-json
$attrib_values = '{":year":{"S","2017"}}' | convertTo-json
aws dynamodb scan --table-name myTest
                  --expression-attribute-names $attrib_names
                  --expression-attribute-values $attrib_values
                  --filter-expression "begins_with(#d,:year)"

Attribute names are just substitution of the real column to a short form. These must begin with the "#" symbol. And these are mandatory if your column is a reserved word. If you don't want to take a chance, then just create a substitution attribute name for all your columns that you want to use in the filter expression. But here are list of all reserved words, http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ReservedWords.html

Attribute values are variable name, definition, and the value that will be used in the filter expression. These variables must begin with the ":" symbol.

Filter-expression can also be your regular comparison operators (<,>,=,>=,<=). For example,
--filter-expression "#d >= :year" would also be a valid comparison. Here's a more detailed (and really complicated) explanation.
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html#FilteringResults

Note that when doing scanning, the entire table is read, so you're not saving any read-times or reservations. And I haven't found a simple way to delete based on some "where" clause, so I just loop through the keys returned from the table scan.

AWS WAF log4j query

How to query AWS WAF log for log4j attacks 1. Setup your Athena table using this instruction https://docs.aws.amazon.com/athena/latest/ug/wa...