Data manipulation

Amazon’s DynamoDB offers the ability to both update and insert data with a single save() method that is mostly exposed by Dynamodb-mapper.

Saving

As Dynamodb-mapper directly exposes items properties as python properties, manipulating data is as easy as manipulating any Python object. Once done, just call save() on your model instance.

save() has 2 optional parameters. When manually set to False, allow_overwrite will only allow the insertion of a new Item. this is done by setting a condition on the keys.

The second parameter expected_values will garantee that the Item is saved only if these values are present in the database. This dict is a bit tricky to use as it needs to be a raw DynamoDB mapping.

  • It supports only string and numers.
  • When a field is set to False, it will ensure that it does not exist.

Hopefully, DynamodbModel class offers an utility method to ease the mapping creation. to_db_dict() converts the current Item to a DynamoDB compatible representation.

Use case: Virtual coins

When a player purchases a virtual good in a game, virtual money is withdrawn from from its internal account. After the operation, the balance must be > 0. If multiple orders are being processed at the same time, we must prevent the lost update scenario:

  • initial balance = 200
  • purchase P1 150
  • purchase P2 100
  • read balance P1 -> 200
  • read balance P2 -> 200
  • update balance P1 -> 50
  • update balance P1 -> 100

Indeed, when saving, you expect that the balance has not changed. This is what expected_values are for.

from dynamodb_mapper.model import DynamoDBModel, autoincrement_int

class NotEnoughCreditException(Exception):
    pass

class User(DynamoDBModel):
    __table__ = u"game-dev-users"
    __hash_key__ = u"login"
    __schema__ = {
        u"e-mail": unicode,
        u"firstname": unicode,
        u"lastname": unicode,
        u"e-mail": unicode,
        u"connexioncount": int,
        #...
        u"balance": int,
    }

user = User.get("waldo")
oldbalance = user.balance
if user.balance - 150 < 0:
    raise NotEnoughCreditException
user.balance -= 150

try:
    user.save(expected_values={"balance": oldbalance})
except ExpectedValueError:
    print "Ooops: Lost update syndrome caught!"

Note: In a real world application, this would most probably be wrapped in Transactions

Autoincrement technical background

When saving an Item with an autoincrement_int hash_key, the save() method will automatically add checks to prevent accidental overwrite of the “magic item”. The magic item holds the last allocated ID and is saved at hash_key=-1. If hash_key == 0 then a new ID is automatically and atomically allocated meaning that no collision can occure even if the database connection is lost. Additionaly, a check is performed to make sure no Item were manually inserted to this location. If applicable, a maximum of MAX_RETRIES=100 attempts to allocate a new ID will be performed before raising MaxRetriesExceededError. In all other cases, the Item will be saved exactly where requested.

To make it short, Items involving an autoincrement_int hash_key will involve 2 write request on first save. It is important to keep it in mind when dimensioning an insert-intensive application.

Know when to use it, when *not* to use it.

Example:

>>> model = MyModel() # model with an autoincrement_int 'id' hash_key
>>> model.do_stuff()
>>> model.save()
>>> print model.id # An id field is automatically generated
7

About editing hash_key and/or range_key values

Dynamodb-mapper let you edit hash_key and/or range_key fields like any other. However, Amazon’s DynamoDB has no support for changing their values. If they are edited, a new item will be saved in the table with these keys. If you indeed meant to change the keys, first delete the item and then save it again. Beware that any item pre-existing at this keys will be overwritten unless allow_overwrite=True in save.

Example:

>>> model = MyModel.get(24)
>>> model.delete() # Delete *first*
>>> model.id = 42  # Then change the key(s)
>>> model.save()   # Finally, save it

There is no plan to protect the key fields in any future release.

Logically group data manipulations

Some data manipulations requires a whole context to be consistent, status saving or whatever. If your application requires any of these features, please go to the transactions section of this guide.

Limitations

Some limitations over Amazon’s DynamoDB currently applies to this mapper. save() has no support for :

  • returning data after a transaction
  • atomic increments

Please, let us know if this is a blocker to you!