Model class

Class definition

class dynamodb_mapper.model.DynamoDBModel(**kwargs)

Abstract base class for all models that use DynamoDB as their storage backend.

Each subclass must define the following attributes:

  • __table__: the name of the table used for storage.

  • __hash_key__: the name of the primary hash key.

  • __range_key__: (optional) if you’re using a composite primary key,

    the name of the range key.

  • __schema__: {attribute_name: attribute_type} mapping.

    Supported attribute_types are: int, long, float, str, unicode, set. Default values are obtained by calling the type with no args (so 0 for numbers, “” for strings and empty sets).

  • __defaults__: (optional) {attribute_name: defaulter} mapping.

    This dict allows to provide a default value for each attribute_name at object creation time. It will never be used when loading from the DB. It is fully optional. If no value is supplied the empty value corresponding to the type will be used. “defaulter” may either be a scalar value or a callable with no arguments.

  • __migrator__: Migration handler attached to this model

To redefine serialization/deserialization semantics (e.g. to have more complex schemas, like auto-serialized JSON data structures), override the _from_dict (deserialization) and _to_db_dict (serialization) methods.

Important implementation note regarding sets: DynamoDB can’t store empty sets/strings. Therefore, since we have schema information available to us, we’re storing empty sets/strings as missing attributes in DynamoDB, and converting back and forth based on the schema.

So if your schema looks like the following:

{
    "id": unicode,
    "name": str,
    "cheats": set
}

then:

{
    "id": "e1m1",
    "name": "Hangar",
    "cheats": set([
        "idkfa",
        "iddqd"
    ])
}

will be stored exactly as is, but:

{
    "id": "e1m2",
    "name": "",
    "cheats": set()
}

will be stored as simply:

{
    "id": "e1m2"
}

Constructors

__init__

DynamoDBModel.__init__(**kwargs)

Create an instance of the model. All fields defined in the schema are created. By order of prioritym its value will be loaded from:

  • kwargs
  • __defaults__
  • mapper’s default (0, empty string, empty set, ...)

We’re supplying this method to avoid the need for extra checks in save and ease object initial creation.

Objects created and initialized with this method are considered as not coming from the DB.

Data access

get

classmethod DynamoDBModel.get(hash_key_value, range_key_value=None, consistent_read=False)

Retrieve a single object from DynamoDB according to its primary key.

Note that this is not a query method – it will only return the object matching the exact primary key provided. Meaning that if the table is using a composite primary key, you need to specify both the hash and range key values.

Objects loaded by this method are marked as coming from the DB. Hence their initial state is saved in self._raw_data.

Parameters:
  • hash_key_value – The value of the requested item’s hash_key.
  • range_key_value – The value of the requested item’s range_key, if the table has a composite key.
  • consistent_read – If False (default), an eventually consistent read is performed. Set to True for strongly consistent reads.

get_batch

classmethod DynamoDBModel.get_batch(keys)

Retrieve multiple objects according to their primary keys.

Like get, this isn’t a query method – you need to provide the exact primary key(s) for each object you want to retrieve:

  • If the primary keys are hash keys, keys must be a list of their values (e.g. [1, 2, 3, 4]).
  • If the primary keys are composite (hash + range), keys must be a list of (hash_key, range_key) values (e.g. [("user1", 1), ("user1", 2), ("user1", 3)]).

get_batch always performs eventually consistent reads.

Objects loaded by this method are marked as coming from the DB. Hence their initial state is saved in self._raw_data.

Parameters:keys – iterable of keys. ex [(hash1, range1), (hash2, range2)]

query

classmethod DynamoDBModel.query(hash_key_value, range_key_condition=None, consistent_read=False, reverse=False, limit=None)

Query DynamoDB for items matching the requested key criteria.

You need to supply an exact hash key value, and optionally, conditions on the range key. If no such conditions are supplied, all items matching the hash key value will be returned.

This method can only be used on tables with composite (hash + range) primary keys – since the exact hash key value is mandatory, on tables with hash-only primary keys, cls.get(k) does the same thing cls.query(k) would.

Objects loaded by this method are marked as coming from the DB. Hence their initial state is saved in self._raw_data.

Parameters:
  • hash_key_value – The hash key’s value for all requested items.
  • range_key_condition

    A condition instance from boto.dynamodb.condition – one of

    • EQ(x)
    • LE(x)
    • LT(x)
    • GE(x)
    • GT(x)
    • BEGINS_WITH(x)
    • BETWEEN(x, y)
  • consistent_read – If False (default), an eventually consistent read is performed. Set to True for strongly consistent reads.
  • reverse – Ask DynamoDB to scan the range_key in the reverse order. For example, if you use dates here, the more recent element will be returned first. Defaults to False.
  • limit – Specify the maximum number of items to read from the table. Even though Boto returns a generator, it works by batchs of 1MB. using this option may help to spare some read credits. Defaults to None
Return type:

generator

scan

classmethod DynamoDBModel.scan(scan_filter=None)

Scan DynamoDB for items matching the requested criteria.

You can scan based on any attribute and any criteria (including multiple criteria on multiple attributes), not just the primary keys.

Scan is a very expensive operation – it doesn’t use any indexes and will look through the entire table. As much as possible, you should avoid it.

Objects loaded by this method are marked as coming from the DB. Hence their initial state is saved in self._raw_data.

Parameters:scan_filter – A {attribute_name: condition} dict, where condition is a condition instance from boto.dynamodb.condition.
Return type:generator

save

DynamoDBModel.save(raise_on_conflict=False)

Save the object to the database.

This method may be used both to insert a new object in the DB, or to update an existing one (iff raise_on_conflict == False).

It also embeds the high level logic to avoid the ‘lost update’ syndrom. Internally, it uses expected_values set to self._raw_data

raise_on_conflict=True scenarios:

  • object from database: Use self._raw_dict to generate expected_values
  • new object: self._raw_dict is empty, set allow_overwrite=True
  • new object with autoinc: flag has no effect
  • (accidentally) editing keys: Use self._raw_dict to generate expected_values, will catch overwrites and insertion to empty location
Parameters:

raise_on_conflict – flag to toggle overwrite protection – if any one of the original values doesn’t match what is in the database (i.e. someone went ahead and modified the object in the DB behind your back), the operation fails and raises ConflictError.

Raises:
  • ConflictError – Target object has changed between read and write operation
  • OverwriteError – Saving a new object but it already existed

delete

DynamoDBModel.delete(raise_on_conflict=False)

Delete the current object from the database.

If the Item has been edited before the delete command is issued and raise_on_conflict=True then, ConflictError is raised.

Parameters:raise_on_conflict – flag to toggle overwrite protection – if any one of the original values doesn’t match what is in the database (i.e. someone went ahead and modified the object in the DB behind your back), the operation fails and raises ConflictError.
Raises ConflictError:
 Target object has changed between read and write operation

Data export

to_json_dict

DynamoDBModel.to_json_dict()

Return a dict representation of the object, suitable for JSON serialization.

This means the values must all be valid JSON object types (in particular, sets must be converted to lists), but types not suitable for DynamoDB (e.g. nested data structures) may be used.

Note that this method is never used for interaction with the database.

Internal data export

Use of _from_db_dict and _to_db_dict methods is discouraged. however they are documented for a very specific use-case. In some edit forms, you want to guarantee that the resource has not been modified elsewhere. Use _to_db_dict to serialize the original object, save it to a hidden field and then use _from_db_dict to reload your object without the actual database call.


classmethod DynamoDBModel._from_db_dict(raw_data)

Build an instance from a dict-like mapping, according to the class’s schema. Objects created with this method are considered as comming from the DB. The initial state is persisted in self._raw_data. If a __migrator__ has been declared, migration is triggered on a copy of the raw data.

Default values are used for anything that’s missing from the dict (see DynamoDBModel class docstring).

Direct use of this method should be avoided as much as possible but still may be usefull for “deep copy”.

Overload this method if you need a special (de-)serialization semantic

Parameters:raw_data – Raw db dict

_to_db_dict

DynamoDBModel._to_db_dict()

Return a dict representation of the object according to the class’s schema, suitable for direct storage in DynamoDB.

Direct use of this method should be avoided as much as possible but still may be usefull for “deep copy”.

Overload this method if you need a special serialization semantic

Auto-increment

class dynamodb_mapper.model.autoincrement_int

Dummy int subclass for use in your schemas.

If you’re using this class as the type for your key in a hash_key-only table, new objects in your table will have an auto-incrementing primary key.

Note that you can still insert items with explicit values for your primary key – the autoincrementing scheme is only used for objects with unset hash_keys (or to be more precise, left set to the default value of 0).

Auto-incrementing int keys are implemented by storing a special “magic” item in the table with the following properties:

  • hash_key_value = -1
  • __max_hash_key__ = N

where N is the maximum used hash_key value.

Inserting a new item issues an atomic add on the ‘__max_hash_key__’ value. Its new value is returned and used as the primary key for the new elem.

Note that hash_key_value is set to ‘-1’ while __max_hash_key__ initial value is 0. This will element at key ‘0’ unused. It’s actually a garbage item for cases where a value is manually added to an unitialized index.

Project Versions

Table Of Contents

Previous topic

Migration class

Next topic

Transactions class

This Page