concrete.util.redis_io module

class concrete.util.redis_io.RedisCommunicationReader(redis_db, key, add_references=True, **kwargs)

Bases: concrete.util.redis_io.RedisReader

Iterable class for reading one or more Communications from redis. See RedisReader for further description.

Example usage:

from redis import Redis
redis_db = Redis(port=12345)
for comm in RedisCommunicationReader(redis_db, 'my-set-key'):
    do_something(comm)

Create communication reader for specified key in specified redis_db.

Parameters:
  • redis_db (redis.Redis) – Redis database connection object
  • key (str) – name of redis key containing your communication(s)
  • add_references (bool) – True to fill in members in the communication according to UUID relationships (see concrete.util.add_references), False to return communication as-is (note: you may need this False if you are dealing with incomplete communications)

All other keyword arguments are passed through to RedisReader; see RedisReader for a description of those arguments.

Raises:Exception – if deserialize_func is specified (it is set to the appropriate concrete deserializer internally)
class concrete.util.redis_io.RedisCommunicationWriter(redis_db, key, uuid_hash_key=False, **kwargs)

Bases: concrete.util.redis_io.RedisWriter

Class for writing one or more Communications to redis. See RedisWriter for further description.

Example usage:

from redis import Redis redis_db = Redis(port=12345) w = RedisCommunicationWriter(redis_db, ‘my-set-key’) w.write(comm)

Create communication writer for specified key in specified redis_db.

Parameters:
  • redis_db (redis.Redis) – Redis database connection object
  • key (str) – name of redis key containing your communication(s)
  • uuid_hash_key (bool) – True to use the UUID as the hash key for a communication, False to use the id

All other keyword arguments are passed through to RedisWriter; see RedisWriter for a description of those arguments.

Raises:Exception – if serialize_func is specified (it is set to the appropriate concrete serializer internally), or if hash_key_func is specified (it is set to an appropriate function internally)
class concrete.util.redis_io.RedisReader(redis_db, key, key_type=None, pop=False, block=False, right_to_left=True, block_timeout=0, temp_key_ttl=3600, temp_key_leaf_len=32, cycle_list=False, deserialize_func=None)

Bases: object

Iterable class for reading one or more objects from redis.

Supported input types are:

  • a set containing zero or more objects
  • a list containing zero or more objects
  • a hash containing zero or more key-object pairs

For list and set types, the reader can optionally pop (consume) its input; for lists only, the reader can moreover block on the input.

Note that iteration over a set or hash will create a temporary key in the redis database to maintain a set of elements scanned so far.

If pop is False and the key (in the database) is modified during iteration, behavior is undefined. If pop is True, modifications during iteration are encouraged.

Example usage:

from redis import Redis
redis_db = Redis(port=12345)
for obj in RedisReader(redis_db, 'my-set-key'):
    do_something(obj)

Create reader for specified key in specified redis_db.

Parameters:
  • redis_db (redis.Redis) – Redis database connection object
  • key (str) – name of redis key containing your object(s)
  • key_type (str) – ‘set’, ‘list’, ‘hash’, or None; if None, look up type in redis (only works if the key exists, so probably not suitable for block and/or pop modes)
  • pop (bool) – True to remove objects from redis as we iterate over them, and False to leave redis unaltered
  • block (bool) – True to block for data (i.e., wait for something to be added to the list if it is empty), False to end iteration when there is no more data
  • right_to_left (bool) – True to iterate over and index in lists from right to left, False to iterate/index from left to right
  • deserialize_func (func) – maps blobs from redis to some more friendly representation (e.g., if all your items are unicode strings, you might want to specify lambda s: s.decode(‘utf-8’)); return blobs unchanged if deserialize_func is None
  • block_timeout (int) – number of seconds to block during operations if block is True; if 0, block forever
  • temp_key_ttl (int) – time-to-live (in seconds) of temporary keys created during scans (amount of time to process a batch of items returned by a scan should be much less than the time-to-live of the temporary key, or duplicate items will be returned)
  • temp_key_leaf_len (int) – length (in bytes) of random part of temporary key (longer is less likely to cause conflicts with other processes but slower)
  • cycle_list (bool) – iterate over list by popping items from the right end and pushing them onto the left end (atomically), note iteration thus modifies the list (although a full iteration ultimately leaves the list in the same state as it began)
Raises:
  • Exception – if key_type is None but the key does not exist in the database (so its type cannot be guessed)
  • ValueError – if key type is not recognized or the options that were specified are not supported for a recognized key type
batch(n)

Return a batch of n objects. May be faster than one-at-a-time iteration, but currently only supported for non-popping, non-blocking set configurations. Support for popping, non-blocking sets is planned; see http://redis.io/commands/spop .

Parameters:n (int) – number of objects to return
Raises:Exception – if key type is not a set, or if it is a set but popping or blocking operation is specified
class concrete.util.redis_io.RedisWriter(redis_db, key, key_type=None, right_to_left=True, serialize_func=None, hash_key_func=None)

Bases: object

Class for writing one or more objects to redis.

Supported input types are:

  • a set of objects
  • a list of objects
  • a hash of key-object pairs

Example usage:

from redis import Redis redis_db = Redis(port=12345) w = RedisWriter(redis_db, ‘my-set-key’) w.write(obj)

Create object writer for specified key in specified redis_db.

Parameters:
  • redis_db (redis.Redis) – Redis database connection object
  • key (str) – name of redis key containing your object(s)
  • key_type (str) – ‘set’, ‘list’, ‘hash’, or None; if None, look up type in redis (only works if the key exists)
  • right_to_left (bool) – True to write elements to the left end of lists, False to write to the right end
  • serialize_func (func) – maps objects to blobs before sending to Redis (e.g., if everything you write will be a unicode string, you might want to use lambda u: u.encode(‘utf-8’)); pass objects to Redis unchanged if serialize_func is None
  • hash_key_func (func) – maps objects to keys when key_type is hash (None: use Python’s hash function)
clear()

Remove all data from redis data structure.

write(obj)

Write object obj to redis data structure.

Parameters:
  • obj (object) – object to be serialized by
  • and written to database, according (self.serialize_func) –
  • key type (to) –
Raises:

Exception – if called on redis key type that is not a list, set, or hash

concrete.util.redis_io.read_communication_from_redis_key(redis_db, key, add_references=True)

Return a serialized communication from a string key. If block is True, poll server until key appears at specified interval or until specified timeout (indefinitely if timeout is zero). Return None if block is False and key does not exist or if block is True and key does not exist after specified timeout.

Parameters:
concrete.util.redis_io.write_communication_to_redis_key(redis_db, key, comm)

Serialize communication and store result in redis key.

Parameters:
  • redis_db (redis.Redis) – Redis database connection object
  • key (str) – name of simple (string) redis key to write communication to
  • comm (Communication) – communication to serialize