concrete.util.file_io module

Code for reading and writing Concrete Communications

class concrete.util.file_io.CommunicationReader(filename, add_references=True, filetype=0)

Bases: concrete.util.file_io.ThriftReader

Iterator/generator class for reading one or more Communications from a file

The iterator returns a (Communication, filename) tuple

Supported filetypes are:

  • a file with a single Communication
  • a file with multiple Communications concatenated together
  • a gzipped file with a single Communication
  • a gzipped file with multiple Communications concatenated together
  • a .tar.gz file with one or more Communications
  • a .zip file with one or more Communications

Sample usage:

for (comm, filename) in CommunicationReader('multiple_comms.tar.gz'):
    do_something(comm)
Parameters:
class concrete.util.file_io.CommunicationWriter(filename=None)

Bases: object

Class for writing one or more Communications to a file

Sample usage:

writer = CommunicationWriter('foo.concrete')
writer.write(existing_comm_object)
writer.close()
close()
open(filename)
Parameters:filename (str) –
write(comm)
Parameters:comm (Communication) –
class concrete.util.file_io.CommunicationWriterTGZ(tar_filename=None)

Bases: concrete.util.file_io.CommunicationWriterTar

Class for writing one or more Communications to a .TAR.GZ archive

Sample usage:

writer = CommunicationWriterTGZ('multiple_comms.tgz')
writer.write(comm_object_one, 'comm_one.concrete')
writer.write(comm_object_two, 'comm_two.concrete')
writer.write(comm_object_three, 'comm_three.concrete')
writer.close()
class concrete.util.file_io.CommunicationWriterTar(tar_filename=None, gzip=False)

Bases: object

Class for writing one or more Communications to a .TAR archive

Sample usage:

writer = CommunicationWriterTar('multiple_comms.tar')
writer.write(comm_object_one, 'comm_one.concrete')
writer.write(comm_object_two, 'comm_two.concrete')
writer.write(comm_object_three, 'comm_three.concrete')
writer.close()
Parameters:
  • tar_filename (str) – If a filename is given, open() will be called with the filename
  • gzip (bool) – Flag indicating if .TAR file should be compressed with gzip
close()
open(tar_filename)
Parameters:tar_filename (str) –
write(comm, comm_filename=None)
Parameters:
class concrete.util.file_io.ThriftReader(thrift_type, filename, postprocess=None, filetype=0)

Bases: object

Iterator/generator class for reading one or more Thrift structures from a file

The iterator returns a (obj, filename) tuple where obj is an object of type thrift_type.

Supported filetypes are:

  • a file with a single Thrift structure
  • a file with multiple Thrift structures concatenated together
  • a gzipped file with a single Thrift structure
  • a gzipped file with multiple Thrift structures concatenated together
  • a .tar.gz file with one or more Thrift structures
  • a .zip file with one or more Thrift structures

Sample usage:

for (comm, filename) in ThriftReader(Communication,
                                     'multiple_comms.tar.gz'):
    do_something(comm)
Parameters:
  • thrift_type – Class for Thrift type, e.g. Communication, TokenLattice
  • filename (str) –
  • postprocess (function) – A post-processing function that is called with the Thrift object as argument each time a Thrift object is read from the file
  • filetype (FileType) – Expected type of file. Default value is FileType.AUTO, where function will try to automatically determine file type.
next()
concrete.util.file_io.read_communication_from_file(communication_filename, add_references=True)

Read a Communication from the file specified by filename

Parameters:
Returns:
Return type:

Communication
concrete.util.file_io.read_thrift_from_file(thrift_obj, filename)

Instantiate Thrift object from contents of named file

The Thrift file is assumed to be encoded using TCompactProtocol

WARNING - Thrift deserialization tends to fail silently. For example, the Thrift libraries will not complain if you try to deserialize data from the file /dev/urandom.

Parameters:
  • thrift_obj – A Thrift object (e.g. a Communication object)
  • filename (str) – A filename string
Returns:

The Thrift object that was passed in as an argument
concrete.util.file_io.read_tokenlattice_from_file(tokenlattice_filename)

Read a TokenLattice from a file

Parameters:tokenlattice_filename (str) – Name of file containing serialized TokenLattice
Returns:
Return type:TokenLattice
concrete.util.file_io.write_communication_to_file(communication, communication_filename)

Write a Communication to a file

Parameters:
  • communication (Communication) –
  • communication_filename (str) –
concrete.util.file_io.write_thrift_to_file(thrift_obj, filename)

Write a Thrift object to a file

Parameters:
  • thrift_obj –
  • filename (str) –