Churro: Simple Filesystem Object Persistence¶
Churro is a simplistic persistent storage for Python objects which stores a tree of hierchically nested objects as folders and flat files in a filesystem. Churro uses AcidFS to provide ACID transaction semantics. Changes to any Churro object tree are only persisted when a transaction successfully commits. Churro uses JSON to serialize objects. Churro is meant to be lightweight and durable. Use of JSON, a universally understood and human readable text file format, insures that data stored by Churro is portable to other applications and platforms across space and time.
In addition to these docs, it couldn’t hurt to look over the AcidFS documentation.
Defining Persistent Types¶
In order for an object to be saved in a Churro repository, it must inherit
PersistentFolder. Attributes of your
persistent objects that you want to be persisted must be derived from
PersistentProperty. Probably the best way to illustrate is by
example, so let’s say you’re writing an application that saves contacts in an
address book. We might write some code that looks like this:
from churro import Persistent from churro import PersistentProperty from churro import PersistentFolder class AddressBook(PersistentFolder): title = PersistentProperty() def __init__(self, title): self.title = title class Contact(Persistent): name = PersistentProperty() address = PersistentProperty() def __init__(self, name, address): self.name = name self.address = address
You can see that defining your persistent types is pretty straightforward. Next you’ll want to open a repository and start storing some data.
Adding Objects to the Repository¶
from churro import Churro repo = Churro('/path/to/folder') root = repo.root() contacts = AddressBook('My Contacts') root['contacts'] = contacts contacts['fred'] = Contact('Fred Flintstone', '1 Rocky Road') contacts['barney'] = Contact('Barney Rubble', '6 Bronto Lane')
Above, we create an instance of
Churro where the
argument is the folder in the filesystem where the repository will live. If the
folder does not exist, it will be created and an empty repository will be
initialized. Otherwise an existing repository will be opened. The call to
repo.root() gets the root folder of the repository,
the starting point for traversing to any other objects in the repository. From
there, adding data to the repository is as easy as instantiating data objects
using folders as Python dicts.
Committing a Transaction¶
So far no data has actually been stored yet. You’ll need to commit a transaction:
import transaction transaction.commit()
PersistentProperty and its subclasses are responsible for
serializing individual attributes of your Python objects to JSON.
PersistentProperty can handle values of any type natively
serializable to JSON. These include strings, booleans, numbers, lists, and
dictionaries. Persistent properties can also hold as values other
Persistent objects, allowing objects to be nested inside of
For other types you’ll need to provide a means for converting the type to
something serializable by JSON and then converting back to a Python object.
This is done by extending
PersistentProperty and overriding
validate() methods. The following is an
actual example from Churro code that illustrates this:
import datetime class PersistentDate(PersistentProperty): def from_json(self, value): if value: return datetime.date(*map(int, value.split('-'))) return value def to_json(self, value): if value: return '%s-%s-%s' % (value.year, value.month, value.day) return value def validate(self, value): if value is not None and not isinstance(value, datetime.date): raise ValueError("%s is not an instance of datetime.date") return value
You can use the new property type in your class definitions:
class Contact(Persistent): name = PersistentProperty() address = PersistentProperty() birthday = PersistentDate() def __init__(self, name, address): self.name = name self.address = address
Mutable Property Values¶
Churro automatically keeps track of which objects have been mutated and saves those objects at transaction commit time. Churro does this by keeping track of when a setter is called on a property and marking that object as dirty. So simply assigning a value to a property will cause that object to get persisted at commit time:
daniela.birthday = datetime.date(2010, 5, 12)
You can find yourself in a situation, however, where the assigned value is a mutable structure and instead of assigning a new value to the property you simply mutate the structure. Let’s say that we add a list of friends to our Contact class:
class Contact(Persistent): name = PersistentProperty() address = PersistentProperty() birthday = PersistentDate() friends = PersistentProperty() def __init__(self, name, address): self.name = name self.address = address self.friends = 
If we have a Contact instance that is clean and the only change we make is to add a friend to the list, Churro will not detect the mutation and the change will not be persisted at commit time:
# This change won't be persisted daniela.friends.append('Katy')
One way to get around this problem is to call the
set_dirty() method on the object that needs to be
# Unless you call this method daniela.set_dirty()
This brute force method is always available, whatever you’re doing. Churro
does, however, provide helpers for the two most common types of mutable data,
dicts and lists. These are
PersistentList respectively. We could rewrite the example
above to a use a
PersistentList instead of a plain Python list:
from churro import PersistentList class Contact(Persistent): name = PersistentProperty() address = PersistentProperty() birthday = PersistentDate() friends = PersistentProperty() def __init__(self, name, address): self.name = name self.address = address self.friends = PersistentList()
Now you don’t need to call
set_dirty() when adding a
friend to a contact’s friend list:
# Don't need to call set_dirty, this change will be persisted daniela.friends.append('Silas')
Churro(repo, head='HEAD', factory=None, create=True, bare=False)¶
repoThe path to the repository in the real, local filesystem.
headThe name of a branch to use as the head for this transaction. Changes made using this instance will be merged to the given head. The default, if omitted, is to use the repository’s current head.
factoryA callable that returns the root database object to be stored as the root when creating a new database. The default factory returns an instance of churro.PersistentFolder. This has no effect if the repository has already been created.
createIf there is not a Git repository in the indicated directory, should one be created? The default is True.
bareIf the Git repository is to be created, create it as a bare repository. If the repository is already created or create is False, this argument has no effect.
Writes any unsaved data to the underlying AcidFS filesystem without committing the transaction.
Gets the root folder of the repository. This is the starting point for traversing to other objects in the repository.
This is the base class from which all persistent classes for Churro must be derived. Only objects which are instances of a class derived from Persistent may be stored in a Churro repository.
Calling this method on a persistent object detaches that object, and its children, from the in memory persistent object tree, potentially allowing it to be garbage collected if there are no other references to the object.
Calling this method alerts Churro that this object is dirty and should be persisted at commit time. It is usually not necesary to call this method from application code, since Churro tries to detect object mutation whenever possible. You may need to call this method from your application code, however, if you use mutable data structures that are not themselves Persistent as values of persistent properties, as Churro has no way of detecting mutations to those structures.
Classes which derive from this class are not only persistent in Churro but have dict-like properties allowing them to contain children which are other persistent objects or folders. Storing an instance of PersistentFolder in a Churro repository, creates a folder in the underlying filesystem, in which child objects are stored. Instances of PersistentFolder are dict-like and are interacted with in the same way as standard Python dictionaries.
Returns the child object of the given name. Returns default if the child is not found.
Returns an iterator over (child object’s name, child object) tuples.
Returns the names of child objects.
Removes the child with the given name from the folder. Raises KeyError if there is no child with the given name.
Returns an iterator over child objects.
A PersistentDict is a Python dict work alike that marks its parent object as dirty whenever it is mutated, solving the problem of using mutable datastructures as values for persistent properties with Churro and eliminating the need to call
set_dirty()in application code when updating the dictionary.
A PersistentList is a Python dict work alike that marks its parent object as dirty whenever it is mutated, solving the problem of using mutable datastructures as values for persistent properties with Churro and eliminating the need to call
set_dirty()in application code when updating the list.
The base type for all persistent properties. This property type can handle any data type as a value that is serializable natively to JSON. Other types are implemented by extending this class and overriding the from_json, to_json, and validate methods.
Converts a value from its JSON representation to a Python object.
Converts a value from a Python object to an object that can be serialized as JSON.
Used at assignment time to validate a value. If a value is not of the proper type and cannot be converted to the proper type, a ValueError is raised, otherwise the valua is returned, including any transformation or coercion that has been performed.
A persistent attribute type that can store instances of datetime.date.
A persistent attribute type that can store instances of datetime.datetime.