A Peek at Three Python Databases: pickleDB, TinyDB and ZODB
Persistence of data plays a critical role in most software applications. This article introduces three databases—pickleDB, TinyDB and ZODB—which are implemented in Python. Let’s explore their unique characteristics and use case scenarios.
The support for a large number of databases is one of the many features that make Python a favourite language for software developers. Apart from supporting general-purpose database systems such as MySQL, Oracle, PostgreSQL, Informix, etc, it has support for many other specific systems also. For example, it has support for embedded databases such as SQLite and ThinkSQL. And it supports graph databases such as Neo4J.
This article explores three databases -- pickleDB, TinyDB and ZODB. These three are implemented in Python and used for specific purposes.
pickleDB
pickleDB is a simple and lightweight data store. It stores the data as a key-value store. It is based on the Python module named SimpleJSON, which allows developers to handle JSON (JavaScript Object Notation) in a simple and faster manner. SimpleJSON is a pure Python implementation with no dependencies; it functions as an encoder and decoder for Python versions 2.5+. The complete documentation on SimpleJSON is available at https://simplejson.readthedocs.io/.
Developed by Harrison Erd, pickleDB is available with the BSD three-clause licence. It may be installed effortlessly with the following command:
$ pip install pickledb
The name pickleDB is inspired by a Python module named pickle, which pickleDB was using earlier. Though the later versions of pickleDB started using the SimpleJSON module, the name pickle was retained.
The following code segment illustrates the basics of using pickleDB.
>>> import pickledb >>> db = pickledb.load('example.db', False)
>>> db.set('key', 'value') True
>>> db.get('key') 'value'
>>> db.dump() True
Some of the popularly used commands of pickleDB are explained below.
LOAD path dump: This is used to load a database from a file.
SET key value: This is the value of a key with the string. GET key: Used to retrieve the value of the key.
GETALL: Used to fetch all the keys in a database.
REM key: Deletes the key.
DUMP: Saves the database from the memory into the file specified with the load command.
Apart from the above mentioned commands there are various other commands. Interested readers can get the complete details from https://pythonhosted.org/pickleDB/ commands.html.
pickleDB can be used for those scenarios in which the key-value store type of format is suitable.
TinyDB
As the name indicates, TinyDB is a compact, lightweight database and is document-oriented. It is written 100 per cent in Python and has no external dependencies. As the official documentation says, TinyDB is a database optimised for your happiness.
The applications which are best suited to TinyDB are small apps for which a traditional SQL-DB server based approach would be an overload. The major features of TinyDB are listed below:
As the name indicates, TinyDB is very small. The complete source code is only 1200 lines.
TinyDB is based on document-oriented storage. This type of storage is adopted in popular tools such as MongoDB. The API of TinyDB is very simple and clean. Primarily, TinyDB has been designed keeping ease of usage in mind. Another likeable feature of TinyDB is the nondependency. In addition, TinyDB supports all the recent versions of Python. It works on Python 2.6, 2.7, 3.3 – 3.5. Extensibility is another major feature of TinyDB. With the help of middleware, TinyDB behaviour can be extended to suit specific needs.
Though TinyDB has various advantages, it is not a single-size-fits-all solution for problems. It has certain limitations as listed below:
TinyDB is not suitable for those scenarios in which high speed data retrieval is the key.
If you need to access the database from multiple processes or threads, then TinyDB won’t be optimal. Similarly, HTTP server based access is another scenario in which it won’t be suitable.
Having seen the pros and cons of TinyDB, let us look at how to use it. As is the case with many other packages, TinyDB can be installed simply with the following command:
$ pip install tinydb
The following code snippet explores how to create and store the values in TinyDB:
from tinydb import TinyDB, Query db = TinyDB('db.json') db.insert({'type': 'OSFY', 'count': 700}) db.insert({'type': 'EFY', 'count': 800})
After the successful execution of the above mentioned snippet, you can retrieve the values as shown below.
To list all values, give the following commands:
db.all()
[{'count': 700, 'type': 'OSFY'}, {'count': 800, 'type': 'EFY'}]
To search and list values, type:
Magazine = Query() db.search(Magazine.type == 'OSFY') [{'count': 700, 'type': 'OSFY'}]
db.search(Magazine.count > 750) [{'count': 800, 'type': 'EFY'}]
For updating the values, use the following commands:
db.update({'count': 1000}, Magazine.type == 'OSFY') db.all()
[{'count': 1000, 'type': 'OSFY'}, {'count': 800, 'type': 'EFY'}]
To remove values, type:
db.remove(Magazine.count < 900) db.all()
[{'count': 800, 'type': 'EFY'}]
To delete all the values, give the following commands:
db.purge() db.all() []
TinyDB enables developers to handle data in two different ways, as listed below:
JSON
In-memory
The default value is JSON, and if you want to change it to in-memory, you need to specify that explicitly.
from tinydb.storages import MemoryStorage db = TinyDB(storage=MemoryStorage)
The detailed documentation on TinyDB API is available at https://tinydb.readthedocs.io/en/latest/api. html.
ZODB
ZODB is a native object database for Python. Its major features are:
Seamless integration between code and database. No separate language is needed for operations related to the database.
No database mapper is required.
ZODB is better suited for the following scenarios: When the developer wants to focus more on the application rather than building lots of database code. When the application has lots of complex relationships and data structures.
When the data read operations are comparatively larger than the write operations.
At the same time, ZODB is not suitable for scenarios in which the application requires a high volume of data write operations.
To install ZODB, use the following command:
$ pip install ZODB
A simple code snippet to establish connection with a database is shown below:
import ZODB, ZODB.FileStorage storage = ZODB.FileStorage.FileStorage('mydata.fs') db = ZODB.DB(storage) connection = db.open() root = connection.root
ZODB allows developers to use a wide variety of storage options, as listed below:
In-memory databases
Local files
Databases on remote servers
Advanced options such as compressed and encrypted storage
The members of the object can be directly stored using ZODB functions. A detailed tutorial for using ZODB is available at http://www.zodb.org/en/latest/tutorial.html. ZODB has support for transactions also.
This article has attempted to provide an insight into the world of Python databases. There are many other databases apart from the three narrated in this article, such as buzhug (http://buzhug.sourceforge.net/) and CodernityDB (http://labs.codernity.com/codernitydb/). The bottomline is that all these database tools provide a Pythonic ambience when you work with them.