Tuesday 1 January 2013

Redis + Python : Connection management and not RTFM

I am fairly familiar with Redis having used it in an earlier project in java.  This time I started to use it in Python given that I was completely sold on its simplicity and performance.  Having spent quite some time I decided to just jump in with a pip install redis.

To create a redis object you use the following code

import redis


mypool = redis.ConnectionPool(connection_class=UnixDomainSocketConnection, 
                                  path = '/var/redis/redis.sock')

r=redis.Redis(connection_pool = mypool)


and then you use the variable r to execute all standard redis commands.  Very simple and clear and having seen a few sample code snippets I was all set or so I thought.  The confusion was that I thought that redis.Redis() represented the connection handle and so I could execute various commands against it.  So to free the connection I looked at the ConnectionPool class and it had a very helpful release() method.  So at the end of my method, I simply did a

mypool.release(r)


Guess what -  that will throw an exception!  This redis object does not represent a connection at all!  All it does is gives you the ability to use the underlying connection pool and execute the command.  you dont have to release the connection as you never got one in the first place.  the Redis() object manages connection acquisition and release transparently.  I figured this out when went through the  source code of the redis-py framework.  So after 4 hours of confusion on why a pool release call can throw exceptions and another couple to read through the redis-py source code. I knew what it was all about.  Something I could have avoided if I had RTFM.  I assumed that redis-py will be a thin wrapper around the redis engine.  It seems to do a bit more than that is what I figured out.

So morale of the story is that each client library to a framework can be very different.  Dont assume that the framework dictates the library interface.  It can provide more layers of abstraction on top.