Redis MDEL command

To delete a key from Redis you have the DEL command.  Very fast and works great.    Now I had to delete a bunch of keys from Redis and guess what there is no MDEL command.  Now the choices are that I can do a KEYS <pattern>* (in the older versions) or the new SCAN calls.  Add Lua to that mix and we can create our own MDEL command.

I have uploaded the file to a gist at

https://gist.github.com/vivekhub/1f84d888edaabafb17a5

Feel free to try it out and submit any comments / changes on the gist.

Installing Ubuntu on a Lenovo Z580 and making Wifi work

Lenovo has a laptop model Z580 that comes with a broadcom chipset for Wifi.  This will not work out of the box with Ubuntu 12.04.04 or Mint.  Even a kernel upgrade to a newer kernel did not help.  Hacking through the documentation I found that there is a section on Ubuntu's website that explains the problem in detail.  Instead of providing the detail on the same here on my blog, Just follow the instructions as outlined in this document.  This will make your Wifi come back alive!


Ubuntu Broadcom chipset issues

Reset the XFCE Desktop on Mint Maya

every so often I attempt something crazy and mess up the fvwm on my XFCE desktop.  This will result in the Window manager misbehaving and it doesn't matter if you logout or reboot because it faithfully saves the settings and restores it.  Here is how you get rid of the messed up settings

First, attempt to clear the cache.

cd ~;rm -rf .cache

This should hopefully clear out the problem once you log out and log back in.   If that doesn't work we have to take more drastic steps.  This involves resetting all your desktop configurations and settings.  Personally I hate it as I have a fairly customized destkop with various window manager settings, panels etc., and this step will make you lose all of them.

cd ~;rm -rf .config

This should clear out all the desktop settings and you should be able to see the old mint desktop back in action.

Hope this helps.

Sequence numbers from pymongo

I was looking for a way to generate in a multi process environment unique sequence numbers.  My development language was python and the underlying database was MongoDB.  I found a fairly simple way to do this in MongoDB.

def getNextEventSeq(idkey):


   nextId = seqtab.find_and_modify(query={'_id':idkey}, 

                                   update = {'$inc' : {'seq':1}},

                                   upsert=True)

   return nextId['seq']        

MongoDB guarantees that find_and_modify() is atomic even across multiple mongos hence this is guaranteed to be threadsafe.

Here is the detailed article on MongoDB website

http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/

Interesting Internet prank...

I just ran into this prank.  What was impressive was that it was done at the TCP / network level by a bunch of bored geeks.  I am sure the number of machines and network ports involved would have been interesting to know. 

If you are on a windows machine try doing this

tracert -h 100 216.81.59.173

If you are a diehard linux fan. try this one

tracert -m 100  216.81.59.173

Wait for a few minutes and watch the fun :)

Renaming a branch in git

Many a time we do the mistake of calling it something and figuring out that we really meant something else.  Git branch names are no exceptions :)  So here is how you rename a git branch

git branch -m oldname newname

Thats it. Short and simple in case you were trying to rename git branches

Javascript has grown and CDN failures - Oh My!

For a long time I considered javascript to be a fairly childish language (and rightfully so since my days of working with Javascript goes back to the original navigator days) and I had actually lost track of the language  for over a decade!  I have been recently looking at the language as it seems to have become this new programming language for the real time web and frankly surprised by the number of concepts, frameworks and capabilities that it has evolved into over the years.

Coming to more practical matters, I had a steep learning curve  - learnt how they managed to not pollute the global namespace with global variables, how they have classes (sort of...) and how they have objects (sort off...) and in the middle of all this feeling that the world of javascript is just one big hashtable.

But duty calls and I have been attempting to use some of the popular frameworks out there - socket.io, json, jQuery and so on.  That is when I discovered these wonderful CDNs from Google, Microsoft and Cloudflare that will provide hosting for all these popular frameworks.  Now there is an entire school of thought on why CDNs are bad, I as always have an opinion of my own that does not quite agree with the general population so I decided to use the CDNs and still wanted to fall-back to my own servers in case the CDN is down  (yes that is an extremely unlikely event but call me whatever - remember we didn't pay for it).

I found an elegant solution in the Javascript language itself and I thought you may want to use.

Here is how I load JQuery from CDN or fallback to the origin servers when I have to...

 <script src='https://ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js' charset='utf-8' type='text/javascript'></script> 

< script type="text/javascript">
// <![CDATA[
            window.jQuery || document.write('<script src="http://i.myorigin.com/fqstatic/js/jquery/jquery.min.js">\x3C/script>')
// ]]> 

Hope this helps 

Getting rid of the brace matching in VIM

For some strange reason VIM has a very bad habit of matching braces rather greedily.  Every time you get on top of a brace, the cursor jumps willy nilly trying to find the matching brace and possibly making strange noises when it cannot find one.

I personally hate that behaviour as much as I like my VIM so here is how you turn it off.  On your ~/.vimrc add the following

let matchparen=1

Thats all there is to it.  No more jumping greedy cursors looking for matching braces ;)

Linux Mint Maya Trackpad trouble

I just got a new thinkpad T430 that has a synaptics trackpad.   Strangely the trackpad would not work correctly once the laptop goes to sleep and wakes up. The little red knob IBM thingy would work fine but the trackpad and the associated buttons were not working fine.    Found a nice workaround for it that I thought I will share.

The idea was to disable and enable the psmouse driver so that it would reset itself.  Here is the script to do the same.  It should be placed in /etc/pm/sleep.d and marked as executable

vivekv@Vivek-Laptop /etc/pm/sleep.d $ cat 71input-reset 
#!/bin/sh
#
# Reload the AT keyboard interface.

case "$1" in
        hibernate|suspend)
                rmmod psmouse
                ;;
        thaw|resume)
                modprobe psmouse
                ;;
        *)
                ;;
esac

Here is my virtual pointer information.

⎡ Virtual core pointer                        id=2    [master pointer  (3)]
⎜   ↳ Virtual core XTEST pointer                  id=4    [slave  pointer  (2)]
⎜   ↳ SynPS/2 Synaptics TouchPad                  id=11    [slave  pointer  (2)]
⎜   ↳ TPPS/2 IBM TrackPoint                       id=13    [slave  pointer  (2)]
⎣ Virtual core keyboard                       id=3    [master keyboard (2)]
    ↳ Virtual core XTEST keyboard                 id=5    [slave  keyboard (3)]
    ↳ Power Button                                id=6    [slave  keyboard (3)]
    ↳ Video Bus                                   id=7    [slave  keyboard (3)]
    ↳ Sleep Button                                id=8    [slave  keyboard (3)]
    ↳ Integrated Camera                           id=9    [slave  keyboard (3)]
    ↳ AT Translated Set 2 keyboard                id=10    [slave  keyboard (3)]
    ↳ ThinkPad Extra Buttons                      id=12    [slave  keyboard (3)]

Linux mint maya outdated ISOs

I just installed Linux mint maya xfce on my new IBM thinkpad T430.  I downloaded the latest ISO from linuxmint.com burned it to a DVD and went through the installation.  The DVD was about 795 MB in size and the installation went fairly smoothly.

Once the installation completed I did the obvious

apt-get update
apt-get upgrade

Boom this hits me now

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages have been kept back:
ginn hplip hplip-data libgnome2-0 libgrip0 libhpmud0 libsane-hpaio linux-generic linux-headers-generic linux-image-generic printer-driver-hpcups printer-driver-hpijs vlc vlc-nox vlc-plugin-notify vlc-plugin-pulse
The following packages will be upgraded:
accountsservice acpi-support activity-log-manager-common alsa-base alsa-utils apt apt-transport-https apt-utils aptdaemon aptdaemon-data aptitude



snipped out a lot of stuff...


585 upgraded, 0 newly installed, 0 to remove and 16 not upgraded.
Need to get 495 MB of archives.
After this operation, 25.7 MB of additional disk space will be used.


This is quite a bit of updates to download right after burning and installing from an ISO only an hour ago.  Not sure why the ISO has not been kept up to date.

Redis + Python : Connection management and not RTFM

I am fairly familiar with Redis having used it in an earlier project in java.  This time I started to use it in Python given that I was completely sold on its simplicity and performance.  Having spent quite some time I decided to just jump in with a pip install redis.

To create a redis object you use the following code

import redis


mypool = redis.ConnectionPool(connection_class=UnixDomainSocketConnection, 
                                  path = '/var/redis/redis.sock')

r=redis.Redis(connection_pool = mypool)


and then you use the variable r to execute all standard redis commands.  Very simple and clear and having seen a few sample code snippets I was all set or so I thought.  The confusion was that I thought that redis.Redis() represented the connection handle and so I could execute various commands against it.  So to free the connection I looked at the ConnectionPool class and it had a very helpful release() method.  So at the end of my method, I simply did a

mypool.release(r)


Guess what -  that will throw an exception!  This redis object does not represent a connection at all!  All it does is gives you the ability to use the underlying connection pool and execute the command.  you dont have to release the connection as you never got one in the first place.  the Redis() object manages connection acquisition and release transparently.  I figured this out when went through the  source code of the redis-py framework.  So after 4 hours of confusion on why a pool release call can throw exceptions and another couple to read through the redis-py source code. I knew what it was all about.  Something I could have avoided if I had RTFM.  I assumed that redis-py will be a thin wrapper around the redis engine.  It seems to do a bit more than that is what I figured out.

So morale of the story is that each client library to a framework can be very different.  Dont assume that the framework dictates the library interface.  It can provide more layers of abstraction on top.

Managing my ubuntu security patches

I use ubuntu server as my key linux development environment.  It is fairly up to date and today in fact that is the topic of my post.  Many a time Ubuntu tells me that there is a whole bunch of updates that I have to install and I dont have the patience to figure out which ones are needed and why.  While I have always wanted to get the security updates installed, I really didnt want to install all the updates.

That is when I found out the gem of a command, unattended-upgrade.  Just issuing this from time to time took care of installing all the security updates and let me decide what I want to install later.  

sudo unattended-upgrade




Hope this post will help you also address this simple but essential update need on your ubuntu instances.




Wrestling with gevent-socketio

I just spent most of last week attempting to get gent-socketio to work. First problem that I ran into was that nobody seems to have used it with adobe flash based fallback on browsers that do not support websocket.   This method is referred to in the socket.io library as flashsocket.  I was pretty confident that nobody has tried it since the swf file required by socket.io was missing in the distribution.  I fixed that by checking in the swf file into the bit repo.   That was the easy problem to fix.

The second challenge was that socket.io has a concept of namespaces which are essentially logical boundaries r messages so that one physical connectioncan be multiplexed by multiple capabilities on a given page. There are two confusing concepts plus a potential bug that really confused me.  In the socket.io world you would use a namespace by specifying it in the io.connect() call on the client. Thus if you wanted to use the namespaces foo and bar. 

You would use the following code

fooconn =io.connect('http://myserver.com:8000/foo')
barconn =io.connect('http://myserver.com:8000/bar')

Now the confusing part is that the client library will use the host and port specified to connect to the server but would use the 'foo' /'bar' as the namespace as part of the socket.io protocol and the actual URL requested  on the server would be

http://myserver.com:8000/socket.io/

If you want to change the URL you have to send a key:value pair as part of the connect call like so

fooconn =io.connect('http://myserver.com:8000/foo', resource:"myurlprefix")

This will make the client library generate an URL of http://myserver.com:8000/myurlprefix/  and the namespace within the protocol would be 'foo'.  If you want to change the URL prefix then you have to change it on the server side as well. You do that by specifying the resource parameter on the server side if you are using a node.js server.  On python gevent-socketio libry theword resource is deprecated and you instead have to use the key 'namespace'!!! 


Well I just couldn't figure out all this complexity myself.  Thanks to Daniel Swarbrick who gave me some very detailed help I finally got my hands around the library. Thanks Daniel.

Hope this little explation gives you a background on what to look out for in gevent-socketio.  I will keep posting on any other gotchas that I runinto when I play with this library.

Using Monit to monitor a Redis instance

I have been using Redis for a long time now (since version 1.x) - the challenge has been how to start and manage an instance and get alerts when it is consuming too much memory.  This is where Monit comes in - it can monitor various Unix daemons and provide a single dashboard to view status, manage restarts etc.,

Here is how I have configured monit to watch redis

 check process redis with pidfile /var/run/redis.pid
   group redis
   start program = "/usr/local/bin/redis-server /etc/redis/redis.conf"
   stop  program = "/usr/local/bin/redis-cli shutdown"
   if failed unixsocket /var/redis/redis.sock
        send "SET MONIT-TEST value\r\n"
        expect "OK"
        send "EXISTS MONIT-TEST\r\n"
        expect ":1"
   then restart
   if 2 restarts within 2 cycles then alert

This worked fairly well with Monit not only monitor an instance but also tries to put a key and test and make sure that the redis instance is running fine.

Hello world

A new domain and a new blog.  This is my new beginning.