What Sublime Text 2 got right.

After pondering on the IDE matter more, the two finalist in my mind that are worth considering for now are PyCharm and Sublime. This week though I just found myself using Sublime more and more instead of using PyCharm. I am not saying that PyCharm is bad or in anyway less than Sublime. I guess I just find Sublime able to fit into my mind easier. 

I began pondering why in a relatively short time Sublime was able to gain such a foothold as well as mindshare in the developer's community. Just search in youtube or google for Sublime vs PyCharm to see what I mean. I feel that Will Bond of Sublime just did a few things right in my opinion.

1. Kept performance of the editor as it's main goal firmly in sight. I just get the feeling that performance always trumped functionality if there was ever a toss up during the development of Sublime. Never once when I used this editor did it feel bloated or sluggish.

2. Sublime just remembered what Nokia realized all those years back when they launched and marketted their multi color phones in the world of drab black color 'professional phones'. Sublime remembers that although the editor is made on a machine that it's main / only audience is a human. Sublime's out of the box colors / fonts and settings until today still remains the most eye pleasing out of the box and normally I just leave it at the default settings.

3. These two things in turn created the third thing in which Sublime gets right which is having an eye pleasing and easy to use forum where the users searching through the knowledge base can get to the information that they are looking for easy and quickly.

4. Create innovative new features such as their right hand code map which I find really useful for moving around while at the same time giving a nod to the old true and tried ways... read Vintage mode. The fact that Vintage mode is built in and not some third party package also gets a big plus from me. For those who don't know Vintage mode is Sublime speak for vim keybindings.

5. Have some pretty informative videos teaching the usage of sublime. Have a look at this Sublime porn (Although it's a bit targeted towards web developers) and tell me are not raring to get on the Sublime band wagon. The guy can really work fast!

6. Allowed and let flourish the tools / nuances of different languages so that developers of any language can just feel at home on Sublime.

I agree totally with people who say that Sublime was created by a developer ... in this case Will Bond for developers. You can feel that Sublime was created trying to scratch some of the itch he himself was trying to scratch and that makes a really powerful product. At this juncture though, I just discovered that certain really nice to use plugins, such as SVN plugin requires you to pay after sometime. While you can always just choose not to use it, but then I prefer it if all of these plugins were just included as part of the main install and just charge me upfront.

Also, I like the fact that Will and gang have developed this so-called 'App Center' by accepting and creating a repo for the community to contribute towards the development of plugins for Sublime. There are paid / free plugins available but seeing as how you can (for now) just keep on using indefinitely the paid plugins I guess you can decide to pay / or not for the plugins that you like. Personally I have paid for Sublime, the SFTP and SVN plugins all by Will. I really hope that Will will walk a non-Apple path by allowing healthy competition if his users decide to come out with free packages that will compete with his commercial ones. 

I will in the end have a look at PyCharm and seriously use it for at least a week to see if I will eventually pay for it, but compared to my first time on Sublime, I just found myself fumbling around more looking for stuff on PyCharm or just stopping my work sometime waiting for some unscheduled index to finish running. As for now it looks as though Sublime as a lead over PyCharm.

State of Python IDEs 2012

The search for a Python IDEs is like for lack of a better analogy like masturbating. Every serious programmer or serious programmer wannabe, more so the novice programmers have done it before or at least thought about it once or twice before. I know it's a bad analogy but you should know what I am trying to get at. We have all done it before one time or another or at least thought hard if there is something outside there better.

The macho among us will claim that we only use the true and tried Vim / Emacs but given an IDE that can do all that both of those can do and more would you not be tempted ? Some would argue that it's not about the tool and we if we put the time into it we can customize something usable out of the free stuff that is available out there. I for one would rather just pay for something like or just have everything available out of the box that is configurable without much effort and I am just a sucker for how good a great editor feels like when you are using it. At a certain point I can almost see the wisdom behind why people would rather just survive without plugins on a vim install ... but use vim without my NerdTree ?! Woe! What kind of life would that be.

Well my search went into serious mode for these past 2 weeks. I walk away from the experience a much frustrated but wiser man. Summary of my findings, which is quite shocking .... none of them are quite ready yet! I thought that this would be the state of Python IDEs say 2 to 3 years ago but it seems that the scenario has not changed much. Here are the few that I looked at:

1. Geany
2. Kate
3. Gedit
4. Spyder
5. Ninja-IDE
6. Anjuta with PyDev
7. Vim

2 and 3 are there because they are so ubiquitous with Linux distributions and I wanted to try them out to see if I can have an editor that comes with almost all Linux installations that I have not spent too much time with them and have something up and running in no time. Alas, finding that pep8 and pyflakes seemed more like a third party integration does not spark the hot fuzziness in me. that I can have out of the box without installing much of anything and just use them. Yes, I did look around and stuff like this for Kate, just did not work correctly either and even vim integration with Gedit looked aged and at last just did not work (Just spent another 30 mins with it to find the location of the plugins folder moving from gnome2 to gnome3 .. yeucchh).

Let's get to the so-called review part of this post. Let's start with the grand daddy of them all:

Vim

I have been using Vim for the longest time and to a certain degree it was great plugging in to my local pyflakes and pep8 to help with my Python code and the reason my love started to wander was because of the way plugins are managed. I used pathogen to manage my plugins and for sometime I was happy until strange errors started popping out here and there because some plugins were having errors. I found it quite hard to debug and found myself sitting at my terminal late one day deleting my plugins one by one until I finally arrived at the plugin that was causing the error. I patched it up and sent the author the patch, yet the whole experience created the seed of thought in my head that there must be a better way of doing things or at least a better editor / IDE out there that can be used.

Truth be told vundle seems promising and after knocking it around a bit it seems like a good way to manage your vim plugins across machines. Read this ... http://www.charlietanksley.net/philtex/sane-vim-plugin-management/

Sublime Text 2

This is the editor that I am still using today. While initially put off by it's text based config / settings files I stuck with it and was happy as it was light and responsive and it had everything that I needed, except maybe a debugger or a prompt that came out of the box. I was so happy with it I bought it and till today it remains my main workhorse. About 3 weeks back something happen though that diminished my love for sublime. I installed a new sublime editor in another machine and pointed it's plugins at my Dropbox install. This way all of my favorite plugins will always be available to me. It was strange, it just hung there refusing to start up. When I deleted the symlink to my Dropbox it just started up normally. There was no error messages, no logs nothing indicating what was wrong. I was quite devastated as like many I hated to have to go and look for another IDE / Editor that I can use but I was really up my creek without a paddle, as googling for the phrase "Sublime can't start because of plugin problems" yielded nothing useful. The other thing that irked my was the fact that I had to lug my plugins around all around in my Dropbox folder. The issue that I had with Vim was not really solved, infact it became worse, because with vim at least it could start up, it just spat some angry error messages at me.

Actually it would be unfair of me to comment on Anjuta with PyDev, Spyder or Ninja-IDE as all of the these IDEs just could not install correctly on my box. Anjuta was just dogged slowed my machine down to a crawl that even firing up a browser to go and look at what was wrong was rendered impossible. Geany I just installed today, but once I found that vim bindings was a wishlist item rather than a configurable item, that really put me off. At this point I am really wondering if PyCharm or WingIDE would be worth the investment. I tried ver 1.0 of PyCharm before and my initial view of it was quite laggy and consumed resources on my machine. Probably would be good for me to try out the 30 day trials of Wing and PyCharm as they can be quite pricey... 99USD for Pycharm and 245USD for WingIDE.

What seems to be going for PyCharm now as I look at it are:

1. Guido seems to be behind it.
2. Some obscure functionality which I usually use which I find hard to find anywhere else like the ability to stash away changes. This is something that is very useful for me which I only find that I could do using git. This is really one of the most attractive points for me to consider getting a license from these guys as it allows me to do this outside of whatever VCS that I am using and I do not need to do it two times.
3. Integrated nosetests and coverage.
4. Integrated pep8/flakes checking for code. Been using these for sometime now and I find them indispensable.

Down side seems to be:

1. Based on Java, read dead slow plus resource hog. Related to this I found this ticket today which looks a bit worrying.

Actually in summary, I can see very little wrong with PyCharm except that it becoming a resource hog which usually is the case for IDEs written in Java, just look at Eclipse. Enabling and configuring PyCharm to get semblance of a working environment for myself was quite easy and the controls seem intuitive without much need to hit google for an answer. Just as I wrote this my whole desktop became unresponsive because of the resource eaten up by PyCharm. The jury is still out on this one. I will post what I eventually do in part 2 of this.

Presenting ... the inupypi server

After using the pypiserver: http://pypi.python.org/pypi/pypiserver/ for sometime to house our internal eggs and libraries, we (one of my colleagues at least, very very quietly I might add) decided add to give back to the Python community by developing our own pypiserver called InuPyPi: http://pypi.python.org/pypi/inupypi/0.2.3 which is based on Flask. Some of the features of our pypi server are:

• A nice(r) web interface for packages.
• Supports multiple eggbasket folder per server.
• Auto searching for updates of packages (planned)
• One click update for outdated packages (planned)
• Command-line tool to upload to server something like 'python setup.py upload' (planned. This will be pretty close to doing)

For those who are interested to contribute the project, the project lives in github: https://github.com/hallaj/inupypi. We will appreciate suggestions and help with our pypi server. 

How do you test your Python apps for production ?

As usual I am just thinking aloud for this post and thinking about the ways to solve a problem that we are having. Haven't really got the answer yet just sharing hoping that some other smart soul out there might offer me their 2 cents. Our apps currently have got pretty good test coverage or so I feel. I would say that our coverage currently is about 95% of our application functionality. We have unit tests, functional tests and even selenium tests (currently trying out splinter to replace selenium). We run them all against our test fixtures and use Jenkins to visualize and notify us whenever the tests fail. While I think there are more rooms for improvements, on the whole I am pretty proud of our whole test architecture.

We freeze our code and only branch our code once all of it's tests hits blue on Jenkins. That's all fine. Hunky Dory. That's when the ice berg reared it's ugly head on our sailing Titanic. The problem is our code failed twice while we were trying to deploy the app (wsgi app) and we only detected it through manual means, eye balling it. Once because our ssl certs failed because of this particular bug: http://code.google.com/p/httplib2/issues/detail?id=202#c2. This bug occurred with httplib2 when using wildcard certs which we don't use in our testing. The whole wsgi app came crumbling down without any of our tests informing us of the wiser. I would be okay if these were some obscure bug that occurred because of some strange use case that we have never considered prior to this, but these are big-pie-on-your-face kind of embarrassing show stoppers. What I want to do is a way to catch show stopper bugs before deploying our code on production servers.

What I am talking about here almost like a full dress rehearsal before deployment. Setting up a dedicated machine for testing that mirrors the live data seems the most obvious way but in someways is too much of a resource hog in my opinion. From what I gather from doing a bit of quick 2 minute research I am not alone:

• http://www.certpal.com/blogs/2010/06/testing-in-production/ and more recently from the Python Planet .. 
• http://www.alexconrad.org/2011/10/proxying-tests.html . This is a start but it still means almost mirroring my live site on to a test box.

This seems like an idea too http://xunitpatterns.com/Test%20Logic%20in%20Production.html. I think overall the main thing to be moving towards is to stream line the test machines to be as close to the live machines as possible and to lessen the 'magic' that we do with test servers to make tests work and determine which environmental variable would be crucial to our application being deployed well and replicating that in our tests. 

What I don't like about this solution is that for 'n live sites with different environment' you need n amount of replicated live environments. Another idea that I am playing around with is the create a suite of 'critical tests' that is run after an upgrade just to ensure the most critical services are running fine. Ideally these tests should be short but it should give you an a pretty good idea if the main services will be running when you go home after an upgrade is performed on the server. The tests should just be a subset of the main battery of tests and should complete in a fraction of the time taken to run the full suite of test, just like one of those old windows installers that run 'self diagnosis' after installation. 

OpenSuSE 12.2 delivers again

Just upgraded OpenSuSE to 12.2 and I am happy to report the upgrade process is flawless as usual. Stuffed in the DVD, settled some packaging conflicts and I was good to go. Half an hour later 12.2 was smiling back at me after rebooting after I restarted my machine. This is by no means a review, just a little tip of the hat to the OpenSuSE guys for making the upgrade process smooth.

I don't think enough is said or enough accolades is paid to hardworking folks that helps me to retain a few more of my already receding follicles after a major upgrade. Thank you guys! Aside from having to re-install my google chrome and my codecs (Why can't they just have an option to do or enable that during upgrade process ??) nothing much of anything was interesting. Boring. Crickets. That's the way a smooth and great upgrade should be.

Really like the work put into the boot up screen and the improvements done to zypper. Overall the box at homes is chugging along at a somewhat higher speed.

Thank you OpenSuSE. Just the other day it crossed my mind that I have had more than a decade affair with OpenSuSE and it just seems get stronger from release to release.

Replacing our eggbasket server

Currently our company uses Christopher Arndt's eggbasket, which seems more than anything to be abandoned-ware since the last release was 2008 and there has not been any changes to the code since then.

It's based on TG and since TG has been going through some tremendous changes since 2008 in terms of it's API, easy_install-ing eggbasket will not work anymore as it's broken 7 days to Sunday. Here, we really love his eggbasket as it's simple to use and really pretty looking, but alas, good things must come to an end and more importantly good things must also be replaced.

I can hear people out there saying that creating our own eggbasket is as easy as statically linking a folder in Apache, but hey I love trying out software and want to give some apps out there a chance ... after trying a few, we decided to go with pypiserver as it seems that this project seems to be still maintained and the most active.

Installing it is as easy as easy_install or pip and running it is dead simple too. There was one based on Django called localshop and it really looked pretty with tons of features to boot but in our case it's an overkill. Localshop even has a feature to limit to access to the eggbasket server using cidr! Really nice stuff but then not really what we need.

About the only gripe that I can probably say about pypiserver is that it does not use templates and just prints html (Remember me saying about it being really simple ? Well, I was not in anyway kidding!), but I guess we can slap on a templating app to it and maybe submit it back to the author, if he wants it. So finally, good bye to Christ's eggbasket and hello to our new eggbasket server by Ralf Schmitt!