Time to update my Python on windows

It's time again to play with my Python on my Windows vm and I thought it was high time I updated it since it was running ver. 2.5. First came the easy part, replacing Python itself. I just downloaded and changed the path for python in window's environment variable.

I ran 'easy_install mysql-python' thinking on other tasks ahead. Hmm .. command now found, fudge around with the paths and found that I had to install the setuptools for windows. Installed it, changed the path to point at PATH;C:\Python27\Scripts and boom now my Python has easy_install. Little did I know more fun lies in wait ...

First in trying out 'easy_install mysql-python' I ran into an error saying "error: Setup script exited with error: Unable to find vcvarsall.bat". Looks like my install was still lacking something. After some digging around, it seems that, that error is equivalent of gcc not found on Linux, so I was off to install MingGW.

Install that, downloaded the sources for MySQLdb from sf.net. They used to provide binaries for MySQLdb but it seems now no more. I ran into another error! Google lead me to this answer: http://www.mail-archive.com/pylons-discuss@googlegroups.com/msg11448.html. Good point. So I finally found my answer here... http://www.codegood.com/archives/129. Thanks codegood or Ioannis Lalopoulos, for saving a few more of those pili on my scalp! Now ... to find that win32com!

Equivalent of python's virtualenv

I use Python's virtualenv nearly everyday in my work and was lamenting about the lack of a well made solution on Ruby's side of the fence, that was until I found this great gem: http://rvm.beginrescueend.com. This is the closest I guess Ruby comes to having virtualenv.

I like the elegant way the author of this too approaches installation of rvm short for Ruby Version Manager. It's just a bunch of bash scripts which does some fancy tap dancing with curl but it elegantly works and everything is in your box in a jiffy. I am eagerly installing rvm on my boxes and will report back on my findings.

CSVs, unicodes and BOMs

Nothing like a little project specs to make the ugliness start climbing out of the woodwork. I was tasked to write this parser to massage some data destined to be imported into a database. As I worked with the csv library a few things started to be painfully evident to me.

There was no support for unicode in the csv module, and as I gathered from here, it doesn't look as though there will be anytime in the near future ...

Just tinkered a little around with a the neighbor over the fence to see what they offered ... here are both of them side by side ...

>>> data = reader(open("data.csv","r"))
>>> for item in data:
...     print item
...

 'NZ(\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf\xa1) >Food(\xe9\xa3\x9f\xe5\x93\x81\xe8\xa1\x8c\xe4\xb8\x9a\xe7\xbb\x84)', '\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf\xa1\xe9\xa3\x9f\xe5\x93\x81\xe8\xa1\x8c\xe4\xb8\x9a\xe7\xbb\x84\xe6\x9c\x80\xe7\xbb\x88\xe7\x94\xa8\xe6\x88\xb7', 'TRUE']                          
['NZ', 'NZ(\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf\xa1) >Technical(\xe5\xb7\xa5\xe4\xb8\x9a\xe8\xa1\x8c\xe4\xb8\x9a\xe7\xbb\x84)', '\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf\xa1\xe5\xb7\xa5\xe4\xb8\x9a\xe8\xa1\x8c\xe4\xb8\x9a\xe7\xbb\x84\xe6\x9c\x80\xe7\xbb\x88\xe7\x94\xa8\xe6\x88\xb7', 'TRUE']                     
['NZ', 'NZ(\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf

irb(main):002:0> require 'csv'                                                                      
=> true                                                                                             
irb(main):003:0> CSV.open('data.csv','r') do |row|                                     
irb(main):004:1* puts row                                                                           
irb(main):005:1> end
诺维信) >Technical(工业行业组) >Starch(淀粉糖行业)                                               
诺维信淀粉糖行业最终用户                                                                            
TRUE
...

Seems like ruby has got the unicode out of the box ... hmm wonder how much effort it would take have that unicode support? Those reading my last 2 posts are probably wondering if I pro ruby or something. I am not. I love Python and use it for most of my tasks and I love recent developments in it ... just that there are a few "nigglies" with it. I want it to improve beyond these few little bumps.

That being said I love the functionality I get when I use DictReader from Python's csv module as it let's me address my columns by name. Useful in situations where you have to reorder the columns. Seems that you can do that too with Ruby with a little work.

The next one that got my goat was this thing about BOMs or Byte Order Marks fecal matter left by Excel on the headers. Not really a problem with Python but rather Excel, still had to deal with that. Ended up sanitizing my headers and stripping off the damn BOM. There was a library to do that but I thought that was a bit of an overkill.

Anyway back to the grindstone ....

List comprehension ruby vs python

Been exploring a bit of list comprehension in Python so today I thought I peeked over the fence to see a bit of what is happening at ruby's end..

I really like how they are doing their list comprehension over there, check this example out:

[1,2,3,4,5].select(&:even?).map(|x| x*3)

That is almost readable code even in English! I would read it as something like this.."From the list 1,2,3,4,5 select even numbers and map it to number times 3". Really nice!

This vs the clunkier Python code ...

[x*3 for x in [1,2,3,4,5] if x%2 == 0]

I mean it both works just that it seems more elegant in ruby. Would be nice if we had that even or odd function in Python. Yeah I know it's trivial to write ... but then following that logic wouldn't it also be trivial to include it in the standard library. Probably my way of emulating this example ain't exactly the best and there are other more succint ways of doing it, if so be great to drop me a line here, but for now hang on while I peek over the fence more :)