Lowkster - Pythonly yours

Come on ubuntu release already

2011-10-09T01:54:00.000-07:00

After looking at the new stuff available in unit tests:

http://docs.python.org/library/unittest.html

I am really sold! At a mere glance I can already see two things that would be supremely useful in cleaning up my tests which is:

doCleanups

and

assert*Equal

Especially the assertTupleEqual and assertListEqual is really really nice where in the difference of the two sets are compared and the diff included in the failed message. We are using Ubuntu on all of our tests machines so I really hope that the stock Python that comes with the next version of Ubuntu is the 2.7.x branch. The assertAlmostEqual seems a bit dodgy to me but then again I still haven't really gone in depth with the docs yet.

What would be a nice addition for me would be 'assertTextFileEqual' or something to that end. This would be a great shortcut for me as I do a lot of text file reading and asserting in my tests.

All in all kudos to the changes made in the unit test module of Python and I like the direction that it's heading.

Where is that missing recursive chown for Python

2011-09-22T07:28:00.000-07:00

A question from a colleague of mine prompted me on a week long search for a recursive chown for Python. Initially I just thought that it was due to his inexperience that he was unable to find an answer as a recursive chown sounds too easy of a function for Python not to have. Alas, however after on and off scouring the net I found that the closest (and the best) answer to what he was looking for is actually This stack overflow question. Ruby has their built into the Fileutils library which really prompted this post. Also on a less cursory glance I notice that the ruby standard lib has some functions which I really wished Python has which would make a probably transition from my bunch of work horse bash script to a Ruby platform that much easier. They have cp, cp_r and even a diff! Nice! Has there been any attempt of making this easier in Python? If someone knows better please enlighten me, as this just seems too easy for Python not to have.

Finally! 2.6.38-9 is it!

2011-06-14T18:57:00.000-07:00

It took 9 revisions but as of kernel 2.6.38-9, my arch box can now reboot and halt fine (just need to put 'rmmod e1000e' in /etc/rc.local.shutdown) and I can live with it for now. For the longest time (about 2 months) after upgrading to 2.6.38 my box just would not reboot or poweroff without hanging and I was actually toying around with idea of replacing my Arch with Fedora 15.

The relevant posts:

1. Arch Forums

2. Kernel bugzilla

I wonder what magicks was sprinkled on this latest stable kernel. Wish there was like someway to read changelogs for revisions on the command line just before I do a 'pacman -Syu', or is there really a changelog maintained for the kernel? From the looks on the kernel bugzilla, it doesn't seem to be moving at all at the moment.

systemd for opensuse

2011-06-10T20:53:00.000-07:00

As a guy who use openSuSE daily for my work, I like the direction that openSuSE has been treading in the past and now. I am not that intune with the community but then the direction they always take seem to walk the fine line between bring in the latest and greatest changes and maintaining a stability that allow for the majority of the users being able to use it for work. While guys like Fedora and Arch seems to throw caution to the wind bring in the changes with seemingly little regard for stability, openSuSE in my books have always been successful at the almost impossible task of maintaining a stable environment with the slew of changes coming from all directions. One thing that comes to my mind when writing this is their decision to maintain kde3 together with kde4 when kde4 was born to aid the transition 4.x.

With this in mind I look forward to the systemd integration into OpenSuSE planned for 12.1. Looks like there are a lot of overwhelming wins using systemd compared to the old systems. I was beginning to think that the OpenSuSE camp was a bit quiet compared to Ubuntu opting for upstart and Fedora going the Plymouth way. The only downside I can see for this is that untypical of something that is usually brought into the OpenSuSE, systemd seems to be the youngest of all the projects and I envision a lot of work ahead to stabilize it down for the future. This is something that I am definitely looking forward to in the Roadmap for openSuSE.

Packaging python packages in opensuse

2011-04-12T10:22:00.000-07:00

While getting one of my packages rejected for inclusion into the python lang repo for openSuSE, the comments made by the reviewer pointed me to find the package: py2pack.

This package made the packaging of Python packages on OpenSuSE brain dead easy by doing all of the heavy lifting, creating a sample spec file that really needs very little editing. Install py2pack, and follow this guide, http://en.opensuse.org/openSUSE:Packaging_Python#The_fast_and_automated_way and put that together with the great openSuSE build service and you will have a great time packaging which is usually a thankless job no one wants to do.

Update:

Wow the ruby packaging method is about the same too: http://en.opensuse.org/openSUSE:Packaging_Ruby

opensuse 11.4 a quick disappointed look... part2

2011-03-13T09:40:00.000-07:00

I will just add closure to this story. After the upgrade botched up, I reluctantly tried to reinstall my server leaving the home volume untouched. That turned out to be a pleasant surprise because of a few things:

1. 'import setup' of partitioning. Love that! Allowed me to copy over my old partition settings. Saves a whole lot of pain.

2. Seamless update after that X worked and everything else worked 'out of the box'.

The only draw back probably is to reinstall a few of the application I have running on the box that is not in the stock install. I am going to put back a few of the points I took away the day before when the upgrade fuched up.

So if you are upgrading from 11.3 to 11.4 you might want to use the reinstall path. It might turned out to be better and smoother.

opensuse 11.4 a quick disappointed look...

2011-03-12T19:12:00.000-08:00

On the upfront, openSuSE's upgrade has always impressed me with it's smoothness. While others falter, openSuSE's upgrade process just chugs along and everything is up and running at the end of it. I did the online upgrade with one of my boxes while the other used the DVD method with 11.3 and both went without a hitch.

Along came 11.4 and I did not hesitate based on my past experiences, but boy was I wrong. Right after the upgrade my box booted up to no Xdisplay, no sax2 (wtf ! I know I should have read the release notes, but then no alternatives ?!). The no display I can deal with but the next bit really took me for a doozer. The network did not start up even though it was in runlevel 5. Initially I thought it was because my network card was left un configured but configuration and 6 reboots later did not solve the problem. I resigned myself to a putting 'rcnetwork start' to /etc/rc.d/boot.local.

I did not find other people with the same problem querying uncle google, but for the life of me I can't get it to work. It would just boot up and say that it has reached runlevel 3 or 5 but then I have to manually start up networking using 'rcnetwork start'. These problems really surprised me as openSuSE prior to this has always good to me and upgrades works right from version 10.3 to 11.3 without problems. I guess I got to stop boasting to people how good openSuSE is ... :(

string substitution in list

2010-12-06T20:35:00.000-08:00

While coding the other day ... I came across this suspiciously looking code that almost look like it would work ...

substituted = ["My name is %s","My name is %s", "My name is %s" ] % ["Earl","Matthew","David"]

So used to looking at string substitution, I almost thought that code could work ... of course in order for that to work correctly ... it would have to be something like this ...

strings = ["My name is %s","My name is %s", "My name is %s" ]
names = ["Earl","Matthew","David"]
substituted = [ string % name for string,name in zip(strings,names) ]

It's string substitution and list substitution .. Repeat that 10 times :). It's always good and enlightening to strip right down to the basics most days. Go on run the first part of the code and see what you get.

Time to update my Python on windows

2010-11-27T08:25:00.000-08:00

It's time again to play with my Python on my Windows vm and I thought it was high time I updated it since it was running ver. 2.5. First came the easy part, replacing Python itself. I just downloaded and changed the path for python in window's environment variable.

I ran 'easy_install mysql-python' thinking on other tasks ahead. Hmm .. command now found, fudge around with the paths and found that I had to install the setuptools for windows. Installed it, changed the path to point at PATH;C:\Python27\Scripts and boom now my Python has easy_install. Little did I know more fun lies in wait ...

First in trying out 'easy_install mysql-python' I ran into an error saying "error: Setup script exited with error: Unable to find vcvarsall.bat". Looks like my install was still lacking something. After some digging around, it seems that, that error is equivalent of gcc not found on Linux, so I was off to install MingGW.

Install that, downloaded the sources for MySQLdb from sf.net. They used to provide binaries for MySQLdb but it seems now no more. I ran into another error! Google lead me to this answer: http://www.mail-archive.com/pylons-discuss@googlegroups.com/msg11448.html. Good point. So I finally found my answer here... http://www.codegood.com/archives/129. Thanks codegood or Ioannis Lalopoulos, for saving a few more of those pili on my scalp! Now ... to find that win32com!

Equivalent of python's virtualenv

2010-11-24T23:52:00.000-08:00

I use Python's virtualenv nearly everyday in my work and was lamenting about the lack of a well made solution on Ruby's side of the fence, that was until I found this great gem: http://rvm.beginrescueend.com. This is the closest I guess Ruby comes to having virtualenv.

I like the elegant way the author of this too approaches installation of rvm short for Ruby Version Manager. It's just a bunch of bash scripts which does some fancy tap dancing with curl but it elegantly works and everything is in your box in a jiffy. I am eagerly installing rvm on my boxes and will report back on my findings.

CSVs, unicodes and BOMs

2010-11-07T19:01:00.000-08:00

Nothing like a little project specs to make the ugliness start climbing out of the woodwork. I was tasked to write this parser to massage some data destined to be imported into a database. As I worked with the csv library a few things started to be painfully evident to me.

There was no support for unicode in the csv module, and as I gathered from here, it doesn't look as though there will be anytime in the near future ...

Just tinkered a little around with a the neighbor over the fence to see what they offered ... here are both of them side by side ...

>>> data = reader(open("data.csv","r"))
>>> for item in data:
...     print item
...

 'NZ(\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf\xa1) >Food(\xe9\xa3\x9f\xe5\x93\x81\xe8\xa1\x8c\xe4\xb8\x9a\xe7\xbb\x84)', '\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf\xa1\xe9\xa3\x9f\xe5\x93\x81\xe8\xa1\x8c\xe4\xb8\x9a\xe7\xbb\x84\xe6\x9c\x80\xe7\xbb\x88\xe7\x94\xa8\xe6\x88\xb7', 'TRUE']                          
['NZ', 'NZ(\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf\xa1) >Technical(\xe5\xb7\xa5\xe4\xb8\x9a\xe8\xa1\x8c\xe4\xb8\x9a\xe7\xbb\x84)', '\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf\xa1\xe5\xb7\xa5\xe4\xb8\x9a\xe8\xa1\x8c\xe4\xb8\x9a\xe7\xbb\x84\xe6\x9c\x80\xe7\xbb\x88\xe7\x94\xa8\xe6\x88\xb7', 'TRUE']                     
['NZ', 'NZ(\xe8\xaf\xba\xe7\xbb\xb4\xe4\xbf

irb(main):002:0> require 'csv'                                                                      
=> true                                                                                             
irb(main):003:0> CSV.open('data.csv','r') do |row|                                     
irb(main):004:1* puts row                                                                           
irb(main):005:1> end
诺维信) >Technical(工业行业组) >Starch(淀粉糖行业)                                               
诺维信淀粉糖行业最终用户                                                                            
TRUE
...

Seems like ruby has got the unicode out of the box ... hmm wonder how much effort it would take have that unicode support? Those reading my last 2 posts are probably wondering if I pro ruby or something. I am not. I love Python and use it for most of my tasks and I love recent developments in it ... just that there are a few "nigglies" with it. I want it to improve beyond these few little bumps.

That being said I love the functionality I get when I use DictReader from Python's csv module as it let's me address my columns by name. Useful in situations where you have to reorder the columns. Seems that you can do that too with Ruby with a little work.

The next one that got my goat was this thing about BOMs or Byte Order Marks fecal matter left by Excel on the headers. Not really a problem with Python but rather Excel, still had to deal with that. Ended up sanitizing my headers and stripping off the damn BOM. There was a library to do that but I thought that was a bit of an overkill.

Anyway back to the grindstone ....

List comprehension ruby vs python

2010-11-03T19:12:00.000-07:00

Been exploring a bit of list comprehension in Python so today I thought I peeked over the fence to see a bit of what is happening at ruby's end..

I really like how they are doing their list comprehension over there, check this example out:

[1,2,3,4,5].select(&:even?).map(|x| x*3)

That is almost readable code even in English! I would read it as something like this.."From the list 1,2,3,4,5 select even numbers and map it to number times 3". Really nice!

This vs the clunkier Python code ...

[x*3 for x in [1,2,3,4,5] if x%2 == 0]

I mean it both works just that it seems more elegant in ruby. Would be nice if we had that even or odd function in Python. Yeah I know it's trivial to write ... but then following that logic wouldn't it also be trivial to include it in the standard library. Probably my way of emulating this example ain't exactly the best and there are other more succint ways of doing it, if so be great to drop me a line here, but for now hang on while I peek over the fence more :)

Who's playing catch up now?

2010-10-28T08:55:00.000-07:00

At one time if I remembered correctly, ruby came up with their gems framework first. I tried it out and really liked it as I got to try out various gems without having to worry if my platform had packages for it or not. At that point the poor Python was left in the dust at the other side of the room without anything remote resembling gems or cshell (Perl).

Fast forward a couple of years and now easy_install and virtualenv and these two great apps are so wide spread. I used them so much and got used to using and depending on them that when I found myself returning to gem to try out watir, a framework for web testing, I found gem to be clunky and unwieldy. Kudos to the Python camp! In such a short time you have left gems shaking it's head disbelief in the dust! Who said snakes can't be fast.

For those who are interested, I had problems installing the gem for watir and encountered all sorts of errors and upon searching for the download point for watir to download it by hand, let's just say I missed PyPi a lot.

Python-Flask packages in opensuse

2010-10-26T08:01:00.000-07:00

Been doing some work and playing around with opensuse's great build service and as a result, my packages has been accepted by the opensuse team. So, now pensuse has python-flask and a few related packages in there.

Use the devel:languages:python and enjoy.

Alternatively use the search http://software.opensuse.org/113/en to find what you need.

Blogging awards

2010-08-29T00:06:00.000-07:00

Geeta from packt informed me that Packt is having this contest for opensource blogs. To quote the site with the contest running on it's fifth installment:

"Now in its fifth year, the Award, formerly known as the Open Source Content Management System (CMS) Award, is designed to encourage, support, recognize and reward not only CMSes but a wider range of Open Source projects."

Nominations start on August 9 and ends on September 17, so get your nominations in! You can read the full announcement here.

easy_install how I wish ...

2010-08-12T19:03:00.000-07:00

I use a lot of easy_install during the day to manage my packages. First off the bat let me just get this off my chest, I hesitated like about 10 times before writing this article, for fear of getting panned all the way to Sunday, but then Sunday isn't really that far away so here goes. Also, before that if you are going to pan me or burn my ass for not reading docs at least at the end of it disseminate some useful info at the end of the burning comment so that everybody can benefit from it. God knows I would appreciate it.

I was doing some stuff on Pylons and for some reason Routes-1.12 was screwing stuff up, so I had to downgrade Routes by doing something like:

easy_install 'Routes<1.12' and it did it's work well enough.

Two thoughts itched at my thoughts though ... the whole experience of downgrading Routes felt a little bit too deja-vuish to be comfortable. Did I not do this before the other day? I could have sworn it was working the other day with Routes-1.11. All of this mental itching started a seed of mistrust for my hithero good simple friend easy_install.

Anyways, I just proceeded but now with an eye on my friend in the corner just to check up on what the macha was doing. I tried to find out the version of route that was running from easy_install, hmmm there was no way of doing that! At this point I just hear those magnums and AK-47s from all of you Python elites ready to blow me to kingdom come, as they always say "easy_install is meant for easy installing!" Okay okay, but is it so very hard to write it so that we can do something like:

'easy_install query Routes' ??

Remember pan me at this point if you want if you know that easy_install can do it somehow with some black magicks but at least let everybody know how. In the end I settled for this:

'python -c 'import routes;print routes.__path__'

Sigh ... I know my ass is just going to be hurting after this ....

New Book from Packt

2010-08-11T02:28:00.001-07:00

Guys from packt have been hitting the web quite frequently and have sent me another book to review, this time Object Oriented programming in Python. Always wanted to read about this.

Anyway you can find a peek here:

Chapter No.7: Python Object Oriented Shortcuts

Enjoy!

My easy buildout example Pt. 1

2010-07-10T08:48:00.000-07:00

For a long time I could not make heads or tails of buildouts just that I kept on hearing that it's supposed to be very easy to make packages and distribute them. So finally this week end I sat down, rolled up the sleeves and decided to really try and understand what I was missing.

My aim, create the world's easiest package with absolutely nothing in it. All I wanted to do was just understand what is going on.

Firstly I needed to create my own package, for this I let the great python paste script do all the heavy lifting for me:


[lowks@bobot-mdv-ng pylons]$ ./bin/paster create -t basic_package buildout_example
Selected and implied templates:
  PasteScript#basic_package  A basic setuptools-enabled package

Variables:
  egg:      buildout_example
  package:  buildout_example
  project:  buildout_example
Enter version (Version (like 0.1)) ['']: 0.1
Enter description (One-line description of the package) ['']: Buildout Example
Enter long_description (Multi-line description (in reST)) ['']: Buildout Example
Enter keywords (Space-separated keywords/tags) ['']: buildout example
Enter author (Author name) ['']: Low Kian Seong
Enter author_email (Author email) ['']: kianseong@gmail.com
Enter url (URL of homepage) ['']: www.lowkster.com
Enter license_name (License name) ['']: BSD
Enter zip_safe (True/False: if the package can be distributed as a .zip file) [False]:
Creating template basic_package
Creating directory ./buildout_example
  Recursing into +package+
    Creating ./buildout_example/buildout_example/
    Copying __init__.py to ./buildout_example/buildout_example/__init__.py
  Copying setup.cfg to ./buildout_example/setup.cfg
  Copying setup.py_tmpl to ./buildout_example/setup.py
Running /home/lowks/virtualenv/pylons/bin/python setup.py egg_info

What the paste script will do is it will ask you a bunch of questions where all the answers is used to fill up your setup.py. Your setup.py now will look something like this:

from setuptools import setup, find_packages
import sys, os

version = '0.1'

setup(name='buildout_example',
      version=version,
      description="Buildout Example",
      long_description="""\
Buildout Example""",
      classifiers=[], # Get strings from http://pypi.python.org/pypi?%3Aaction=list_classifiers
      keywords='buildout example',
      author='Low Kian Seong',
      author_email='kianseong@gmail.com',
      url='www.lowkster.com',
      license='BSD',
      packages=find_packages(exclude=['ez_setup', 'examples', 'tests']),
      include_package_data=True,
      zip_safe=False,
      install_requires=[
          # -*- Extra requirements: -*-
      ],
      entry_points="""
      # -*- Entry points: -*-
      """,
      )

Wow! How did that just happen ?? Well the paster script as promised did all the heavy lifting and filled up the setup.py with all the input you just put in just now. Isn't that magical ?

If you are doing in a virtualenv, do a 'easy_install zc.buildout' to get started on getting all the buildout stuff. Remember that you will definitely find other guides somewhere else to do this. I am just writing this down as an exercise and a reminder to myself.

After you finish installing zc.buildout, to sprinkle the buildout magic in your current source directory, cd into the source directory which for this example would be the buildout_example and then all you have to do is run '../bin/builout init'. It is really really easy!

[lowks@bobot-mdv-ng buildout_example]$ ../bin/buildout init
Creating '/home/lowks/virtualenv/pylons/buildout_example/buildout.cfg'.
Creating directory '/home/lowks/virtualenv/pylons/buildout_example/bin'.
Creating directory '/home/lowks/virtualenv/pylons/buildout_example/parts'.
Creating directory '/home/lowks/virtualenv/pylons/buildout_example/develop-eggs'.
Generated script '/home/lowks/virtualenv/pylons/buildout_example/bin/buildout'.

That's almost it! Thanks to your fairy god mother buildout script, you will find a bunch stuff already created. Most of the guides I read was too heavy as it tried to get too much stuff done now ... so I am going to cover more in the second part. For now just pat your back in that you are 3/4 done! Go have a great cup of puerh tea!

For the next part we will see what each part in the buildout.cfg and at the end we will see how to finish the buildout process plus create a section for nosetests.

Did you know?

2010-06-27T00:40:00.000-07:00

There are some of my overseas friends that are genuinely surprised when they visit my homeland Malaysia, to find that people are actually civilized, not swinging from tree to tree in loin cloths. To these people, with a sense of pride I even venture to tell them "We even have an healthy OpenSource movement !". That being said, this is not a posting about pride.

As I was driving back home I thought about the point of this post, what I wanted to say. I guess I just wanted to let my other Python and OpenSource audience or brothers on the other side of the globe know what is the state of acceptance of Python and other OpenSource technologies in most of the corporate sectors over here. I summarized this list after talking and being turned down by a few corporate companies recently for employment. Here goes ...

Did you know ...

Most of the corporate customers, banking sectors, basically the people that matters still think that most of the OpenSource technologies such as Python, Ruby, MySQL is unsecure and reject it based on that? "unstable" is the word they use. PhP which runs probably the majority of the websites in the world is shunned for security reasons while they force their standards of using Java for all of their internal applications!
While some banks accept that they cannot be a total island and reject OpenSource technologies, they require all components you use in a project to be listed down. I don't know why, probably they want to scrutinize the list and make sure that everything is secure before accepting any "unstable" application. This vs. the fact that M$ Products all around that are used all around their establishment which do not even reveal any of the components used in their products!
All of these guys complain about the fact that languages like Python should not be used based on the fact that it's not officially supported and at the same time accept Java? Ermmm .... I do hope that Oracle signed an agreement with Sun before buying it that it will never close down support for Java even if it gets unprofitable.
Almost all of the major financial institutions that I know of that reject OpenSource use products based on OpenSource such as F5, HP and Bluecoat to run their daily operations. Should I be afraid now?

It is my hope that since this blog has a audience of somewhat more than 3 people that the CEO or the decision makers of these institutions chance upon this blog and realize how flamingly stupid their decision for rejecting OpenSource based on these FUD is. At least if you want to so damn secure then you should be fair and insists that all of your products such as M$ products reveal all of their components and source code so that you can audit it (If you really have so much resources to audit all of the things you use). When you succeed in doing that then the good thing is after that you can set up your own freaking software company. Come on, meet me half way here I would really like to believe that the people which I trust to take care of my money at least know their mouth from their asshole when it comes to the apps they are using to run their daily operations!

For No. 4, don't you think that since these Products use the same OpenSource tools that you rejected, that they would also be unstable or unsecure? When will you learn that it's not the tool you use that makes the app unstable, IT'S HOW YOU BUILD OR PROGRAM IT????? Then based on this logic all of those applications should be rejected too right? Why don't you?

After thinking about it for some time, I decided to start working with some local Universities to educate their students by offering my time to do small projects. I am meeting up with a lot of resistance but I am optimistic. Although my small ship is sailing alongside behemoths like M$ who sponsor everything from labs to the tissue our snot nosed university graduates use to wipe their nose with, the acceptance is growing. More of these students are and have heard about OpenSource technologies at least and a few of the major education institutions have started to accept OpenSource technologies.

To change the whole apocalyptic outlook for being an "OpenSource consultant" now, fresh graduates will need jobs waiting for them. Show them that it pays to pick up this knowledge or study and they will. Now, the few misfits that dare to venture down the OpenSource hole, always have this unemployment cloud hanging over their heads when they graduate. Herein lies a problem. If this was to go right, these students who invest their time and energies needs jobs, good high paying jobs when they go out and dare to put the current deal breaker word "OpenSource" on their resumes. For that to happen, we need some MNCs out there to start realizing that the space between their ears would be better employed to think with instead of the hole where excrement comes out from. I really hope that this slap on the face really lands on the right audience as I really hope that one day Malaysia too can come out of this intellect and choice stranglehold that some proprietary vendors think of putting us in.

I would like one day to be able to proudly exclaim "This app was built using OpenSource technologies" like the way it should be instead of having to hide behind generalizations or something out right lying.

Mandriva pt 2

2010-06-25T08:29:00.000-07:00

Another little thing that Mandriva paid attention to that makes me like it that much more. After reading the comments to the earlier post, I enabled the backports and installed chrome from the backports, checked the flash plugin out of the box and guess what? It just works!

Great job mandrivians! It's small touches like this that shows you care and make repeat customers (like me) as well as new ones! Btw, I am still trying out the multitude of guides on how to get flash working on Fedora 13 x64 without success.

Moving to mandriva?!

2010-06-23T00:59:00.000-07:00

Never thought I would be writing this, but I am actually considering moving my Linux boxes (based mainly in OpenSuSE and Fedoras) to Mandriva! During my hey days we always considered Mandrake (it's name then) something of a play distribution or a distribution for Linux beginners and should not be used for 'serious' work.

Fast forward about 10 years, after a name change I don't know why suddenly I felt an urge to pick up the latest version of Mandriva 2010 to give it the proverbial spin. After playing around with the bought version of Mandriva (yes I actually bought a copy!). I really started liking it. Let me just list down a few of the reasons why.

There is a wealth of packages. I was not left wanting for any specific packages. Most of the packages I needed was there.
For a rpm based distribution the package manager is blindingly fast and contains a couple of things done right I wish was in zypper and yum. One of them being the ability to pass as an argument the bandwidth that is to be consumed by urmpi (their package manager) during updates or installation. This I found to be very useful in my old office who was sharing bandwidth with another company! They also do some smart things to speed things up during updates such as by default not downloading the description of patches unless needed (when clicked on)
The network manager is cool. I like how Mandriva does their network manager. It's independant of the DE. For example, if I am logged into XFCE and wanted to move over to KDE or some other DE, and I am already connected wirelessly, after logging out and in, I am still connected because the network manager is separate from the DE. Oh! How I wish the other distros would do this simple little thing!
Their look and feel is quite standard across all DEs and WMs. Now the one small bug in the ointment is that I found that LXDE, the new favorite new kid on the block does not work correctly.

I am trying my best to find fault with Mandriva 2010 and can find little (The 'little' here would be it does not have a package for chromium-browser!), so I am seriously considering moving over to Mandriva from what I am using now. Just downloaded the Free version and am thinking hard why should I be keeping Fedora 13 on my laptop now, though truth be told Fedora 13 does a lot of things right too and after the little debacle of getting flash-plugin working correctly is sitting happily on my X61 now. Ah ... decisions, decisions!

New Book review Django- Django E-Commerce by Jesse Legg

2010-06-22T10:36:00.000-07:00

Folks at Packt sent me another book to review which is Django E-Commerce. Right off the bat I got some comments regarding how relevant this book might be with so many applications based on Django that do ecommerce floating out there.

For me though, I find this book is suited for those people who want to use Django building an Ecommerce site by using the tools provided by a default installation of Django. It also gives insight into how the ecommerce tools are built. It starts from the best place possible which is from a working example.

Each section is explained carefully from the overview to the technical implementation of the module. This is very useful and I learnt a thing or two reading most of the explanation here.

The payment process here employs the 'google checkout' processor to process the payment. What is covered here is also the use of one of Django 'secret sauce' which is views and each section and code is explained. The steps on how to lift a normal Django component to enterprise level functionality is explained starting from using the most basic way of implementation. This approach is taken in explaining the search module and it's great! I like it this way as it's a progression approach and programmers will learn why code and approaches evolve the way they do. Great way of learning! Here too the author explains and shows how to use some of the most famous Django plugins to achieve what we want to do.

Using Sphinx for an E-Commerce site is also used here and the value here is that the example can be easily expanded to create more complex examples or sites. Later in the reporting section integration to salesforce is shown. I really like this as these examples are practical examples that can be used to create a real E-Commerce site that works and avoid walking too much on the academic side of things. This is the main strength of this book. Amazon, JScript and a lot of other tools are also shown.

I will definitely be keeping this book handy on my book shelf for reference in building an E-Commerce app and I highly recommend it.

Check out the book here: http://www.packtpub.com/django-1-2-e-commerce-build-powerful-applications/book?utm_source=blog.lowkster.com&utm_medium=bookrev&utm_content=blog&utm_campaign=mdb_003437

A Linux type?

2010-05-22T10:31:00.000-07:00

As part of what we do, we help business people migrate over from their usually rotten Windows XP, Vista and what ever else incarnation of Redmond OS over to Linux. We found that some people really took to Linux while some got stuck in purgatory complaining constantly in the transition period. I found that certain types of personality as well as expectations makes the whole process smooth and dare I say even enjoyable?

I found that the more open the new user is to a new OS, being more open to forget about Windows they are the more enjoyable the process is. "This used to work so much better on my laptop in Windows ..." kinda attitude will almost guarantee a world of pain.
Having a positive attitude helps ... as in you are home ... your printer does not work instantly ... instead of just chalking it up to the new OS being bad just give it a whirl and think of the fun of trying out something new will make it enjoyable.
Trusting your new OS as well as your new found friend (me) when it comes to buying peripherals will help during the transition.

Those with the attitude above really took to Linux like fish to water and it sorta became their bragging rights in among their friends. Remember too that these are pure business people without any prior Linux experience. So it's gratifying to me to over hear them in conversation with their friends "Aiya ... you are still using that slow piece of crap .. see my new Open Source OS! No virus can touch it and it's blazing fast!" It's nice to know we did one for Linux advocacy in our own way as well as fill our pockets up a bit :).

Another book to review! (Django E-Commerce)

2010-05-20T06:58:00.000-07:00

The nice folks at Packt has sent me another book to review. This time the book is Django 1.2 E-Commerce. The sample title looks great which includes setting up an e-commerce site in 30 mins using the usual suspects from the Django framework: Admin, generic views and using Checkout from google. The example looks practical and easy to follow ... makes me like this book already. For those of you who want to get a whiff of what this book is like ... here is a link to the sample chapter.

https://www.packtpub.com/sites/default/files/7009-chapter-2-setting-up-shop-in-30-minutes_0.pdf

Repoze BFG

2010-05-05T01:35:00.000-07:00

I have been trying repoze BFG's tutorials the past few days and I must say that I like it just judging from the tutorials. Firstly I was very surprised the tutorials worked out of the box ... after being hammered around by the tutorials from Grok, so it was a nice surprise. Looking at the examples too it doesn't make my eyes bleed _so_much_ so I am definitely interested to know more about Repoze.

Developed by http://agendaless.com/ , on the whole the project has a good solid feel to it offering tons of documentations on it's site. Of course you cannot run that far away from the mind-melting Zope framework but then hey! trust me, it looks a whole lot better! It says that Repoze is inspired by Zope, Pylons and Django and I would say it shows in the code produced.

According to the site, Repoze is MV rather than MVC as there is not much of a 'Controller' to talk about. I am currently actively looking for a project where I can really put Repoze to the metal to see how it fits me in a project environment.

As someone said long time ago to me ... all frameworks suffer from the same disease ... it can really do the 90% fast and great but it's the last 10% that is going to blast the skin from your bones ... okay so I am paraphrasing, but you get my point. So, I am going to put this one through the paces hopefully initially with a smallish project that is not so heavy with the funky specs with just one or two bumps on the road to stretch the framework a bit.

PyCon APAC

2010-04-15T02:03:00.001-07:00

Well they say that a picture paints a thousand words ... so here we go

I think I might go this year just to see how it is.

Update on grok web development tryout

2010-04-12T22:59:00.000-07:00

After dancing around the pale moonlight with a newly slaughtered chicken in my birthday suit, offering fruits to the grok gods, finally today the gods were kind on me and helped me along to find this solution that works! The closest I got the example in the grok book to work is to install the grokproject and start a new project without errors, but when I try to start up the devel server, all sorts of strange manner of monsters jumped out at me.

Just have a look at this : archive.netbsd.se What needs to be done is to edit the paster template and everything will work fine. I guess either this bug has not been looked into or is still an open bug.

Heh! I spoke too soon, I got to the main page of grok management but as soon as I try to do anything, I get this:

"TypeError: character mapping must return integer, None or unicode"

Hmmm ... google gods say this:

Not really a solution

Not much of a solution it seems, just that Python-2.6 is not supported. I am using grokproject, but how can I see what version of grok is grokproject using. As the mail says, errors like this is really really off putting and does not help with building confidence with the project!

Grok 1.0 Web Development

Review of Grok Web Development 1.0 Part 1

2010-04-05T00:04:00.000-07:00

I like Zope/Plone. I repeat I really like Zope and Plone. Though here in this blog I might sometimes balk at the monolithic-ness of the 2.x era of them, fundamentally I like the concepts that they put forth. When things like Django, Turbogears rolled around thanks to Ruby On Rails, the call for Zope to recreate itself was sounded and did they respond! Zope3 was almost a ground up rewrite. At that point I had moved on to Django for one of my projects but I looked back longingly sometimes at the useful components put out by Zope. Then someone told me about Grok. I looked at it and liked it alot (okay maybe I still have reservations about Zodb) and it looked to me very much similar to how Django and others were playing the game, except that Grok came to the party with very advanced clubs and spears inherited from it's day in Zope-2.x! So when I got the chance to review the Grok book from Packt, I jumped at it. I wanted to find out what is the innards of Grok and how to or can I deploy an enterprise project using Grok.

The review of this book got very much delayed due to the fact that grok, like all other Zope stuff needed Python2.5 or Python2.4 to get working correctly. Now in this day and age of every distro racing to be the latest, bestest and most up to date, it was a challenge for me to get a Python2.5 or Python2.4 easily. I ended up going the VMWare route and installing a copy of Centos5.4 which still spots Python2.4.3 by default. This probably would be the biggest stumbling block of any newbie trying out the book the first time. The examples all require you to have a copy of Python2.5 which can be a challenge to find or install right. Once I had the components, I started on my way by using easy_install and then virtualenv which is the Python Virtual Environment. Again, these would require a little fore knowledge of Python stuff so this book might not be totally for the newbie programmer who don't know anything about the Python programming language. A bit of introduction to Grok, as far as I can remember. Grok came about when there was a little discord in the community about the monolithic-ness of Zope-2.x, when light frameworks such as Django, Turbo Gears and others started coming on to the scene. You can say that the Grok project was created to allow programmers or fans of Zope to enjoy a modular development environment offering the best of Zope where programmers can now pick and choose pieces of it can apply it to their work. I must say that the pieces of Zope when good is truly good and most often than not I find myself recreating some of the functionality of Zope in my projects using other frameworks. What could have been done is to release a VMware image of a working environment to work on. Definitely a boon in terms of speed up of trying out the examples!

The book starts out the example by using Virtualenv (Python's virtual environment) and easy_install which is a good move as Grok's packages are not readily available and probably installing it would not be the easiest thing to do in the programming world. Getting a relative newbie caught up in installation of Grok when all they want to do is to bring it for a test drive would be painful to say the least. I tried lots of environment including Ubuntu which resulted in errors preventing the example from working until at last I found Centos 64 bit which worked with the examples. The process of finding this out was maddening! Perhaps grok is still under development but what I suspect is that most of grok's dependencies or libraries is still playing catch up with the example in the book. This might be off putting to a Grok newbie venturing into the land of Grok development. The examples were the most enjoyable part of the book where the beginner could feel that they were starting a project from scratch.

The admin tab in the applications for Grok is touched here and covered in passing highlighting all the main features of all the tabs. In the views section, those who are new to Zope Page Templates and the TAL (Template Attribute Language) will get a nice and concise introduction to some of the functionalities of using ZPT with Grok. Those who are Zope / Plone developers could probably skip this section without too much difficultly unless you are like me who want a refresher's course on ZPT. This deep into the book I really like the examples thrown my way and it's really refreshing for me to see that one of my favorite Python application have reinvented themselves and this bodes well for it.

Catch more of this review in part 2....

Getting mtp device (Sony Walkman) to automount in mandriva 2010 powerpack

2010-04-03T10:36:00.001-07:00

Currently I am running Mandriva 2010 powerpack on my laptop. Everything worked correctly and great and there are a lot of things I like about Mandriva. Today while trying to get my Sony walkman to automount as a usb device it just did not work automagically like how it did in my OpenSuSE. I debugged it a bit and found that it was recognized as a usb device but it just did not mounted automatically (I use KDE). Shot up nautilus and everything worked just great.

Being in a frisky mode today, I was undaunted and continued to search around for the solution. Found the solution in a Mandriva forum here:

MTP Solved Forum

A bit outdated yes but I deciphered what I could from there, and found that the solution is to install mtpfs. Just do a 'urpmi mtpfs' as root and your Sony Walkman should automount automagically from thereon after and everything will be hunky dory on your Mandriva baby.

Btw, I am also quite impressed with the number of packages offered in Mandriva. Nearly all of the Python packages I am used to is in here and the package installer is space age fast!

Sneak peak preview of upcoming book

2010-02-22T03:53:00.000-08:00

My friends at packt has sent another book for me to review! The Grok-1.0 is up and I have been looking for this as I have been looking for just an excuse to get my elbows dirty up to Python oil again!

You can see a preview of the book here:

Preview

Even better a sample chapter here:

Chapter 5

This is going to be good!

Python training in Malaysia

2009-12-17T20:26:00.001-08:00

A guy contacted me recently regarding training. Me tried of course to look for local trainers here but some were either too expensive or just strangely did not reply emails. So it boiled down to having to do it myself. My junior programmer (or should I say sys admin since he said he wants nothing to do with programming) asked essentially can we survive being a Python trainer in Malaysia. Sadly, the answer is still no. Training here are few and far apart. After hearing my answer, his face showed a kind of demeanor that your neighbor would have if you drove up the drive way with your broken down Toyota claiming that this is dream car.

He must have been thinking that Java as a language is still better than Python since he can at least see some Java jobs on jobstreet and stuff. Generally that is still the whole conception of Python in Malaysia. Talks do help. We just need more. At least nowadays when I talk to corporates about Python they don't ask me why I suddenly bring wildlife into a conversation when they asked me what would the ideal programming language that their IT department should adopt for high productivity. I should hit the university level more to create more awareness. I have thought classes for corporates who were full of Python doubters and at the end of the class, all the Java people walked out at least respecting Python for what it can do and actually contemplating using Python in their environment.

It's really gratifying to see programmers who were initially so skeptical about Python suddenly light up to the possibilities of using Python for their work after being in the Python training with me. That to me is priceless and I wish that I could do more of these kind of Python gigs.

I think that awareness is the key for us Pythonistas to be able to survive doing the thing that we love so much here. I think that here at least if more people knew about the existence of Python it would really help and then I can finally tell that doubting junior system administrator ... "Yes! There is a future being a Python trainer!"

Project Documentation

2009-12-03T00:53:00.000-08:00

Currently I am in the process of passing down my project to some juniors and I find the need to document it. Me being the lazy ass programmer that I am obviously started to look for something that would allow me to complete my documentation in as least steps and most painless way as possible. Found two main applications that I am using now:

1. Sphinx -> For main documenting work.

2. Happydoc -> For parsing and generating code documentation.

Both are quite easy to use and allows documentation to be generated in various format, which includes latex, html or pdf. Nice thing is both supports rest type reStructured text formatting which I just absolutely adore and is great to work with. I am especially impressed with Sphinx and the beautiful documents generated by Sphinx, coming with it's own search and everything. It's just what the lazy programmer doctor ordered. I am still evaluating both of these programs and am still open to other choices but so far these fit the bill nicely.

django-evolution aka orm pain part 2

2009-11-11T17:48:00.000-08:00

Finally bit the bullet and tried out a new application to help me do the changes I needed to do on the db and boy am I glad I did. django-evolution is a joy to use. It's easy to install with no strange dependencies and it did everything out of the box as described. Initially somethings did trip me up such as the django-evolution's initialization actually looks only at the models.py and does not actually bother what is really in the db, as in say the models.py you define a field that is not actually in the db during the first 'python manage.py syncdb' to create the django-evolution table, it does not actually know that and will just carry on not knowing that there are discrepancies between the models.py and the actual db.

After I figured that part out, it was easy. Just add the fields and their attributes, re-run an update script on my data and everything is done. Now my selects are much faster. Initially a 3k row db select was taking 50s and now using the caching functionality of select_related() it comes back in a blazing 4s. Ah here too initially I got tripped a bit as after putting in all the relationship I noticed that it still did not do a cache select_related until I did some digging and found that all columns defined with 'null=True' is not cached by select_related(). Looking at the actual sql generated by the Django CRM, it looks proper and very well done. I am still checking out the little nooks and crannies of the application but so far I give django-evolution the thumbs up! I am actually looking at the code to see if during the initial syncdb django-evolution can actually peer into the db to see if the definitions are actually kosher before proceeding.

When you play outside of django's ORM ... you get pain!

2009-11-09T00:53:00.000-08:00

For one of my projects, I forgot to implement the relationship of one my models. I thought I could get away with it by writing a tag that made a query to the db upon being fed a string in the report. To my horror, this caused the report to take about 50seconds to generate for a paltry 3000 record db! Upon tailing the logs of query.log, I found that the multitude of query to the db was causing the problem. It was basically hitting the db about 3000+ times for a single page report. That is when I found out the good thing about Django's select_related(). The problem now is, I do not have that relationship and need to build that relationship into the db. Not exactly fun with a db that is already populated with data. Led me to thinking, while Django's ORM might be great for stuff like calling data from another table, but would it also incur a performance hit everytime data is being queried. I have heard Django being used for high traffic sites but then are these high traffic sites?

I tried looking around but I could not find a satisfactory answer to this: "What if you wanted to model data for an existing MyISAM table which does not have the relationship?" It would be nice if Django provided a mechanism to recreate the relationship between MyISAM tables that is implemented at the MiddleWare. Say if I were to go the Django way now, would I have to dump out the data, re-implement the tables in a 'Django-ic' model then reimport back my data?

Currently I am feeling that in order to play in Django ORM's park. I am being forced to play by it's rules. If I opt to write my sqls raw then the ORM takes back it's ball and refuses to let me even venture into it's side of the park and I am left to recreate back a lot of the convenient functionality that was provided by the ORM layer. There doesn't seem to be a nice middle ground and that just sucks.

Patching Fun - Update

2009-09-18T09:03:00.000-07:00

Looking around for a solution for a cleaner way of patching up my code on the Windows server, without installing the unixtools for windows, I eventually arrived at the python-patch tool. This nice little (less than 600 lines) tool allows you to have patch like capability and all deliciously packed in a Python script. Thanks techtonik!

So now I can just packup the patch.py Python script together with my patch file, and Walla! the admin at the other side can just run something like:

python patch.py diff_patch

to patch up my code. With a little modification, the patch.py can even generate a results.log for me to review the patch process. If anything were to go south, I can just send the reverse patch.

Now .... should I just get the script to autorun .... hmmm...

Patching fun (on windows) 101

2009-09-17T12:18:00.000-07:00

I have sent about 100 odd patches to a system I am working on and in this post I gather together some of the lessons I have gathered along this path. I write all of my patches in Python and after much hair pulling sessions over the phone with customers am happy to say the sacrificed hair did go along to help me have a epiphany of what would be the best way to help customers who are willing to help us apply patches along. You can say it's a collection of what to do and what not to do tested in real life project scenarios.

First off and of ultimate importance is to get into the right frame of mind before you write a patch to be sent to a customer. The frame of mind you should be in is ... remember that paying customers never ever want to help you apply patches so if you run across one that is willing (like in my case), thank Allah, Buddha, the Universe or whatever deity you pray to for your lucky break! Your (paying) customer does not owe you anything, in fact you are in their debt if they even entertain your patch request. Get in this frame and your chances of writing a successful patch will be increased two fold.

Okay, here are some dos for writing patches. These steps below are for systems that are not accessible remotely and a bit physically too far or demanding for your to travel to.

1. Always recreate (as much as possible) the environment in which to run the patch. This means that if your customer's site has a G Drive (windows) your machine in which you are creating your patch better have that. I have had some great time tracking down why a patch does not work when it magically died while trying to open up a log file on Drive G in a machine that only has C and D drives. This includes in the case of windows of using the same binaries of libraries that your code belongs to. Just recently I had a good time on the phone with a customer who claims that come functionality was not working whereas using we were both using the same script! He insisted that my script was not working while I kept on suspecting that the problem lies between the computer and the chair. Finally I found the problem was due to an outdated binary build that he was using. If having the same build is no longer possible, at least go through the changelog of whatever library you are using to get prepared for whatever problems that might arise.

2. Always have backups of each of your patching steps. In fact it's best to include a sort of un-patch patch just incase your patch goes south. The steps to backup the important files should be done by your script and should not depend on your customer. So something like 'Please backup and replace file bla-bla" is not recommended because how do you know what your customer is backing up the original file to. Here you are thinking that they backed it up to something like .bak and they backed it up to .bak. A lot of fun will ensue in the case if you have to write a revert script later. Step no. 2 can be summarized as your customer should do as least as possible in the patching process short of running a patch script.

3. Generate error and result log of your patch process and ask the good will of your customers to send those logs back to you. This will help in determining if your patch process is successful.

4. Describe in your email what should the customer see in the event that your script is successfully run. I find here the quickest and surest way is to include a screenshot of what they should see. Typing out the results is error prone and it's hard to describe what they should see plus do you think your customers got that much time to read a long drawn out email?

5. Test, test, test and test each of your steps at least 4 times before sending the patch out. No testing is too much in this kind of scenario especially if you are dealing with live data.

6. Always include a way in which to identify what build are you customers currently running. This could be as simple as a BUILD text file indicating what BUILD are they currently running or in the case of a web application, a build number nicely tucked away at the top left hand or right hand corner of the screen. Everyone of your patches should update this BUILD number. This is instrumental if you were on the phone with your customer and want them to help you identify how updated are their source code.

Hopefully this post will help me remember the things that I did right and those that I painfully learned not to do again. It would be great too if it can help you avoid those pot holes that have claimed the lives of a few of those hair on my head. Happy patching!

import scripts using python

2009-09-07T11:48:00.000-07:00

Today I finished another import script using python. The import medium is from Excel to MySQL db. I began my importing exploits using Python and have used it ever since. After doing so many import scripts using Python as my main language I could not think of using anything else. While some other people might be using Bash, I have been using Python ever since the beginning I got my feet wet in doing data migration for systems. Some of the reasons why I like Python for this task are:

1. The wealth of libraries available. I particularly like the csv module which I use heavily in my imports the DictReader module in the csv module also makes short of a lot work during imports. I just cannot imagine using something else.

2. The dicts, the lists and tuples are indispensable and great little tools in sifting through data.

3. Python's clean and structured program allows me to pass the script to juniors or allows me to look back at the import script to either use it at another location or improve it 3 months down the line.

Then the question is why don't more people write import scripts in python? Well I can think of a few cons when it comes to using Python in import scripts and these are:

1. For old ass machines such as old style Solaris or HP/UX Python is just not installed by default. While you can install a copy of python on these boxes, sometimes you do not have that liberty and besides the first rule of any importing job is not to add more applications on the system you are trying to import data into.

Hmm I think I can only think of one valid reason why using Python would not give you and edge. I am using Python till this day importing all my data and have not faced any issues yet. How is your experience ?

My Personal take Python 31 on Windows

2009-08-30T04:38:00.000-07:00

Downloaded and installed Python 3.1 on Windows to kick around the tyres a bit on the new version of Python. Currently I am getting myself used to one of the biggest changes where print is now a function. This means that stuff like this:

print "testing"

will not work anymore. This will now give you an error :


>>> print "testing"
  File "", line 1
    print "testing"
                  ^
SyntaxError: invalid syntax

The help function has also been made a function. This means that ....

help will not work too and will just end up ...



>>> help print()
  File "", line 1
    help print()

Now you will have to put up the parentheses like so ... help(print)

The map function will also not work like it used to. I am still trying to find out how it works.



>>> def print_me(line):
...     print (line)
...
>>> print_me("stuff")
stuff
>>> a = ["print","me","now"]
>>> map(print_me,a)
map object at 0x01765CD0

These means too most of my code would be borked .. sigh! Stuff that I have been using for my client's code like Django and Zope will need to be updated too. This is all I have explored for now. Will write more as I encounter more.

Update: I updated the title a bit after the first comment. Tsk tsk some people can so be so uptight these days.

Django 1.1

2009-07-27T21:12:00.001-07:00

Django 1.1 is nearing a release and from what I can see about the new features the two most exciting one (to me at least) are improvements from the models end that is the "managed" model and the "proxy" models. Managed models would be good for existing tables that should not be managed by django. This is useful in some of my projects in which I am using django for database work but do not necessarily need django to manage it. Now if only the database evolve stuff or database versioning stuff: http://paltman.com/2008/jul/03/managing-database-changes-in-django/ could be integrated into the core, now that would be superb!

Talking about migration too, is this project: http://code.google.com/p/dmigrate/ still alive. Doesn't look like there are any downloadable stuff. The examples are half of what I need what I further need is the ability to undo db migration changes between revisions.

ModelAdmin readonly fields

2009-06-20T11:58:00.000-07:00

Finalizing my web application, there was a few outstanding patches left to do. One of this I always felt strange why it was left out in the open. Django's admin interface form by default allows you to edit the primary key of the model. I always felt strange why it allowed this as it led to unexpected behavior. My users would edit the primary key expecting the changes to be done on the record which of course did not happen as django created another new object with the new edited id.

In solving this bug I was hoping that django's admin model allowed for something like 'readonly = ( 'id_field')' or something like to that respect. Just thinking out aloud but shouldn't this be automatically done for all primary key fields in models.py ? Shouldn't the primary key be made non editable by default unless explicitly stated in the definition? Even then should the primary key be allowed to be edited? If a user were to be allowed to edit the primary key, would it really be editing or 'cloning' a new object with similar data but differing ids? I haven't really done a thorough search yet but I think by default all the fields is open to edit in the current django's ModelAdmin. Of course you hide the field away by chucking the field in the exclude tuple in your model's admin declaration but what if the user wanted to see that field? I was actually quite surprised when a declaration of fields that should be readonly in my ModelsAdmin did nothing. That was the first way I thought of doing it anyway.

The best would be a new tuple called readonly that would declare fields that are readonly in the change_form. To cut a long story short, I found someone had already created a snippet for it here. Works perfectly and it's exactly what I was looking for. The questions that remain however are:

1. Why isn't this in the core?
2. Should you actually allow primary key fields to be edited ?!

Ownership and responsibility

2009-06-16T00:21:00.000-07:00

One thing that most smaller companies or any company for that matter really treasures are employees who take ownership of what they do. This is because employees that take ownership often will go the extra mile for a project because they feel that they own the project. In my opinion this is hard to create in a employer-employee based company because of the inherent weakness of this structure. Employees are normally paid monthly. Whether they take on more or less projects, their salary will still come in, so the less they do the better off they are. Taking on the responsibility of a project will only give them more headache and more to answer to.

The boss who wants to create this kind of situation must be brave enough to take the leap into somewhat unchartered region. Profit share. You want partners start treating your employees like partners and I am sure they will rise up to the challenge. Trust them and allow them to make mistakes. Mistakes are the single most important asset for a small rising star. They are your mistakes and they are there to show you the way. Learn from them and cease making them mistakes. Mistakes are good. Expect to make many of them. The more you make the higher the chances of success. You know in the beginning your staff will screw up, except their work anyway making as least modifications as possible. Asking your staff to make a 10 page writeup only to at last use the last 2 sentences will be a killer on morale and will create the "I-don't-give-a-shit" employees. Let them make mistakes and take a chance with them. They will appreciate it.

Follow through on your words. Expecting people to own a project without any compensation or carrot will just create a situation where your people will feel exploited and words will sound hollow. Who in their right minds want to take on extra work for nothing? Bosses or companies owners who realize this early will be able to tap into the entrepreneur side of their employees and create a situation where their employees will naturally take on extra responsibilities without fuss. As the owner or the boss you main job is to step out of the way of your people as they do their work offering your support along the way. At the end of the day if your people feel that they have achieved something with the least help from you as possible then you are successful in creating people who take on ownership of projects and you in the would have gained leverage.

Google Docs Office killer Not?!

2009-06-11T05:35:00.001-07:00

Those who know me a bit might know that I sell Google Apps as one of the many interest in business. Lately however I find that I cannot seriously sell Google Docs and expect them to use google docs to replace Microsoft Office. Google docs might just be good for doing simple documents and nothing more than that.

I say that for a few reasons:

1. Documents cannot exceed 50k. Now if an organization were to depend on Google docs entirely how is it able to do some serious document work if it has this limitation?

2. For simple formatting google docs is superb. In fact I like it so much I tried living of it exclusively for a little while until I found that for real work like doing quotes and invoicing you had to know html to edit the templates. Now how can you expect a company's secretary or the admin staff to know how to edit html?! Fail! Complete utter fail!

3. There is no way to password protect a document easily. Now I do not need comments telling with a webpage detailing the hows of doing this across a few pages of hacking. The point here is that it cannot be done by customers so it's a fail.

Until these issues are solved or at least worked around, google don't even expect to be able to dislodge Microsoft Office. Infact, looking at the technical challenges that you have to surmount, I am beginning to think that it might not even be worth the fight.

As for now I will continue to sell my services configuring Google Apps but I will concentrate more on pushing the email and the Calendaring.

Django CRM?

2009-06-10T02:28:00.001-07:00

Few of our customer came up asking for some crm functionality from us. Quite a few actually. These are not full blown sugarcrm kind of requirements. Actually they are very simple functionality. Looking at these requirements, it made me think that going for something like sugar would be like taking three steps back to gain a step forward. I would have to spend my time editting out functionality and it would be a dumbing down process all over again. Rather than go this route I thought it would be nice to probably use Django to build them this.

Looking around here are some of the links I found relating Django to CRMs:

http://www.tschitschereengreen.com/blog/index.php/2007/10/15/django-based-crm/
http://code.google.com/p/django-projects-crm/
https://launchpad.net/django-cream/+download

Seems that most of them are either abandoned or not very usable. Sigh ...

That's about all that I found for django crm. The rest of the solutions are too much of an overkill in terms of hacking time or completeness. Okay now my favorite part ... back to the Django drawing board.

Django talk

2009-06-02T00:47:00.000-07:00

Just finished my Django talk yesterday and I must say while a few dude and dudettes walked out, there was generally quite good interest (better than I thought) from a few young people in the crowd. Initial talks are also underway to form a PUG or Python User Group locally here in Malaysia. I would really like to see this happen. The response has been quite good infact I would go so far as to push to even include Python in some programming curriculum in our universities.

That is the only way I would see Python growing and getting more acceptance. On that note too I also mentioned during my talk for frameworks like ROR, Django, TG and it's like to gain more acceptance in the industry, more articles or howtos concentrating on enterprise level deployments of these frameworks have go to be available.

Something like: http://enterprise.djangoproject.com?

I would really like to see Django grow from it's current girth to something that could rival all the J2EE stuff out there.

Nice link here to some good django slides ... damn could have used those for my talk :).

P/S -Thanks to all those readers who came to the talk :)

Review of Django 1.0 Template Development

2009-05-17T20:33:00.000-07:00

Initially when Packt Publishing sent this ebook Django 1.0 Template Development by Scott Newman to review, I did what any self respecting Django entreprenuer with 3 project down his pants would do. I skimmed through the book to the code sections and read through the code and read about 10 lines after that. The blink that I obtained reading the book this way was not too good and I felt some of the explanation in certain sections lacking.

BAD mistake! This book SHOULD NOT and I repeat SHOULD NOT be read that way. After I hunkered down and read the book cover to cover, it was then I really began to like it. So now, Django projects later, I wish that I read the book the right way the first time, because it has given me so many ideas to improve my Django code. So my advice, get this book, read it cover to cover and then revisit, bookmark or underline sections that you will revisit later. This way I found out was the way to get the maximum benefit out of this great Django reference. I really like reading technical books that know how to put forth advanced concepts with sounding too technical. Good programmers do not automatically make good teachers and the ones that do are the rare breed. On that note I loved the simple way in this book was presented. So, my kudos to Scott for making this an easy understandable read.

This is a book for Django developers who already have the basics and have been playing around with the Django examples from the website. The writing is to the point and easy to understand and practical examples are littered around to help the Django developers get their heads around what is being discussed. Perfect book to have by your side to try out code as your read. While the examples are simple, you could easily incorporate the examples into a real project.

My guess is that this book was not really meant for the total novice to Django. This book is meant for the mid level Django developer who wants introduction so some deeper concepts in Django especially those who want to find out more about Django's templates. That being said however, some sections seem to be crying out for more explanation.

The urls.py section has a very useful section regarding splitting up of the urls.py file for bigger projects. I really like this! It shows a bit about the whys and hows about proper Django deployment that is not usually covered the normal docs and mostly has to be scraped of mailing lists. The section about views is covered briefly but does contain a few useful examples and explains the concepts of requests and responses well using a few of Django's built in functions. Now what would have made this part of the book exceptional would be if the author covered best practices of deploying enterprise Django applications such as splitting the views.py into apps to avoid the views.py getting too big or unwieldy. The generic views section gives a good and solid explanation regarding one of Django's more powerful functionality that is the generic views. Middling Django developers should find this section of the book a useful reference as well as guide in their projects.

The next chapter is all about Context and how to use them in Django. I like the examples here and they work well to explain the concepts about Context. What the author could have done to solidify the examples here would be probably to include an example of how a normal call to an object would be done versus the method of using the Context object. The examples in this section are simply superb! They assume minimal knowledge from the beginning and build up to cover some of the more complex topics in Django without ever letting the example getting too heady. One thing I learnt here is the alters_data attribute that is actually included in all the templates preventing a method being called from the templates. It serves as an introductory to this functionality but I would have liked it more if a bit more explanation was given as how this attribute is put together with the save and delete methods. A bit more explanation about what kind of data is carried in the request object would be nice here. Scott teaches us to write our own context processor with a simple example of returning the user's ip address.

In my opinion Django's tags and filters are most of the time underestimated in the light of some other more powerful templating engine for example ZPT in Zope, but understanding them will make you understand the power of them and how much you can really do. I rely on these guys most of the time in my projects and they are great to use. Here Scott goes through most of the tags and filters that you probably would use in your project life and more. If you ever wanted to understand what is the difference between tags and filters read the book! I understand now and he does it in a one liner! The next chapter is equally good focusing on loading and inheriting templates. Don't blast over this chapter thinking it's not important (I was tempted to). Scott cover one or two things you might not have known about your settings.py. I learnt a thing or two here namely the fact that you can get away by even distributing your template files as eggs! Template inheritance in Django is another one my favourites and Scott does a good job covering the basics using easy to understand diagrams, much like how I would explain the concept of template and inheritance to a template newbie, read it through though as Scott delves a bit deeper into templates with a few advanced concepts. The concept of multiple templates are covered next and the simple examples covered here are so easy to understand I could not wait to just to my Django instance to knock out the examples and try the out! Extending the section on creating a section for mobile content, Scott shows us how to employ the use of session variables in writing a middleware to check for mobile user agents. Good stuff this, you could if you wanted to copied section of this code and make it parf of your site.

In the custom filter section, Scott offers a simple example and the nice thing here is he slowly builds in advanced concepts without getting too Django technical. The example is easy to follow and beginners will easily pick up filter writing skills. One issue I ran into when trying out the examples is that this section does not cover some of the strange error messages that the template filters shoots out if you do not follow the syntax exactly. On that note this book could do well with some examples showing the errors that might be encountered in the examples.

One of Django's “killer app” is the admin interface. With just a one liner you get a beautiful web interface to edit your objects with all the forms and validation thrown in. While this is all good, the templates for admin interface takes some digging in and sometime to understand. I used it for one of my projects and let's just say one or two times I had to think hard just to get a column to align. Doing some wishful thinking I wish now I have a Delorean so that I could take a copy of this ebook and pass it to my head scratching self in the past. That would have really sped things up and made life easier for me as Scott does a superb job here of explaining the intricacies of the admin templating system.

On the whole I would gladly add this book my collection of Django books as a good source of reference. While it does not cover all aspects of Django, the section that it does cover, it covers thoroughly and I find that it's very useful in satisfying my curiosity to find out more about Django's templating system, filters and tags. The information that it presents is very practical and good to be applied in projects.

Unexpected behavior of python's ConfigParser

2009-05-12T20:44:00.000-07:00

While working on some python code today I ran into some scoping issues that involved ConfigParser. The issue that made the process of finding and fixing the issue in the code harder is an unexpected behavior (on my end at least).

The situation is like this. My script relies on a configuration file that resides in the same directory as my script. For example sake let's call this file example.conf. During the execution of my script however, the directory scope of my script is no longer at my script main directory, so when it tries to find example.conf again it fails. This however is the part that put me off. The part that put me off is the fact that the error coming from ConfigParser is not really descriptive.

Take this example for example:

">>> config_file = 'does_not_exist_stuff.conf'
>>> config_parser = SafeConfigParser()
>>> config_parser.read(config_file)
[] <------------- shouldn't this part shot up some error messages?
>>> config_parser.get('config1','test_config')
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python2.6/ConfigParser.py", line 531, in get
raise NoSectionError(section)
ConfigParser.NoSectionError: No section: 'config1'"

The thing I am trying to say here is when I try to feed SafeConfigParser a config file that does not exist should it complain? I understand that the onus should be on my code to check whether or not the config file exists but then an error saying "No section:" does not really help too much in finding the error fast. I am writing this post just hoping some one would chip in and show me if the way I am using ConfigParser is wrong or something if not I am considering writing a patch for ConfigParser.

On the other hand I found another Python configuration parser:http://pyfig.alecwh.com/. Going to look at this later on.

Sum for the range of a column in Excel

2009-04-11T20:09:00.000-07:00

Working on my current project, one requirement came out which made me all the more appreciative of django's nice array of built-in filters. I generated an Excel report for my clients which is generated from html. Good and nice. Then one of them said, hey can the Excel report have a row which outputs the sum of the whole column of say the column E? Something usual and easy in Excel but stumped me for a while of how to do it from html to excel. For the most part it was trying to figure out how to include that SUM formula at my html code.

A little visit to M$ support center got me the help I needed and then it was back to nice plethora of filters django has .... Turns out that you can include formula into a cell (or a table column in this case) by simply doing something like this '=SUM(E2:E70)' just after the opening tags for your column. This part actually took the longest for me as I did not really know how to do it correctly. My last row in this case was dynamic because it depended on the number of rows generated by my query. So all I did next was some little magicks like this ...

'=SUM(E2:E{{query.count|add:1}})', which takes the count of the results and adds one to it because Excel rows start from a 1 while the count function returns counts starting from row 0. Just something very simple. Don't really want to tax the brain during the weekends y'know ... anyway just a small thank you to django for really great tool box of useful built-in filters. You really made this post shorter than it ought to have been :).

Another strange django unicode error solved!

2009-03-19T10:01:00.000-07:00

Today while trying to my final product to one of our customer's an ugly little error shot back at me when I tried editting any of the imported records into the database. "django ascii codec can't encode ..." bla bla it would complain and refuses to render the said template. Tracking down the problem down, I found it to the __str__ method being returned one of my models.

After changing the database encoding for my MySQL database ( sucky on windoze ), it still stubbornly spat back the same error at me. Finally after searching around more I found the smart_str method's usage here.

Kudos! It works now. So instead of returning the normal 'str(self.name)' I do a 'smart_str(self.name)' and everything is back to it's hunky doriness!

Multi row transaction capabilities

2009-01-26T05:29:00.000-08:00

The project in which I wrote a python script to process rows in an Excel spreadsheet interfacing with a MySQLdb and fronted on the web end using django is coming to a close. There is just one more requirement to go. They want rollback capabilities with the ability to completely rollback all actions on a particular excel file if one of the rows in the file buffs up or errors out. This is so that they can correct just that one row in that Excel file then feed the same Excel file to my Python process script without worry about the sanctity of the data of the rest of the rows.

This means that I have to somehow implement a multi-row rollback functionality or application level transaction. Python's MySQLdb library supports rollback but only on each transaction. I need a "all or nothing" like functionality. This is something new for me, so I scouted around on google to get some inspiration or to see if someone else had done the same thing. I found this small transaction class by Harald Hoyer at his site. Planning to try out his stuff to see if they can fulfill my requirements. Instinctively, I feel the final solution that I come up with will be a bit different.

The only other way I can see this happening is to maintain another database table keeping track of all actions that are performed at the row level in the input Excel Forms eg: row1 : insert, row2: update. Then if the script detects an error or Exception while processing the form it will perform the reciprocal or rollback to all of the actions that have been performed on the data on the Excel file. So far whatever I have is still being worked out in the noggin' coming close to being implemented.

Update:
Something that I seem to have overlooked is that since I am using django for the web front maybe I can also use it's ORM at the backend. It all depends on how much work it would be to ORM-ize my existing code base.

Lenovo's ideapad

2009-01-23T08:24:00.000-08:00

Today I was tasked by the better half to bring a lenovo ideapad S10 to life. My date with the ideapad got off to a rough start when during the initial filling in of the customary username and password and other stuff, the power died on me. From there on the puppy was not too happy with me and refused to boot up slapping me in the face with a "Installation not complete" dialog box and proceeded to reboot every time I clicked on the 'Ok' button or tried to cancel it. Good stuff. Tried the reboot-click-on-dialog cycle for about 3 to 4 times before calling yer-olde' friendly tech support dude who began the conversation with "Don't mean to be offensive..." (I mean what was I supposed to do?! It doesn't even come with a damned rescue disk!) Ever notice how people always tend to start getting nasty every time after saying that.... ?! Yeehawe!

Okay, so after a little bit of good vibes rescue generally telling how manly he sounded on the horn, the guy tells me that there is a rescue button there that can reset the baby back to it's factory settings ... Ah! So that is what that little non-standard button is fer! The funny thing about this button is that it will only work if the unit is turned off! Tried it during boot up and no way Jose! The button was like non responsive, just like when you tell your gf _not_ to go MNG just this once for your date. It only worked after I finally hunkered down to read the damn manual! Now, why didn't I just save myself the aggravation and did that from the beginning? Why? I don't know, I just want to be cool like everyone?!

At this point the lappie is working fine installed with a printer and a hax0red version of office 2007. Hey! don't be barking down this pair of pants asking why not openoffice! When the customer is your gf and she ain't in the mood of hearing about your FSS evangelism, you shut up and and nod man! The customer is always right, doubly so when she is your better half!

On the performance of this little guy I can only say this ... "It's slow and steady". Stuff works but there is just that little bit of lag before starting up, hey give it a break it's just that little atom of a processor under the hood man! I am quite happy with the performance considering that we only forked out Rm 1,783 with printer thrown in! I can see the appeal this kind of lappie would have for the ever on the go exec that just wants to occasionally check his/her mail or do the usual proposal on the run. For the rest of us that needs that performance juice from their daily machine, and do more than that stay clear!

Well, at least it's been fun and I am still in her good books :)

Gmail migration script done!

2009-01-11T08:57:00.000-08:00

Hello there again! Nothing like a little good olde' scripting to usher in the new year. As most of you will recall I was working on a script to migrate email from an existing imap server on to google app. The only thing that my script needed to do was transform folders into labels. Easy peasy I thought sure someone else has encountered this. Boy, was I wrong. I could not for the life of me find anything of the sort out there!

Well, no matter. I proceeded to create my own script from scratch and after about 2 hours my script could already scan my existing imap server, list all the folders and re-create them as labels on my gmail account! It was then that I stumbled across this piece of genius xiao duo so graciously left of the web. Their script was made to migrate emails from an existing gmail account to another gmail account. I found that the script they had complemented mine to a T! Talk about Laws of Attraction!! Great! All I had to do now was to transplant my script using my vim scalpel on to my script and faster than you can say Fedora 10's release name I had a working super duper gmail migration script!

Everything works as it should. About the only drawback I noticed with my script was that with TmNut's great connection, the connection timed out on me a few times and after restarting the script it started screening the folders right from the top again. Maddening! This is one of the improvements I have noted down, but for now everything works just fine. I can't attach any code here, so if for some reason you need the script just drop me a comment or a line here.

Catch ya laterz.

Why I use Linux.

2008-12-31T07:44:00.000-08:00

Today while trawling around on the opensuse mailing list I found this gem of a quote from one of the user's answers that summarize really well why I love and continue to use Linux and why any other thing would feel wrong ....

"""
I use openSUSE and Linux because it allows me to use my computers the
way I want. I have some almost new hardware and some really old
stuff. I'm typing this on a Thinkpad laptop with a P3/1Ghz and 256mB
RAM while I have 4 ssh sessions into my server running 2 movie
re-encodes and doing file maintenance in the other 2. I've got 2
directories on my server mounted on this and 1 directory on this
laptop mounted on the server to copy files with. If I was using MacOS
or Windows I'd barely be able to do anything.
""" - Larry Stotler

Thank you Larry for converting my thoughts into such simple but expressive words.

There have been mixed reviews for OpenSuSE 11.1, some lauding it while others panning it for a disaster almost as bad as the 10.1 release mostly surrounding KDE-4 desktop choice. The desktop environment is so much a part of a distro nowadays that normal users do not usually make the distinction between what is the distro and what is the desktop environment. For them, KDE or OpenSuSE they are both the same. For me I applaud the OpenSuSE team for taking a brave new step ahead! KDE-4 has it's faults no doubt but as usual I put my total trust in the OpenSource community to mold into something better than the current 3.5.x. I will still look forward to keeping OpenSuSE as my main working distro.

Finally to all the readers of this blog. Merry belated Christmas and a Happy Profitable new year ahead!

opensuse 11.1 and python 2.6

2008-12-28T20:33:00.001-08:00

Just upgraded my X61's opensuse from 11.0 to 11.1 and found that the opensuse team bumped up the python version from python-2.5.x to python-2.6. Woohooops! Definitely won't win them any new fans in the Python camp for that move. For one deluge no longer works as it uses Python-2.5.x. As a comparison, although the folks over at the Fedora-10 camp is also known for their up to date apps, they still maintained python at version 2.5.2 to maintain a semblance of sanity with some of the apps that need python 2.5. I am just going through the multitude of Python applications I have on my box to see which one greets me with a "Goodbye cruel world" bomb. Why oh why do you have to be like that opensuse?! Well on the upside the whole desktop experience has improved and I had a relatively smooth upgrade path.

Something else too, kde-4.1 which used to be pretty clunky and unusable in 11.0 has been improved and after a while of getting used to it, I am beginning to think it's pretty good.

Well, when the lizard gives you lemon make lemonade I guess ... time to revisit ktorrent. I just hope everything else is intact.

Script to import mail from existing email account into google

2008-12-20T10:39:00.000-08:00

I am writing a script to help our customers migrate from their current mail to google apps' gmail. Found a host of scripts on the net but finding one that works correctly not to mention even work proved a challenge. Too many of the script just fell short of what I needed. There were perl scripts, ruby scripts and also python scripts.

All of them just did not have the one thing I needed that is to convert the imap folders into gmail's labels. What about folders that have more than one level? Well I came up with the idea to convert them this way:

Original structure:
Level 1
=> Level2
Into a label : Level1_Level2

The normal mail migrate script that comes with gmail does not seem to convert imap folders into labels. Currently I am looking into using the libgmail library to do what I want. Ran into some trouble installing the source copy because it was looking for python-mechanize and one of the dependancy of python-mechanize is python < 2.5! Sheesh! Well poking around using google I found an older package of libgmail that did not need python-mechanize. Hopefully to be able to come up with early prototype by tomorrow. Has anyone built something similar?

Some great functions in django templates

2008-12-06T08:02:00.000-08:00

In my quest to capitalize the title for my application, I found some great functionality in django's templates.

To 'humanize' data there is contrib package called 'humanize'. For the docs you can find it here.

For some built in functions look here. I really like some of the humanize functions after spending so much time to implement some of the functions found here. While django's template language might not be as powerful as others they sure have some pretty useful functionality here!

Return of zope?

2008-10-27T22:01:00.000-07:00

First off the bat, let me just this post is not meant to add more gasoline to the django vs Plone wars. It is just meant to read as my thoughts surrounding the Experience of using Zope and Plone and trying to use them both in my company.

It's no secret now my company is using django for most of our web related projects. Although I use mostly django now, I am like the kid who left his good pal to play at the other side of the fence. I constantly look back and think if we could still fit zope in somewhere in our projects. I think back right to the beginning when I did my first web project. Unlike many I did not start with php. I started with zope because my company at the time wanted to vary our skill base. I had many a sleepness night trying to wrap my head around the zope concepts and paradigms. I got my ass handed back to me many times at the #zope channel by MacYet yet strangely I persisted and slowly I grew to like zope.

Then came Plone. In my next company I deployed a Customer Complaints Form applications for them using Plone and Archetypes (it was called cmftypes at that time). Most of the time I did not know what I was doing yet at the end I succeeded in completing the project and the last I synced with my cronies there they are still using that application for their daily operations. This enamored my confidence in Zope and Plone. I also did a few other projects using Plone and generally found the going easy and liked using it although there was always a voice telling me to go deeper into the engine to see how everything ticked.

Coming from these positive experiences, I did not hesitate to recommend Plone to deploy a web membership system for a political party for a project that we had secured. This was when the nightmare began which chipped away nearly all of my confidence for using Zope/Plone as a development platform for my company. Here are some of the issues I ran into:

Technical Issues

1. Plone was over complete for the project. As with a lot of other projects I tried to shoehorn Plone into, I found that it could do the 90% of the requirements easily without even breaking a sweat but the last 10%, which usually is the most essential and mission critical ones required a huge re-engineering of some of the components inside of Plone. It was just too daunting of a task to do. To make it worse it was tasks that seemed very easy to non-Plone developers. "Can I make the font for this automatically generated font smaller here","Could you please remove this green drop-down here and place it here" .... etc. etc. Most of the time I had to explain to my principal his technical point of view and the Plone point of view. This created a lot of heated and opinionated exchanges and needless to say at the end both of us were unhappy. Most of the time I found that I either had to make some little change to some Plone component of Product that would cost me a lot of time because I had to understand how everything fit together before I could modify it. Other times I found that I had to strip out features that were not needed to start with. Deploying Plone for our prototyping stage created problems for us as users saw some of the components of Plone and started coming out with extra requirements based on what they saw.

2. ZODB Performance. This one really stuck a knife in my gut. We were not deploying the next Slashdot but ZODB needed to at least handle 30,000 to 100,000 objects at once. ZODB was too much of a blackbox at this stage for us to optimize when it when it went to the land of the Slo-Zo. I kept on packing the database up and found bottlenecks all around the place. I can totally relate to how this guy feels and why he too chose to go for a ZODB-less Zope. My partner who is a Dba felt pretty helpless without me when all his knowledge of optimizing rdbms was moot when the heart of Plone was ZODB which did not even have a SQL like syntax. At 30,000 objects it was taking 20 seconds to come back with queries! This alarmed a lot of people but I did not have good answers for them. Trying to run Plone on MySQL did not seem very optimal too as a lot of Plone's magicks and functionality seem too tied down to ZODB. I will concur here and admit I did not know enough of ZODB to make it really work for us, yet from all angle that we were looking at this, Plone seemed more and more of a wrong choice.

Non Technical Issues

1. There was just no developers around! I could find PhP developers by just talking to aunt Tilly that lives across the street but Python/Zope/Plone developers was a rare breed! Even if I found them, they were very expensive and usually overseas. However, to be fair most of those PhP developers that we found through Aunt Tilly lacked finesse and were not of standard. However, having some developers is still better than nothing at all and this was one of the reasons I had to surrender to using PhP. The trick now was not to code the application myself rather to leverage my time by hiring programmers to do it. My partner and I found that there was no way to deploy the project using Zope/Plone if we did not have anyone we can hire or outsource it to. The sheer pervasiveness of PhP won out here.

It was at this time when I started to look at other alternative frameworks that we could use that was a bit closer to the metal and allowed for easy deployment. I found django. Django was a joy from the start. Following the documentation allowed me to be productive from the go and for once I could understand what I was doing! There weren't 120 stacks of libraries I had to wade through to get something working for once I found that I could pass some of my half finished stuff to my developers and they could take over the project ! Like many coming to django there were of course some paradigm shifts I had to get used to namely the lack of some functionality in the django templates but I soon got used to that. jquery was also integrated into my pages easily and for the first time in a long time I know exactly what each line of code in my application did.

django is certainly no Php not by a long shot but at strangely I found that explaining django to a developer easier going than trying to carpet bomb their brain with Zope.

Recently, my projects started growing in size and after my views.py and my models.py started getting unwieldy and spinning off too many apps I started looking around to see if other people faced the same thing and I found other dissidents around. Most of the voices out there seem to say that django is good for small projects but found that to create re-usable apps using django they seemed to be re-creating zope! Although with django I do not feel that I am "fighting the framework" I could see where these voices are coming from. This is where I could almost see Zope developers snickering at a corner as I could almost hear them say "I told you!".

Certain developments too have happened and a few articles I have read, pointed to Zope3 being a totally different beast to Zope2. I read a few of the examples using the zope3 architecture and I must say they look a world easier but would I be able to come up with my own stuff? I honestly do not know. I think too the move to run zope with wsgi without ZODB is a good one. I am challenged to think of a situation where using ZODB would be advantageous to using a RDBMS. Rant on here. I still do not know where we are going yet and I am going to take zope3 for a thorough drive before making my decisions.

Some other opinions:

What django can learn from Zope

Time taken to create a trivial app. <-- Trivial applications or play code is less useful as most of the time they usually fall on the side of the "barebones" frameworks like django, ror or pylons.

I wrote this post because I know as time goes along and to remain agile and more importantly fast, with more projects under our belt, re usability will be a feature that we want. Currently we are doing fine with django but will the future paint a come back for Zope into our organisation ? Personally I hope so.

datepicker without admin

2008-10-25T02:08:00.000-07:00

Django's admin interface is one of the really nice things about django because of it's functionality and useful tools. Nearly all of the reviews I have read about django have mentioned a good thing or two about the admin interface. There is however sometimes a need to go beyond or implement a lesser version of admin interface. This is where the adventure of living outside the realm of comfort for me begins ...

I have been trying to live outside the comfort of the admin interface for django and have found other niceties outside of the admin interface namely generic views, forms and others. I have been using these tools to build a web inventory cum sales application for one of our clients and happy to say it has been fairly successful.

Generic views are really powerful and really helps to shave off development time. There are however withdrawal symptoms my code is feeling to the admin interface. I have reimplemented some of the css niceties and javascript in my code. One of the one I am looking for is to re implement the cool datepicker admin has every time it encounters a DateTime or a Date Field.

I have tried a few things towards this goal:

1. Included all the javascript from the admin interface for my page. This just failed silently and my text box looked just the same.
2. Passing the model.DateTime a widget argument in which I call the widgets from django.contrib.admin.widgets. This resulted in a unexpected keyword error.

It would be nice if the django admin team can do a small writeup or a break down on how the admin interface was implemented. This would be really nice as there are some really good stuff in there which can also be useful in it's bits and pieces.

Update on the python windows saga

2008-10-08T04:54:00.001-07:00

The fiasco finally ended, my code is working and the customers are at least quiet. My requirements were simple, my excel trawler trawled the excel pumping stuff into the database. Then after that it had to copy the excel file based on the outcome of the process. So if it failed it had to copy the input excel file into a 'fail' and conversely into a 'pass' folder if it did it's job right. Here is the catch. The customer wanted the whole directory structure of the input file to be created in the fail/pass folders. I thought "Heck this is simple! Just do a 'cp --parents /path/to/input/file /[pass | fail]/folder' What could be easier?"

Boy was I wrong. All of the windblows solutions did not fit right. I found this page by Tim Golden which I thought was golden until I tried out most of it's stuff and it did not exactly work right for me at least. Essentially it boiled down to:

1. If it copied the directory tree(shutil.copytree) it did not copy the file right. If it saw a file in the path, it would just copy the file and the did not do the sub folders and the parent folders. Augh! Maddening!!

2. I could not just do a system call to do 'cp --parents' because this was a windblows machine. So I thought initially os.system stuff was out. It drove me nuts something so simple could not be achieved by any of the windows command line stuff like copy or xcopy or robocopy.

Finally I cheated. I downloaded a windows port of cp, and just a os.system call on it. That too had it's fair share of problems such as you could not do a 'cp --parents c:/some/thing c:/some/one' because it tried to do a c:/some/one/c: and bombed out because it did not like the colon. Looking at this I used the os.path.splitdrive and split up the stuff then did a copy.

I know this is not clean enough for you purist out there, but heck it works and that is all that counts ... to customers at least.

Spaces in directory names ... I really love you not!

2008-10-07T19:19:00.000-07:00

Have I mentioned I really love Python? Have also mention that I really hate Windows? I did not? Well here I go again ... I REALLY HATE WINDOWS. This time the target of my scorn or anger is the stupid spaces in folder names windows users are so apt to give to their folders. It screws my programs and special attention has to be given to quote them to work...

Grrrr, all sorts of funny stuff happen at this stage. I am wondering how the heck I am going to quote the whole filename if I am doing something like below:

import shutil

for file in os.listdir("/home/lowks"):

shutil.copy(file,"/tmp")

I tried 'shutil.copy('%s',"/tmp") but it spat out errors at me. Is this because the string is a unicode string? If so how do you convert the file path to a raw string? I really hope if someone has experience doing this to shine some light this way.

Ops ... it seems I made a little error there, it seems to work but not in my longer script. My script basically has to copy a file including it's parent folders into a fail / success directory based on the outcome of running the script. I tried shutil.copytree but it does not work and fails silently, which prompted my spaces-in-directory-name hate. I am wondering aloud here there must be an easier way of doing this, a one liner thing.

I think also I have found the error with my script. It is still a space related issue where if there is a space in the folder name, then it automatically treats the last string as a file name and shutil will bomb out when it tries to open up that file for copying or moving, like something below:

Traceback (most recent call last):
File "process.py", line 321, in
,failure_directory)
File "D:\python25\lib\shutil.py", line 80, in copy
copyfile(src, dst)
File "D:\python25\lib\shutil.py", line 46, in copyfile
fsrc = open(src, 'rb')
IOError: [Errno 13] Permission denied: u'C:\\import2\\testing folder'

In the case above, the 'testing folder' is actually a folder but shutil have interpreted the 'folder' as the filename hence the exception.

Ah ask and ye shall receive, in windows there is a short file name for all files for example C:\Program Files is equals to C:\Progra~1 or something to that effect. This same short filename can be used in python if you use the win32api, in a one liner like this:

import win32api

long_file_name='C:\Program Files\I am a file'
short_file_name=win32api.GetShortPathName(long_file_name)

This works. If you are incline to read more about this go here.

django and some plone

2008-09-24T19:12:00.000-07:00

There seems to be some rumblings on the Plone front for us currently. For our main stable of development however, we have moved away although not totally from Plone to django. The main reason here is that for Plone deals to be feasible for us, we either have to:

1. Lock requirements down really tight so that the juniors can have a go at it .... or

2. Sic some seniors at it.

In my own experience, it seems that while Plone can be easy if worked within a tight parameter, try swaying a bit out of those parameters, when you have to peer under the hood and you are in a world of pain. It's hard to gain leverage using Plone this way. django on the hand has been actually quite young developer friendly for us and they end up being actually more productive compared to when they are forced to make some funky changes in Plone. Most of the business applications that we build require a light framework and is normally built from ground up. This means that Plone is sometimes an overkill for these apps which is a shame because Plone almost always appear like a good fit in the beginning. In the end, to be able to shoehorn Plone in means a lot stripping functionality and code out.

For what it's worth we still try to train up our guys to do basic Plone stuff but only time will tell if it makes business sense to invest any time in Plone/Zope.

A simple find in python ?

2008-08-01T00:29:00.000-07:00

Continuing on my saga with my Python importer for xls, I ran into a situation where I had to dig in recursively into a directory structure and do something with Python. There seemed to be nothing easy in Python such as a 'find . -name '*.xls' -exec perform_task {} \;'. All of the solutions or suggestions that I have run across so far from stuff found in google seems to need me to perform some sort of mental or code acrobats. This is not to create a flame, I am just an intermediate in this so I am welcoming suggestions to do this if you people have any.

scraping data from excel files

2008-06-22T21:55:00.000-07:00

I am currently still in the midst of writing my excel scraper to scrape data from excel files. There are two types of encrypted excel files that I need to scrape from. Both of them contain similar data but are structured differently.

The first problem that I encountered is that the data position are not standard and scattered all around the place, for example in the first type of file the data starts reading from line 10 to 31 while the second type of file starts reading from 11 to 31. The form looks neat and good to the human eye but is a coordinate hunting nightmare programmatically! While both of the files contain similar data, I found that one method to scrape data is insufficient because of the tiny differences in the location of the data. I cannot use odbc methods to run sql statements on them as they are encrypted, so I am stuck doing coordinate targeting.

After much doodling around with positions of data, I finally decided to write a Form object that takes a record_type an argument. When I instantiate a record_type of say 'A' type it will then load up the profile of the 'A' type object which is a dictionary that contains all the information of where most of the data are on the form. This dictionary is my implementation of a profile. So from there my object will know where all the important bits of information are based on the profile from the dictionary.

Anyone have done something similar in a better way?

surrounded by win32com's python and not liking it

2008-06-20T05:49:00.000-07:00

I am using the win32com library to write an application to populate data that are in encrypted excel files that are kept in a specific folder into a mysql database. Most of the part is there except that I do not find much of win32com library documentation around. One part of my code really irks me and it's the excel visible part. Everytime it is processing an excel file, it has to open up the file and close it up again. When I set the visible part of the code to 0 or false, my application doesn't run correctly anymore. Some of my code are as follows :

import win32com.client
xl=win32com.client.Dipatch('Excel.Application')
xl.Visible=1

Grr ! I wonder why this thing doesn't work if you set the Visible to 0. Can't this work in the background? Woe !

Enterprise project lessons

2008-06-13T21:37:00.000-07:00

Being in the technical lead for an enterprise project teaches you a thing or two. I was wearing those shoes recently for one of our projects recently and here is what I learnt :

1. If you are going to be using a framework of any sort you better know it inside out, left to right and top to bottom. When funky modifications are asked for or if your application starts to slow down you better know exactly or roughly where to start tweaking. This is the reason too why my partner feels that the bigger the project, the closer to the metal the framework used has go to be.

2. If the scope of the project is large and you suspect you might not have enough developers, never never never choose to use a tool which leaves you with one choice when it come to developers. For our case we had to give in and go the php route for this particular project. In my company we have shunned php in the past choosing not to use for any projects, but we had to bow in to pressure this time because it was just too hard trying to find capable and dependable python developers where we are. Once we caved in and accepted php, we found a pool of capable php programmers.

3. I can never over emphasize the next one. Never never never never under estimate the scope of a project especially if it pays a lot. When the customer says "I just need a web page to capture data" always ask for more information. It might not just be the application that is complex, it might even turn out to be the setup itself that is going to pose a challenge. In our case, we had to deploy multiple instances of our application in a few remote places to allow data entry people to key data. At the end of the day we had to merge the data all back into a central database.

4. For any project always look for a capable PM. It might lower your bottom line, but trust me it's worth it. A good PM will save you 3 am calls while at the same time allow you to multi task in the back ground and do what you do best. Fortunately for us, we have found such a candidate in my old business associate though we cannot use him for this project fearing our customer might just freak out seeing a new PM on the job !

5. Always have a old school unix or Linux who knows all the old text mangling tricks loaded and ready. In this case it was my partner. Even when doing seemingly unrelated stuff like web application never under estimate the usefulness of these guys. We had to move about 50,000 records trapped in excel into the database and his skills proved invaluable ! (Not that I am that much behind now, but I had to concentrate on getting the web end up)

In the end though, through all the late nights and sweat, this project has thought invaluable lessons in project management as well as resource planning.

python unicode sucks

2008-06-08T21:16:00.000-07:00

I have had some bad experiences with python unicode on my current project and suffice to say I a tad nervous about using python for my next project. The unicode support is just horrendous and is very limited. I tried googling for unicode+ and nearly everytime it came back with multiple hits ! I read about a version of python in cvs that has unicode support built in but then I noticed the word 'cvs' **shrudder** My experiences might not be that complete so I would appreciate input from other senior snake wranglers out there.

django pluggables

2008-06-05T02:43:00.000-07:00

I have struck goldmine ! Bored today with another plone innard hacking session I began hunting around on the internet for a django plugins site. I came across this site called http://www.djangoplugables.com/ ! This is exactly what I am looking for with a cute little plug in socket as a logo! I am astounded by the speed in which django has grown !

I have coded some applications with django the past and I can say I like the feel of using it. It is has what I need in terms of a web application framework and is closer to the python metal than plone or zope will ever hope to be. Don't get me wrong, I still have a soft spot for zope and I truly hope that zope 3's modularity will be more elegant than the monolithic monstrosity that zope 2.x is.

Today while looking at the pluggable site, has given me much more confidence of adopting django as one of the stable technologies to deliver web applications in my company's offerings. Coming from a background that have used both I can attest to why a senior python programmer would love django. I hope though the django community get it right and do not start on the path of sticking stuff together to create another zope 2.x !

index vs. metadata

2008-06-04T07:58:00.000-07:00

Newbies hacking up scripts for the first time in a zope or plone environment might be confused at some of the terminology, heck I know I was ! Anyway, one that took me some time to grok not to mention numerous emails bugging people on the mailing list to understand is the difference between portal catalog indexes and metadatas. I will just say it in my own words here as I understand it now.

Index are keywords you add to portal catalog to allow you to create query based on them. Eg. query['Title'] = 'lowkster'. Here, 'Title' would be an index. The easiest way to see all the index in your portal_catalog would be to use the zmi. Go to the root of your site using the zmi and then access your portal catalog using the link provided by the left hand side menu. Here youwill find a tab called 'Indexes'. Here are the listing of all the Indexes in your portal's catalog. You can also add a new index in your portal catalog using this page.

Metadatas on the other hand are stuff that you want to show in your query results. For example you query your portal catalog and in the result is a metadata called 'Books'. You can then in your page template display this by calling it as such '. To set one of the fields in your content type as a metadata field, you have to add the variable 'isMetaData=True' to your field definition. You can also add a new metadata to the portal catalog using the zmi.

Anyway, forgive me if my way of explaining indexes and metadatas is wrong. This is just my way of understanding the difference between the two. Feel free to correct me if I am wrong.

getting tapping to work on fc9

2008-05-15T09:32:00.000-07:00

Currently i am running fc9 on my trusty old acer and one of the initial irritants I found was that tapping on the touchpad no longer works (In opensuse it worked just fine out of the box). This is a real annoying bug as I am used to tapping my way around using my touchpad. Scouring the fedora forums, I finally chanced upon this entry in the redhat bug list. Well, well, well it turns out the "bug" isn't really a bug after all. A long story short basically your package of synaptics from fedora has a patch that has disabled tapping on touchpads because some wiseguy thought that it was for the best since no one really likes tapping on a touchpad.

So a guy called Bob Kashani was kind enough to rebuild the packages and remove the the offending patch. All you have to do is download his rpm from :

http://www.ocf.berkeley.edu/~bobk/packages/

Run a rpm -Uvh, do a restart of the Xorg by doing the old ctrl-alt-backspace and you are good to go :)

Thanks bob for the package. It works great here !

ATVocabulary Manager with MasterSelectWidget

2008-04-30T00:32:00.000-07:00

I am using MasterSelectWidget in one of my projects to create the master-slave/master-slave combo of select widgets in my form It then occured to me how nice would it be if I could get my vocabulary from ATVocabularyManager which I have already installed on my site. Well, I lied it's not actually nice... it's one of the requirements :). Anyway I tried a few times to point my dynamic vocabulary at ATVocab's method but it failed to load up the Select widget with it's values. I wonder if this has been done by anyone ...

Archecsv

2008-04-21T00:58:00.000-07:00

Recently while working on a plone project I found the need to import a whole lot of csv based data into my plone site while creating objects on the fly. I found just the thing to fit this need in ArcheCSV. This is a nice little product that you can add on to your plone site and can then import data from your csv files in batch and map it on to existing AT based Objects on your site. Good one this ! Just remember update your catalog for your objects to show up and start using them. Using this Product and Archgenxml one has little code left to touch ... :)

Ouch ! Torrentspy

2008-03-29T19:11:00.000-07:00

Ouch ! It's a sad day indeed today when I tried visiting torrentspy and was greeted with this :

One can only guess the real reason behind the close but like they say it's been a wild and most often than not a wonderful ride. Good bye torrentspy.

Sles 10 Training

2008-03-26T21:57:00.000-07:00

Just finished a three day training on SuSE Linux Enterprise Server 10 for some iperintis folks. It was a nice change of pace for work and the interesting thing that I found was the tremendous amount of improvements that the SuSE distro has undergone. From it's butt slow updater in the 10.1 version to today's zypper and tolerable updater, I must say I do not have to strain to see the improvements that the team has done. The students were quite impressed to see that installing opensuse and sles was even easier than windows and were just tickled pink over some of the tools that they were shown. This first training is for the fundamentals level while next week they will be going through the intermediate and advanced levels. Good luck people !

YUI datatables does not work for IE ?

2008-03-20T09:35:00.000-07:00

I have been playing around with the datatables from Yahoo UI package for one of my customers. Much to my surprise though the code does not work for IE. I have been googling around for somebody reporting the same issue but so far I haven't found anything to the effect. I am just following the same example from the YUI site. On using YUI for my work, everything is good but then there are a few issues I am running into :

1. caching on the datatable itself does not lend itself to easy doing.
2. compatibility issues between browsers doesn't lend itself friendly to enterprise solutions.

Going to look more into this.

What has happened to plonecollectorng ?

2008-01-27T23:24:00.000-08:00

One my favorite product on plone is the plonecollectorng. I like all the functionality that it offers. It was imo ahead of it's time. I was actually planning to use it to build part of company's portal as the customer care center but as I found out later it seems that plonecollectorng has disappeared from the plone world. When I ask on the channel it's people who answer me only do so in whispers and would rather not have anything to do with it and even went so far as to suggest me a new collector called poi. I felt so much like Charleston Heston in Planet of the Apes !

I am sorry I lost touch with the plone world a little while, but can anyone actually explain to me what has happened to plonecollectorng ?

We are live on the unoffical python planet

2008-01-26T03:31:00.000-08:00

Python programming has been slow this few weeks as I am busy working up business deals all around, with three plone presentations set up for next week. Life is busy but good. We have generated some local interest here in Malaysia especially with the ISO standardizations in using plone as their document management system. With plone's multitude of functionality, a full blown document management system is just a few clicks away.

On the blog side I happy to report that this blog is now part of the unofficial python planet community ! So we are live on two channels now ! Yayy !!

Plone - redirecting authenticated users to member folder after login

2008-01-09T14:26:00.000-08:00

Another in the useful-hack-department

Task :

After login you want authenticated users to be directed straight to their folders. Normally, after login, the users are redirected to the where ever they wanted to go in the first place.

Solution :

The file that controls what happens after a user logs in is the file CMFPlone/skins/plone_login/login_next.cpy. So customize this file and edit it.

Basically what is happening here is, login_next.cpy will look for a variable called came_from. If the came_from variable is set, then it will redirect the user to whatever came_from is set as. At this point I knew what I had to do, the challenge was to find the right methods to achieve what I want. I needed the method to tell me what is the authenticated user's home folder. After some digging around I found it.

Just add this line to the login_next.cpy, somewhere at line 25 :

home_url = membership_tool.getAuthenticatedMember().getHomeFolder().absolute_url()
came_from = home_url

It's pretty evident what this two lines does, it basically calls the membership_tool's methods to determine the authenticated user's home folder's url and set's came_from to that value. After adding those two lines in there, login and logout again and all of your authenticated user's should be redirected to their home folders after login. Happy hacking !

plone-3.0.5 is out and it's great

2008-01-08T20:48:00.000-08:00

Plone 3.0.5 is out, finally ! I was waiting for this release for it to solve a bug with archetypes where when I used argouml to generate a plone type, it's required fields did not work correctly. Well, I tested the release out on my lappie and now it works like a charm. Damn this is sweet !

For the uninitiated few who have been living under a big tempurung for like a year, archgenxml is product for plone which generates plone products from your uml diagrams. So what this means to you o' tired developer is, YOU DON'T HAVE TO TOUCH ONE LINE OF CODE to have a working plone product at your disposal.... YES ! Don't adjust your browser text size, you read me right the first time. YOU DO NOT HAVE TO EVEN TOUCH ONE LINE OF CODE. Just design your application using something like argouml, feed it to argenxml and BOOM BAM in as long as it takes for you to restart your plone instance, you have your product sitting there waiting for you.

Take your first step on your journey here and start taking on as many plone projects as you can ... woohoo !! Damn this product is so good it should not be free !

Zope's best kept secrets

2008-01-08T15:46:00.000-08:00

Stumbled upon this gem of an article during one of my searches, the article does a decent job of summarizing what are the best features inside zope but wonders why zope is still the best kept secret. Reading that reminded me of a video I saw at plone.tv. Check it out here. Okay, so this is a plone screencast, but I shrudder to think of zope's future if the same people are involved. I mean come on, this guy is supposed to be involved in marketing plone and he is boring me to tears ! That ain't supposed to be happening. I am supposed to feel inspired after one of these right ? Heck at least the grok presentation was funny ....

I mean I can understand if English is not his language, but give a little more enthusiasm than that huh ... ?! Put me up infront of a stage of people and I can sell plone better than that pal. More thought should be given into zope / plone marketing if this is the state of the team involved with marketing it now.

More good words for z3

2008-01-08T09:49:00.000-08:00

Electric Duncan some more good things to say about zope3 or z3 here, this time it's vs django. To a certain extent I do agree with some of the things mentioned in that article. At times I do feel that I am trying to bend django to my will when I was coding my application, but alas, I am too green in z3 to give any meaningful comment regarding the comparison. Here, duncan mentions something that piqued my interest - Coding his application in z3, with learning curve thrown in took him two days ! Now that is a wow-ser for me !

Although at the back of my mind the article look more slanted towards zope than django which might explain why the author felt more at home with z3, but then it's a good read after all and has inspired me more to go towards giving z3 another serious look for my web development work.

grok zope

2008-01-07T19:31:00.000-08:00

Just finished my grok.zope.org tutorial and much kudos to the author for making zope easy. The tutorial is well paced and newbies should have no problems understanding some of the concepts of zope, although to be honest when the part about the persistence come rolling along, it would pay to have some zope experience.

To me it is good to see zope reinvent ( or in this case re architect ) itself to meet the demands of web application frameworks of today. It used to be that people never looked twice at zope when you needed to deploy a basic web application because they did not want the extra baggage that came with zope ( esp the 2.x branch ) although it came with a host of useful features that would really help to shorten development time of any web application. Now with zope3, I am confident of zope being the cog that can fit into much much more wheels out there.

Good stuff ! If you haven't checked out what the whole bruhaha is surrounding zope3, do yourself a favor and drop by grok.zope.org and try out the tutorials, the very least is you will walk away understanding the zope framework and architecture a whole lot better.

Installing zope3

2008-01-06T22:47:00.000-08:00

Wanting to try out the tutorials over at grok.zope.org ( ** CORRECTION - Actually you don't need zope3 to tryout grokproject - My mistake **) which seems really really yummy, I decided to buckle up and install zope from source on my trusty old lappie. It just requires a few steps and shouldn't be scary to all ye old sys admins out there. So, this short howto is meant for those who just dipped the big toe into the big zope pool and who are just trying to get their heads around the whole architecture.

1. Get the relevant sources. In this case python-2.4.4 from python.org and zope-3.3.x source from zope.org

2. Building python. This one is easy. Just untar the python package and then issue the commands : ./configure --prefix=/opt/python24;make;make install This will install the python binaries in the /opt/python24 folder. The reason why we have to build the python 2.4 binaries is because zope3 works best with the python-2.4.x branch.

3. Next you have to build zope. This involves untarring the zope source and then in the source directory issue the command : ./configure --prefix=/opt/zope3 --with-python=/opt/python24/bin/python. This will tell zope to install it's binaries in /opt/zope3 and use the python in /opt/python24/bin/python instead of the system PATH.

3. Create the zope instance : './opt/zope3/bin/mkzopeinstance -d /opt/zope3/instance -u [username]:[secret] where username and secret is the username and password of your choice when you want to access your zope instance.

4. Starting up your zope instance : 'cd /opt/zope3/instance/bin; ./zopectl start'. Open up your browser and try to access 'http://localhost:8080/manage'. If your installation is successful, it should prompt you for a username and password pair. Fill in the username and password from step 3.

Congrats ! You have just successfully installed zope 3 !

getting to pesky foreign key data in django

2008-01-06T18:56:00.000-08:00

Optimizing databases is a good practice and should be done for any enterprise applications, but as all things go there is a price to pay. The price you pay is when calling back and displaying the data into one can be real PITA. django is a damn good framework and allows you to do work fast with a pedal-to-the-metal feel but then certain amounts of work still has to be done when you have foreign keys defined.

Say you have a database with two tables : contacts and address. The relationship of the two tables are of the one to many kind, I could never really wrap my head around the one-many many-one many-many thingy so let me just say it in layman so I understand it better myself. "One contact can have one or more than one address"

class Contacts(models.Model):
..... bla bla

class Address(models.Model):

customer_id=models.ForeignKey(Contacts,edit_inline=models.STACKED,num_in_admin=2)

So in the code above you have basically linked both the tables together using a ForeignKey relationship. the 'edit_inline' ... mumbo jumbo just tells django to add the address field into the main Contacts form during creation and modification and not as a pop up link ( This is important and could be a clincher for normal users ).

Calling all the data back for display

Okay so now you are honky dory with a great admin form for you to add data into your database, now the problem starts when you want to call all the data back for display in your template.

Don't worry django comes with little helper functions, you just have to do a few more steps to display your foreignkey data.

For my case, in my view.py I do something like this :

from stock.models import Contacts

def main(request,id):
contact_list = Contacts.objects.get(id=id)
return render_to_response('stock/mytemplate.html',{'contacts':contact_list})

Probably the html formatting would made my code not valid in python, but just check ur identation especially after the function definition.

The few lines of code above basically expects you to send it an id ( from the url using the GET method ) and then get the Contacts object and then packs up that data object and sends you merrily along to your template with that.

Okay, a bit about the Contact objects. Inside your contact objects if you have a foreign key relationship it automatically create a method for you to access the data. The method is called _set. So following our example, we know that we have a relationship with the table Address, so we can expect that each contact object will have a method called 'address_set'. Still okay so far ? Lets go ....to get the data you normally call it like this :

contact_list.address_set.select_related()

This returns an orm object inside a list, so on your template you will call it like this :

{% for address in contact.address_set.select_related %}

Address: {{address.address|escape}}

{% endfor %}

That will bring up and display all your address for that particular contact. Done ! Easy peasy, well for me at least just comment me if think more clarification...

Plone 3.0.5 !!

2008-01-06T09:22:00.000-08:00

Waiting for the 8th of January which basically means that Plone 3.0.5 will be released. I am waiting for this release because it fixes a bug in Archetypes which currently causes validation on required fields in a form generated by Archgenxml to fail. After learning the few curves of using ArchgenXML, I must say I am bought hook and sinker on this product. It's really good stuff. With just the drawing of the uml class diagrams allows a developer to generate Plone code, almost too good to be true.

I want to be able to cover the Archgenxml topic in one of my classes on Plone. Now I am still working on the course structure of how the plone course will be like. Will keep you guys posted regarding this too.

A different view of blogger

2008-01-06T08:33:00.000-08:00

After having a gym session and talking to geekool, I am having a different view of blogger. I used to like having my own plone site and managing it, but after chin seng explained how it would make more sense and less headaches just having your blog in blogger, it just hit home to me.

One thing about having my blog here is that no one can come and tell me that need to move it or need to shut it down or anything like that. So, I am going to put my blog up here from now on and this will be the central point of dissemination of all information.

On projects, currently I am still working on a web project using django and I am learning up the little nooks and crannies of the language. It would seem that most of the project's repo is placed in google code repositories. I am using a calendar event but it seems to be buggy. But while researching for this, I found this project that seems a bit more promising. Will look into this and report later on this.