Why I use Linux.

Today while trawling around on the opensuse mailing list I found this gem of a quote from one of the user's answers that summarize really well why I love and continue to use Linux and why any other thing would feel wrong ....

"""
I use openSUSE and Linux because it allows me to use my computers the
way I want. I have some almost new hardware and some really old
stuff. I'm typing this on a Thinkpad laptop with a P3/1Ghz and 256mB
RAM while I have 4 ssh sessions into my server running 2 movie
re-encodes and doing file maintenance in the other 2. I've got 2
directories on my server mounted on this and 1 directory on this
laptop mounted on the server to copy files with. If I was using MacOS
or Windows I'd barely be able to do anything.
""" - Larry Stotler

Thank you Larry for converting my thoughts into such simple but expressive words.

There have been mixed reviews for OpenSuSE 11.1, some lauding it while others panning it for a disaster almost as bad as the 10.1 release mostly surrounding KDE-4 desktop choice. The desktop environment is so much a part of a distro nowadays that normal users do not usually make the distinction between what is the distro and what is the desktop environment. For them, KDE or OpenSuSE they are both the same. For me I applaud the OpenSuSE team for taking a brave new step ahead! KDE-4 has it's faults no doubt but as usual I put my total trust in the OpenSource community to mold into something better than the current 3.5.x. I will still look forward to keeping OpenSuSE as my main working distro.

Finally to all the readers of this blog. Merry belated Christmas and a Happy Profitable new year ahead!

opensuse 11.1 and python 2.6

Just upgraded my X61's opensuse from 11.0 to 11.1 and found that the opensuse team bumped up the python version from python-2.5.x to python-2.6. Woohooops! Definitely won't win them any new fans in the Python camp for that move. For one deluge no longer works as it uses Python-2.5.x. As a comparison, although the folks over at the Fedora-10 camp is also known for their up to date apps, they still maintained python at version 2.5.2 to maintain a semblance of sanity with some of the apps that need python 2.5. I am just going through the multitude of Python applications I have on my box to see which one greets me with a "Goodbye cruel world" bomb. Why oh why do you have to be like that opensuse?! Well on the upside the whole desktop experience has improved and I had a relatively smooth upgrade path.

Something else too, kde-4.1 which used to be pretty clunky and unusable in 11.0 has been improved and after a while of getting used to it, I am beginning to think it's pretty good.

Well, when the lizard gives you lemon make lemonade I guess ... time to revisit ktorrent. I just hope everything else is intact.

Script to import mail from existing email account into google

I am writing a script to help our customers migrate from their current mail to google apps' gmail. Found a host of scripts on the net but finding one that works correctly not to mention even work proved a challenge. Too many of the script just fell short of what I needed. There were perl scripts, ruby scripts and also python scripts.

All of them just did not have the one thing I needed that is to convert the imap folders into gmail's labels. What about folders that have more than one level? Well I came up with the idea to convert them this way:

Original structure:
Level 1
=> Level2
Into a label : Level1_Level2

The normal mail migrate script that comes with gmail does not seem to convert imap folders into labels. Currently I am looking into using the libgmail library to do what I want. Ran into some trouble installing the source copy because it was looking for python-mechanize and one of the dependancy of python-mechanize is python < 2.5! Sheesh! Well poking around using google I found an older package of libgmail that did not need python-mechanize. Hopefully to be able to come up with early prototype by tomorrow. Has anyone built something similar?

Some great functions in django templates

In my quest to capitalize the title for my application, I found some great functionality in django's templates.

To 'humanize' data there is contrib package called 'humanize'. For the docs you can find it here.

For some built in functions look here. I really like some of the humanize functions after spending so much time to implement some of the functions found here. While django's template language might not be as powerful as others they sure have some pretty useful functionality here!

Return of zope?

First off the bat, let me just this post is not meant to add more gasoline to the django vs Plone wars. It is just meant to read as my thoughts surrounding the Experience of using Zope and Plone and trying to use them both in my company.

It's no secret now my company is using django for most of our web related projects. Although I use mostly django now, I am like the kid who left his good pal to play at the other side of the fence. I constantly look back and think if we could still fit zope in somewhere in our projects. I think back right to the beginning when I did my first web project. Unlike many I did not start with php. I started with zope because my company at the time wanted to vary our skill base. I had many a sleepness night trying to wrap my head around the zope concepts and paradigms. I got my ass handed back to me many times at the #zope channel by MacYet yet strangely I persisted and slowly I grew to like zope.

Then came Plone. In my next company I deployed a Customer Complaints Form applications for them using Plone and Archetypes (it was called cmftypes at that time). Most of the time I did not know what I was doing yet at the end I succeeded in completing the project and the last I synced with my cronies there they are still using that application for their daily operations. This enamored my confidence in Zope and Plone. I also did a few other projects using Plone and generally found the going easy and liked using it although there was always a voice telling me to go deeper into the engine to see how everything ticked.

Coming from these positive experiences, I did not hesitate to recommend Plone to deploy a web membership system for a political party for a project that we had secured. This was when the nightmare began which chipped away nearly all of my confidence for using Zope/Plone as a development platform for my company. Here are some of the issues I ran into:

Technical Issues

1. Plone was over complete for the project. As with a lot of other projects I tried to shoehorn Plone into, I found that it could do the 90% of the requirements easily without even breaking a sweat but the last 10%, which usually is the most essential and mission critical ones required a huge re-engineering of some of the components inside of Plone. It was just too daunting of a task to do. To make it worse it was tasks that seemed very easy to non-Plone developers. "Can I make the font for this automatically generated font smaller here","Could you please remove this green drop-down here and place it here" .... etc. etc. Most of the time I had to explain to my principal his technical point of view and the Plone point of view. This created a lot of heated and opinionated exchanges and needless to say at the end both of us were unhappy. Most of the time I found that I either had to make some little change to some Plone component of Product that would cost me a lot of time because I had to understand how everything fit together before I could modify it. Other times I found that I had to strip out features that were not needed to start with. Deploying Plone for our prototyping stage created problems for us as users saw some of the components of Plone and started coming out with extra requirements based on what they saw.

2. ZODB Performance. This one really stuck a knife in my gut. We were not deploying the next Slashdot but ZODB needed to at least handle 30,000 to 100,000 objects at once. ZODB was too much of a blackbox at this stage for us to optimize when it when it went to the land of the Slo-Zo. I kept on packing the database up and found bottlenecks all around the place. I can totally relate to how this guy feels and why he too chose to go for a ZODB-less Zope. My partner who is a Dba felt pretty helpless without me when all his knowledge of optimizing rdbms was moot when the heart of Plone was ZODB which did not even have a SQL like syntax. At 30,000 objects it was taking 20 seconds to come back with queries! This alarmed a lot of people but I did not have good answers for them. Trying to run Plone on MySQL did not seem very optimal too as a lot of Plone's magicks and functionality seem too tied down to ZODB. I will concur here and admit I did not know enough of ZODB to make it really work for us, yet from all angle that we were looking at this, Plone seemed more and more of a wrong choice.

Non Technical Issues

1. There was just no developers around! I could find PhP developers by just talking to aunt Tilly that lives across the street but Python/Zope/Plone developers was a rare breed! Even if I found them, they were very expensive and usually overseas. However, to be fair most of those PhP developers that we found through Aunt Tilly lacked finesse and were not of standard. However, having some developers is still better than nothing at all and this was one of the reasons I had to surrender to using PhP. The trick now was not to code the application myself rather to leverage my time by hiring programmers to do it. My partner and I found that there was no way to deploy the project using Zope/Plone if we did not have anyone we can hire or outsource it to. The sheer pervasiveness of PhP won out here.

It was at this time when I started to look at other alternative frameworks that we could use that was a bit closer to the metal and allowed for easy deployment. I found django. Django was a joy from the start. Following the documentation allowed me to be productive from the go and for once I could understand what I was doing! There weren't 120 stacks of libraries I had to wade through to get something working for once I found that I could pass some of my half finished stuff to my developers and they could take over the project ! Like many coming to django there were of course some paradigm shifts I had to get used to namely the lack of some functionality in the django templates but I soon got used to that. jquery was also integrated into my pages easily and for the first time in a long time I know exactly what each line of code in my application did.

django is certainly no Php not by a long shot but at strangely I found that explaining django to a developer easier going than trying to carpet bomb their brain with Zope.

Recently, my projects started growing in size and after my views.py and my models.py started getting unwieldy and spinning off too many apps I started looking around to see if other people faced the same thing and I found other dissidents around. Most of the voices out there seem to say that django is good for small projects but found that to create re-usable apps using django they seemed to be re-creating zope! Although with django I do not feel that I am "fighting the framework" I could see where these voices are coming from. This is where I could almost see Zope developers snickering at a corner as I could almost hear them say "I told you!".

Certain developments too have happened and a few articles I have read, pointed to Zope3 being a totally different beast to Zope2. I read a few of the examples using the zope3 architecture and I must say they look a world easier but would I be able to come up with my own stuff? I honestly do not know. I think too the move to run zope with wsgi without ZODB is a good one. I am challenged to think of a situation where using ZODB would be advantageous to using a RDBMS. Rant on here. I still do not know where we are going yet and I am going to take zope3 for a thorough drive before making my decisions.

Some other opinions:

What django can learn from Zope

Time taken to create a trivial app. <-- Trivial applications or play code is less useful as most of the time they usually fall on the side of the "barebones" frameworks like django, ror or pylons.

I wrote this post because I know as time goes along and to remain agile and more importantly fast, with more projects under our belt, re usability will be a feature that we want. Currently we are doing fine with django but will the future paint a come back for Zope into our organisation ? Personally I hope so.

datepicker without admin

Django's admin interface is one of the really nice things about django because of it's functionality and useful tools. Nearly all of the reviews I have read about django have mentioned a good thing or two about the admin interface. There is however sometimes a need to go beyond or implement a lesser version of admin interface. This is where the adventure of living outside the realm of comfort for me begins ...

I have been trying to live outside the comfort of the admin interface for django and have found other niceties outside of the admin interface namely generic views, forms and others. I have been using these tools to build a web inventory cum sales application for one of our clients and happy to say it has been fairly successful.

Generic views are really powerful and really helps to shave off development time. There are however withdrawal symptoms my code is feeling to the admin interface. I have reimplemented some of the css niceties and javascript in my code. One of the one I am looking for is to re implement the cool datepicker admin has every time it encounters a DateTime or a Date Field.

I have tried a few things towards this goal:

1. Included all the javascript from the admin interface for my page. This just failed silently and my text box looked just the same.
2. Passing the model.DateTime a widget argument in which I call the widgets from django.contrib.admin.widgets. This resulted in a unexpected keyword error.

It would be nice if the django admin team can do a small writeup or a break down on how the admin interface was implemented. This would be really nice as there are some really good stuff in there which can also be useful in it's bits and pieces.

Update on the python windows saga

The fiasco finally ended, my code is working and the customers are at least quiet. My requirements were simple, my excel trawler trawled the excel pumping stuff into the database. Then after that it had to copy the excel file based on the outcome of the process. So if it failed it had to copy the input excel file into a 'fail' and conversely into a 'pass' folder if it did it's job right. Here is the catch. The customer wanted the whole directory structure of the input file to be created in the fail/pass folders. I thought "Heck this is simple! Just do a 'cp --parents /path/to/input/file /[pass | fail]/folder' What could be easier?"

Boy was I wrong. All of the windblows solutions did not fit right. I found this page by Tim Golden which I thought was golden until I tried out most of it's stuff and it did not exactly work right for me at least. Essentially it boiled down to:

1. If it copied the directory tree(shutil.copytree) it did not copy the file right. If it saw a file in the path, it would just copy the file and the did not do the sub folders and the parent folders. Augh! Maddening!!

2. I could not just do a system call to do 'cp --parents' because this was a windblows machine. So I thought initially os.system stuff was out. It drove me nuts something so simple could not be achieved by any of the windows command line stuff like copy or xcopy or robocopy.

Finally I cheated. I downloaded a windows port of cp, and just a os.system call on it. That too had it's fair share of problems such as you could not do a 'cp --parents c:/some/thing c:/some/one' because it tried to do a c:/some/one/c: and bombed out because it did not like the colon. Looking at this I used the os.path.splitdrive and split up the stuff then did a copy.

I know this is not clean enough for you purist out there, but heck it works and that is all that counts ... to customers at least.

Spaces in directory names ... I really love you not!

Have I mentioned I really love Python? Have also mention that I really hate Windows? I did not? Well here I go again ... I REALLY HATE WINDOWS. This time the target of my scorn or anger is the stupid spaces in folder names windows users are so apt to give to their folders. It screws my programs and special attention has to be given to quote them to work...


Grrrr, all sorts of funny stuff happen at this stage. I am wondering how the heck I am going to quote the whole filename if I am doing something like below:

import shutil

for file in os.listdir("/home/lowks"):

shutil.copy(file,"/tmp")


I tried 'shutil.copy('%s',"/tmp") but it spat out errors at me. Is this because the string is a unicode string? If so how do you convert the file path to a raw string? I really hope if someone has experience doing this to shine some light this way.

Ops ... it seems I made a little error there, it seems to work but not in my longer script. My script basically has to copy a file including it's parent folders into a fail / success directory based on the outcome of running the script. I tried shutil.copytree but it does not work and fails silently, which prompted my spaces-in-directory-name hate. I am wondering aloud here there must be an easier way of doing this, a one liner thing.

I think also I have found the error with my script. It is still a space related issue where if there is a space in the folder name, then it automatically treats the last string as a file name and shutil will bomb out when it tries to open up that file for copying or moving, like something below:

Traceback (most recent call last):
File "process.py", line 321, in
,failure_directory)
File "D:\python25\lib\shutil.py", line 80, in copy
copyfile(src, dst)
File "D:\python25\lib\shutil.py", line 46, in copyfile
fsrc = open(src, 'rb')
IOError: [Errno 13] Permission denied: u'C:\\import2\\testing folder'

In the case above, the 'testing folder' is actually a folder but shutil have interpreted the 'folder' as the filename hence the exception.

Ah ask and ye shall receive, in windows there is a short file name for all files for example C:\Program Files is equals to C:\Progra~1 or something to that effect. This same short filename can be used in python if you use the win32api, in a one liner like this:

import win32api

long_file_name='C:\Program Files\I am a file'
short_file_name=win32api.GetShortPathName(long_file_name)

This works. If you are incline to read more about this go here.

django and some plone

There seems to be some rumblings on the Plone front for us currently. For our main stable of development however, we have moved away although not totally from Plone to django. The main reason here is that for Plone deals to be feasible for us, we either have to:

1. Lock requirements down really tight so that the juniors can have a go at it .... or

2. Sic some seniors at it.

In my own experience, it seems that while Plone can be easy if worked within a tight parameter, try swaying a bit out of those parameters, when you have to peer under the hood and you are in a world of pain. It's hard to gain leverage using Plone this way. django on the hand has been actually quite young developer friendly for us and they end up being actually more productive compared to when they are forced to make some funky changes in Plone. Most of the business applications that we build require a light framework and is normally built from ground up. This means that Plone is sometimes an overkill for these apps which is a shame because Plone almost always appear like a good fit in the beginning. In the end, to be able to shoehorn Plone in means a lot stripping functionality and code out.

For what it's worth we still try to train up our guys to do basic Plone stuff but only time will tell if it makes business sense to invest any time in Plone/Zope.

A simple find in python ?

Continuing on my saga with my Python importer for xls, I ran into a situation where I had to dig in recursively into a directory structure and do something with Python. There seemed to be nothing easy in Python such as a 'find . -name '*.xls' -exec perform_task {} \;'. All of the solutions or suggestions that I have run across so far from stuff found in google seems to need me to perform some sort of mental or code acrobats. This is not to create a flame, I am just an intermediate in this so I am welcoming suggestions to do this if you people have any.

scraping data from excel files

I am currently still in the midst of writing my excel scraper to scrape data from excel files. There are two types of encrypted excel files that I need to scrape from. Both of them contain similar data but are structured differently.

The first problem that I encountered is that the data position are not standard and scattered all around the place, for example in the first type of file the data starts reading from line 10 to 31 while the second type of file starts reading from 11 to 31. The form looks neat and good to the human eye but is a coordinate hunting nightmare programmatically! While both of the files contain similar data, I found that one method to scrape data is insufficient because of the tiny differences in the location of the data. I cannot use odbc methods to run sql statements on them as they are encrypted, so I am stuck doing coordinate targeting.

After much doodling around with positions of data, I finally decided to write a Form object that takes a record_type an argument. When I instantiate a record_type of say 'A' type it will then load up the profile of the 'A' type object which is a dictionary that contains all the information of where most of the data are on the form. This dictionary is my implementation of a profile. So from there my object will know where all the important bits of information are based on the profile from the dictionary.

Anyone have done something similar in a better way?

surrounded by win32com's python and not liking it

I am using the win32com library to write an application to populate data that are in encrypted excel files that are kept in a specific folder into a mysql database. Most of the part is there except that I do not find much of win32com library documentation around. One part of my code really irks me and it's the excel visible part. Everytime it is processing an excel file, it has to open up the file and close it up again. When I set the visible part of the code to 0 or false, my application doesn't run correctly anymore. Some of my code are as follows :

import win32com.client
xl=win32com.client.Dipatch('Excel.Application')
xl.Visible=1

Grr ! I wonder why this thing doesn't work if you set the Visible to 0. Can't this work in the background? Woe !

Enterprise project lessons

Being in the technical lead for an enterprise project teaches you a thing or two. I was wearing those shoes recently for one of our projects recently and here is what I learnt :

1. If you are going to be using a framework of any sort you better know it inside out, left to right and top to bottom. When funky modifications are asked for or if your application starts to slow down you better know exactly or roughly where to start tweaking. This is the reason too why my partner feels that the bigger the project, the closer to the metal the framework used has go to be.

2. If the scope of the project is large and you suspect you might not have enough developers, never never never choose to use a tool which leaves you with one choice when it come to developers. For our case we had to give in and go the php route for this particular project. In my company we have shunned php in the past choosing not to use for any projects, but we had to bow in to pressure this time because it was just too hard trying to find capable and dependable python developers where we are. Once we caved in and accepted php, we found a pool of capable php programmers.

3. I can never over emphasize the next one. Never never never never under estimate the scope of a project especially if it pays a lot. When the customer says "I just need a web page to capture data" always ask for more information. It might not just be the application that is complex, it might even turn out to be the setup itself that is going to pose a challenge. In our case, we had to deploy multiple instances of our application in a few remote places to allow data entry people to key data. At the end of the day we had to merge the data all back into a central database.

4. For any project always look for a capable PM. It might lower your bottom line, but trust me it's worth it. A good PM will save you 3 am calls while at the same time allow you to multi task in the back ground and do what you do best. Fortunately for us, we have found such a candidate in my old business associate though we cannot use him for this project fearing our customer might just freak out seeing a new PM on the job !

5. Always have a old school unix or Linux who knows all the old text mangling tricks loaded and ready. In this case it was my partner. Even when doing seemingly unrelated stuff like web application never under estimate the usefulness of these guys. We had to move about 50,000 records trapped in excel into the database and his skills proved invaluable ! (Not that I am that much behind now, but I had to concentrate on getting the web end up)

In the end though, through all the late nights and sweat, this project has thought invaluable lessons in project management as well as resource planning.

python unicode sucks

I have had some bad experiences with python unicode on my current project and suffice to say I a tad nervous about using python for my next project. The unicode support is just horrendous and is very limited. I tried googling for unicode+ and nearly everytime it came back with multiple hits ! I read about a version of python in cvs that has unicode support built in but then I noticed the word 'cvs' **shrudder** My experiences might not be that complete so I would appreciate input from other senior snake wranglers out there.

django pluggables

I have struck goldmine ! Bored today with another plone innard hacking session I began hunting around on the internet for a django plugins site. I came across this site called http://www.djangoplugables.com/ ! This is exactly what I am looking for with a cute little plug in socket as a logo! I am astounded by the speed in which django has grown !

I have coded some applications with django the past and I can say I like the feel of using it. It is has what I need in terms of a web application framework and is closer to the python metal than plone or zope will ever hope to be. Don't get me wrong, I still have a soft spot for zope and I truly hope that zope 3's modularity will be more elegant than the monolithic monstrosity that zope 2.x is.

Today while looking at the pluggable site, has given me much more confidence of adopting django as one of the stable technologies to deliver web applications in my company's offerings. Coming from a background that have used both I can attest to why a senior python programmer would love django. I hope though the django community get it right and do not start on the path of sticking stuff together to create another zope 2.x !

index vs. metadata

Newbies hacking up scripts for the first time in a zope or plone environment might be confused at some of the terminology, heck I know I was ! Anyway, one that took me some time to grok not to mention numerous emails bugging people on the mailing list to understand is the difference between portal catalog indexes and metadatas. I will just say it in my own words here as I understand it now.

Index are keywords you add to portal catalog to allow you to create query based on them. Eg. query['Title'] = 'lowkster'. Here, 'Title' would be an index. The easiest way to see all the index in your portal_catalog would be to use the zmi. Go to the root of your site using the zmi and then access your portal catalog using the link provided by the left hand side menu. Here youwill find a tab called 'Indexes'. Here are the listing of all the Indexes in your portal's catalog. You can also add a new index in your portal catalog using this page.

Metadatas on the other hand are stuff that you want to show in your query results. For example you query your portal catalog and in the result is a metadata called 'Books'. You can then in your page template display this by calling it as such '. To set one of the fields in your content type as a metadata field, you have to add the variable 'isMetaData=True' to your field definition. You can also add a new metadata to the portal catalog using the zmi.

Anyway, forgive me if my way of explaining indexes and metadatas is wrong. This is just my way of understanding the difference between the two. Feel free to correct me if I am wrong.

getting tapping to work on fc9

Currently i am running fc9 on my trusty old acer and one of the initial irritants I found was that tapping on the touchpad no longer works (In opensuse it worked just fine out of the box). This is a real annoying bug as I am used to tapping my way around using my touchpad. Scouring the fedora forums, I finally chanced upon this entry in the redhat bug list. Well, well, well it turns out the "bug" isn't really a bug after all. A long story short basically your package of synaptics from fedora has a patch that has disabled tapping on touchpads because some wiseguy thought that it was for the best since no one really likes tapping on a touchpad.

So a guy called Bob Kashani was kind enough to rebuild the packages and remove the the offending patch. All you have to do is download his rpm from :

http://www.ocf.berkeley.edu/~bobk/packages/

Run a rpm -Uvh, do a restart of the Xorg by doing the old ctrl-alt-backspace and you are good to go :)

Thanks bob for the package. It works great here !

ATVocabulary Manager with MasterSelectWidget

I am using MasterSelectWidget in one of my projects to create the master-slave/master-slave combo of select widgets in my form It then occured to me how nice would it be if I could get my vocabulary from ATVocabularyManager which I have already installed on my site. Well, I lied it's not actually nice... it's one of the requirements :). Anyway I tried a few times to point my dynamic vocabulary at ATVocab's method but it failed to load up the Select widget with it's values. I wonder if this has been done by anyone ...

Archecsv

Recently while working on a plone project I found the need to import a whole lot of csv based data into my plone site while creating objects on the fly. I found just the thing to fit this need in ArcheCSV. This is a nice little product that you can add on to your plone site and can then import data from your csv files in batch and map it on to existing AT based Objects on your site. Good one this ! Just remember update your catalog for your objects to show up and start using them. Using this Product and Archgenxml one has little code left to touch ... :)

Ouch ! Torrentspy

Ouch ! It's a sad day indeed today when I tried visiting torrentspy and was greeted with this :


One can only guess the real reason behind the close but like they say it's been a wild and most often than not a wonderful ride. Good bye torrentspy.

Sles 10 Training

Just finished a three day training on SuSE Linux Enterprise Server 10 for some iperintis folks. It was a nice change of pace for work and the interesting thing that I found was the tremendous amount of improvements that the SuSE distro has undergone. From it's butt slow updater in the 10.1 version to today's zypper and tolerable updater, I must say I do not have to strain to see the improvements that the team has done. The students were quite impressed to see that installing opensuse and sles was even easier than windows and were just tickled pink over some of the tools that they were shown. This first training is for the fundamentals level while next week they will be going through the intermediate and advanced levels. Good luck people !

YUI datatables does not work for IE ?

I have been playing around with the datatables from Yahoo UI package for one of my customers. Much to my surprise though the code does not work for IE. I have been googling around for somebody reporting the same issue but so far I haven't found anything to the effect. I am just following the same example from the YUI site. On using YUI for my work, everything is good but then there are a few issues I am running into :

1. caching on the datatable itself does not lend itself to easy doing.
2. compatibility issues between browsers doesn't lend itself friendly to enterprise solutions.

Going to look more into this.

What has happened to plonecollectorng ?

One my favorite product on plone is the plonecollectorng. I like all the functionality that it offers. It was imo ahead of it's time. I was actually planning to use it to build part of company's portal as the customer care center but as I found out later it seems that plonecollectorng has disappeared from the plone world. When I ask on the channel it's people who answer me only do so in whispers and would rather not have anything to do with it and even went so far as to suggest me a new collector called poi. I felt so much like Charleston Heston in Planet of the Apes !

I am sorry I lost touch with the plone world a little while, but can anyone actually explain to me what has happened to plonecollectorng ?

We are live on the unoffical python planet

Python programming has been slow this few weeks as I am busy working up business deals all around, with three plone presentations set up for next week. Life is busy but good. We have generated some local interest here in Malaysia especially with the ISO standardizations in using plone as their document management system. With plone's multitude of functionality, a full blown document management system is just a few clicks away.

On the blog side I happy to report that this blog is now part of the unofficial python planet community ! So we are live on two channels now ! Yayy !!

Plone - redirecting authenticated users to member folder after login

Another in the useful-hack-department

Task :

After login you want authenticated users to be directed straight to their folders. Normally, after login, the users are redirected to the where ever they wanted to go in the first place.

Solution :

The file that controls what happens after a user logs in is the file CMFPlone/skins/plone_login/login_next.cpy. So customize this file and edit it.

Basically what is happening here is, login_next.cpy will look for a variable called came_from. If the came_from variable is set, then it will redirect the user to whatever came_from is set as. At this point I knew what I had to do, the challenge was to find the right methods to achieve what I want. I needed the method to tell me what is the authenticated user's home folder. After some digging around I found it.

Just add this line to the login_next.cpy, somewhere at line 25 :

home_url = membership_tool.getAuthenticatedMember().getHomeFolder().absolute_url()
came_from = home_url


It's pretty evident what this two lines does, it basically calls the membership_tool's methods to determine the authenticated user's home folder's url and set's came_from to that value. After adding those two lines in there, login and logout again and all of your authenticated user's should be redirected to their home folders after login. Happy hacking !

plone-3.0.5 is out and it's great

Plone 3.0.5 is out, finally ! I was waiting for this release for it to solve a bug with archetypes where when I used argouml to generate a plone type, it's required fields did not work correctly. Well, I tested the release out on my lappie and now it works like a charm. Damn this is sweet !

For the uninitiated few who have been living under a big tempurung for like a year, archgenxml is product for plone which generates plone products from your uml diagrams. So what this means to you o' tired developer is, YOU DON'T HAVE TO TOUCH ONE LINE OF CODE to have a working plone product at your disposal.... YES ! Don't adjust your browser text size, you read me right the first time. YOU DO NOT HAVE TO EVEN TOUCH ONE LINE OF CODE. Just design your application using something like argouml, feed it to argenxml and BOOM BAM in as long as it takes for you to restart your plone instance, you have your product sitting there waiting for you.

Take your first step on your journey here and start taking on as many plone projects as you can ... woohoo !! Damn this product is so good it should not be free !

Zope's best kept secrets

Stumbled upon this gem of an article during one of my searches, the article does a decent job of summarizing what are the best features inside zope but wonders why zope is still the best kept secret. Reading that reminded me of a video I saw at plone.tv. Check it out here. Okay, so this is a plone screencast, but I shrudder to think of zope's future if the same people are involved. I mean come on, this guy is supposed to be involved in marketing plone and he is boring me to tears ! That ain't supposed to be happening. I am supposed to feel inspired after one of these right ? Heck at least the grok presentation was funny ....

I mean I can understand if English is not his language, but give a little more enthusiasm than that huh ... ?! Put me up infront of a stage of people and I can sell plone better than that pal. More thought should be given into zope / plone marketing if this is the state of the team involved with marketing it now.

More good words for z3

Electric Duncan some more good things to say about zope3 or z3 here, this time it's vs django. To a certain extent I do agree with some of the things mentioned in that article. At times I do feel that I am trying to bend django to my will when I was coding my application, but alas, I am too green in z3 to give any meaningful comment regarding the comparison. Here, duncan mentions something that piqued my interest - Coding his application in z3, with learning curve thrown in took him two days ! Now that is a wow-ser for me !

Although at the back of my mind the article look more slanted towards zope than django which might explain why the author felt more at home with z3, but then it's a good read after all and has inspired me more to go towards giving z3 another serious look for my web development work.

grok zope

Just finished my grok.zope.org tutorial and much kudos to the author for making zope easy. The tutorial is well paced and newbies should have no problems understanding some of the concepts of zope, although to be honest when the part about the persistence come rolling along, it would pay to have some zope experience.

To me it is good to see zope reinvent ( or in this case re architect ) itself to meet the demands of web application frameworks of today. It used to be that people never looked twice at zope when you needed to deploy a basic web application because they did not want the extra baggage that came with zope ( esp the 2.x branch ) although it came with a host of useful features that would really help to shorten development time of any web application. Now with zope3, I am confident of zope being the cog that can fit into much much more wheels out there.

Good stuff ! If you haven't checked out what the whole bruhaha is surrounding zope3, do yourself a favor and drop by grok.zope.org and try out the tutorials, the very least is you will walk away understanding the zope framework and architecture a whole lot better.

Installing zope3

Wanting to try out the tutorials over at grok.zope.org ( ** CORRECTION - Actually you don't need zope3 to tryout grokproject - My mistake **) which seems really really yummy, I decided to buckle up and install zope from source on my trusty old lappie. It just requires a few steps and shouldn't be scary to all ye old sys admins out there. So, this short howto is meant for those who just dipped the big toe into the big zope pool and who are just trying to get their heads around the whole architecture.

1. Get the relevant sources. In this case python-2.4.4 from python.org and zope-3.3.x source from zope.org

2. Building python. This one is easy. Just untar the python package and then issue the commands : ./configure --prefix=/opt/python24;make;make install This will install the python binaries in the /opt/python24 folder. The reason why we have to build the python 2.4 binaries is because zope3 works best with the python-2.4.x branch.

3. Next you have to build zope. This involves untarring the zope source and then in the source directory issue the command : ./configure --prefix=/opt/zope3 --with-python=/opt/python24/bin/python. This will tell zope to install it's binaries in /opt/zope3 and use the python in /opt/python24/bin/python instead of the system PATH.

3. Create the zope instance : './opt/zope3/bin/mkzopeinstance -d /opt/zope3/instance -u [username]:[secret] where username and secret is the username and password of your choice when you want to access your zope instance.

4. Starting up your zope instance : 'cd /opt/zope3/instance/bin; ./zopectl start'. Open up your browser and try to access 'http://localhost:8080/manage'. If your installation is successful, it should prompt you for a username and password pair. Fill in the username and password from step 3.

Congrats ! You have just successfully installed zope 3 !

getting to pesky foreign key data in django

Optimizing databases is a good practice and should be done for any enterprise applications, but as all things go there is a price to pay. The price you pay is when calling back and displaying the data into one can be real PITA. django is a damn good framework and allows you to do work fast with a pedal-to-the-metal feel but then certain amounts of work still has to be done when you have foreign keys defined.

Say you have a database with two tables : contacts and address. The relationship of the two tables are of the one to many kind, I could never really wrap my head around the one-many many-one many-many thingy so let me just say it in layman so I understand it better myself. "One contact can have one or more than one address"

class Contacts(models.Model):
..... bla bla

class Address(models.Model):

customer_id=models.ForeignKey(Contacts,edit_inline=models.STACKED,num_in_admin=2)


So in the code above you have basically linked both the tables together using a ForeignKey relationship. the 'edit_inline' ... mumbo jumbo just tells django to add the address field into the main Contacts form during creation and modification and not as a pop up link ( This is important and could be a clincher for normal users ).

Calling all the data back for display

Okay so now you are honky dory with a great admin form for you to add data into your database, now the problem starts when you want to call all the data back for display in your template.

Don't worry django comes with little helper functions, you just have to do a few more steps to display your foreignkey data.

For my case, in my view.py I do something like this :


from stock.models import Contacts

def main(request,id):
contact_list = Contacts.objects.get(id=id)
return render_to_response('stock/mytemplate.html',{'contacts':contact_list})

Probably the html formatting would made my code not valid in python, but just check ur identation especially after the function definition.

The few lines of code above basically expects you to send it an id ( from the url using the GET method ) and then get the Contacts object and then packs up that data object and sends you merrily along to your template with that.

Okay, a bit about the Contact objects. Inside your contact objects if you have a foreign key relationship it automatically create a method for you to access the data. The method is called _set. So following our example, we know that we have a relationship with the table Address, so we can expect that each contact object will have a method called 'address_set'. Still okay so far ? Lets go ....to get the data you normally call it like this :

contact_list.address_set.select_related()

This returns an orm object inside a list, so on your template you will call it like this :

{% for address in contact.address_set.select_related %}

Address: {{address.address|escape}}

{% endfor %}



That will bring up and display all your address for that particular contact. Done ! Easy peasy, well for me at least just comment me if think more clarification...

Plone 3.0.5 !!

Waiting for the 8th of January which basically means that Plone 3.0.5 will be released. I am waiting for this release because it fixes a bug in Archetypes which currently causes validation on required fields in a form generated by Archgenxml to fail. After learning the few curves of using ArchgenXML, I must say I am bought hook and sinker on this product. It's really good stuff. With just the drawing of the uml class diagrams allows a developer to generate Plone code, almost too good to be true.

I want to be able to cover the Archgenxml topic in one of my classes on Plone. Now I am still working on the course structure of how the plone course will be like. Will keep you guys posted regarding this too.

A different view of blogger

After having a gym session and talking to geekool, I am having a different view of blogger. I used to like having my own plone site and managing it, but after chin seng explained how it would make more sense and less headaches just having your blog in blogger, it just hit home to me.

One thing about having my blog here is that no one can come and tell me that need to move it or need to shut it down or anything like that. So, I am going to put my blog up here from now on and this will be the central point of dissemination of all information.

On projects, currently I am still working on a web project using django and I am learning up the little nooks and crannies of the language. It would seem that most of the project's repo is placed in google code repositories. I am using a calendar event but it seems to be buggy. But while researching for this, I found this project that seems a bit more promising. Will look into this and report later on this.