Return of zope?

First off the bat, let me just this post is not meant to add more gasoline to the django vs Plone wars. It is just meant to read as my thoughts surrounding the Experience of using Zope and Plone and trying to use them both in my company.

It's no secret now my company is using django for most of our web related projects. Although I use mostly django now, I am like the kid who left his good pal to play at the other side of the fence. I constantly look back and think if we could still fit zope in somewhere in our projects. I think back right to the beginning when I did my first web project. Unlike many I did not start with php. I started with zope because my company at the time wanted to vary our skill base. I had many a sleepness night trying to wrap my head around the zope concepts and paradigms. I got my ass handed back to me many times at the #zope channel by MacYet yet strangely I persisted and slowly I grew to like zope.

Then came Plone. In my next company I deployed a Customer Complaints Form applications for them using Plone and Archetypes (it was called cmftypes at that time). Most of the time I did not know what I was doing yet at the end I succeeded in completing the project and the last I synced with my cronies there they are still using that application for their daily operations. This enamored my confidence in Zope and Plone. I also did a few other projects using Plone and generally found the going easy and liked using it although there was always a voice telling me to go deeper into the engine to see how everything ticked.

Coming from these positive experiences, I did not hesitate to recommend Plone to deploy a web membership system for a political party for a project that we had secured. This was when the nightmare began which chipped away nearly all of my confidence for using Zope/Plone as a development platform for my company. Here are some of the issues I ran into:

Technical Issues

1. Plone was over complete for the project. As with a lot of other projects I tried to shoehorn Plone into, I found that it could do the 90% of the requirements easily without even breaking a sweat but the last 10%, which usually is the most essential and mission critical ones required a huge re-engineering of some of the components inside of Plone. It was just too daunting of a task to do. To make it worse it was tasks that seemed very easy to non-Plone developers. "Can I make the font for this automatically generated font smaller here","Could you please remove this green drop-down here and place it here" .... etc. etc. Most of the time I had to explain to my principal his technical point of view and the Plone point of view. This created a lot of heated and opinionated exchanges and needless to say at the end both of us were unhappy. Most of the time I found that I either had to make some little change to some Plone component of Product that would cost me a lot of time because I had to understand how everything fit together before I could modify it. Other times I found that I had to strip out features that were not needed to start with. Deploying Plone for our prototyping stage created problems for us as users saw some of the components of Plone and started coming out with extra requirements based on what they saw.

2. ZODB Performance. This one really stuck a knife in my gut. We were not deploying the next Slashdot but ZODB needed to at least handle 30,000 to 100,000 objects at once. ZODB was too much of a blackbox at this stage for us to optimize when it when it went to the land of the Slo-Zo. I kept on packing the database up and found bottlenecks all around the place. I can totally relate to how this guy feels and why he too chose to go for a ZODB-less Zope. My partner who is a Dba felt pretty helpless without me when all his knowledge of optimizing rdbms was moot when the heart of Plone was ZODB which did not even have a SQL like syntax. At 30,000 objects it was taking 20 seconds to come back with queries! This alarmed a lot of people but I did not have good answers for them. Trying to run Plone on MySQL did not seem very optimal too as a lot of Plone's magicks and functionality seem too tied down to ZODB. I will concur here and admit I did not know enough of ZODB to make it really work for us, yet from all angle that we were looking at this, Plone seemed more and more of a wrong choice.

Non Technical Issues

1. There was just no developers around! I could find PhP developers by just talking to aunt Tilly that lives across the street but Python/Zope/Plone developers was a rare breed! Even if I found them, they were very expensive and usually overseas. However, to be fair most of those PhP developers that we found through Aunt Tilly lacked finesse and were not of standard. However, having some developers is still better than nothing at all and this was one of the reasons I had to surrender to using PhP. The trick now was not to code the application myself rather to leverage my time by hiring programmers to do it. My partner and I found that there was no way to deploy the project using Zope/Plone if we did not have anyone we can hire or outsource it to. The sheer pervasiveness of PhP won out here.

It was at this time when I started to look at other alternative frameworks that we could use that was a bit closer to the metal and allowed for easy deployment. I found django. Django was a joy from the start. Following the documentation allowed me to be productive from the go and for once I could understand what I was doing! There weren't 120 stacks of libraries I had to wade through to get something working for once I found that I could pass some of my half finished stuff to my developers and they could take over the project ! Like many coming to django there were of course some paradigm shifts I had to get used to namely the lack of some functionality in the django templates but I soon got used to that. jquery was also integrated into my pages easily and for the first time in a long time I know exactly what each line of code in my application did.

django is certainly no Php not by a long shot but at strangely I found that explaining django to a developer easier going than trying to carpet bomb their brain with Zope.

Recently, my projects started growing in size and after my views.py and my models.py started getting unwieldy and spinning off too many apps I started looking around to see if other people faced the same thing and I found other dissidents around. Most of the voices out there seem to say that django is good for small projects but found that to create re-usable apps using django they seemed to be re-creating zope! Although with django I do not feel that I am "fighting the framework" I could see where these voices are coming from. This is where I could almost see Zope developers snickering at a corner as I could almost hear them say "I told you!".

Certain developments too have happened and a few articles I have read, pointed to Zope3 being a totally different beast to Zope2. I read a few of the examples using the zope3 architecture and I must say they look a world easier but would I be able to come up with my own stuff? I honestly do not know. I think too the move to run zope with wsgi without ZODB is a good one. I am challenged to think of a situation where using ZODB would be advantageous to using a RDBMS. Rant on here. I still do not know where we are going yet and I am going to take zope3 for a thorough drive before making my decisions.

Some other opinions:

What django can learn from Zope

Time taken to create a trivial app. <-- Trivial applications or play code is less useful as most of the time they usually fall on the side of the "barebones" frameworks like django, ror or pylons.

I wrote this post because I know as time goes along and to remain agile and more importantly fast, with more projects under our belt, re usability will be a feature that we want. Currently we are doing fine with django but will the future paint a come back for Zope into our organisation ? Personally I hope so.

datepicker without admin

Django's admin interface is one of the really nice things about django because of it's functionality and useful tools. Nearly all of the reviews I have read about django have mentioned a good thing or two about the admin interface. There is however sometimes a need to go beyond or implement a lesser version of admin interface. This is where the adventure of living outside the realm of comfort for me begins ...

I have been trying to live outside the comfort of the admin interface for django and have found other niceties outside of the admin interface namely generic views, forms and others. I have been using these tools to build a web inventory cum sales application for one of our clients and happy to say it has been fairly successful.

Generic views are really powerful and really helps to shave off development time. There are however withdrawal symptoms my code is feeling to the admin interface. I have reimplemented some of the css niceties and javascript in my code. One of the one I am looking for is to re implement the cool datepicker admin has every time it encounters a DateTime or a Date Field.

I have tried a few things towards this goal:

1. Included all the javascript from the admin interface for my page. This just failed silently and my text box looked just the same.
2. Passing the model.DateTime a widget argument in which I call the widgets from django.contrib.admin.widgets. This resulted in a unexpected keyword error.

It would be nice if the django admin team can do a small writeup or a break down on how the admin interface was implemented. This would be really nice as there are some really good stuff in there which can also be useful in it's bits and pieces.

Update on the python windows saga

The fiasco finally ended, my code is working and the customers are at least quiet. My requirements were simple, my excel trawler trawled the excel pumping stuff into the database. Then after that it had to copy the excel file based on the outcome of the process. So if it failed it had to copy the input excel file into a 'fail' and conversely into a 'pass' folder if it did it's job right. Here is the catch. The customer wanted the whole directory structure of the input file to be created in the fail/pass folders. I thought "Heck this is simple! Just do a 'cp --parents /path/to/input/file /[pass | fail]/folder' What could be easier?"

Boy was I wrong. All of the windblows solutions did not fit right. I found this page by Tim Golden which I thought was golden until I tried out most of it's stuff and it did not exactly work right for me at least. Essentially it boiled down to:

1. If it copied the directory tree(shutil.copytree) it did not copy the file right. If it saw a file in the path, it would just copy the file and the did not do the sub folders and the parent folders. Augh! Maddening!!

2. I could not just do a system call to do 'cp --parents' because this was a windblows machine. So I thought initially os.system stuff was out. It drove me nuts something so simple could not be achieved by any of the windows command line stuff like copy or xcopy or robocopy.

Finally I cheated. I downloaded a windows port of cp, and just a os.system call on it. That too had it's fair share of problems such as you could not do a 'cp --parents c:/some/thing c:/some/one' because it tried to do a c:/some/one/c: and bombed out because it did not like the colon. Looking at this I used the os.path.splitdrive and split up the stuff then did a copy.

I know this is not clean enough for you purist out there, but heck it works and that is all that counts ... to customers at least.

Spaces in directory names ... I really love you not!

Have I mentioned I really love Python? Have also mention that I really hate Windows? I did not? Well here I go again ... I REALLY HATE WINDOWS. This time the target of my scorn or anger is the stupid spaces in folder names windows users are so apt to give to their folders. It screws my programs and special attention has to be given to quote them to work...


Grrrr, all sorts of funny stuff happen at this stage. I am wondering how the heck I am going to quote the whole filename if I am doing something like below:

import shutil

for file in os.listdir("/home/lowks"):

shutil.copy(file,"/tmp")


I tried 'shutil.copy('%s',"/tmp") but it spat out errors at me. Is this because the string is a unicode string? If so how do you convert the file path to a raw string? I really hope if someone has experience doing this to shine some light this way.

Ops ... it seems I made a little error there, it seems to work but not in my longer script. My script basically has to copy a file including it's parent folders into a fail / success directory based on the outcome of running the script. I tried shutil.copytree but it does not work and fails silently, which prompted my spaces-in-directory-name hate. I am wondering aloud here there must be an easier way of doing this, a one liner thing.

I think also I have found the error with my script. It is still a space related issue where if there is a space in the folder name, then it automatically treats the last string as a file name and shutil will bomb out when it tries to open up that file for copying or moving, like something below:

Traceback (most recent call last):
File "process.py", line 321, in
,failure_directory)
File "D:\python25\lib\shutil.py", line 80, in copy
copyfile(src, dst)
File "D:\python25\lib\shutil.py", line 46, in copyfile
fsrc = open(src, 'rb')
IOError: [Errno 13] Permission denied: u'C:\\import2\\testing folder'

In the case above, the 'testing folder' is actually a folder but shutil have interpreted the 'folder' as the filename hence the exception.

Ah ask and ye shall receive, in windows there is a short file name for all files for example C:\Program Files is equals to C:\Progra~1 or something to that effect. This same short filename can be used in python if you use the win32api, in a one liner like this:

import win32api

long_file_name='C:\Program Files\I am a file'
short_file_name=win32api.GetShortPathName(long_file_name)

This works. If you are incline to read more about this go here.