Return of zope?

First off the bat, let me just this post is not meant to add more gasoline to the django vs Plone wars. It is just meant to read as my thoughts surrounding the Experience of using Zope and Plone and trying to use them both in my company.

It's no secret now my company is using django for most of our web related projects. Although I use mostly django now, I am like the kid who left his good pal to play at the other side of the fence. I constantly look back and think if we could still fit zope in somewhere in our projects. I think back right to the beginning when I did my first web project. Unlike many I did not start with php. I started with zope because my company at the time wanted to vary our skill base. I had many a sleepness night trying to wrap my head around the zope concepts and paradigms. I got my ass handed back to me many times at the #zope channel by MacYet yet strangely I persisted and slowly I grew to like zope.

Then came Plone. In my next company I deployed a Customer Complaints Form applications for them using Plone and Archetypes (it was called cmftypes at that time). Most of the time I did not know what I was doing yet at the end I succeeded in completing the project and the last I synced with my cronies there they are still using that application for their daily operations. This enamored my confidence in Zope and Plone. I also did a few other projects using Plone and generally found the going easy and liked using it although there was always a voice telling me to go deeper into the engine to see how everything ticked.

Coming from these positive experiences, I did not hesitate to recommend Plone to deploy a web membership system for a political party for a project that we had secured. This was when the nightmare began which chipped away nearly all of my confidence for using Zope/Plone as a development platform for my company. Here are some of the issues I ran into:

Technical Issues

1. Plone was over complete for the project. As with a lot of other projects I tried to shoehorn Plone into, I found that it could do the 90% of the requirements easily without even breaking a sweat but the last 10%, which usually is the most essential and mission critical ones required a huge re-engineering of some of the components inside of Plone. It was just too daunting of a task to do. To make it worse it was tasks that seemed very easy to non-Plone developers. "Can I make the font for this automatically generated font smaller here","Could you please remove this green drop-down here and place it here" .... etc. etc. Most of the time I had to explain to my principal his technical point of view and the Plone point of view. This created a lot of heated and opinionated exchanges and needless to say at the end both of us were unhappy. Most of the time I found that I either had to make some little change to some Plone component of Product that would cost me a lot of time because I had to understand how everything fit together before I could modify it. Other times I found that I had to strip out features that were not needed to start with. Deploying Plone for our prototyping stage created problems for us as users saw some of the components of Plone and started coming out with extra requirements based on what they saw.

2. ZODB Performance. This one really stuck a knife in my gut. We were not deploying the next Slashdot but ZODB needed to at least handle 30,000 to 100,000 objects at once. ZODB was too much of a blackbox at this stage for us to optimize when it when it went to the land of the Slo-Zo. I kept on packing the database up and found bottlenecks all around the place. I can totally relate to how this guy feels and why he too chose to go for a ZODB-less Zope. My partner who is a Dba felt pretty helpless without me when all his knowledge of optimizing rdbms was moot when the heart of Plone was ZODB which did not even have a SQL like syntax. At 30,000 objects it was taking 20 seconds to come back with queries! This alarmed a lot of people but I did not have good answers for them. Trying to run Plone on MySQL did not seem very optimal too as a lot of Plone's magicks and functionality seem too tied down to ZODB. I will concur here and admit I did not know enough of ZODB to make it really work for us, yet from all angle that we were looking at this, Plone seemed more and more of a wrong choice.

Non Technical Issues

1. There was just no developers around! I could find PhP developers by just talking to aunt Tilly that lives across the street but Python/Zope/Plone developers was a rare breed! Even if I found them, they were very expensive and usually overseas. However, to be fair most of those PhP developers that we found through Aunt Tilly lacked finesse and were not of standard. However, having some developers is still better than nothing at all and this was one of the reasons I had to surrender to using PhP. The trick now was not to code the application myself rather to leverage my time by hiring programmers to do it. My partner and I found that there was no way to deploy the project using Zope/Plone if we did not have anyone we can hire or outsource it to. The sheer pervasiveness of PhP won out here.

It was at this time when I started to look at other alternative frameworks that we could use that was a bit closer to the metal and allowed for easy deployment. I found django. Django was a joy from the start. Following the documentation allowed me to be productive from the go and for once I could understand what I was doing! There weren't 120 stacks of libraries I had to wade through to get something working for once I found that I could pass some of my half finished stuff to my developers and they could take over the project ! Like many coming to django there were of course some paradigm shifts I had to get used to namely the lack of some functionality in the django templates but I soon got used to that. jquery was also integrated into my pages easily and for the first time in a long time I know exactly what each line of code in my application did.

django is certainly no Php not by a long shot but at strangely I found that explaining django to a developer easier going than trying to carpet bomb their brain with Zope.

Recently, my projects started growing in size and after my views.py and my models.py started getting unwieldy and spinning off too many apps I started looking around to see if other people faced the same thing and I found other dissidents around. Most of the voices out there seem to say that django is good for small projects but found that to create re-usable apps using django they seemed to be re-creating zope! Although with django I do not feel that I am "fighting the framework" I could see where these voices are coming from. This is where I could almost see Zope developers snickering at a corner as I could almost hear them say "I told you!".

Certain developments too have happened and a few articles I have read, pointed to Zope3 being a totally different beast to Zope2. I read a few of the examples using the zope3 architecture and I must say they look a world easier but would I be able to come up with my own stuff? I honestly do not know. I think too the move to run zope with wsgi without ZODB is a good one. I am challenged to think of a situation where using ZODB would be advantageous to using a RDBMS. Rant on here. I still do not know where we are going yet and I am going to take zope3 for a thorough drive before making my decisions.

Some other opinions:

What django can learn from Zope

Time taken to create a trivial app. <-- Trivial applications or play code is less useful as most of the time they usually fall on the side of the "barebones" frameworks like django, ror or pylons.

I wrote this post because I know as time goes along and to remain agile and more importantly fast, with more projects under our belt, re usability will be a feature that we want. Currently we are doing fine with django but will the future paint a come back for Zope into our organisation ? Personally I hope so.