Friday, September 18, 2009

Patching Fun - Update

Looking around for a solution for a cleaner way of patching up my code on the Windows server, without installing the unixtools for windows, I eventually arrived at the python-patch tool. This nice little (less than 600 lines) tool allows you to have patch like capability and all deliciously packed in a Python script. Thanks techtonik!

So now I can just packup the Python script together with my patch file, and Walla! the admin at the other side can just run something like:

python diff_patch

to patch up my code. With a little modification, the can even generate a results.log for me to review the patch process. If anything were to go south, I can just send the reverse patch.

Now .... should I just get the script to autorun .... hmmm...

Thursday, September 17, 2009

Patching fun (on windows) 101

I have sent about 100 odd patches to a system I am working on and in this post I gather together some of the lessons I have gathered along this path. I write all of my patches in Python and after much hair pulling sessions over the phone with customers am happy to say the sacrificed hair did go along to help me have a epiphany of what would be the best way to help customers who are willing to help us apply patches along. You can say it's a collection of what to do and what not to do tested in real life project scenarios.

First off and of ultimate importance is to get into the right frame of mind before you write a patch to be sent to a customer. The frame of mind you should be in is ... remember that paying customers never ever want to help you apply patches so if you run across one that is willing (like in my case), thank Allah, Buddha, the Universe or whatever deity you pray to for your lucky break! Your (paying) customer does not owe you anything, in fact you are in their debt if they even entertain your patch request. Get in this frame and your chances of writing a successful patch will be increased two fold.

Okay, here are some dos for writing patches. These steps below are for systems that are not accessible remotely and a bit physically too far or demanding for your to travel to.

1. Always recreate (as much as possible) the environment in which to run the patch. This means that if your customer's site has a G Drive (windows) your machine in which you are creating your patch better have that. I have had some great time tracking down why a patch does not work when it magically died while trying to open up a log file on Drive G in a machine that only has C and D drives. This includes in the case of windows of using the same binaries of libraries that your code belongs to. Just recently I had a good time on the phone with a customer who claims that come functionality was not working whereas using we were both using the same script! He insisted that my script was not working while I kept on suspecting that the problem lies between the computer and the chair. Finally I found the problem was due to an outdated binary build that he was using. If having the same build is no longer possible, at least go through the changelog of whatever library you are using to get prepared for whatever problems that might arise.

2. Always have backups of each of your patching steps. In fact it's best to include a sort of un-patch patch just incase your patch goes south. The steps to backup the important files should be done by your script and should not depend on your customer. So something like 'Please backup and replace file bla-bla" is not recommended because how do you know what your customer is backing up the original file to. Here you are thinking that they backed it up to something like .bak and they backed it up to .bak. A lot of fun will ensue in the case if you have to write a revert script later. Step no. 2 can be summarized as your customer should do as least as possible in the patching process short of running a patch script.

3. Generate error and result log of your patch process and ask the good will of your customers to send those logs back to you. This will help in determining if your patch process is successful.

4. Describe in your email what should the customer see in the event that your script is successfully run. I find here the quickest and surest way is to include a screenshot of what they should see. Typing out the results is error prone and it's hard to describe what they should see plus do you think your customers got that much time to read a long drawn out email?

5. Test, test, test and test each of your steps at least 4 times before sending the patch out. No testing is too much in this kind of scenario especially if you are dealing with live data.

6. Always include a way in which to identify what build are you customers currently running. This could be as simple as a BUILD text file indicating what BUILD are they currently running or in the case of a web application, a build number nicely tucked away at the top left hand or right hand corner of the screen. Everyone of your patches should update this BUILD number. This is instrumental if you were on the phone with your customer and want them to help you identify how updated are their source code.

Hopefully this post will help me remember the things that I did right and those that I painfully learned not to do again. It would be great too if it can help you avoid those pot holes that have claimed the lives of a few of those hair on my head. Happy patching!

Monday, September 7, 2009

import scripts using python

Today I finished another import script using python. The import medium is from Excel to MySQL db. I began my importing exploits using Python and have used it ever since. After doing so many import scripts using Python as my main language I could not think of using anything else. While some other people might be using Bash, I have been using Python ever since the beginning I got my feet wet in doing data migration for systems. Some of the reasons why I like Python for this task are:

1. The wealth of libraries available. I particularly like the csv module which I use heavily in my imports the DictReader module in the csv module also makes short of a lot work during imports. I just cannot imagine using something else.

2. The dicts, the lists and tuples are indispensable and great little tools in sifting through data.

3. Python's clean and structured program allows me to pass the script to juniors or allows me to look back at the import script to either use it at another location or improve it 3 months down the line.

Then the question is why don't more people write import scripts in python? Well I can think of a few cons when it comes to using Python in import scripts and these are:

1. For old ass machines such as old style Solaris or HP/UX Python is just not installed by default. While you can install a copy of python on these boxes, sometimes you do not have that liberty and besides the first rule of any importing job is not to add more applications on the system you are trying to import data into.

Hmm I think I can only think of one valid reason why using Python would not give you and edge. I am using Python till this day importing all my data and have not faced any issues yet. How is your experience ?