April 7th, 2008

Gmail uploadr

[UPDATE] Run in Ruby 1.8.7 since Mail (mailread.rb) was removed from the Ruby 1.9.3 Standard Library.

… moving to Google Hosted (Google Apps), there was the need to upload some thousands of e-mail messages. Since they were (mt) MediaTemple Berkeley mbox stores, i scp’d them locally (backup) and them imported them into Evolution local folders.
The problem was the copy/move to the google hosted imap mail server. Evolution couldn’t finish the task on severall tries. Mozilla Iceape (Seamonkey) had a similar behaviour, suffering from server disconnects, thus stopping the operation. Because i had severall folders, the task needed lot’s of attention, and i wanted the machine to do it all by itself…

Solution: let’s code !! What response is the server sending… ?

Shugo Maeda’s Ruby net/imap.rb class (stdlib) is simple and trusty.
Not wasting lots of time in it, i made a new class with the following usage, to send specified mbox file mail messages to a gmail (google hosted) account:

$ ./2gmail.rb MBOXFILE GMAILFOLDER [STARTINGMESSAGE]

If GMAILFOLDER doesn’t exist, gets created (i prefered messages in temporary folders) upthere.
If it breaks, you can:

$ tail 2Gmail.log

to see the last message sent, and restart the process in that message. I could have threaded the whole thing, but this is done just once, so lets keep in focus…
Usual problems during the tests, msg headers/body separation, timestamps, etc…
Class Mail in mailread.rb does a great job creating an Hash for the Message Headers. Sweet…

Everything is logged, so that it’s easy to ‘grep’ for errors in the 2Gmail.log file.
I had one only problem on a message with a 25MB attachment (gmail didn’t accept it).
All the messages are now up there, nobody knows for certain where (EU ? US ? both ?…)

The code needs one or two more Exception Handling, but it works very well. I had 6 xterm’s uploading at the same time. And yes, I could have openned 6 tabs in Gnome Terminal since we have them for ages, Mr. Leopard ones! Hey, we can even detach Tabs to independent Terminal windows via drag-n-drop (like in Epiphany, way way long ago… way…)

There are some thousands more messages to be uploaded, from ancient backups… maybe next week.

Operation screenshot: jump to flickr at Flickr as usual
Wanna try ? Get it here (ready for Debian & Ubuntu. For other distros, check line 1 or just prepend ruby interpreter on call)

$ chmod u+x 2gmail.rb
$ ./2gmail.rb MBOXFILE GMAILFOLDER [STARTINGMESSAGE]

Gist: