Monday 10 May 2010

Authenticated downloads using WGET

Today I needed to download an Adobe Livecyle ES 2 trial version directly to a headless Amazon EC2 instance. So I wanted to use wget and for most downloads, direct downloads this is pretty easy: wget http://www.somesite.com/download.zip. The problem I had, was that you need to be logged into the Adobe site to be able to download the trial. The login basically means that cookies are created and because wget supports cookies I just needed a way to capture the cookies in the correct format to some file on disk.

This can be easily done using the Firefox Cookie Exporter add-on. I just needed to open the Adobe site in Firefox, log in and click Tools > Export Cookies... . This will save all the current cookies to a file on disk. This file can then be used together with the --load-cookies switch present in wget to submit all the cookies present in the file when doing a web request.