Thursday 9 September 2010

Installing MySQL on RHEL4 without a RHN account


I needed to install a recent MySQL version, 5.1.50 in this case, on a server Red Hat Enterprise Linux 4. As I normally do on the binary equivalent CentOS I tried the yum command, but this apparently isn't available in version 4 of the operating system. Some Googling made clear that on RHEL 4 you need to use the up2date command instead of yum.

When I tried this new command I was presented with a screen that ask for a valid Red Hat Network (RHN) account, which I don't have as the server is owned and was installed by another consultancy firm that works for this customer.


I know I could ask the customer to give me the necessary information or at least try to find it or get it from the other firm, but that would take too long. But, as is usually the case, there's always another way - in this case - do a more manual install using an RPM package.

Two RPMs are needed: MySQL-server-community-5.1.50-1.rhel4.i386.rpm and MySQL-client-community-5.1.50-1.rhel4.i386.rpm, both the 32-bit ones for RHEL4. Installing them should be easy using e.g. rpm -i MySQL-server-community-5.1.50-1.rhel4.i386.rpm, but as usual this isn't the case and I get an error:

error: Failed dependencies: MySQL conflicts with mysql-4.1.22-2.el4.i386

A bit more Googling turned up this link in the MySQL bug tracker that described the same issue and error message. After reading through the issue a solution was found: remove the old MySQL version with the following command: rpm -e mysql-4.1.22-2.el4 --nodeps.

After this I was able to issue two rpm -i commands for the RPM files I downloaded and the MySQL server and client were installed and the MySQL was started. After this you just need to change the MySQL root password for security reasons using mysqladmin -u root password newpassword and you're good to go.

Wednesday 8 September 2010

The future of this blog

I know I haven't been blogging lately, but due to time constraints I can't find enough time for long post. That's why I've been trying out Twitter lately and I must say I like it. You can follow me on Twitter under the nickname planetsizebrain.

However this blog isn't exactly dead. Whenever I encounter something interesting in the future that's worth blogging about and that can't be expressed in a tweet, I'll be posting it here on the blog.

Monday 10 May 2010

Authenticated downloads using WGET

Today I needed to download an Adobe Livecyle ES 2 trial version directly to a headless Amazon EC2 instance. So I wanted to use wget and for most downloads, direct downloads this is pretty easy: wget http://www.somesite.com/download.zip. The problem I had, was that you need to be logged into the Adobe site to be able to download the trial. The login basically means that cookies are created and because wget supports cookies I just needed a way to capture the cookies in the correct format to some file on disk.

This can be easily done using the Firefox Cookie Exporter add-on. I just needed to open the Adobe site in Firefox, log in and click Tools > Export Cookies... . This will save all the current cookies to a file on disk. This file can then be used together with the --load-cookies switch present in wget to submit all the cookies present in the file when doing a web request.

Wednesday 28 April 2010

Find and replace in MySQL

This week I needed to do some find and replace on a column value in a MySQL table. I had a table that contained a column of the form name (id) and I needed to replace the id part with a fixed value.

When I started looking into a solution I found that MySQL supports
regular expressions, but sadly enough they aren't supported in the replace statement. Since I hadn't got a lot of time I had to do it quick and dirty by looking up the positions of the parentheses and then use those positions in combination with the substring function. That resulted in the following query:

UPDATE table SET column = REPLACE(column, SUBSTRING(column, LOCATE('(', column), LOCATE(')', column) - LOCATE('(', column) + 1), 'newvalue')
Just replace table, column and newvalue with the correct values and you're good to go. It also might be a good idea to try the query first in the following form just to be safe, as this won't change anything yet:

SELECT column, REPLACE(column, SUBSTRING(column, LOCATE('(', column), LOCATE(')', column) - LOCATE('(', column) + 1), 'newvalue') FROM table

Tuesday 9 February 2010

Fixing session fixation in Liferay on Tomcat

The last months I've been working on a Liferay 5.2.5 (embedded Tomcat 6.0.18) portal implementation for a customer and about a week ago they had a security expert do a penetration test on it. One of the biggest remarks was that the portal had a problem with session fixation, a problem that can allow a malicious third party to hijack a portal session.

A short description of the problem is that it is possible for an attacker to provide someone with a URL that contains a session ID that he has retrieved. When the unsuspecting user clicks the URL and logs in, the portal will use the provided session ID, thus enabling the attacker to request URLs with the same session ID and get results as if he had logged in.

The cause of the problem is usually that the web application doesn't change the session ID for a successful login, but continues to use the provided one. The solution is simple, change the session ID just before or after a login, the implementation of this proved to be a bit more difficult.

I first looked at using the login.events.pre or login.events.post events (see portal.properties) that Liferay provides, but invalidating the session, creating a new one and copying over the old session information to the new session in one of these custom actions didn't solve the problem. When using a custom pre login action invalidating the session caused a problem in Liferay's MainServlet because that keeps a reference to the old session and tries to set an attribute on it after the pre login events have been executed causing an IllegalStateException. So I tried a custom post login action since the MainServlet doesn't do any session manipulation after the post login events, but eventhough the action ran without exceptions, the session ID didn't change.

So I switched to plan B: a servlet filter. I used my post login action code in a pretty straightforward servlet filter and configured Liferay to use it, but the result was the same: the code was being executed, but the session ID stubbornly stayed the same even after invalidating the old session contrary to what this post claims (no fingerpointing, just an observation, could be a difference in Tomcat versions).

So what to do now, plan C? Some Googling brought one more possibility: a custom Tomcat Valve. By this time I feld I was getting into 'obscure hack' territory. But I still decided to give it a twirl since we're using Tomcat, no change of web container is foreseen for the far future and we're in control of the enviroment. I quickly threw together a Maven project in Eclipse, put the code attached to the blog post I found in it, packaged it, put the resulting JAR file in tomcat/lib/ext, added the valve definition to tomcat/conf/context.xml and restarted Liferay. Again the code was being executed just fine, but ... the session ID remained the same. Unbe-f*cking-lievable. 3 strikes and I'm out.

Or maybe not. I complained to a collegue about my trials and tribulations trying to solve the session fixation problem and with the provided information he was able to find a blog post with a variation of the Tomcat Valve solution. The solution is largely similar to the previous one, except that it manipulates the request/session a bit more. Applying these changes to the code of the previous Valve, packaging it again and redeploying it brought me to my fourth test and this time 'the bar was green' so to speak. This time the custom Valve seemed to work as expected and produce a different session ID between opening the first page of the portal and going to the login page.

No only one task was left to do: take the best parts off the 2 custom Valves and create an uberValve. One Valve to rule them all:

package your.company.valves;

import java.io.IOException;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;

import javax.servlet.ServletException;

import org.apache.catalina.Session;
import org.apache.catalina.connector.Request;
import org.apache.catalina.connector.Response;
import org.apache.catalina.valves.ValveBase;
import org.apache.juli.logging.Log;

/**
* Valve to regenerate HTTP Session ID's. Based on information
* available in the following 2 links:
*
* http://mikusa.blogspot.com/2008/06/tomcat-authentication-session-fixation.html
* http://www.koelnerwasser.de/?p=11
*/
public class FixSessionFixationValve extends ValveBase {

private static final String INFO = "your.company.valves.FixSessionFixationValve/1.0";

private String url = null;

public String getInfo() {
return INFO;
}

public void setUrl(String url) {
this.url = url;
}

public String getUrl() {
return url;
}

public void invoke(Request request, Response response) throws IOException, ServletException {
Log logger = container.getLogger();

if (url != null && !"".equals(url) && request.getRequestURI().contains(getUrl())) {
// step 1: save old session
Session oldSession = request.getSessionInternal(true);
Map<String, Object> oldAttribs = new HashMap<String, Object>();
Map<String, Object> oldNotes = new HashMap<String, Object>();

if (logger.isDebugEnabled()) logger.debug("Old session ID: " + oldSession.getId());

// Save HTTP session data
Enumeration names = oldSession.getSession().getAttributeNames();
while (names.hasMoreElements()) {
String name = (String) names.nextElement();
oldAttribs.put(name, oldSession.getSession().getAttribute(name));
}

// Save Tomcat internal session data
Iterator it = oldSession.getNoteNames();
while (it.hasNext()) {
String name = (String) it.next();
oldNotes.put(name, oldSession.getNote(name));
}

// step 2: invalidate old session
request.getSession(true).invalidate();
request.setRequestedSessionId(null);
request.clearCookies();

// step 3: create a new session and set it to the request
Session newSession = request.getSessionInternal(true);
request.setRequestedSessionId(newSession.getId());

if (logger.isDebugEnabled()) logger.debug("New session ID: " + newSession.getId());

// step 4: copy data pointer from the old session
// to the new one. Restore HTTP session data
for (String name : oldAttribs.keySet()) {
newSession.getSession().setAttribute(name, oldAttribs.get(name));
}

// Restore Tomcat internal session data
for (String name : oldNotes.keySet()) {
newSession.setNote(name, oldNotes.get(name));
}
}

getNext().invoke(request, response);
}
}

This valve is configurable with one parameter, url, that is used to signal when the session ID needs to be invalidated and recreated. In the case of Liferay, I'm using /c/portal/login.

<?xml version='1.0' encoding='utf-8'?>
<Context useHttpOnly="true">

<!-- Default set of monitored resources -->
<WatchedResource>WEB-INF/web.xml</WatchedResource>

<Valve className="your.company.valves.FixSessionFixationValve" url="/c/portal/login" />

</Context>

And that's it for today folks. It was late enough yesterday due to one hell of a deploy that continued to well after midnight, so I'm calling it a night.

Update 30/03/2010: after testing the valve a bit it seemed that it didn't work exactly as wanted, because the URL I was using to detect a login isn't called in all cases. The fix for this is not to detect a URL, but a POST parameter. For this we need to do 2 things: change the code of the valve a bit and move the valve configuration from context.xml to server.xml.


package your.company.valves;

import java.io.IOException;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.Iterator;
import java.util.Map;

import javax.servlet.ServletException;

import org.apache.catalina.Session;
import org.apache.catalina.connector.Request;
import org.apache.catalina.connector.Response;
import org.apache.catalina.valves.ValveBase;
import org.apache.juli.logging.Log;

/**
* Valve to regenerate HTTP Session ID's. Based on information
* available in the following 2 links:
*
* http://mikusa.blogspot.com/2008/06/tomcat-authentication-session-fixation.html
* http://www.koelnerwasser.de/?p=11
*/
public class FixSessionFixationValve extends ValveBase {

private static final String INFO = "be.belgacom.enable.security.FixSessionFixationValve/1.0";

private String parameterName = null;
private String value = null;

public String getInfo() {
return INFO;
}

public String getParameterName() {
return parameterName;
}

public void setParameterName(String parameterName) {
this.parameterName = parameterName;
}

public String getValue() {
return value;
}

public void setValue(String value) {
this.value = value;
}

@SuppressWarnings("unchecked")
public void invoke(Request request, Response response) throws IOException, ServletException {
String param = request.getParameter(getParameterName());
if (param != null && getValue().equals(param)) {
Log logger = container.getLogger();

// Save old session
Session oldSession = request.getSessionInternal(true);
Map<String, Object> oldAttribs = new HashMap<String, Object>();
Map<String, Object> oldNotes = new HashMap<String, Object>();

if (logger.isDebugEnabled()) logger.debug("Old session ID: " + oldSession.getId());

// Save HTTP session data
Enumeration names = oldSession.getSession().getAttributeNames();
while (names.hasMoreElements()) {
String name = (String) names.nextElement();
oldAttribs.put(name, oldSession.getSession().getAttribute(name));
}

// Save Tomcat internal session data
Iterator it = oldSession.getNoteNames();
while (it.hasNext()) {
String name = (String) it.next();
oldNotes.put(name, oldSession.getNote(name));
}

// Invalidate old session
request.getSession(true).invalidate();
request.setRequestedSessionId(null);
request.clearCookies();

// Create a new session and set it to the request
Session newSession = request.getSessionInternal(true);
request.setRequestedSessionId(newSession.getId());

if (logger.isDebugEnabled()) logger.debug("New session ID: " + newSession.getId());

// Copy data pointer from the old session to the new one. Restore HTTP session data
for (String name : oldAttribs.keySet()) {
newSession.getSession().setAttribute(name, oldAttribs.get(name));
}

// Restore Tomcat internal session data
for (String name : oldNotes.keySet()) {
newSession.setNote(name, oldNotes.get(name));
}
}

getNext().invoke(request, response);
}
}

Use the following configuration in server.xml:



Friday 5 February 2010

Recursive delete files/directories on OSX

When working with Java projects in Eclipse, or some other IDE, checked out from a source code repository such as CVS or SVN, you sometimes need to remove the IDE's project files or files created by the repository system. On the command line there's an easy way to do this using one of the following 2 code snippets:

  • Files: rm -rf `find . -name '*.project'`
  • Directories: rm -rf `find . -type d -name '.svn'`

Wednesday 27 January 2010

Ubuntu 9.10 display resolution problem

1,5 years ago I switched from a Windows laptop to a MacBook Pro and haven't really looked back since, ... except at home I still have a water cooled desktop PC that's about 7 years old and still runs Windows XP and I absolutely loathe that operating system. In my honest opinion Windows 2000 Professional was the best OS Microsoft ever made. My desktop PC ran on it for 4 years without the dreaded BSOD and within a week of installing XP it started presenting me all too frequently with them.

Since I wasn't really using the PC that much I tolerated this for some time, but two weeks ago 2 of the 3 hard drives in it failed and while repairing it I decided to install Ubuntu 9.10 which completed my goal of getting rid of Windows. The installation of Ubuntu went pretty painless and quick, with just one hitch: I couldn't configure a higher screen resolution than 800x600. That really sucks ass, especially on a 19inch CRT.

Installing the hardware drivers didn't really solve the problem, in fact it only made it worse, because afterwards I could only select 640x480 as the highest resolution. After reverting to the previous settings I started farting around with the xorg.conf file, but it turned out that that wasn't really my forté as I fucked up the configuration in such a way that in the end I just got a black screen.

You'd think that after all these years a desktop linux distro such as Ubuntu would get configuring your screen resolution as good/user friendly/easy as it is in Windows! So I pretended I wanted to be a millionaire and phoned a friend. He came over and also messed around a bit with the xorg.conf, but also couldn't get a better resolution. What we did discover however was that Ubuntu recognized my ancient Geforce2 M X400 without a problem, but not my CRT. It's some cheap piece of crap from some long-defunct Taiwanese manufacturer and that doesn't help autodiscovering settings one bit.

So I almost gave up, but decided to Google one last time and found a forum post with a possible solution: download a different linux live cd, check if that one provides better resolutions and if it does find the xorg.conf and copy the contents of that one into Ubuntu. I tried several, Knoppix, Linux Mint and CentOS, and with the last one I hit pay dirt! In CentOS I was able to change my monitor and resolution to a significantly better one by using some menu's in System > Preferences and System > Administration.

So I emailed myself the file /etc/X11/xorg.conf from within the livecd, rebooted back into Ubuntu, got the file from my email, put it in the correct location and a log off/log on later I had the resolution I wanted.


And so others won't have to go through the same troubles, I present to you the xorg.conf I got:


# Xorg configuration created by system-config-display

Section "ServerLayout"
Identifier "single head configuration"
Screen 0 "Screen0" 0 0
InputDevice "Keyboard0" "CoreKeyboard"
EndSection

Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
Option "XkbModel" "pc105"
Option "XkbLayout" "us"
EndSection

Section "Monitor"

### Comment all HorizSync and VertSync values to use DDC:
Identifier "Monitor0"
ModelName "Monitor 1024x768"
### Comment all HorizSync and VertSync values to use DDC:
HorizSync 31.5 - 61.0
VertRefresh 50.0 - 75.0
Option "dpms"
EndSection

Section "Device"
Identifier "Videocard0"
Driver "nv"
EndSection

Section "Screen"
Identifier "Screen0"
Device "Videocard0"
Monitor "Monitor0"
DefaultDepth 24
SubSection "Display"
Viewport 0 0
Depth 24
Modes "1280x800" "1152x864" "1152x768" "1024x768" "800x600" "640x480"
EndSubSection
EndSection

Friday 22 January 2010

Eurostar Oopsie

It seems that forgetting to translate that one message happens to everyone, even Eurostar:

Thursday 21 January 2010

Liferay + FancyBox + IE8 = problems

The last few days I've been trying to integrate PDF in a nice way into Liferay by showing them in a Lightbox alike way (more on the full integration in a future post). I first tried to use the Liferay.Popup, but that just gave me a headache. So finally I settled on using FancyBox 1.2.6.

Getting it to work on OSX/Safari wasn't a problem, Firefox also was a breeze, but IE8 was a different story. It was the first browser to cause a real problem, a Javascript error to be exact: Object doesn't support this property or method. The strange thing was that the examples on the FancyBox website would work correctly in IE8. So what to do now?

The error line and column pointed to some browser detection code (if you can call IE a browser) in the FancyBox Javascript file:
var ieQuirks = null, IE6 = $.browser.msie && $.browser.version.substr(0,1) == 6 && !window.XMLHttpRequest, oldIE = IE6 || ($.browser.msie && $.browser.version.substr(0,1) == 7);
This is default JQuery browser detection code, so I couldn't figure out why this would cause a problem? But some Googling turned up a comment of Nate Cavanaugh to a post about Liferay/IE/Browser detection problems. In his comment he mentions that Liferay has had its own Javascript browser detection code since ages. So I decided to change the FancyBox code to use the Liferay browser detection instead of the JQuery one:
var ieQuirks = null;
var IE6 = Liferay.Browser.isIe() && Liferay.Browser.getMajorVersion() == 6 && !window.XMLHttpRequest;
var oldIE = IE6 || (Liferay.Browser.isIe() && Liferay.Browser.getMajorVersion() == 7);

And what do you think: the error disappeared. I'm no Javascript expert and so I can't explain what exactly causes the Javascript error, maybe a strange Liferay/JQuery/browser interaction (it's not the first one we've encountered), but frankly I don't care since it works now and I don't have to time to investigate further. On to the next problem...