I'm using the default WordPress gallery and I'm encountering a bug in IE. It looks like the first gallery image floats to the right or is placed to the left of something else. This only happens in IE8 & 9 (yes, it somehow works in IE7!).
I've tried removing the caption from the last gallery item. I've also tried moving the large image the the end and neither have any effect. Perhaps this is more of a CSS issue than a WP issue.
Here's the live site.