I'm not sure why this bug sneaked in with patch #10245, but here's what's up:
http://dev.rubyonrails.org/attachment/ticket/10245/fix_firefox_scrolling_bug.diff
document.viewport.getScrollOffsets[0] ? that will always return undefined. This was mistaken for a fix since the undefined sort of fixes the problem, but it's not the expected result.
I guess what he was trying todo is:
scrollOffsets = document.viewport.getScrollOffsets(),
elementOffsets = $(element).cumulativeOffset(),
max = scrollOffsets[0] - document.viewport.getHeight();
But that doesn't solve anything. Hmm.. I'll have to look at this a bit more to see what you are expecting to get from max before I patch this up.