Since it is not mentioned anywhere that the ancestor cannot be document, I thought it would be OK to simply check if the element is a descendant of a frame's document.
This used to work in Prototype 1.5.x, but not in Prototype 1.6, where the Element.descendantOf method was improved.
In any case, the new code of Element.descendantOf looks strange:
if (!nextAncestor) {
do { ancestor = ancestor.parentNode; }
while (!(nextAncestor = ancestor.nextSibling) && ancestor.parentNode);
}
nextAncestor is initialized ancestor.nextSibling, but if it is not defined, or null ("if (!nextAncestor)"), ancestor is assigned to "ancestor.parentNode" (which, as we recall, is undefined/null) in a loop, where the loop condition checks ancestor.nextSibling. However, the check is first done after the next execution, which is too late, since ancestor was already assigned to null/undefined.
An obvious simply workaround is to call Element.descendantOf(document.body) which solves the problem in most of the cases, but there is still a problem.