I have long known of a bug in my flagship plugin CorVex--for some reason some geometry was turning out corrupt. I couldn't see anything in the code that cause the particular segments to be special. But this week I finally spent some time and got to the bottom of the issue.

Consider this line of code:

Angle = acos ( dot( normal1 normal2))

It seems straightforward. How can it possibly fail? It doesn't seem like it should be able to fail, but sometimes you get this result: Angle = -1.#IND

I'm not a math guru, so I don't know the cause. My best guess is that there might be some rounding errors that happen with some particular normals which cause the normals to not be fully normalized.

My fix for this problem is to rewrite the code like this:

local d = dot( normal1 normal2)
case of (
 (d < -1.0):( d = -1.0)
 (d > 1.0):( d = 1.0)
 default:()
)
Angle = acos d