Don't know whether anyone has mentioned this before, but it could be a dodgy wheel speed sensor.
I had one pack up (twice) on my old BMW causing similar symptoms. Also, it wouldn't change up into any gear over 3rd unless I shifted it manually which was a bit worrying first time it happened.
If you haven't done so already, its probably worth a trip to Lexus to get the fault codes read.