It has happened twice now, in two different places. It is not only without the clutch down either.
Also, I got in my car and sat on the phone for 5 minutes, then tried to start it and had an absolute nightmare!! In the end I got out, locked it and re-entered before it worked!
It may just be an issue with that particular key? I am going to try the other one and see if the same happens. If so Ill take it into Lexus and see if they know.