Take some of both for a blast before deciding!
The 4cyl is very buzzy, with little vibrations appearing in all parts of the car. It really gets up and goes from 5000rpm on, (210bhp) and has a distinctly better turn-in from the lighter engine and the mass a little further back. They have crappy upholstery, unheated and non-electric seats. They are the cheap sporty version. You have to work to drive them smoothly around town, especially with the six-speed box and the grabby clutch.
The six-cyl is smooth and torquey, lovely to drive and effortless around town. However the power lack is VERY noticeable. (150bhp) Maybe the TRD 156KW 6cyl or the supercharger is the answer! The trim inside is a step up, especially in the Lexus with the heated leather seats and the sat-nav.
I drove about six from various car dealers before deciding on mine. I soon decided on the 4cyl six-speed, and then picked which of the dozen available was the best value.
Still love it to bits!