Thanks all. I ended up buying it after speaking to a Lexus tech who explained most of what I'd experienced (and what some of you guys pointed out):
Electric power steering vs hydraulic in the 350 & 300
FWD bias causing torque steer vs the RWD bias in the 350/300 - Colin, your description matches perfectly!
Sensitivity to cheap / mismatched tyres (in terms of pressure, tread, or direction)
FWD + electric + CVT vs the above for a different launch experience
I pumped the tyres up to 2.2 bar all round and the drive was much better. I've also ordered a set of four CrossClimate SUVs to be fitted over the weekend along with an alignment to improve things further, and will look into calibrating the steering with software after that.