I've owned a 2005 IS300 Sportcross for six months after five years with an IS200 Sportcross and I definitely prefer the bigger engine. Much more torque, more relaxed cruising, and a few more toys to boot, such as cruise control, auto headlight adjustment, full leather seats and parking sensors. Also self-dimming heated mirrors, but mine don't self-dim any more and new ones are £600 a pair inc VAT! Agree with the comment about sticking calipers, I've had one, the telltale sign is a squeak at low speeds that goes away when you brake. Otherwise it is showing every sign of being just as reliable as its predecessor. Check that the washers and rear wiper motor are all working OK, I had some trouble with these on the other car and good secondhand items are like hen's teeth.
The Navigator version had built-in sat nav but mine doesn't, apparently late models don't because Lexus ran out!
Downsides? Fuel consumption is higher, but not dramatically so, about 3mpg less than the IS200. Also the lower profile tyres are noisier and give a stiffer ride, but they sharpen up the steering considerably so I reckon that is a fair trade-off.
A truly excellent tool, fulfills so many roles so well.