Your decision should rest on the type of driving you use the car for !
I clocked up over 33k miles on my '02' Sport, and I now have a '04' IS300 Auto, I would never go back to manual gears, but this is because I am driving All day EVERY day. So, if you only drive in the evenings and weekends I can highly recommend the Sport, BUT the main thing I disliked about it was the fact that it had 6 gears and a VERY close ratio gearbox (and i mean the revs would only drop about 400rpm between gear changes) and i was changing gears more often and when you are driving all day and around town it becomes very tiring (especially with a 2nd gear clonk!). The other thing is that during motoray driving, the revs are at 3000 rpm and the speed is only 64mph ! and it would REALLY need a 7th gear. ;)
The auto box on the IS300 suits me best and im not being bias towards the IS300 the fact that its a much faster car, because its soooo new i havent even floored it yet, so I havent yet experienced the power side of it.
You would lose the L.S.D. by going for the SE auto, but hey ! In the 33k miles i drove my Sport, it never once had to be used.
Auto for me !
P.S. I drove a SE auto once and the acceleration from standing felt quicker than the manual.