Finally figured it out. The lexus manual you posted makes it more complicated (thank you for helping out)
Basically.
1. Sit in your car
2. Press and hold lock and unlock at the same time till it beeps.
Your display will show all doors unlocked.
That's it.
I think where I was going wrong was I wasnt sitting in the car at the time when I was pressing both buttons on the remote. I was standing outside the car.