I checked again and actually the button on the dashboard button does work.
It is only the external button that does not work and it looks damaged (see picture)
Do you think replacing this piece of plastic might fix the issue? I see it is available here
https://www.amazon.co.uk/Aramox-Rx400H-Release-Opening-2004-2009/dp/B08Q3P41FT/ref=sr_1_fkmr0_1?crid=N2BIQN6JPL58&keywords=lexus+rx+400+boot+button&qid=1705674547&quartzVehicle=842-11018&replacementKeywords=400+boot+button&sprefix=lexus+rx+400+boot+button%2Caps%2C66&sr=8-1-fkmr0