I'd take the ivory leather, but not the cloth. I'd be worried it would stain too easily. Furthermore, I like the ivory when paired with the wood trim as it makes the interior look more rich, warm and inviting; but not the metal -- it's too "cold." But this is a personal preference.
To see a picture, here's a link to the lexus.jp website. If you go to the bottom of the page and click the 4th picture from the left, there is a shot of the ivory fabric.
http://lexus.jp/models/is/design/color.html