How Are You Similar to Me?