Race plays a central role in Thomas' account of his life. (His bitterness about racial slights and racial mistreatment may surprise some readers.) I am "White" and I have not experienced racial slights and discrimination. However, poverty is also an integral part of the story that Thomas tells. I found this part of Thomas' story to be particularly gripping. This is surely because I also grew up poor. (Some of the time Thomas was poorer than I was, but -- believe it or not -- much of the time I was poorer than he was.)
Justice Thomas has little affection for Yale Law School, which is where he got his law degree. This is partly because he felt patronized by Yale faculty members and by his fellow law students and because he deeply resented being patronized because of his race. But part of his bitterness about Yale stems, I think, from his bitter experiences with poverty.
Because Thomas had been poor, he was relatively "unpolished" when he entered Yale Law School (e.g., no college vacation trips to Egypt to do archeology or to Florence to study Renaissance architecture). And because of his relative poverty while at Yale, he could not afford much ordinary entertainment while he was there. I suspect that both of these factors embittered him about Yale.
Although many observers in the legal profession may sympathize with discomfort experienced by a student at a law school because of the student's poverty, they may find it hard to understand why such discomfort would make someone so bitter about a place such as Yale, which, after all, admitted Thomas and tried -- by its lights -- to treat him and similar students well. But this sort incomprehension about Thomas' feelings about Yale exists, I think, because very, very few people at institutions such as Yale and Harvard (or in the legal profession as a whole) have experienced deep poverty.
I went to Yale College and Harvard Law School. By and large, I loved both of them. But only by and large. I vividly remember not having enough money to go on a date or even to go to a movie for months at a time. I remember, just as vividly, feeling unpolished -- and, worse yet -- being unpolished in comparison with my fellow students. These are hurts that can be forgotten and overcome -- but these sorts of hurts are not easily forgotten, and they are perhaps never fully overcome.
P.S. The above comments do not constitute an endorsement of Justice Thomas' theory of law and judging, his approach to constitutional law, or an endorsement of any other similar thing.