I don’t think segregation was a minor setback, and I would be very surprised if I heard someone say that. However, until something is brought to your attention, you’re going to be incognizant. Why are minorities so much more aware of racism? Because minorities are the ones who experience it. So often, slavery and (de)segregation are presented as chapters in history books that don’t explore the long-term effects of either. If a white person (or another non-black) doesn’t learn about it in school or experience it firsthand, their views could become skewed (understandably). Some people grow aware. Some don’t.