The fact of the matter remains that SOMEONE has to break the color barrier for others to play. And it's not like he was a slouch athletically. Dude not only broke records, he was the first black student body president.
How can you be "too early" to break the color barrier? Not only was he an absolute monster, if he doesn't break the color barrier at Miami, someone else has to, or we don't have a very significant majority of all the great players we've had.