Not completely because of how much pre-college scouting goes on and, like you said, did not hurt Ja Morant a bit. Ivy League schools still schedule big games too. Scouts are looking for traits and skillsets which translate no matter the level of competition. Whether that be Ja's athleticism or Trey Lance's accuracy (North Dakota State qb). Similarly Grant Riller, a point guard from Charleston, is rising up draft boards. Didn't matter much for Dame or CJ.
What I do believe is that Wendell Carter considered Duke playing UNC and making a Final Four when making his decision. Playing on TV, legacy of Duke lottery picks, playing in front of a rabid fanbase, playing with other future NBA players, playing against other future NBA players, etc. That stuff hurts Harvard's chances against Duke/Stanford more than the level of competition hurts an elite prospect's chance of getting drafted high.