Your work colleagues sound like smart people, you should listen to them more.
Good athletes can be found all over the world, the difference is how they are trained. In the America, you go to high school and play your sport on the side. You go to college, and play your sport on the side. You only go full-time when you go pro (except baseball I think).
In Latin America and Europe, for soccer, if you have skills at a young age they pull your *** out of school and you do nothing but play soccer 24/7 in a development league. The best time to develop any motor skill is when you're a kid, but in USA they split their time with school.
That's why I read these articles about how the US is improving at soccer and I'm like "nope." The only way you ever catch up to Europe and Latin America is if they wake up one day and decide it's more important for little Johnny to learn how to read than dribble a soccer ball. But for now, that ain't happening. And if your kid is practicing 3 hours a day and the Brazilians are practicing 12 hours when they're kids and their motor skills are developing, that's the difference and it ain't changing.
It's also why if you had an American college football team from the 80's and they played one from today, the 80's team would destroy today's team. Put the '87 Hurricanes in a time machine and have them play last year's Alabama team, and the Canes would destroy them. Most would disagree - people by nature just want to believe that things are always improving and moving forward, they would talk about training methods etc getting better, but that's not true. The fact is they practiced longer and harder in the 80's, because the NCAA allowed it. The new rules limiting practice time makes for worse athletes.
As for Olympic sports, US is the best because you have Title 9 and so you have money going in to it. Other countries (besides maybe the old USSR) don't have academy models for olympic sports like they do for soccer, so that's not really relevant to compare.
So that's a long answer all just to say.... how good you are at a sport is directly related to how long and hard you practice. Especially when you're young.