The South Africa under-23 squad is taking part in an Olympic qualifying tournament that take place in Egypt from November 8-22. The top three finishers will qualify for the 2020 Tokyo Olympic Games.
Cameroon, Ghana, Ivory Coast, Zambia, Nigeria and Mali will also take part in the tournament that kicks off in Cairo on Friday.
The SAFA technical director says there are teams refusing to release their players for the important tournament.
"The teams aren't willing to release players," Tovey tells snl24.com/kickoff.
"We tried to talk and arrange... it hasn't just been started now, it started a long time ago, months back talking to the teams, talking to coaches but when it came to the teams they don't want to release the players, which is quite upsetting.
"Yes, we understand that obligation-wise and rule-wise they are not obligated to because it's outside of the FIFA window, but at the same time we've got to realise that you need to assist the national teams you know. That's how players grow, that's how players become better, that's how players come into shop windows, and the clubs benefit.
"I mean these players haven't just got to become 22 years of age. They played in the under-17 national team, they've gone through our development structures, they've gone through the under-20 national team some of them. So they've gone through those World Cups... now to not release them is disappointing you know. It's not for the benefit of the country.
"I know that teams are playing games to win the league, for the top eight, relegation. But at the same time things can happen, a player can get injured and you'll have to do without him. So that's how life is. So not just me, I mean as a country we are disappointed in the attitude of some of the clubs.
"Some of the clubs have been wonderful, they have tried to work with us, tried to assist us, and have assisted us in the past. But ja as it goes it's very disappointing."
Asked how many players are currently in Egypt, 'Mokoko' admitted is not enough.
"I don't know, around 13, 14... two arrived last night. We are supposed to have 21. I don't want to name teams, some teams have been working with us, some haven't been working with us. I'm just disappointed overall you know.
"I know it's awkward. CAF they've got to also think about how they position these tournaments because you want your best players at the tournament, and the way it has been done now we can't get our best players at the tournament. So that needs to be looked at.
"You can't go and win the AFCONs without your best players. Now there's comments 'Why don't you use so many Motsepe Leagues, so many NFD, so many this that, MDC'... you can't do that. You've got to have your best players otherwise you can't come close to winning competitions.
"The standards are high, these are players that play in the top leagues in the world and are playing in club leagues in Africa. People must realise you can't undermine national teams."