Download ZIP (60.8 MB)

Testing and Issues

You can test this entry and submit issues during the testing period of the VoIP Contest, Round 1 contest.

Entries with serious issues will not be able to win the contest, but even minor issues might be important for overall results.




The rating software is based on voice recognition with custom audio models and vocabulary, generated using built-in scripts (along with secondary rating modules based on silence and length recognition).
The call software is a very simple OOP wrapper around libtgvoip, making use of libopusenc (must be compiled by calling `make`) and libopusfile to encode and decode OGG OPUS files.
You have not added any comments yet...
by rating


Fair Wombat Nov 14, 2019 at 11:05
The submitted tgvoiprate app got a score of 551 and took the 2nd place in the scoring table, receiving 4⭐️.
Minor issue #issue8875 and external dependencies of pocketsphinx voice recognition (free open source) -0.2⭐️
Final result is 3.8⭐️
@Classy Bee, thanks for the explanation. You are right, because it is true that "the speech is still very understandable even with distortion". Please find attached files [1] used for comparison above. So in this case the high score, produced by your different approach (based on PocketSphinx speech recognition engine), makes sense. However we can see that for the same input ViSQOL-based and PESQ-based algorithms produce much lower score. Which makes even more interesting the question of how to select the best metric/algorithm for speech audio quality comparison in VoIP systems.
P.S. Cannot attach .ogg / .zip / .tar.gz here.
Debian GNU/Linux 10
Borik Bobrujskov Oct 28, 2019 at 15:52
a little bug: dependencies in deb-packages.txt must be one per line
Debian GNU/Linux 10 \n \l
Classy Bee Oct 29, 2019 at 09:11
Oh boy I forgot about that
Borik Bobrujskov Oct 28, 2019 at 15:56
And much more significant issue: your tgvoipcall depend on wich you are not add to deb-packages.txt and i don't found it in Debian repo by apt search libopusenc :( I can't run your file
Debian GNU/Linux 10 \n \l
Classy Bee Oct 29, 2019 at 09:12
I mentioned this in the readme: libopusenc is only available under Debian sid, run `make` to automatically build and install all required dependencies including libopusenc.
For a distorted ogg tgvoiprate returns 4.66 while others tgvoiprate return 1.98 (ViSQOL-based) and 1.95 (PESQ-based).
Debian GNU/Linux 10
Classy Bee Oct 29, 2019 at 09:53
This is because I'm using a different approach with speech intelligibility detection (which makes more sense to detect the quality of voice calls) using custom voice recognition (and not much distorsion detection, apart from a silence counter).

The high rating might be because the speech is still very understandable even with distortion, which would indicate a not-so-bad call (would you mind attaching the used files?).
You can add a third parameter.log to the command line of tgvoiprate to log and see the individual component ratings and recognized text.
Fair Wombat Nov 1, 2019 at 10:55
make fails with following error:

cd libopusenc && autoreconf -vfi && mkdir -p build && cd build && ../configure --prefix=/usr && make -j && sudo make install && cd ../..

autoreconf: '' or '' is required

make: *** [Makefile:71: /usr/include/opus/opusenc.h] Error 1

and src/libopusenc folder is empty.
Debian GNU/Linux 10 (buster)
Classy Bee Nov 2, 2019 at 19:10
It seems like I forgot to clone the libopusenc submodule in the tarball I submitted.
You can fetch the full correct submission with git clone --recursive; make will work then.
Fair Wombat Nov 7, 2019 at 19:38
tgvoiprate returns values 0.0-1.0 for some input, while in the task "the rating should be expressed as a real number from 1.0 to 5.0". How should the output be adjusted from 0.0-5.0 to 1.0-5.0?
Debian GNU/Linux 10 (buster)
Classy Bee Nov 12, 2019 at 20:29
Oh, that's right, I probably got confused with the 0-6 rating ;)
Anyway, this can be easily fixed by substituting line 272 of src/rater.cpp:

return final;
return ((final / 5.0) * 4.0) + 1.0;
Nobody added any issues yet...