Archives July 2014

In the late 90s, it seemed to be a sort of magic to make a mobile phone ringing without typing the number of the recipient explicitly on the keypad of another phone. At the GSM World exhibition in Cannes (today’s Mobile World Congress, which has been moved to Barcelona), an exhibitor even had a magician at his booth to impress visitors with this kind of trick. “It is not illusion, it is technology” said the magician at the end of his show.

Central test lab – probes with UEs in the live network

What could fascinate visitors to a fair about 15 years ago was nothing else then a computer controlled modem making a call. This is the principle, up on which Qosmotec’s functional test system LTS-F is based. A compact computer board controls up to 8 modems. All this is put into a 19 inch chassis and such a test probe can be placed anywhere, where power supply and an internet connection is available and can execute test cases in live networks without the presence of an operator. The remote probe, or even a whole bunch of them, can be controlled from a computer at a central location and the executed steps, taken measurements and achieved results are logged, gathered and analyzed there. “This is extremely helpful for executing international roaming tests, much better than sending staff in all kinds of countries, where tests shall be executed”, explains Qosmotec’s technical director Axel Voigt. “Just the UE chassis needs to be shipped and plugged in and will automatically get into contact with the central control unit.”

Test probes with industrial modems allow tests in live networks controlled from a central location

Industrial modems instead of commercial handsets

Axel Voigt and his team have vastly extended the features and capabilities of the remote probes over the last years. “The reason, why we use industrial modems inside the probes instead of commercially available handsets is not merely a matter of compactness and construction, it is because of the test results that we want to provide our customers. These modems are made for remote control purposes and allow to extract information that the user would typically see on the display of his phone. This approach is much more effective than using complex techniques like image or voice recognition.” Modems for all kinds of mobile technology are available, covering all relevant frequency bands for 3G and 4G in Europe as well as America and Asia. Also American CDMA and EV-DO standard are supported.

The SIM array provides the platform for international roaming tests. Up to 13 SIM boards can be plugged into one rack.

No simulation, no modification, just mapping of SIM cards

It makes the remote tests even more convenient that the modems do not necessarily need to be equipped with SIM cards before shipping the remote unit. All needed SIM cards can be kept together in a SIM array with the LTS central control unit and can be mapped via software before test starts. This reduces planning effort for test execution as well as the amount of modem equipment needed drastically, because the same mobile can work sequentially in various operators’ networks without any physical interaction. This is the basis for the full automation of IREG test cases.

LTS-F enables testers to run remote controlled tests with UE test probes spread over several countries

Though it is just a very simple concept of spatial separation from the hardware components mobile and SIM cards, it is sometimes really difficult to explain, what it does. “Very often, people think that very mysterious things happen when transmitting the SIM card information to the modem”, tells Axel Voigt about serious misinterpretations that he has already been confronted with. But there is no simulation of a SIM card, nor is there any modification of the information on the SIM cards. A picture that we really like is the representation of a SIM card in form of a butterfly, symbolizing that the SIM is flying from one location to another. The only prerequisite is a stable internet connection between central unit and remote probe.

IREG test automation is one of the most important applications of LTS-F. Qosmotec can provide ready-to-use test sets for IR24, 35 and 60 specifications. But LTS-F is also very intensively used by Virtual Network Enablers and Operators (MVNEs and MVNOs). They use it for functional tests in those countries, where they are offering their services and just need to send a probe there, while they can execute all tests from their respective headquarters.

Mean Opinion Scores (MOS) describe the average between perfect pitch and tone deaf

Every September tensions are running high at Germany’s mobile network operators. At this time of year the special interest magazine “Chip” publishes its annual mobile network test. A good score in this self proclaimed “toughest mobile network test in Germany” is a boost to the sales departments, a bad result a public relations problem. Even in the age of smartphones and LTE, voice quality is a benchmark in this trial, as Chip stated in 2013: “To most users voice calls are more important than LTE.”

Rating of voice quality is a subjective matter

A conclusion that is confirmed by Qosmotec’s executive board member Dr. Dieter Kreuer: “No matter how important data rates have become, the main purpose of mobile networks is to enable voice calls.” But testing voice quality in fully automated test systems is something easier said than done. Unlike human beings automated test systems cannot distinguish whether a sound pattern is interrupted by signal loss or because a speaker has finished his sentence. But how to measure the quality of sounds?

In a voice quality test case users can select any voice sample and specify a MOS limit, which is relevant for the test assessment

The acurate ear of a musician is more likely to have perfect pitch, than somebody, who has stayed too long in the discotheque the evening before. And exactly this is the key for the so called “Mean Opinion Score” (MOS), the most important performance indicator for the rating of voice transmission. Based on a standardized algorithm, it averages the opinion that you would get, when a certain number of representatively selected people would listen to the same sound sample. Each of them would have a different opinion, but in total, their rating would give a good picture on the sound quality. It is also distinguished between a referenced rating, e.g., direct comparison between the original and the transmitted sound, and the unreferenced rating, which means the rating of the transmitted sample without knowledge of the original source.

Standardized solutions for voice quality tests

Voice quality testing is an integral part in Qosmotec’s test automation system LTS. Dr. Kreuer explains how Qosmotec has addressed the issue: “To make voice quality tests in LTS possible, we integrated the algorithm for tests according to the PESQ standard.” Perceptual Speech Quality (PESQ) is the standard that has been defined by the ITU Telecommunication Standardization Sector (ITU-T) to rate voice as described above. In LTS, we use referenced voice testing, so that results really reflect the transmission via the mobile network and are not influenced by badly recorded voice samples on the input side. Basically, the tester can use any voice sample in any language for testing. For example, the PESQ mechanism can also be used to compare voice samples with each other. If a tester wants to check for example, wether an announcement is correct (or if it is spoken in the correct language),he can use PESQ to compare the received message with the expected announcement. If those differ from each other, this will lead to a bad Mean Opinion Score.  For real network quality tests, we recommend to use a sample that comprises various languages as well as combinations of men and women speakers, which is ideal to provide a Mean Opinion Score with high granularity.

LTS lists all voice quality KPIs and depicts them in graphics.

Challenge on the hardware setup

However, integrating a PESQ algorithm into the LTS software alone does not solve all issues concerning voice quality tests in automated end-to-end systems. The hardware setup also has to be taken into consideration. With commercial handsets, it is almost impossible to execute automated tests for voice quality, because analogue voice would have to be digitalized first. This in turn leads to an additional loss of quality that is not caused by network transmission and is therefore irrelevant for testing. For this reason, Qosmotec uses industrial terminals for voice testing instead of commercial phones. These have a digital sound interface and therefore allow recording the voice as it is received.

Evolving technologies require new rating mechanisms

To ensure Qosmotec’s customers can avoid being slated in the press for their speech quality, Kreuer and his team are making LTS fit for the future of voice testing: “Testing various facilities of voice, for example, High Definition Voice or Voice over LTE, which is going to come soon, are the next issues for voice tests. Our task is to provide easy and effective mechanisms to test these technologies. We are currently integrating the new POLQA (Perceptual Objective Listening Quality Analysis) mechanism which adopts PESQ for VoIP and is used to test voice transmission over packet switched standards.”