Telestax Blog

Applicata JN Test Suite Platform Catches a Bug in RestcommONE Days before MWC 2015

Due to the humongous interest in the Dangerous Demo (video recording) originally shown at TADSummit 2014, TruPhone and its Open Source partners decided to work together on a new version for MWC 2015, which will be more spectacular and dangerous than ever. Due to the sophisticated nature of the demo, involving partners from cellular towers to core IMS network and app servers, the team decided to invite Applicata,  known for its cutting edge cloud telephony service testing product – Applicata JN Test Suite.

The Applicata JN Test Suite is an open platform for prototyping, automated testing and monitoring of services in GSM and IMS networks. In the context of the Dangerous Demo JNTS is utilized to run various testing scenarios against RestcommONE applications. The main demo is happening at the Canonical booth in Hall 8.1 at App Planet Stand CC8.20. In-depth demonstration of the Applicata JN Test Suite can be found at MWC on Applicata’s Stand 7H10.

Demo booth visitors will be able to place a call to a UK-based phone number on Truphone’s network. The call will be routed through Metaswitch’s Project Clearwater open-source IMS platform into the interactive voice response (IVR) system running on Telestax’s RestCommONE.

RestCommONE is a next generation Cloud Communications Platform. It allows web developers to rapidly build voice, video, WebRTC, USSD, SMS, fax and rich messaging applications. RestcommONE opens new revenue streams for service providers by exposing existing SS7 and IMS core network assets to application developers.

The Bugs

In the process of developing the new demo, JNTS caught a bug in RestcommONE which would have been difficult to locate by other means. In fact JNTS caught two bugs. One in RestcommONE and the other one in the demo app that runs on RestcommONE.

The RestcommONE runtime bug was that when RestcommONE was sending a SIP MESSAGE out to a registered SIP user, the message was populated with a local RestcommONE host IP address, instead of an IP address that is reachable from the SIP client.
The second bug – the demo app bug would have been nearly impossible to catch prior to demo day, without JNTS.  The demo app used an absolute URL in one place for the demo. When the app scaled out with the underlying cloud infrastructure elements,  the RestcommONE instance corresponding to the absolute URL went down, because of a zabbix misconfiguration. So the jnts tests which were successful prior to testing scalability started to fail, because RestCommONE couldn’t reach the absolute URL. Thus it was aborting the call way too early. JNTS caught that.

Before and After

For the earlier demo version used at TADSummit 2014, the team used sipp for automated load testing, but there was not tool available for functional testing of the integrated system. While sipp helped discover some performance bottlenecks, it did little to help simulate complete call paths the way that actual users would use the system. All non-trivial usage scenarios and edge cases had to be documented on paper and manually run each time any component of the demo system was updated. A very tedious and error prone process.

Sure enough there was a glitch during the TADSummit demo, which affected some of the users calling into the demo system. If we had JNTS available at the time, the glitch could have been prevented.

Amazingly the de-facto standard for end-to-end telephony system testing for most real world networks is no different than the labor intensive process used for the earlier demo version. Even the largest, well funded, multi-national telecoms write their test specifications on paper and expect solution providers to execute them manually as part of the acceptance criteria. Clearly a process with misaligned incentives.

JNTS changes that by applying Continuous Integration and Continuous Delivery best practices, long adopted by mainstream software developers,  to telephony networks and applications. The tool did not disappoint, as it caught two tricky bugs prior to MWC with enough time available to fix them. There is no 100% guarantee that everything will go smooth when the demo is live next week, but it sure feels better to know that a capable safeguard like Applicata JNTS is continuously watching out for problems.

[gravityform id=”9″ title=”true” description=”false”]

More details about the testing environment and the JN Test Suite script that implements the testing logic are described in more details below in Section JN Test Suite – Restcomm Demo Application Testing Environment .

Applicata JN Test Suite Overview

The current market dynamics raise extreme pressure upon network operators for continuous development and changes in technologies, introducing innovative services, establishing new partnerships, improving the quality of service and customers satisfaction.

With the increased competition between operators, customers become more sensitive about the quality of the provided services.

Mobile number portability service makes it easy for a complaining customer to move to another operator. Inbound roamers can switch to another local network even easier. This new competitive environment  raises operators’ interest in constant monitoring and control of their network and service quality. Proactively detecting eventual degradation as early as possible has direct impact on their revenues.

The key to overcoming the challenges is that the network and services should be regularly and pro-actively tested. A system capable of running automated tests, storing the results and analyzing the results is highly desirable.

JNTS provides a flexible and cost effective solution for automated functional testing of services in GSM and UMTS networks. It keeps these activities away from manual and time consuming testing, reduces costs, and produces measures and reports of service quality and availability

Manual testing is highly time consuming, costly and ineffective. Without special automated system testing activities are often neglected and this opens the door for bad network performance and bad quality of the provided services.

JNTS is easy to use with open functionality for service prototyping, automated testing and monitoring. It exports an interface for quick integration with third party management consoles.

Based on Telestax’ open source protocol stacks, JNTS is highly interoperable and adaptable to a wide range of network environments. The solution supports a wide range of protocols and can simulate the functionality for different nodes in the networks. This feature provides flexibility for running different tests initiated from the core network and/or from probes installed in the radio access network.

Typical Implementations

 Service Prototyping

JNTS is a quick, cost effective and easy service prototyping tool. The prototype can be effectively used as a proof of concept at early stages of a project. It can check and eventually modify the planned service application logic.

Network nodes simulation

New service development requires an environment where the developed service can establish controllable communication with the remote nodes in the GSM/UMTS/IMS networks.

XML scripts are created and deployed to JN Test Suite Platform for simulating the required behavior of the remote nodes in the network that communicate with the service under development.

This scenario brings full featured and controllable network behavior at the developer’s desk and eliminates the need to connect to a real network during the development.

Automated end-to-end and acceptance testing

A set of end-to-end and/or acceptance tests can be developed in XML scripts and deployed to JN Test Suite Platform.

The test can be easily run and repeated in automatic mode eliminating the effort, time and costs associated with manual testing.

Monitoring of services in production

XML scripts that simulate service requests can be created and deployed to JNTS, for constant periodic monitoring of services in production. The corresponding results will be collected and used for calculating service availability, KPIs, and SLA terms and conditions.

International Roaming Testing

Tests according to IR documents simulate roamers’ activity. Tests can run in probes distributed in the networks, or the activity can be simulated in the core network.

Applicata specializes in the design, development, installation and integration, of systems and software for telecommunications.

For more information on Applicata JNTS, please visit http://www.applicata.bg/jnts.html

JN Test Suite – RestcommONE Demo Application Testing Environment

The following message sequence diagram describes the use of JN Test Suite for testing and monitoring the RestcommONE Demo Application.

Sequence Diagram

The testing logic at JN Test Suite side is implemented as an XML script which is loaded and run by JNTS during testing.

The script with the test logic contains a finite state machine described using State Chart XML syntax. JN Test Suite platform expands the standard State Chart XML syntax with custom tags that cover telco protocol specific messages and parameters.

The State Chart XML script that controls the testing logic deployed in the JN Test Suite management console is shown in the next screenshot.

JNTS Script

The test result page in JN Test Suite management console is shown in the next screenshot.

JNTS Results

 

UPDATE: JNTS Voice Test app is now featured in the RestcommONE App Store.

 

Get awesome content in your inbox every week.

Give it a try. It only takes a click to unsubscribe.