Testing with SIPp

Testing is an important task when you are working on a new service and you want to check everything before deploying it in a production environment. For Session Initiation Protocol (SIP) one key testing tool is definitely SIPp, open source software that can be used for testing purposes. It is able to behave as SIP User Agent Client (UAC) as well as User Agent Server (UAS) hence you can use it these ways:

• SIPp is used as UAC and calls FreeSWITCH (IVR, voice applications, and more)
• SIPp is used as UAS and is the endpoint being called (by another FreeSWITCH extension for instance)
• SIPp is used on both sides of a single call (the caller and the callee) to build completely automated tests

Running scenarios

One of the strengths of SIPp is that it is highly customizable: the user writes scenario files in XML format that details all communication steps. An XML scenario file basically describes every SIP message the tool has to send and every response it is supposed to get. Some variables are also available so you do not have to take care of things like tags and call-id stuff. This flexibility in building scenarios means you can simulate many different things: from a basic call to elaborated services such as IVRs, voicemail calls (with user authentication and message recording), and so on.

Let's see a simple scenario with SIPp as UAC. In a vanilla FreeSWITCH setup, call extension 1001 as user 1000. The scenario file would look like the following (please note that SIP messages after first INVITE have been deleted to make the XML content shorter):

<?xml version="1.0" encoding="ISO-8859-1" ?>
<!DOCTYPE scenario SYSTEM "sipp.dtd">
<scenario name="FreeSWITCH: call extension 1001">
<!-- we send the intial INVITE -->
<send retrans="500" start_rtd="mer">
<![CDATA[

      INVITE sip:1001@203.0.113.100 SIP/2.0
      Via: SIP/2.0/[transport] [local_ip]:[local_port];branch=[branch]
      From: 1000 <sip:1000@[local_ip]:[local_port]>;tag=[pid]SIPpTag00[call_number]
      To: 1001 <sip:1001@203.0.113.100:[remote_port]>
      Call-ID: [call_id]
      CSeq: 1 INVITE
      Contact: sip:1000@[local_ip]:[local_port]
      Max-Forwards: 70
      Subject: Test_call
      Content-Type: application/sdp
      Content-Length: [len]

      v=0
      o=user1 53655765 2353687637 IN IP[local_ip_type] [local_ip]
      s=-
      c=IN IP[media_ip_type] [media_ip]
      t=0 0
      m=audio [media_port] RTP/AVP 0
      a=rtpmap:0 PCMU/8000

    ]]>
</send>
<!-- we expect to receive a trying -->
<recv response="100" optional="true" rss="true"/>
<!-- the FreeSWITCH server should ask us to auth your request -->
<recv response="407" auth="true" rss="true"/>
<!-- we ack the 407 message -->
<send crlf="true">
    [...]
</send>
<!-- we send again the INVITE with the auth field -->
<send retrans="500">
<![CDATA[
    [...]
      [authentication username=1000 password=1234]
    [...]
    ]]>
</send>
<!-- we expect to receive a trying -->
<recv response="100"/>
<!-- we expect to receive a provisional 180, but it's not sure yet -->
<recv response="180" optional="true"/>
<!-- we expect to receive a provisional 183, but it's not sure too -->
<recv response="183" optional="true"/>
<!-- we expect 1001 to pick up the call -->
<recv response="200" rtd="mer" rss="true"/>
<!-- we ack the 200 OK SIP message (if the call has been picked up) -->
<send crlf="true">
    [...]
</send>
<!-- we pause, but we could instead send media here by playing a pcap file -->
<pause milliseconds="1000" crlf="true"/>
<!-- we hang up the call -->
<send retrans="500">
    [...]
</send>
<!-- FreeSWITCH should reply 200 OK -->
<recv response="200" crlf="true"/>
<!-- various scenario parameters -->
<ResponseTimeRepartition value="100, 200, 500, 1000, 1500, 2000, 5000, 10000"/>
<CallLengthRepartition value="10, 100, 500, 1000, 5000, 10000, 20000"/>
</scenario>

To run such a scenario is an easy task to accomplish, you only need to run the sipp binary:

./sipp -sf call_1001.xml -i 192.168.1.10 -m 1 203.0.113.100

Here the parameter -sf specifies the scenario file, -i specifies the local IP address, -m specifies the number of test calls to start (here, only one) and the last address represents the FreeSWITCH server, but many other parameters are available. A complete report is printed out at the end.

SIPp provides a command-line user interface that displays real-time information about the scenario being run so the user can check if everything is running as expected even if the tests are not finished. The command line interface can be used to increase/decrease the frequency and the number of calls per second (cps) to start, if more than one is set with the -m option.

In addition to signaling, SIPp is able to handle media traffic/RTP packets, so even voice QoS testing can be done with it, such as evaluating the MOS (Mean Opinion Score) of a test call. To implement it in the previous scenario, the following instruction could be introduced once the call is established:

<nop><action><exec play_pcap_audio="scenario/test.pcap"/></action></nop>

If you want to use SIPp as a UAS (as per the last two use cases mentioned in the introduction), meaning the SIPp instance receives a call made by your SIP server, you will need to register SIPp to FreeSWITCH (pretending SIPp is a phone) to make it reachable from a SIP server point of view. This can be done with a dedicated scenario; several such examples are available on the Internet.

Load testing

SIPp is handy for load testing. Indeed, the tool is able to start as many concurrent instances as the hardware can handle so it is very easy to simulate hundreds or even thousands of simultaneous calls. For instance, two hosts can each run a SIPp instance with a simple call scenario: one starts a call towards the FreeSWITCH server that itself creates the leg towards the second host that finally picks up the call for x seconds. Instances and calls of this scenario can be slowly increased, and the tester can then find the limit of the hardware hosting FreeSWITCH when the results start to deviate from the expectations (because of too many dropped packets for instance). Moreover, it is possible to enable time trackers, an important SIPp feature during load test because it displays how long the duration was between two (or more) steps (for example between the moment when the call is initiated and the moment when it is answered).

If you decide to run such tests, make sure the hosts running the SIPp instances are not the bottleneck though.