Howto:SIP Interop Test Description

This test description is obsolete, for information on the new SIP Test method see: Howto:Innovaphone_SIP_Provider_Tests

innovaphone SIP Interop Tests

Summary

This article describes the SIP provider Tests done by innovaphone when testing the compatibility of a provider. The features are divided in three classes: KO-criteria,Important and optional criteria. If the provider does not pass a test marked as KO-criteria, the provider will not be accepted in the category recommended SIP Provider. The only exception for KO-criteria will be the support of both the G711alaw and G711ulaw coder, it's only required for a SIP Provider to support one of them (G711a or G711u) to pass the requirements. If the provider does not pass a test marked as Important then this should be reported in the Summary of the testreport and in the 3rd-party-product page.

To differentiate the quality of the offered service, the provider receives an additional rating. The current rating value ranges between 0 and 187 points. The test success rate is displayed as a percent value. It is calculated by using the following formula: (rating points achieved / rating points total) * 100.

Before we start running the test calls we should ask the SIP provider if the call path is always the same or if there is any type of calls scenarios that could have different path/behaviour (i.e. Call between two accounts of the same SIP provider). If the behaviour it's the same for all calls we should run the tests as described, if the provider mentions that some call scenarios are different and could not work or behave differently, we should run additional tests calls on that scenarios.

Test Name	Importance	Rating points
Registration
SIP over UDP	KO	5
SIP over TCP	Optional	2
SIP over TLS - SIPS	Optional	2
Numbering scheme
Inbound(Provider->Innovaphone)	KO	5
Outbound(Innovaphone-> Provider)	KO	5
CGPN can be suppressed	Important	3
CLIP no screening	Important	3
Check for SIP Provider errors
IP Fragmentation	KO	5
Large SIP-messages	KO	5
SDP crypto attribute supported?	Optional	2
SDP fingerprint attribute supported?	Optional	2
SDP ICE attribute supported?	Optional	2
SDP Video media supported?	Optional	2
Basic Call
NAT Detection	Important	3
Call using SRTP	Optional	2
Call using G711A	KO*	5
Call using G711U	KO*	5
Call using G729	Important	3
Call using G722	Optional	2
Loop In Call(Innovaphone-> Provider->Innovaphone)	Important	3
Overlapped Sending	Optional	2
Early Media Channel	KO	5
Fax using T.38	Important	3
Fax using G.711	Important	2
T.38 Transcoding by the provider	Optional	3
Reverse Media Negotiation	Important	3
Long time call possible(>30 min)	KO	5
External Transfer	KO	5
Mobility Call	Important	3
WebRTC Call	Important	2
Voice Quality	KO	5
Redundancy Mechanism?	Important	3
DTMF
DTMF tones sent correctly via RTP-Events(RFC 2833)	KO	5
DTMF tones received correctly via RTP-Events(RFC 2833)	KO	5
Hold/Retrieve
Call can be put on hold	KO	5
Held end hears music on hold / announcement from PBX	Important	3
Transfer with consultation
Call can be transferred	KO	5
Held end hears music on hold	Important	3
Transfer with consultation (alerting only)
Call can be transfered	KO	5
Held end hears music on hold	Important	3
Call returns to transferring device if the third endpoint is not available	KO	5
Blind Transfer
Call can be transferred	Important	3
Held end hears dialling tone	Important	3
CFU / CFB Transfer
Call can be forwarded	KO	5
Held end hears dialling tone	KO	5
CFNR / Blind Transfer (alerting only)
Call can be transferred or forwarded	KO	5
Held end hears dialling tone	KO	5
Broadcast Group & Waiting Queue
Caller can make a call to a Broadcast Group	KO	5
Caller can make a call to a Waiting Queue	KO	5
Announcement if nobody picks up the call	KO	5

For test results, see the list of tested SIP providers.

Test Scenario

This scenario describes a setup where the PBX and phones are in a private network.

There are 3 major configuration variants of the SIP trunk, which one is used depends on the test results. The variants are:

• the SIP trunk is configured without Media Relay and without exclusive coder. This is the case if all tests were successful
• the SIP trunk is configured with Media Relay and STUN but without exclusive coder. This is the case when the test for "NAT Traversal" fails
• the SIP trunk is configured with Media Relay and exclusive coder. This is the case when the test for "Reverse Media Negotiation" fails

Registration

SIP over UDP(KO)

Purpose: Check if provider offers the possibility to transport SIP messages via UDP.

Test description:

• Configure the SIP interface for SIP over UDP(SIP).

SIP over TCP(optional)

Purpose: Check if provider offers the possibility to transport SIP messages via TCP. SIP over TCP consumes less CPU resources on the gateway/PBX and is therefore preferred is some scenarios(e.g. hosting)

Test description:

• Configure the SIP interface for SIP over TCP(TSIP). If TSIP is supported by the provider but SIPS is not, use TSIP for all tests.

SIP over TLS - SIPS(optional)

Purpose: Check if provider offers the possibility to transport SIP messages via SIPS, i.e. encrypt SIP messages using TLS.

Test description:

• Configure the SIP interface for SIPS. If SIPS is supported by the provider, use SIPS for all tests.

Numbering scheme

Purpose of all Numbering scheme tests is to determine the number format(CGPN,CDPN) expected and used by the provider. Moreover the configuration of the SIP-Interface SIP Interop Tweaks must be determined, i.e. the value of the options of From Header when Sending INVITE, Identity Header when Sending INVITE and if the usage of the P-Asserted-Identity-Header(SIP Tweak /pai on) is required. You will can use the first test for inbound calls to determine the CGPN and CDPN format used by the provider. It is very probable that the provider expects the same format for outbound calls. If the number format is known, you will have to try all permutations of the 3 SIP-Interface-options mentioned above, until you have a combination that works for all KO-tests and for the most other tests in this section.

Hint: The SIP-interface options above influence only outbound calls. As a result, you don't have to repeat the inbound call test for all permutation tests.

Inbound(Provider -> Innovaphone)(KO)

Purpose: Test if the provider forwards the call to the PBX using the correct DDI - number(CDPN).

Test description:

• Test 1:
  • Phone in the PSTN(International number) calls Tel1.
  • Check if the CDPN is forwarded correctly to the PBX.
  • Check the Syslog to find out the CGPN and CDPN used by the provider.
    • Check if the provider uses the "+" - number format(e.g. +41...) or the "00"- number format(e.g. +41...) for the CGPN.
    • Check if the provider uses an international number format(e.g. see above) or the "0"- number format(e.g. 07031...) for the CDPN.

• Test 2:
  • Phone in the PSTN(National number) calls Tel1.
  • Check if the CDPN is forwarded correctly to the PBX.
  • Check the Syslog to find out the CGPN and CDPN used by the provider.
    • Check if the provider uses the "+" - number format(e.g. +41...) or the "00"- number format(e.g. +41...) for the CGPN.
    • Check if the provider uses an international number format(e.g. see above) or the "0"- number format(e.g. 07031...) for the CDPN.

Outbound(Innovaphone -> Provider)(KO)

Purpose: Test if the provider handles the CDPN correctly and forwards the call to the PSTN phone. Additionally check if the PSTN phone displays the CGPN (trunk number + extension) correctly.

Test description:

• Tel1 calls phone in the PSTN.
• Check if the PSTN phone display shows the correct CGPN.

CGPN can be suppressed(Important)

Purpose: Test if provider accepts anonymous calls

Test description:

• Tel1 is configured to not send his CGPN. 'User Setup'->'Number Presentation' = Off
• Tel1 calls a phone in the PSTN. The display of the PSTN phone should show as Calling Party: 'anonymous'

CLIP no screening(Important)

Purpose: Test if provider accepts calls having a different CGPN as assigned to the SIP trunk. Test if the provider accepts also international CGPNs for CLNS or only national CGPNs.

Test description:

• Relay mapping is changed for this test, so that calls to the provider will have a random CGPN.
• Tel1 calls a phone in the PSTN. Check if the PSTN-phone displays the configured CGPN.

Check for SIP Provider errors

The tests here will check for different errors caused by the provider SIP-server or network equipment. E.g. some providers SIP server cannot handle ICE-candidates in the SDP, therfore you must activate "No ICE" on the SIP-interface.

IP Fragmentation(KO)

Purpose: Check if the provider can handle fragmented SIP-messages. IP packets larger than the MTU (Ethernet = max. 1500 bytes) get fragmented in smaller pieces. This is critical for SIP/UDP messages. We will force the PBX to fragment our IP-packets, by setting the MTU to a low value.

Test description:

• http://ip-of-pbx/!config add IP0 ETH0 /mtu 576
• http://ip-of-pbx/!config write, followed by a reset of the PBX
• make a call from Tel1 to the PSTN-phone. You can trace with wireshark and double-check that the PBX is sending fragmented IP-packets.
• when finished testing, remove the option by http://ip-of-pbx/!config rem IP0 ETH0 /mtu and http://ip-of-pbx/!config write, followed by a reset of the PBX

If this test fails, the provider must support SIP over TCP. If it doesn't, then the provider is not recommended.

Large SIP-messages(KO)

Purpose: Check if provider can handle large SIP-messages. We will use a large SIP packet(> 1500 bytes) to test if the provider can handle this messages.

To generate a large packet, we will use large display names in objects. Test description:

• (only for this test)activate the Interworking option in the route to the SIP-Provider(RSx->SIPx)
• create two objects with a Display-Name of 99 characters(max. value). The first object has a CFU to the second, the second to the PSTN-phone.
• for Tel1 create also a Display-Name of 99 characters(only for this test)
• From Tel1 call the first object.

If this test fails the provider is not recommended.

SDP crypto attribute supported? (optional)

Purpose: Check if the provider can handle SDES encryption attributes in the SDP. This test doesn't require the provider to support encryption, but tests if the provider SIP-server can handle this type of messages without aborting the call.

Test description:

• configure the SIP-Trunk for MediaRelay, SRTP Cipher AES128/32 and SRTP Key Exchange SDES.
• Tel1 calls a phone in the PSTN
• check that audio is sent in both directions

SDP fingerprint attribute supported? (optional)

Purpose: Check if the provider can handle DTLS encryption attributes in the SDP. This test doesn't require the provider to support encryption, but tests if the provider SIP-server can handle this type of messages without aborting the call.

Test description:

• configure the SIP-Trunk for MediaRelay, SRTP Cipher AES128/32 and SRTP Key Exchange DTLS.
• Tel1 calls a phone in the PSTN
• check that audio is sent in both directions

SDP ICE attribute supported? (optional)

Purpose: Check if the provider can handle ICE-candidates in the SDP. This test doesn't require the provider to support ICE, but tests if the provider SIP-server can handle this type of messages without aborting the call.

Test description:

• make sure you use an v11-phone or webRTC-client to setup the call.
• Tel1 calls a phone in the PSTN
• check that audio is sent in both directions

SDP Video media supported? (optional)

Purpose: Check if the provider can handle video-media attributes in the SDP. This test doesn't require the provider to support video calls, but tests if the provider SIP-server can handle this type of messages without aborting the call.

Test description:

• configure a myPBX-Windows-client for video-calls and with Tel1 as corresponding device.
• Tel1 calls a phone in the PSTN
• check that audio is sent in both directions

Basic Call

NAT Detection(Important)

Purpose: In scenarios were Media Relay is not used and the PBX and phones are behind a NAT router, the SIP provider must support a NAT detection mechanism. The provider will receive SIP packets containing private IP addresses, this private IP addresses have to be replaced by the provider with the corresponding public addresses. If this test fails, the RTP-endpoints (PBX and phones) must be configured to use a STUN server. In order for NAT-Traversal using STUN to work, the NAT-Router must support full-cone or restricted NAT. If the NAT Detection test is passed successfully, don't configure a STUN server at the RTP-endpoints and also not at the SIP-interface.

Test description:

• Tel1 is in a private network behind a NAT router.
• Tel1 calls a phone in the PSTN
• check that audio is sent in both directions

Note: If the provides supports NAT detection but requires that the RTP endpoint is always the PBX, the RTP-endpoints (PBX and phones) should be configured to use a STUN server (i.e. as if NAT Detection is not supported). As an alternative, in case that STUN is not possible, you can configure the SIP Trunk with Media-Relay - without STUN configuration. As a result of using Media Relay, the amount of concurrent calls is much lower. In both cases we still set the result as NOK for this point.

call using SRTP(optional)

Purpose: Check if provider offers the possibility to encrypt audio data(RTP) using SRTP

Test description:

• Configure the SIP interface for SRTP and check the call screen(Gateway/Calls) if the call is using SRTP(Lock Symbol). If SRTP is supported by the provider, use it for all tests.

call using g711a(KO*)

Purpose: Test of capability to handle G711A RTP - Streams.

Test description:

• Tel1 is configured with G711A exclusive coder preference.
• Tel1 calls a phone in the PSTN

Note: * The provider must support at least one of the G711-Variants(a or u-law), preferably both are supported.

call using g711u(KO*)

Purpose: Test of capability to handle G711U RTP - Streams

Test description:

• Tel1 is configured with G711U exclusive coder preference.
• Tel1 calls a phone in the PSTN

Note: * The provider must support at least one of the G711-Variants(a or u-law), preferably both are supported.

call using g729(Important)

Purpose: Test of capability to handle G729 RTP - Streams

Test description:

• Tel1 is configured with G729 exclusive coder preference.
• Tel1 calls a phone in the PSTN

call using g722(optional)

Purpose: Test of capability to handle G722 RTP - Streams. At this moment (13/12/2011) the test only could be done when no media-relay is used and tested with a loopback call.

Test description:

• Tel1 and Tel2 is configured with G722 exclusive coder preference.
• Tel1 calls to Tel2 DDI number so call will go to the Provider and come back.
• Check if call is establish and g722 is used.

Loop In Call(Innovaphone -> Provider -> Innovaphone)(Important)

Purpose: Test if the provider handles a loop in call correctly. This test is done by calling out to a number of the same account.

Test description:

• Tel1 calls Tel2 using DDI ( call goes via SIP Provider).
• Check if the Tel2 phone display shows the correct CGPN.
• Check if the CDPN is forwarded correctly to the PBX.
• Check if we have 2way audio.

Overlapped sending(optional)

Purpose: Test if the provider supports Overlap Dialling. If not Enblock Dialling must be configured on the SIP - trunk.

Test description:

• Route from PBX to SIP-provider is not configured with 'Force Enblock'.
• Tel1 calls a phone in the PSTN

early media channel(KO)

Purpose: Test if the transmission of RTP - packets is possible before the call has been accepted (200 OK) by both endpoints. This feature is used when the caller receives announcements or dialtones from the SIP - Provider or from the PBX (i.e. 'The dialled number is incomplete.')

Test-1 description:

• Tel1 calls a not existent(incomplete) phone number in the PSTN.
• PSTN-Provider will play an announce, i.e. 'Number incomplete'.
• The SIP-provider forwards the announcement and terminates the call setup procedure.

Test-2 description:

• Incomning call from SIP provider to a WQ - w/o connect flag.
• the WQ will play the announcement to the calling PSTN phone before the call is connected.
• Check if the announcement is heard on the PSTN phone.

Fax using T.38(Important)

Purpose: Test of capability to handle T.38 signalling/data. Without this feature Fax over IP is not possible.

Test description:

• Fax1 sends a message to a fax machine in the PSTN.
• Fax1 receives a message from a fax machine in the PSTN.

Fax using G711(optional)

Purpose: Test of capability to use G711 Audio Streams to handle fax.

Test description:

• Port of Fax1 is configured without T.38
• Fax1 sends a message to a fax machine in the PSTN.
• Fax1 receives a message from a fax machine in the PSTN.

T.38 Transcoding by the provider(Important)

Purpose: Some providers don't transcode T.38 fax - calls in their RTP-server but do only relay the received RTP packets to other customers or other SIP-providers. The innovaphone fax-server is able to send G3 fax to and receive G3 fax from T.38 compliant remote parties. It is not able to support remote parties which provide G.711 (a.k.a. transparent mode (T30/G.711) fallback or software fax) based fax only.

As a result, even though the provider supports T.38 but doesn't do T.38 transcoding, fax calls sent by the innovaphone fax-server to devices that support only G711 based fax will fail. This test is only necessary/useful if the provider supports T.38.

Test description: Ask provider if he does T.38 transcoding. If you cannot get the information from the provider, a possible test would be:

• connect two fax-machines on one IP2x/IP302, configure the TEL1 interface for G711 and on the TEL2 interface "Enable T.38" .
• make a loop call from TEL1 to the TEL2, by calling from the fax machine connected to TEL1 the external PSTN-number of the TEL2 interface.
• the call will be routed through the SIP-provider. Check the Gateway/Calls screen of the IP2x/IP302 for the protocol used on the TEL2-call. If T.38 is used, the provider does transcoding.

Reverse Media Negotiation(Important)

Purpose: Test if Reverse Media Negotiation is implemented by the provider. If this test fails, media relay and an exclusive coder setting must be configured for this provider, resulting in a much lower amount of concurrent calls.

Test description:

• Tel1 calls Tel2. Tel2 picks up the call.
• Tel2 makes a blind transfer to a phone in the PSTN.
• This is the quickest way to send a INVITE message without SDP over the SIP-Trunk.

Long time call possible(KO)

Purpose: Test if provider interrupts call after session timer expires (RFC 4028). It is possible that the provide awaits Re-Invite after the timer expires. However the session timer is not implemented by innovaphone, so we will not send a Re-Invite.

Test description:

• Tel1 calls a phone in the PSTN
• call is longer than session expire time-out.(usually 1800 seconds)

External Transfer(KO)

Purpose: In scenarios were Media Relay is not used, the provider will have to connect both calls by short-circuiting his RTP ports. This may not be supported by the provider, as a consequence Media Relay is mandatory in such cases.

Test description:

• A phone in the PSTN calls Tel1
• Tel1 picks up the call and transfers it to a second PSTN phone.
• Manual test of voice quality by speaking/listening on both ends.

Mobility Call(important)

Purpose: Test if DTMF signals going from the Provider(mobile phone) to the PBX are sent and interpreted correctly. When a call is forked via the Mobility object, DTMF - tones are sent only via SIP - Info messages. If this test fails, media relay must be configured for this provider, resulting in a much lower amount of concurrent calls.

Test description:

• Tel1 calls Tel2. Tel2 has a Fork destination(with Mobility) in the PSTN.
• PSTN - phone sends ** (Mobility DTMF digits for R-key), check if Tel1 is put on hold.

WebRTC Call(optional)

Purpose: Test if the provider supports ICE and DTLS-SRTP. When a call is created by an WebRTC endpoint, ICE and DTLS-SRTP are required by the remote RTP-endpoint. If this test fails, media relay must be configured for this provider, resulting in a much lower amount of concurrent calls.

Test description:

• create an WebRTC endpoint on your PBX and make a call to the PSTN.

Voice Quality OK?(KO)

Purpose: Simple test of overall voice quality during a call.

Test description:

• Tel1 calls Tel2. Tel2 picks up the call.
• Manual test of voice quality by speaking/listening on both ends.

Redundancy Mechanism(Important)

Purpose: Check if provider offers a redundancy mechanism. For example if a gateway with a trunk to the SIP provider fails, incoming and outgoing calls could still work using a secondary gateway.

Test description:

• Ask provider about his redundancy mechanism and describe the concept. Test it if possible.

DTMF

DTMF tones sent correctly via RTP-Events(RFC 2833)(KO)

Purpose: Test if DTMF signals going from PBX to Provider are received and interpreted correctly.

Test description:

• Tel1 calls phone in the PSTN.
• Check if the DTMF - tones are received at the PSTN phone(using SOAP). For manual test create a Waiting Queue with Map on some PBX connected via PSTN.
• trace on the phone Tel1(All TCP/UDP) and check the RTP-stream for RTP-events

DTMF tones received correctly via RTP-Events(RFC 2833)(KO)

Purpose: Test if DTMF signals going from Provider to PBX are received and interpreted correctly.

Test description:

• PSTN Phone calls Tel1
• Check if the DTMF - tones are received at Tel1(using SOAP). For manual test create a Waiting Queue on the tested PBX with Map.
• trace on the phone Tel1(All TCP/UDP) or trace at the PBX if you call the Waiting Queue and check the RTP-stream for RTP-events

Hold/Retrieve

When a call is put on hold, users normally expect to hear some kind of music/announcement signalling them that they should wait. However there are two possibilities. The PBX generates the announcement or the provide generates it.

To test a PBX generated announcement, use the R - key to hold a conversation. This type of holding is tested in the Hold/Retrieve , Transfer with consultation and Transfer with consultation (alerting only) scenario.

To test a provider generated announcement, use the redial key to hold the conversation. This is used when doing a blind transfer.

Call can be put on hold(KO)

Purpose: Test if provider handles hold signalisation by Re-Invite correctly.

Test description:

• Tel1 calls phone in the PSTN.
• Tel1 presses the 'R-Key'. Test if call is on hold(MoH/announcement).
• Tel1 presses again the 'R-Key'. Test if call is retrieved and conversation is continuable.

Held end hears music on hold/announcement from PBX(Important)

Purpose: Test if provider handles hold using the sendonly attribute correctly. The MoH will be transmitted by the PBX to the provider and must be then forwarded to the waiting phone.

Test description:

• Tel1 calls phone in the PSTN.
• Tel1 presses the 'R-Key'. Test if call signalisation is correct and MoH is audible on waiting phone.

Transfer with consultation

Call can be transferred(KO)

Purpose: Test of call-transfer with consultation(via R-key)

Test description:

• use test-matrix from SIP-Provider test template

Held end hears music on hold(Important)

Purpose: Test for MoH/Announcement on hold phone.

Test description:

• use test-matrix from SIP-Provider test template

Transfer with consultation (alerting only)

Call can be transferred(KO)

Purpose: Test of call-transfer with consultation

Test description:

• use test-matrix from SIP-Provider test template

Held end hears music on hold(Important)

Purpose: Test for MoH/Announcement on hold phone.

Test description:

• use test-matrix from SIP-Provider test template

Call returns to transferring device if the third endpoint is not available(KO)

Purpose: Test if returning calls are handled correctly by the provider.

• Tel1 calls phone in the PSTN.
• Tel1 presses the 'R-Key' and dials the number of Tel2.
• Tel1 doesn't wait for Tel2 to pickup the call and hangs up. PSTN phone changes its status from 'hold' to 'active'.
• Tel2 doesn't answer the call. Depending on the configured Recall 'Timeout' the call falls back to Tel1.
• Tel1 answers calls and checks if audio is working in both directions

Blind Transfer

Call can be transferred(Important)

Purpose: Test of blind transfer (Redial-Key)

Test description:

• use test-matrix from SIP-Provider test template

Held end hears music on hold(Important)

Purpose: Test for MoH/Announcement on hold phone.

Test description:

• use test-matrix from SIP-Provider test template

CFU / CFB Transfer

Call can be forwarded(KO)

Purpose: Test of Call Forward Unconditional or Busy

Test description:

• use test-matrix from SIP-Provider test template

Calling party hears dial tone(KO)

Purpose: Test for Ringing on caller Phone.

Test description:

• use test-matrix from SIP-Provider test template

CFNR / Blind Transfer (alerting only)

Call can be transferred or forwarded(KO)

Purpose: Test of blind transfer (Redial-Key) or Call Forward no Response.

Test description:

• use test-matrix from SIP-Provider test template

Calling party hears dial tone(KO)

Purpose: Test for Ringing on caller Phone.

Test description:

• use test-matrix from SIP-Provider test template

Broadcast Group & Waiting Queue

Caller can make a call to a Broadcast Group(KO)

Purpose: Test of basic functionality of the Broadcast feature, using Re-Invite.

Test description:

• PSTN Phone calls a 'Broadcast Group' number.
• Tel1 and Tel2 are ringing. Tel1 picks up the call.
• Test if audio channels between PSTN phone and Tel1 are established correctly.

Caller can make a call to a Waiting Queue(KO)

Purpose: Test of basic functionality of the Waiting Queue feature, using Re-Invite.

Test description:

• PSTN Phone calls a 'Waiting Queue' number.
• Tel1 and Tel2 are ringing. Tel1 picks up the call.
• Test if audio channels between PSTN phone and Tel1 are established correctly.

Announcement if nobody picks up the call(KO)

Purpose: Test if announcement feature at Waiting Queue is working correctly.

Test description:

• PSTN Phone calls a 'Waiting Queue' number.
• Tel1 and Tel2 are ringing.
• PSTN phone hears an announcement from the PBX, i.e. 'All operators are busy. Please hold the line.'
• Tel1 picks up the call.
• Test if audio channels between PSTN phone and Tel1 are established correctly.