Course12:Basic - Operations: Difference between revisions
(New page: {{#moodlebook: Master Templates / V12 Templates / Basic | Operations | 121 }}) |
m (Protected "Course12:Basic - Operations" [edit=sysop:move=sysop]) |
(No difference)
|
Revision as of 11:49, 24 March 2016
Explains how to setup an innovaphone PBX system so that it can be maintained smoothly.
Monitoring and Debugging
- make sure you notice problems before your end users do
- gather as much information to understand the nature and cause of any problems
![](https://class.innovaphone.com/moodle2/filter/wikilink/imm6.png)
Events
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
There are a
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Some of them indicate bad user behaviour such as
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Others may indicate a misconfiguration of the system such as
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Yet others will likely report hardware problems such as
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Also, temporary performance problems in your network could create an event, such as
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
In any case, events shown in the event log must be examined carefully and it needs to be determined if they indicate a problem that need to be fixed.
Clicking on the event-code listed in the Code column will open a popup window with some useful context information. For example, for the aforementioned type 0x00010002,
Protocol | SIP |
AOR | <sip:a@x> |
Comment | Timeout |
will be shown.
Alarms
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
In a managed installation, there should be no entries in the alarm list at any time!
The event and alarm list is always kept in local flash memory(not on the FLASH drive, SSD or CF-card). The number of entries to keep can be configured in the
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
All alarms are sent via SNMP traps. The SNMP trap delivers: code,severity,txt (e.g interface down) and alarmsource (e.g IP0/ETH1). This can be useful if the customers has a SNMP network management tool in use.
![](https://class.innovaphone.com/moodle2/filter/wikilink/imm6.png)
- unplug the cable from your IP411LEFT's BRI1 interface
- have a look at your
Maintenance / Diagnostics / Events
You should see analarm stating that on RELAY/BRI1 the interface is down
The event list will show you a list of recent events. However, if you look at the alarms which currently are raised will be shown.
- plug the ISDN cable in to the BRI1 interface of your IP411LEFT (the other end of this cable should be plugged in to BRI3 of your IP811)
- the BRI1 LED of your IP411LEFT should light now and ...
- ... there is a new event saying that
the alarm is cleared
- also, in the Alarms list, the cleared alarm disappeared
Syslog
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
- PBX/Gateway Calls - calls and their flow in the PBX or Gateway
- Gateway Routing - number mapping and call routing in Relay
- H.323/SIP-Registrations - endpoint/interface registration successes or fails
- Administration - configuration changes performed on the device
- and more
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
![Further Hints (Further Hints)](https://class.innovaphone.com/moodle2/filter/wikilink/hints.png)
- tick on all logging check marks that relate to information relevant to the problem
- tick off any logging check mark that creates irrelevant information
- try to trim down the log so that it fully captures the reproduced problem but nothing else
- clearly indicate the situation captured in the syslog (not: there is a problem, see syslog. Rather: in the syslog, there is a call coming from TEL1, calling party is xyz, called party is abc. It is sent to extension -xx and terminates with cause code 21. Expected behaviour though is for the call to alert at the phone registered with user efg)
- do not put it into a MS Word or MS Excel file
- do not send a screen shot of the syslog screen
- You can easily save the syslog with copy and paste and send it as a text file
![](https://class.innovaphone.com/moodle2/filter/wikilink/imm6.png)
![smile](https://class.innovaphone.com/moodle2/pix/s/smiley.gif)
- unplug the BRI1 ISDN cable from your IP411LEFT
- open
Maintenance / Diagnostics / Logging - tick the Gateway Calls check-mark and click OK
- click on syslog, the page will show you all log entries in real time
- now call 0123 from your IP232 (Edward Hyde)
- you should see a channel not free message on the phone and hear
- tone
At this point, a number of log entries will appear in your browser window.
- copy the syslog in a file
It should look similar to this one. There is in fact a good deal of useful information in it:
20160122-074142 PBX0 7 Edward Hyde(14:edward.hyde) setup-> (0123)
Edward calls 0123
20160122-074142 PBX0 8 Trunk() peer-> Edward Hyde(14:edward.hyde)
Call is routed to the Trunk
20160122-074142 PBX0 8 Trunk(0123) <-setup Edward Hyde(14:edward.hyde)
20160122-074142 CALL 0 Alloc
Call is sent to the gateway layer (relay)
20160122-074142 CALL 0 A:Call -> / RB1::->*::
It origins from interface RB1 (which is BRI1's registration at Trunk)
20160122-074142 CALL 0 B:Call 14:edward.hyde@class.innovaphone.com->123 / RB1:14:Trunk->TONE:123:
Sent to the TONE interface to provide dial-tone if necessary
20160122-074142 PBX0 8 Trunk(0123) call-proc-> Edward Hyde(14:edward.hyde)
20160122-074142 PBX0 7 Edward Hyde(14:edward.hyde) <-call-proc (0123)
20160122-074142 CALL 0 B:Rel 14:edward.hyde@class.innovaphone.com->123 / RB1:14:Trunk->TONE:123: Cause: Resources unavailable, unspecified
rejected by the TONE interface (as we already have further dialled digits, no dial-tone is necessary)
20160122-074142 CALL 0 B:Call 070317300914:edward.hyde@class.innovaphone.com->123 / RB1:14:Trunk->BRI1:123:
so it is routed to BRI1
20160122-074142 CALL 0 B:Rel 070317300914:edward.hyde@class.innovaphone.com->123 / RB1:14:Trunk->BRI1:123: Cause: Requested circuit/channel not available
rejected by BRI1, as there is no channel available (cause code)
20160122-074142 CALL 0 A:Rel 070317300914:edward.hyde@class.innovaphone.com->123 / RB1:14:Trunk->BRI1:123:
as there are no other lines configured which could be tried, the call is rejected
20160122-074142 CALL 0 Free
20160122-074142 PBX0 8 Trunk(0123) rel-> Edward Hyde(14:edward.hyde)
20160122-074142 PBX0 7 Edward Hyde(14:edward.hyde) <-rel (0123)
So, log entries are not for the nerd only
![wink](https://class.innovaphone.com/moodle2/pix/s/wink.gif)
Tracing encrypted Signalling
In such installations, you can also get traces for trouble shooting (either with text trace or wireshark) and send them to us for analysis. However, as they are encrypted, we cannot read them! (no, there is no manufacturer back-door)
While we usually have no intention to listen to your talks (so SRTP is not a problem), we usually do want to analyze you call signalling. As a result, traces from encrypted communication are useless to us. To circumvent this problem, you must tick the All IPv4 TLS Traffic (or All IPv6 TLS Traffic) check-mark in the tracing page when obtaining traces. This will enclose an uncrypted copy of the trace signalling data.
![screenshot.png](https://class.innovaphone.com/moodle2/filter/wikilink/screenshot.png)
![Screenshot: Encrypted Tracing - Course_Data\\Operational_Basics/tls-tracing.png](https://class.innovaphone.com/moodle2/file.php/533/Course_Data%5C%5COperational_Basics%2Ftls-tracing.png)
If you want to see the encrypted signalling messages in Wireshark, you can mark a specific packet and apply the
![www.png](https://class.innovaphone.com/moodle2/filter/wikilink/www.png)
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Traces
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
As opposed to syslog, traces are permanently saved into an in-core ring buffer. This implies that
- you can obtain information about events that happened before you obtain the trace (provided you obtain it before they get overridden in the ring buffer)
- when you reproduce a problem and then obtain the trace, the trace will include information prior to your reproduction of the problem (which is probably not useful)
- tracing always consumes CPU resources, not only while obtaining a trace
- obtain the trace, thereby clearing the trace buffer
- reproduce the problem
- obtain the trace again
Trace is also a simple text based tool and the notes made about obtaining and providing it to support made for the syslog do apply here too.
Trace Levels
Like for logging messages, trace messages can be configured. This is done in
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Some more exotic trace flags are available on a special, hidden page. You can reach it using the special URL http://x.x.x.x/debug.xml.
![Further Hints (Further Hints)](https://class.innovaphone.com/moodle2/filter/wikilink/hints.png)
Tracing and CPU Load
Tracing does consume CPU cycles. More tracing consumes more CPU cycles.
![Further Hints (Further Hints)](https://class.innovaphone.com/moodle2/filter/wikilink/hints.png)
Be sure to turn of any tracing check-marks both in
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Trap Traces
Writing to the trace ring buffer is disabled when the device traps. Also, the trace ring buffer is not cleared on a re-start. Thus, after a re-start, you can obtain the trace from before the re-start, showing the trap situation.
When your device traps, you can be sure that support will ask for the trap trace!
When tracing is disabled due to a previous trap, the pre-trap trace buffer is kept until the next reboot. For this reason, you can obtain it even days later. However, it is important that you copy the trap trace immediately. As soon as you have retrieved the trap trace for the first time, the trace buffer is cleared and normal tracing commences.
![Further Hints (Further Hints)](https://class.innovaphone.com/moodle2/filter/wikilink/hints.png)
Wireshark
From V6 SR2, the trace mechanism was greatly enhanced by the
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
![download.png](https://class.innovaphone.com/moodle2/filter/wikilink/download.png)
When the Enable RPCAP check mark is ticked in the
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Usually, you will also tick All TCP/UDP Traffic and All IPv4 TLS Traffic in the tracing tab, so that Wireshark will capture all network traffic from and to the device. However, wireshark will also capture all other trace information too. Wireshark's advanced filtering and analysis capabilities allow to capture even huge traces in a life system and drill down the information so that the problem reproed can be isolated from all the irrelevant information. To begin with, try to capture a normal call, then look at Telephony / Voip Calls.
It is generally important to tick the All IPv4 TLS Traffic check-mark, as nowadays, most of the signalling is (or at least should be) encrypted. Setting this option will include the decrypted version of this traffic within the trace. Encrypted signalling traffic traces are useless when it comes to trouble shooting!
Recommended Version
There has been a big wireshark rewrite lately (version 2.x), but at the time of this writing, it is quite buggy still. We therefore recommended to use the last stable version 1.x release (this is the one linked to on our
![download.png](https://class.innovaphone.com/moodle2/filter/wikilink/download.png)
The innovaphone DLL
innovaphone provides a so-called plug-in for wireshark, the innovaphone-dll. This dll provides support for innovaphone trace messages in a wireshark capture.
It is part of the V6 Voice Mail, TAPI, SoftwarePhone, Operator download on
![download.png](https://class.innovaphone.com/moodle2/filter/wikilink/download.png)
![download.png](https://class.innovaphone.com/moodle2/filter/wikilink/download.png)
You must
![screenshot.png](https://class.innovaphone.com/moodle2/filter/wikilink/screenshot.png)
Using the rpcap Interface
![](https://class.innovaphone.com/moodle2/filter/wikilink/imm6.png)
- be sure you have installed the innovaphone-dll (see previous chapter)
- open wireshark
- click on
Capture Options
- un-check your
local Ethernet/WLAN interfaces
- click on
Manage Interfaces
- click on
New and enter rpcap://x.x.x.x/trace in the the Pipe: field
- click on Save and Close
- You may see a popup saying
The capture session could not be initiated . This is because you have not yet enabled rpcap on the device. Go to
Maintenance / Tracing andtick the Enable RPCAP, All IPv4 TCP/UDP Traffic, All IPv4 TLS Traffic, H.323 Signaling and BRI1 check-marks (do not forget to click on the OK button!)
- restart the capture (you can click on the
Start a new live capture icon)
![screenshot.png](https://class.innovaphone.com/moodle2/filter/wikilink/screenshot.png)
While Wireshark still captures packets, we will simulate an issue (the same as before). So
- unplug the ISDN cable from BRI1 of your IP411LEFT
- From Edward's phone, call 0123
- stop the capture by clicking on the
Stop the running live capture icon
You will now see even more captured packet traces in Wireshark and you may get totally confused. Fortunately, Wireshark cannot only display captured packets. It can do some fair amount of analysis.
So let us dig out the call information from all this packets!
- select
Telephony / VoIP Calls
- scroll down in the list until you find the call that is towards 0123 (which is what we have called in our test call)
- You will find a call to 0123 followed by a call to 123 (this is the same call, when it is passed to the gateway level (where the trunk access code is already stripped)
- select both and
click on Flow
![screenshot.png](https://class.innovaphone.com/moodle2/filter/wikilink/screenshot.png)
![Screenshot: Wireshark Call Flow Analysis - Course_Data/Operational_Basics/wireshark12.png](https://class.innovaphone.com/moodle2/file.php/533/Course_Data%2FOperational_Basics%2Fwireshark12.png)
You don't think you will ever understand all this stuff?
No problem! Simply make sure the call that you want to debug is actually part of the trace, then
![screenshot.png](https://class.innovaphone.com/moodle2/filter/wikilink/screenshot.png)
Storing and Consolidating Syslog and Events
Also, in a distributed system such as a VoIP PBX, it is often difficult to determine where the problem actually would be logged. For this reason, it is clearly desirable to consolidate all events, alarms and sylogs to a single place. You can do this by relaying this type of information to a single web server for storage. This will usually be one of your innovaphone devices that has a CF card installed.
Relaying is configured in the
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
This way, all events, alarms and syslog entries are permanently sent to the central device. All events and alarms will show up in the
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Remote events and alarms are not stored in the Flash memory of the receiving device (i.e. your central event/alarm server). If the central event server is restarted, all remote events are lost. Alarms are retransmitted by the generating device (2 minutes after reboot and every 30 minutes), updating the alarms list on the central device. If you have to reboot the central event/alarm server, make sure to use the Save option to store the Event-list.
![](https://class.innovaphone.com/moodle2/filter/wikilink/imm6.png)
Using DHCP for basic Configuration
innovaphone devices thus feature a number of configuration options that can be
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Generally, when a devices receives an option via DHCP, it overwrites the configuration found in the device itself. This makes it possible to overwrite default options. The options received by DHCP are shown in each devices Current Lease area in the
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Which DHCP Server to use?
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Much easier however is to use innovaphone's
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Running a "private" DHCP Server
In such cases, the innovaphone DHCP server can be used to serve innovaphone clients only. This is enforced by setting the Reserved and same Vendor Clients only check mark on the
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
Please note that it is usually not required to set the Server Identifier or Selected Server only field.
How to find correct option values
To help with this, you can have a look at the
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
To find out the right settings, you would thus
- turn off DHCP client mode in the phone (to make sure there are no settings received with DHCP)
- configure the phone according to your needs using the phone menus or the web user interface
- copy the DHCP option values from
Phone/State/DHCP-Options to your DHCP option settings
It is also possible to use
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
The vendor class identifier and the vendor options can be entered as character strings or in hex representation.
Creating Announcements
There have been a number of methods to create audio files for announcements, including /DRIVE/CFX or the softcod.exe utility. However, the only recommended method now is the audio converter
![link_intern.png](https://class.innovaphone.com/moodle2/filter/wikilink/link_intern.png)
Creating good Source Audio Files for Announcements
For best results, you have to create high quality source (.wav) audio files before conversion.
For quick (and dirty) creation of voice-only announcements, you can simply use the phone and here is how
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
If you need to produce high-quality announcements, here is your source of advice:
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)
However, there are of course also commercial suppliers of high-quality custom announcements. See
![fish-help.png](https://class.innovaphone.com/moodle2/filter/wikilink/fish-help.png)