Howto:How to implement large PBXs

innovaphone PBX installations scale from very few users (such as with an IP202) to several thousand users. Here is how to design and configure innovaphone devices to support many users.

Applies To

This information applies to

• All innovaphone V5 PBX platforms

build 05-5842 and later.

More Information

Technical issues to be considered

There are a few issues to be considered when configuring a large PBX:

• CPU power

The platform must be able to cope with all processing requirements to serve the user base. The basic requirement is to support the number of calls created. The ability of a given platform can be measured as its “business hour call attempts” capability, a figure which essentially indicates the number of calls per hour the platform can handle.

Naturally, not only calls take CPU cycles. Major scenarios to consider here include user registration, replication and SOAP/CTI applications (such as TAPI)

• Main Memory

The platform must feature enough main memory to serve the user and call scenarios. Rough “memory/feature” figures will be given.

• Flash Memory

There must be enough persistent memory to store the system configuration. Although the amount of memory used largely depends on the details of the configuration, a rule-of-thumb can be given for the “memory/user” ratio.

• Configuration

The configuration must be appropriate to the intended usage scenario. Guidelines are given for various scenarios.

• Number of channels

Naturally, the number of gateway channels (e.g. to the ISDN or to the DECT network) must be appropriate. The number is a function of the so-called “erlang” value. This issue will not be discussed here.

The information presented in this article is based on practical verification by means of a sample configuration. The setup is made up of 6000 user entries, 10 location entries, 500 speed dial entries and a number of other entries such as trunks etc.

The configuration is attached for your reference.

The standard Testsetup User

The standard Testsetup User (Details)

All measurements given in the remainder of this article apply to the IP800. Since the IP800 both in terms of processor architecture and processing power is comparable to the IP3000, the results apply to the IP3000 too.

Theoretical up to 10000 User PBX can be implemented with IP6000:

Recommended number of users supported

The following table include figures for the maximum number of “registrations” supported by the various innovaphone PBX platforms. These figures are intended as a rough estimate “rule of thumb” guideline.

Platform	Maximum number of registrations as per datasheet	Recommended maximum number of registrations	Recommended maximum number of defined objects	Physical maximum number of defined objects	Default limit for number of defined objects
IP3000 / IP3000 DD	1000	1000	2500	2500	2000
IP800	200	1000	6500	8000	2000
IP400	200	100	100	350	350
IP202	20	100	100	350	350
IP21	k.A.	100	100	350	350

The Recommended maximum number of defined objects refers to the number of entries in the LDAP database. The Recommended maximum number of registrations refers to the maximum number of H.323/SIP peers which can be registered at the same time.

The recommended figures are pretty conservative, so as to make sure that a configuration obeying the figures will work well under normal circumstances.

The number of registered users is reduced for the IP400 due to the limited CPU resources available. That is, the IP400 can support 200 registrations in fact, however, in certain usage situations, such as use of broadcast groups or waiting queues, missing CPU cycles could result in slow service.

All figures are related to a single platform (that is, innovaphone box). Larger installations can be designed using several boxes running related PBXs (a.k.a. locations). See below for details.

The IP202 and IP21 technically could support more users than indicated in the table above under certain conditions. However, this is not supported and can only used “at own risk”.

The default limitation for the number of defined objects (see table above) cannot be increased for the IP21, IP202 and IP400. On the IP800 and IP3000 it can be increased to the physical limit using the

mod cmd FLASHDIR0 /max nitems

config command.

Hardware characteristics

Generally, the IP800 and IP3000 are CPU-wise the fastest innovaphone platforms (with the IP3000 being roughly 10% faster that the IP800). The IP800 is 8 times faster than the IP400. The IP400 is thus not ideally suited to design large PBXs.

The IP800 has twice the main memory than the IP3000 and 4 times the flash memory. Due to this, the IP800 is best suited to run large PBXs (both in terms of defined objects in the LDAP database and the CPU), whereas the IP3000 is good for many registrations but not as capable to huge numbers of object definitions.

IP202 and IP21 have a rather limited capability to store object definitions but more than enough CPU and main memory to cope with the intended target scenarios.

Flash Memory

Flash memory is mainly used to store object definitions in the LDAP database. So the number of defined objects is calculated as a function of the flash memory used to define the objects.

The actual space used by an object definition heavily depends on the number and size of attributes set for the object since the objects are stored “packed” in flash memory. For example, users with long names will consume more space than those with short names. Also, users with a defined call forwarding will consume more space. Because of this, in a given scenario, the number of objects that can be stored in the platforms available flash memory can vary considerably. However, since in normal operation, object definitions are changed frequently, you need to calculate a rather large amount of memory per entry. We recommend to plan for 512 bytes per object entry. This should be sufficient even for unusual complicated configurations. In a given installation, it is well possible that a platform supports significantly more defined objects than indicated in the table above.

Please note that the number of defined objects is not identical to the number of users supported. This is because there are usually a lot of non-user objects defined. As an example, speed dial entries will count into the total number of defined objects. The actual ratio of supported users to defined objects obviously is depending on the installation at hand. Still, as a rule of thumb, you should calculate at least 33% more defined objects than supported users. That is, if you intend to support 1000 users, you may want to calculate for 1400 defined objects.

Also, you need to be aware that the LDAP database needs to keep track of deleted items for replication purposes. Such items are kept for a while in flash memory (although they are not seen in the user interface any more). The amount of memory required to store such items obviously depends on the deletion rate of objects. Especially when using automated tools to delete and create a large number of objects, you may create a lot of “deleted” items which consume flash space. For this reason, when you create a new object database in a PBX which already had a significant number of objects before, make sure first to do a “clear PBX config”. This will physically clear the LDAP database whereas any other method (including ad-hoc replication from a master PBX database) will create a deleted object for each previously existing object.

You can query the current flash memory usage with

mod cmd FLASHMAN0 info

Main Memory

Main memory is obviously required for any kind of operation in the PBX. However, there are some prominent situations which require sufficiant memory. These include amongst others

• System boot

Depending on the firmware size (the firmware is store compressed in flash and thus will be uncompressed into main memory at boot time) and the basic configuration, there is an initial main memory requirement of slightly less than 3MB currently.

During boot, all objects are retrieved from the LDAP database and shadowed in main memory. This will take significantly more space than it consumes in flash memory. The exact amount again depends on the specific object configuration. As a rule of thumb, you can expect 2.7kB per object (based on the same configuration complexity assumed above for flash memory usage). However, during boot time, there is an additional need for main memory of slightly less than twice the flash memory used. We recommend to calculate for approximately 3.5kB per user main memory thus.

• Endpoint Registration

When an endpoint is registered with the Gatekeeper (PBX), it will consume an additional 1.6kB main memory.

• CDR delivery

CDRs to be delivered via HTTP are buffered if delivery cannot take place immediately. The buffer space reserved is 300kB. Since there are 2 CDR interfaces and the LOG interface is treated identically, 900kB must be calculated for buffering.

• Call signaling

Concurrent call signaling will consume further main memory. The exact amount of memory obviously vastly depends on the exact nature of the calls (e.g. internal calls, calls to local interfaces, broadcast calls, etc.). We have measured a test setup where 100 registered clients constantly setup and tear down calls so that there is an average of 15 calls at a time. All calls were monitored by a SOAP application (such as TAPI).

This creates a memory load of an additional 700kB.

Please note that this is a heavy duty load scenario since long lasting calls will consume much less memory load.

You can query the current memory usage with

mem

CPU

The overall performance of a PBX can be measured and given as busy hour call attempts (BHCA) figure. We have measured 2 scenarios:

• Pure gatekeeper mode

This is the same scenario as in the “Main memory / Call Signaling” section above. 100 clients are registered with the PBX and constantly setup and tear down calls so that there is an average of 15 calls at a time. Both ends of all calls terminate in an IP endpoint, so external call signaling issues such as those induced by an ISDN interface are not considered. No SOAP monitoring.

The IP800 performed with 4.57 calls per second with a CPU load of 30%. This amounts to 15 calls per second with a 100% CPU load and thus approximately 50.000 BHCA.

• Combined gatekeeper and gateway mode

Same scenario as above except that additionally all ISDN interfaces are busy with calls from IP to ISDN with a 60ms packet size. This base load creates 6% CPU load.

The IP800 performed with 45.000 BHCA in this scenario.

The IP3000, having approximately the same CPU power, has more ISDN channels to drive when in gateway mode (30 instead of 8). The respective figures for the IP3000 do differ thus. The base load (all 30 channels active with 60ms packets) is 21%. The IP3000 performs with little better than 40.000 BHCA in this scenario.

For both platforms, the base load increases significantly if the packet size is reduced. For example, the IP3000’s base load increases to 42% when 30 20ms channels are active. It is thus recommended to avoid small packet sizes.

An IP800 with 6500 defined objects will take about 20 seconds to boot.

You can query the current CPU load with

mod

To reset the CPU tick counter, use

mod clr

LDAP performance

LDAP is used both for replication of PBX and DECT gateway configurations and for name and number resolution in the IP200.

Using current firmware, there are no performance issues with the LDAP replication.

LDAP name and number resolution performance largely depends on the number of entries defined in the PBX/LDAP database. Based on a setup with 1000 user and 500 speed dial entries configured, the IP800 performs with a maximum of 16 LDAP connect/query/result/disconnect cycles per second.

Assumed that there is one such cycle per call, this will effectively cut down the BHCA to 50%.

It is recommended to use an external LDAP server such as MetaDirectory for large (> 1000 users) PBXs thus.

The number of concurrent LDAP sessions is limited to 30 in a standard configuration. When a new session is requested, older idle sessions will be terminated. However, replication sessions are never terminated so this may result in the ability to create a new session requested. The IP200’s LDAP client will terminate its query session after 3 seconds, however with many phones, the PBX may run out of LDAP sessions. You may want to increase the number of sessions using the

config change LDAPSRV0 /max_session n

config file option (with 300 being the maximum for n).

Small PBX

For smaller PBXs (1000 users and less), a single IP800 or IP3000 can serve as a platform. If redundancy is desired, one platform can be used as master PBX, the other can be used as gateway and standby PBX (so as to distribute gateway and gatekeeper load, especially for the IP3000).

Medium PBX

If the PBX has to support more users, then we recommend to use one IP800/IP3000 per 1000 PBX users. If a uniform numbering scheme is desired and all users shall be maintained in a single PBX, we recommend to add an IP800 which will then hold the complete PBX/LDAP database. This will be the “master PBX” and all other platforms will be defined as “slave location” and register with the master PBX. Their configuration will be replicated from the master PBX, restricted to their specific location data. This is because both hosting the complete configuration and serving a large number of registrations at the same time will decrease performance. The actual endpoint registrations will be only on the slave locations so that each platform needs to handle no more than 1000 registrations at a time.

Each endpoint will register with its defined location PBX. You can freely assign endpoints to locations at will, provided the number of endpoints per location does not exceed 1000.

The master PBX should be used as standby PBX for all locations, so that if a single location PBX fails, the master PBX will take over the 1000 registrations.

While the endpoints register with their location PBX, the IP200s LDAP connection should be configured such that the central master location PBX is used as LDAP server. This is to make sure all IP200 will see the complete phone book. As discussed above, you may want to consider an external LDAP server in which case this should be configured as LDAP connection.

Up to 6000 users can be supported this way.

Large PBX

If a uniform numbering plan and centralized administration is not an issue, large PBXs can be design virtually without limits.

A tree of PBX locations would be configured, each of them storing its own endpoints and/or sub-locations. Each single location can support up to 1000 endpoints.

Since there is no single instance that knows all of the endpoints, the locations must be defined using the “prefix” mode so that calls can be routed according to the called number by traversing the location tree towards the leafs.

All endpoints must register with the correct PBX as defined by the extension prefix-chain down from the root/master location PBX.

Replication

As noted above, when replicating a larger location PBX from a master PBX, the location PBX’s replication settings must restrict the data to be replicated to their own location.

Especially when a location I using a smaller platform such as an IP202 or IP400, this configuration is critical in order not to overflow the locations LDAP capacity.

The same applies to the DECT gateways IP600 and IP1500. They will normally replicate all users, not just the DECT users. However, usually not all the location users are DECT users too and the number of location users may not fit into the number LDAP database of the DECT gateway. You can restrict the objects replicated to the real DECT users using a special configuration of the DECT gateways LDAP replication client. Assumed the DECT users are configured such that the “DECT-GW” is set to “dect” in the PBXs user entries. You would then configure the “Location” in the “LDAP Replication” settings of the DECT gateway to

(pbx=<gw *name=”dect”*>)

This configuration is taking advantage of the fact that a location search attribute starting with a ‘(‘ is considered a complete LDAP search expression. The “DECT Gateway” section of a PBXs user configuration is stored in an LDAP attribute such as

pbx=<gw name="dect" dsp="user" ipei="12345"/>

Known Problems

PBX Groups are PBX-local objects. Users registered on different PBXs/locations belonging to groups with the same name are this effectively members of different groups. The other way round, in a single group there cannot be members registered with different PBXs/locations. If you need to add a PBX user to a remote PBX, you will need to create a co-registration on that PBX and add this registration user to the group.

The TAPI service provider currently maintains a single connection to a single PBX. It will thus not be able to monitor users on more than one PBX/location. (ckl 12:22, 11 January 2011 (CET) this changed with TAPI V7)

FIXME: "This article urgently needs a V6,7,8,9 rewrite"

Relevance for V6 Firmware and later

Most of the conclusions presented in this article still hold true for V6 and V7 firmware. You should just calculate slightly less available memory for the devices as the firmware grows and thus leaves less memory for operation. Also, when looking at the IP2000/6000 you need to know that it has 4 times more memory than the IP800 and is 4 times faster than the IP800.

Relevance for V7 and later

A IP2000/IP6000 can handle up to 10.000 Objects. Group limitation is up to 2048 Objects in a group

Relevance for V9 and later

To do a rough estimate for the IP6010, you should assume that an IP6010 has 16 times RAM, 6 times CPU cycles and 4 times flash memory, compared to the IP800.