Reference7:Configure Active Directory Replication

From innovaphone-wiki

Jump to: navigation, search
There are other versions of this article: Howto | Reference7 (this version)

Contents

Applies To

This information applies to

  • innovaphone PBX V7 and later
  • Microsoft Windows Server Platforms

Tested with MS Windows 2000 Server and MS Windows 2003 Server.

More Information

Active Directory (AD) replication allows to import Windows Users into an innovaphone PBX as user objects. It is a read-only replication. Nothing is going to be written into the AD.

An attribute mapping mechanism allows to map arbitrary AD-attributes into arbitrary innovaphone-attributes.

Configuration

The configuration can be divided into the two tasks AD Settings and Attribute Mapping. This article focusses on Attribute Mapping.

  • Maps for incoming attributes must be configured. An In-Map controls which content of which incoming attribute goes into a runtime symbol table. You can define up to 40 in-maps.
  • Maps for outgoing or local attributes must be configured. An out-map controls which runtime symbol table entry fills a local attribute.

AD Settings

Please see Reference7:Configuration/LDAP/Replicator for common server settings as IP address, DN, User, Password, LDAP Filter.

Replicating in a Domain Forest

Domain Forest, built up by 3x DCs
Domain Forest, built up by 3x DCs

A company foo.bar maintains three domain controllers (DC).

  • A root DC for the naming context dc=foo, dc=bar
  • A sub DC for the naming context dc=sales, dc=foo, dc=bar
  • A sub DC for the naming context dc=support, dc=foo, dc=bar


The task may now be to replicate all users within dc=foo, dc=bar recursing down into all sub-domains. By default this isn't possible, because MS designed a DC to only answer LDAP search requests with search results it is authoritative for.

The role of a global catalog (GC) is supposed to overcome this limitation. A DC acts as a GC if contacted on TCP port 3268 or 3269 for LDAPS/TLS.

Therefore..if it's desired to replicate from a forest of DCs

  • Enter as LDAP/Replicator/Server the destination as [IP Address]:[port]
    • IP Address shall be the IP address of the root DC
    • port must be 3268 or 3269 for LDAPS/TLS


Note: Be sure to have read below Replicated Objects Lost After erroneous Changes in Active Directory

Attribute Mapping

There are a list of In-Maps and a list of Out-Maps. That's it. In-Maps allow for regular expressions[1]. Out-Maps merely consist of symbol names or literals (constants).

  1. On reception of of a source-ldap object, all existing attributes will be investigated and a corresponding in-map is being looked-up for each attribute
  2. If there are many in-maps per attribute, all maps will be executed sequentially and the value assignment will be placed (if the pattern matches)
  3. In case of a matching map, then values will be maintained within a symbol table (see grammar: 'identifier' production)
  4. Values within the symbol table will be overwritten (if already existing)
  5. After completion, the target-ldap-object will be generated by means of the Out-Maps. If a value application without a value in the symbol table will be found (i.e. no In-Map matched and wrote something into the symbol), then the source-ldap-object is to be discarded

Because of the possible discard of a source object in step #5 above, you must provide a default value for all objects which should be replicated although there is no value in it for an attribute used in an Out-Map! (see in the examples below how to do that)

Grammar For In-Maps

An in-map is a pair of <source-attr-name> (An AD-attribute name) and <assignment_pattern>. Approximate Grammar:

    assignment_pattern  ::= <symboldefinitions> ':' <regexp>
                        |   <symboldefinitions>

    symboldefinitions   ::= <symboldefinitions> <symboldefinition>
                        |   <symboldefinition>

    symboldefinition    ::= <identifier> '=' <value_expression>
                        |   <identifier>

    value_expression    ::= '/' <VALUES> '/'

    VALUES              ::= <VALUES> <VALUE>
                        |   <VALUE>

    VALUE               ::= '\'  <NUM>              # Back Reference
                        |   '\'' <ALLCHARS> '\''    # Literal
                        |   <ALLCHARS>              # Const, synonymous to Literal

    identifer           ::= '%' <ALNUM>

    regexp              ::= <ALLCHARS>

    ALNUMS              ::= ALNUMS ALNUM
                        |   ALNUM 

    ALNUM               ::= ['a'-'z'|'A'-'Z'|'0-'9']

    ALLCHARS            ::= [.*]

In-Map Examples, Maps for telephoneNumber

  • %dw=/\1/:07031 12345-(.*)

that assigns the extension to the symbol %dw

  • %dw=/\2/%root=/\1/:07031(.*) -(.*)

this assigns the extension to the symbol %dw and the root-/subscriber number to %root.

  • If the <value_expression> was skipped, it defaults to \n, wher n is the running index of the symbol_definition (starting with 1). The second example from above can therefore be written as: %root%dw:07031(.*) -(.*)
  • A default value for a symbol may be defined by simply applying an an always-match-constant-value. That is, for instance for telephoneNumber

%dw=/0/:.*

  • Because of the rule, that a missing regexp defaults to :(.*) , this can be written as

%dw=/0/

  • If an attribute value is to copied straight, one simply writes

%e164

  • which is identical with

%e164=/\1/:(.*)


If you are not familiar with regex or will test something, this external tool may help.

Grammar For Out-Maps

An out-map is a pair of <destination-attr-name> (name of an innovaphone attribute) and <destination_values>. Approximate Grammar:

    destination_values  ::= <destination_values> <destination_value>
                        |   <destination_value>

    destination_value   ::= <identifier>
                        |   '\'' <ALLCHARS> '\''    # Literal
                        |   <ALLCHARS>              # Const, synonymous to Literal

This grammar allows to fill the e.g. local cn-attribute not only with a single identifier, but with an intermixed concatenation of several identifiers and literals alike e.g.: "%sn', '%givenName" - yielding for instance: "Doe, Jon".

Example

The following example focusses on the generation of the e164-, node- and loc-attribute.

  • Only Sindelfingen-Numbers (+49(7031)...) will match
  • The numbering node (a.k.a. node-attribute) will then be set to root.
  • The hosting PBX (a.k.a. loc-attribute) will be set to sifi.

Within the AD exists..: Btw, the Filter was configured to: (&(objectclass=User)(telephoneNumber=*))

            Peter's telephoneNumber:   +49(7031)12345-75
            John's telephoneNumber:    +49(7031)12345-74
            Mary's telephoneNumber:    +49(7031)12345-43

Map configuration underneath Configuration/LDAP/Replicator:

In Maps
Source Attribute    Assignment Pattern                                    Description
--------------      ------------------                                    -----------
cn                  %cn
telephoneNumber     %tel%loc=/sifi/%node=/root/:\+49.*7031.*12345-(.*)    Sindelfingen numbers with leading '+' at begin: then backref=1 into %tel. Constant=sifi into %loc. Constant=root into %node.
displayName         %dn

Out Maps
Dest.-Attribute     Destination Value
--------------      ----------------
cn                  %cn
e164                %tel
loc                 %loc
node                %node
dn                  %dn

This is how the output is written into the flash:

mod cmd FLASHDIR0 add-item 102 (cn=Peter Schmidt)(repsrc=ad)(guid;bin=2B1DAA4655AE244D845734951F5F2F1B)(node=root)(loc=sifi)(dn=Peter Schmidt)(e164=74)(usn=3675) 
mod cmd FLASHDIR0 add-item 102 (cn=John Doe)(repsrc=ad)(guid;bin=904AF5506116354E9E86BE9A6C5D67FF)(node=root)(loc=sifi)(dn=John Doe)(e164=75)(usn=3676) 
mod cmd FLASHDIR0 add-item 102 (cn=Mary Fernandez)(repsrc=ad)(guid;bin=575743792731EE478EFB40754885BAAB)(node=root)(loc=sifi)(dn=Mary Fernandez)(e164=43)(usn=3678)

Note: According to the Out-Maps Grammar a local attribute can be synthesized from multiple symbols and literals. Here comes an example for the aggregated synthesis of the local CN attribute.

In Map
Source Attribute    Assignment Pattern                                    Description
--------------      ------------------                                    -----------
...
sn                  %sn                                                   Surname (e.g. Fernandez) into %sn
givenName           %gn                                                   Name (e.g. Mary) into %gn
...

Out Map
Dest.-Attribute     Destination Value
--------------      ----------------
...
cn                  %sn', '%gn                                            
...

yielding:

mod cmd FLASHDIR0 add-item 102 (cn=Fernandez, Mary)(repsrc=ad)(guid;bin=575743792731EE478EFB40754885BAAB)(node=root)(loc=sifi)(dn=Mary Fernandez)(e164=43)(usn=3678)

AD Attributes Of Interest

The following list names the attributes to be configured as Source Attribute within an In-Map or to be referenced within the LDAP filter:

  • cn: The Common Name
    • single-valued
    • possible usage: for cn attribute
  • sn: The Surname
    • single-valued
    • possible usage: for DECT display name dn attribute
  • givenName: The Given Name.
    • single-valued
    • possible usage: for DECT display name dn attribute
  • telephoneNumber: The primary business telephone number. To be found on a user's General tab.
    • single-valued
    • mostly in international form with leading plus sign (e.g. +49(7031)12345-44)
    • possible usage: for e164 attribute
  • memberOf: Group memberships. Groups are named here by their Distinguished Name (DN).
    • multi-valued
    • possible usage: for root attribute
    • Note:Though there might be desires - PBX groups must still be configured manually. PBX groups cannot be configured through AD Replication.
  • samAccountName: The logon name
    • single-valued
    • possible usage: for h323 attribute
  • mail: Email Address
    • single-valued
    • possible usage: for email[2] attribute
    • Example
      In Maps
      Source Attribute Assignment Pattern Description
      -------------- ------------------ -----------
      mail %mail AD's attribue "mail" will be read
       
      Out Maps
      Dest.-Attribute Destination Value
      -------------- ----------------
      email %mail


Notes on replicating the new-in-V10 email User Attribute

  • proxyAddresses: Email Address
    • multi-valued
    • possible usage: for email[2] attribute

From V10 upwards, a new user attribute email is present in a PBX user entry (see E-Mail in PBX/Objects). This is stored as a distinct attribute on the LDAP level, so it can be replicated from the active directory. Please note that since V9, it is recommended to use the users primary email addresses' user part as h323 attribute (e.g. if the users primary email address is user@innovaphone.com, then the h323 attribute should be set to user). If you do so, there is no need to set the email attribute. However, if you set the h323 to something else, you should set the email attribute in addition.

To do so, you first need to determine how the user's primary email address is stored in your AD. We recommend to use AD's proxyAddresses attribute. This is a multi-valued attribute. Each value contains an address prefixed with a protocol specifier (e.g. smtp: or sip:). If there are multiple addresses for a single protocol, the primary address for this protocol has the protocol specifier in capital letters (e.g. SMTP:). SMTP addresses in the proxyAddresses attribute always include the domain part (e.g. @innovaphone.com). As from V9 up it is recommended to use the customers domain as PBX System Name property and have the Use as Domain check-mark ticked, the value in the email attribute shall consist of the user-part only (e.g. user instead of user@innovaphone.com).

Having said this, appropriate replication patterns are

Source Attribute          Assignment Pattern                          Description
---------------- ------------------ -----------
proxyAddresses %mail=/\1/:SMTP:(.*) full email address if domain does not match
proxyAddresses %mail=/\1/:SMTP:(.*)@sample.com simplified email address if domain matches
 
Dest. Attribute Destination Value
---------------- ------------------
email %mail

In many companies, multiple email addresses are used per user. For example, a user Fred Firestone with windows account name (samAccountName) ffi may have the proxy addresses smtp:ffi@sample.com and SMTP:ffirestone@sample.com. If the windows account name shall be used as value for the h323 attribute, you can note the fact that the h323 attribute contains another valid email address by prefixing the value of the email attribute with .;. Appropriate mappings could thus look like:

Source Attribute          Assignment Pattern                          Description
---------------- ------------------ -----------
sAMAccountName %sAMAccountName use windows account as Name
proxyAddresses %mail=/\1/:SMTP:(.*) full email address if domain does not match
proxyAddresses %mail=/\1/:SMTP:(.*)@innovaphone.com simplified email address if domain matches
proxyAddresses %othermail=/\1/:SMTP:(.*)@otherdomain.com additional email for domain (optional)
 
Dest. Attribute Destination Value
---------------- ------------------
email .';'%mail';'%othermail apply h323 (Name) as valid email
h323 %sAMAccountName set h323 to windows account

It is unclear (to us :-)) if AD's email attribute always holds the primary SMTP address.

Tips&Tricks

Investigating/Dumping An AD Object

If you want to learn or study about how the AD stores user objects and which attributes are available per user, the tool ldifde.exe[3] will be of help. Ldifde.exe is a Microsoft command line program which is part of a Windows Server installation. It allows to dump the complete AD content or parts of it into LDIF[4] files.

To Dump the AD object for a Windows user John Doe into a file out.txt

  • On a Windows Server open up a command line box.
  • Enter:
ldifde -s 127.0.0.1 -r "(sn=doe)" -f out.txt

Btw, from the file you can also learn which Distinguished Names (DNs) are available. A DN may be required as user name within the common server settings on the LDAP/Replicator page.

To Dump all AD user objects into a file out.txt

  • On a Windows Server open up a command line box.
  • Enter:
ldifde -s 127.0.0.1 -r "(objectclass=User)" -f out.txt

Deleting AD-Replicated Objects

You may delete up to 100 objects in one instance.

  • Proceed to Reference7:Configuration/LDAP/Expert
  • Enter the LDAP filter (repsrc=*) into the search edit field an click on Show
  • Check the column selector to select all displayed objects
  • Click Delete within the toolbar. A confirmation dialog will be shown.
  • Confirm

Display Non-AD-Replicated Objects

Out-Filtering Disabled Windows Accounts

Windows maintains the numerical attribute userAccountControl[5] for each user object within the AD. If Bit 2 is set, the user account was disabled.

A NOT-Filter term, featuring a matching-rule, ensures that this bit isn't set, i.e. that it is not a disabled account:

  • (!(userAccountControl:1.2.840.113556.1.4.803:=2))

A complete AND-filter may therefore look like this one:

  • (&(objectclass=User)(telephoneNumber=*)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))

Filtering For Group Memberships

The multi-value AD attribute memberOf carries the information which groups a user is belonging to. Groups are named within the AD by their Distinguished Name (DN), instead of by their friendly name. This is how the AD could be storing a group membership Berlin:

  • memberOf: CN=Berlin,CN=Users,DC=innovaphone,DC=sifi

If you wanted to filter for members of the group Berlin, the following AND-filter would do the job:

  • (&(objectclass=User)(memberOf=CN=Berlin,CN=Users,DC=innovaphone,DC=sifi))

Note: The filter must be crafted for an equality match. According to MSDN memberOf does not allow for substring/wildcard matches.

Merging Old PBX Users Into An AD-Replicated Database

If AD-replication is about to be administrated for a PBX with an existing portfolio of user objects, the question may arise for how to retain those existing objects. This section explains the background and the required administrative steps.

When looking into a PBX's downloadable configuration file an existing object may grossly look as the following example:

mod cmd FLASHDIR0 add-item 102 (cn=Martin Streller)(dn=Martin S.)(h323=mst)(e164=173)(loc=va-sifi)(pbx=etc,..)(pbx=and so on,..)(node=root)(guid;bin=8D7A0AB9E909D311B539009033080010)(usn=106540)

Various informations are stored within individual attributes cn(common name), dn(display name), h323(diallable name), e164(number), loc(pbx name), pbx(groups, diversions,...), node (numbering node). One attribute - the guid - is the actual identifier for an entire pbx object. It is the actual hook or grip for an object. Once assigned a guid will remain static for the whole object's lifetime.

With the AD-replication the guid is also going to play the same role as an identifier. However, the guid will be taken from the AD. In detail from the AD attribute objectGUID, which is semantically the same as the guid of a pbx object. Therefore the necessity arises to manually tweak the guid as if set by an AD-replication run. After applying this required step the resulting and modified object example from above may look alike:

mod cmd FLASHDIR0 add-item 102 (cn=Martin Streller)(dn=Martin S.)(h323=mst)(e164=173)(loc=va-sifi)(pbx=etc,..)(pbx=and so on,..)(node=root)(repsrc=ad)(guid;bin=4C7BA6C8568CAF4D85BA37486EF39A26)(usn=106540)

The alert reader will notice that not only the guid was changed. An additional attibute repsrc with the value ad joined the set of attributes. This repsrc-attribute is required internally for all AD-replicated objects. It helps to separate AD-replicated objects from all of the other objects.

With that background in mind, the following recipe summarizes the steps to accomplish.

  • Download and save the entire configuration of the box.
  • Configure the AD-replication.
  • Download the resulting new configuration.
  • Open a copy of the old configuration within a suitable editor.
    • For all affected objects, replace the guid attribute with the respective guid from the new configuration.
    • For all affected objects, add the attribute repsrc with the value ad.
    • Copy the AD-replication settings: from the new configuration, copy the line beginning with
      config change LDAPREP0 /server...
    • Copy the AD-replication password: from the new configuration, copy the line beginning with
      vars create LDAPREP0/AUTH ...
  • Upload the edited old configuration into the box and reboot.

Assigning Default/Dummy Content Into a Variable

Sometimes not all AD objects to be replicated, contain all the required attributes to fill values into all of the "Out Maps"-attributes. In order to avoid empty attributes, a dummy content may be provided. The following example outlines how to handle e.g. objects without an email address.

In Maps
Source Attribute    Assignment Pattern                                    Description
--------------      ------------------                                    -----------
cn                  %cn%mail                                              cn is used as content for cn and also as default content for mail attribute. *
mail                %mail                                                 A real email will override the default content. Must follow in-map with default assignment.
..

Out Maps
Dest.-Attribute     Destination Value
--------------      ----------------
cn                  %cn
email               %mail
..

* Since mail is a unique attribute in the PBX, you cannot assign it a static Assignment Pattern like %cn%mail=/default@example.com/ , however you could use it for other non-unique PBX-attributes 

LDAP Decoding Buffer is limited to 24kB

The LDAP Decoding Buffer is set to 24kB. As a result, all the attributes fetched for an AD user is not allowed to exceed this value. This could be an issue if an AD user is in a lot of groups because the memberOf Attribute is fetched from the AD as well. If this is the case you need to reduce the number of groups or the length of their names.

Limitation of the AD Replication config line

Each config line has a limit. Consequently it’s not possible to add an unlimited amount of in/out-maps. But don’t be too worried because in order to exceed this limit you will have to add an enormous number of maps. For further information and an online calculator please follow this link

Troubleshooting

Watch out for the LDAP/Replicator-Status page. This page is going to display errors alike

  • Invalid Credentials, implying wrong credentials for the LDAP authentication.
  • Cannot Connect, meaning the configured IP address may be wrong.
  • Connect Timeout, indicating the AD may be down.
  • Map Config Not Ok!, invalid "In Maps"/"Out Maps" configuration

If there are non-specific errors listed, a further click on Flash Directory Status may reveal error messages from the flash directory module acting below the replication module.

Replicator-Status
Replicator-Status

Suddenly, New Objects Aren't Replicated. What Can I Do?

Few support cases dealt with customers missing the replication of newly created user objects. This section outlines a strategy to identify a root cause. It is based on the MS Ldap client ldifde[3].

Create a batch file and prepare and adapt to your needs, a command line for ldifde as follows:

ldifde -a "<me>\<domain>" "<pw>" -s 192.168.0.5 -g -d "DC=<my-company>,DC=com" -r "(&(objectclass=User)(telephoneNumber=*))" -v  -f result.txt -j c:\tmp
  • -a user, password credentials for the LDAP bind authentication
  • -s Server IP address
  • -t optional port
  • -g disable some MS specifics
  • -d The root DN underneath which to search
  • -r The LDAP filter
  • -v Print verbose informations to the console
  • -f The file, results will be written to
  • -j The path, where the log should be stored, it's mendatory

The paramaters should be aligned to the configuration underneath Services/LDAP/Replicator. In addition it might be a good idea to reduce the size of the file result.txt by providing the command line option

  • -l "cn" This option asks the AD only for the attribute cn.

Then watch-out for the missing objects. If they're still not present in the result set, then modifiy the LDAP filter -r. Simplified filters may be:

  • "(objectclass=User)" Ask for all user objects
  • "(sn=Schmidt)" Ask for the surname/family name Schmidt
  • "(sn=Sch*)" Ask for all surnames/family names starting with Sch

If the missing objects eventually appear in the result set, then remove the -l option, in order to retrieve the objects in their entirety with all attributes. Look at the object's attribute and try to draw a conclusion.

Possible causes reported from the field were:

  • Group membership didn't fit
  • Telephone number wasn't administrated correctly

Known Problems

Freshly Deleted AD-Objects Aren't Deleted Locally

Although the replicator receives change notify results from the AD, the AD might not send notifications about the deletion of objects. The replicator will be deleting affected objects only after a stop+start of the replication session.

Lab experiments showed that the AD sent such deletion notifications only, if the replicator had authenticated with administrator credentials - which practically isn't recommendable.

To recover from this peculiarity the following procedure should be implemented:

  • Please obey paragraph Out-Filtering Disabled Windows Accounts under the Tips&Tricks section.
  • Don't delete AD objects as the first step of the administration procedure for user objects scheduled for deletion.
  • Instead: Disable such objects as the first step.

Increasing CPU Load if poll Timer is set

If you have an AD Forrest you need to configure a poll timer, because the Notify mechanism isn’t working. If you configure a one minute poll timer, a full replication is done every minute. This causes a high CPU load with low priority. Furthermore the AD has a mechanism to reestablish a connection if the MaxConnIdleTime is running out. If the connection is reestablished a full replication is done. The default value is 900 seconds (15 minutes). You can change this value with Ntdsutil.exe.

Replicated Objects Lost After erroneous Changes in Active Directory

When the LDAP replication succeeds and a PBX object is found not to be present in the replication result any more, it will be removed from the PBX. This ensures that objects removed from the AD are removed in the PBX also.

However, if an object is missing from the replication result by accident (e.g. because the replication is based on AD groups and the user is mistakenly removed from the group), the corresponding PBX object will be removed too. When the problem is fixed, the object will be re-created again. However, all object properties which are not derived from the AD will be lost (in other words, you will end up with a naked object with all manually set properties lost). To recover from such a situation, the AD problem needs to be fixed and the most recent PBX backup must be applied then.

Note that erroneously changing the LDAP access from Global Catalog(GC) Mode to Non-GC Mode will create such a situation too. A GC on port 3268 usually covers a broader database than an AD working in non-GC mode. Therefore it is highly likely to lose just some or even all replicated objects after the GC-mode has been turned off. Re-enabling the GC-mode alone does not recover from this situation for the reasons explained above. In this case, restoring the most recent backup will fix the issue, as this backup will also restore the correct LDAP replicator configuration.

Replication fails due to 1,Common Operations Error

The most likely reason for this issue is a conflict during replication. An object with the same cn but a different GUID already exists at the PBX. This is usually the case if an object is created manually and afterwards an object with the same cn should be replicated from the AD. To find out which object is the source of the conflict, please create an All Ipv4 TCP/UDP traffic trace and filter the trace for: innovaphone contains "add_item error"

Image: ActiveDirectory_add_item_error.png

The debug output directs you to the conflicting cn. Keep in mind that you’ll need the Innovaphone Wireshark dll to see the innovaphone debug messages.

Windows Server Updates in March 2020 - LDAP Server

Microsoft will publish in March 2020 changes via Windows update for all Server Versions which will change the default domain policies in the domain controller. After such an update, the LDAP connection to the Active Directory Server will be possible only through TLS - you can enable TLS in Services/LDAP/Replicator. For more information regarding this issue, please read this article: Support:Windows_Server_Updates_in_March_2020_-_LDAP

Notes

  1. Regular Expression, http://en.wikipedia.org/wiki/Regular_expression. We use POSIX 1003.2 regular expressions syntax, see e.g. https://www.freebsd.org/cgi/man.cgi?query=re_format&sektion=7&apropos=0&manpath=FreeBSD+10.3-RELEASE+and+Ports for a detailed description
  2. 2.0 2.1 From on V10: email attribute contains a user's email address
  3. 3.0 3.1 Using LDIFDE, http://support.microsoft.com/kb/237677 or LDIFDE, http://technet2.microsoft.com/WindowsServer/en/Library/32872283-3722-4d9b-925a-82c516a1ca141033.mspx?mfr=true
  4. RFC2849, LDAP Data Interchange Format
  5. userAccountControl, http://support.microsoft.com/?scid=kb%3Ben-us%3B305144
Personal tools