Howto:PHP based Update Server V2

From innovaphone wiki
Revision as of 20:55, 21 November 2016 by Ckl (talk | contribs) (→‎classes/class)
Jump to navigation Jump to search

Applies To

Tools clipart.png FIXME: "work in progress"

This information applies to

  • all innovaphone devices
  • web server running PHP 5 (e.g. Linux Application Platform)

All Versions.

By default, the Update Manager mechanism reads a file that corresponds to the device type (e.g. update-ip222.htm). While this makes sense (update scripts may vary by device type), it is sometimes tedious, as you have to edit a huge amount of files which typically are (at least partly) identical.

Here is a PHP script that can be used as an Update Server that allows you to simplify the handling of update scripts. It also includes a mechanism that makes sure that all devices always have the same firmware installed as a given master device has. Finally, it implements a straight forward configuration backup scheme.

This article describes version 2 of this update server. The previous version is described in Howto:PHP_based_Update_Server. The enhancements are

  • Status user interface showing all known devices
  • Ability to roll out custom device certificates
  • Ability to provide configuration files to MTLS-authenticated devices only (e.g. in order to keep certain configuration settings secure)
  • Hide configuration files from public read access

More Information

Requirements

The update server script requires a web server with working PHP 5.3 or higher platform. It has been tested with Apache and ligHTTPD (on a Linux Application Platform). On IIS, neither the certificate roll-out nor the MTLS authentication or configuration backup works with IIS.

The platform running the PHP scripts must have a valid time setting!

Features

The update server can

  • update all your devices with boot code and firmware that corresponds to the versions running on a reference device of your choice
  • save backups of your devices configuration. Backups are saved only if the configuration changed since the last backup
  • invoke your own update scripts depending on
    • various phases (e.g. you can have staging scripts which are only executed once and normal scripts which are executed in normal operation later on)
    • devices classes (e.g. you can have different scripts for phones and gateways)
    • environments (e.g. you can have scripts for your internal devices and others for devices in home offices)
    • the device serial number
  • roll out customer specific device certificates
  • roll out update scripts to devices only that have identified themselves with a valid device certifictae (MTLS)

Installation

On the Linux Application Platform

Here is how you would install it on the LAP. Some of the steps may not be necessary if you don't want to use all features.

On the Linux Application Platform, you would

  • open the LAP's file system using a SFTP client such as e.g. WinSCP (if you have not yet changed it, the default credentials will be root/iplinux, see Concept Linux Application Platform). Note that you must use SFTP rather than WebDAV, as WebDAV will not give access to the executable web server files
  • create a new directory underneath /var/www/innovaphone/mtls, e.g. /var/www/innovaphone/mtls/update
  • copy the following files in to this directory
    • update.php
    • config.xmlconfig.dtd
    • user-config.xml
  • create s sub-directory called web, copy the following files in to this directory
    • cer.png
    • hide.jpg
    • touch.gif
    • trash.gif
    • update.png
    • style.css
  • create a sub-directory called admin, copy the following files in to this directory
    • admin.php
  • change owner and group of all files and directories to www-data
  • log in to the LAP's admin UI (if you have not yet changed it, the default credentials will be admin/linux, see Concept Linux Application Platform) and open the its Administration/Web Server/Change web server properties and public access to the web/webdav configuration UI. Add the following paths to Public Web Paths (assuming your base directory is called update):
    • /mtls/nupdate
    • /mtls/web
make sure that no other sub-directories of update are listed
make sure you have no trailing '/' in any of these paths
this will make sure the update server's admin user interface is not accessible to the public

On an Apache Server running on the Windows Operating System

Tools clipart.png FIXME: "to be done"

Configuration

Preparation

If you intend to deliver your update scripts with MTLS (that is, with HTPS und mutual certificate check), you will need to have the full name of your certificate Authority (CA) as it is noted as Common Name (CN) in your CA's certificate. Also, you will need a PEM version of your CA's public key (a PEM version of a certificate is a text file that begins with a line like -----BEGIN CERTIFICATE-----).

Basic Parameters

All tweakable parameters are set in user-config.xml. You should start with just setting the info attribute of the master tag and leave everything else as is.


Testing

To test your installation, you can simply use your browser simulating your device (it is convenient to use your browsers view source function to inspect the result).

Cache

Open the update URL with the #t argument replaced by the simulated device type (e.g. http://172.16.10.65/update/update.php?type=ip222). You will see an update script. The first line will look like 

# failed to use cached data (cache not current), retrieving info online
# phase: ...

This is normal, as there is no cache yet. Now simply open the URL again (by hitting F5 e.g.). You will now see 

# firmware build info cache is current

If so, your firmware info retrieval and your cache works.

if not, there is something wrong with your master/info URL or your web server has no write access to the cache sub-directory.

Update Scripts

You should now see a line like 

# phase: staging, nextphase: update, environment: intern, type: ip222, classes: phone

in the update script. This shows that the scripts for the first phase staging, the default (i.e. first) environment intern and for devices of class phone will be delivered.

The next lines will list the potential update scripts for this situation: 

# possible files (from C:\sources\wiki-src\sample\php-update-server\sample\sources):
# staging-phone-intern.txt (does not exist)
# staging-phone-all.txt (does not exist)
# staging-all-intern.txt (does not exist)
# staging-all-all.txt (does not exist)
# all-phone-intern.txt (does not exist)
# all-phone-all.txt (does not exist)
# all-all-intern.txt (does not exist)
# all-all-all.txt (does not exist)

Of course, as you have no scripts yet, all options are tagged as (does not exist). If one or more of such scripts exist, they will be executed in the order listed.

Backup

The next lines show you where the device backup will be sent to: 

# backup
mod cmd UP0 scfg http://172.16.10.65/update.php?mode=backup&hwid=#h ser hourly /force 1

You can see that the backups are sent every hour (/force 1) if this has been configured in the backup tag (see above).

Firmware

The next block shows the required update of the firmware or boot code: 

# current firmware (unknown) does not match required firmware (120663)
mod cmd UP0 prot http://172.16.10.65/fw/120663/ ser 120663
# current boot code (unknown) does not match required boot code (120502)
mod cmd UP0 boot http://172.16.10.65/fw/120502/ ser 120502
# make sure remaining update scripts are executed with up-to-date firmware
iresetn

In this example, the reference device is running firmware build 120663 and boot code build 120502. The update server does not know the devices current firmware, so it creates an update command for both. You can now add a &PROT=120663 query arg to the URL. This tells the update server the firmware your device is running (a real life device will include this argument automatically, so there is no need to configure it anywhere). When you retrieve the script again, no update command for the firmware will be included: 

# current boot code (unknown) does not match required boot code (120502)
mod cmd UP0 boot http://172.16.10.65/fw/120502/ ser 120502
# make sure remaining update scripts are executed with up-to-date firmware
iresetn

Next Phase Switching

As you have 2 phases defined in your config file, the script will instruct the device to switch to the next phase (update): 

# warping from 'staging' to 'update' (url=http://172.16.10.65/update.php?type=#t&phase=update)
config add UP1 /url http%3A%2F%2F172.16.10.65%2Fupdate.php%3Ftype%3D%2523t%26phase%3Dupdate
config add UP1 /no-dhcp
config write
config activate

# turn on fast polling
mod cmd UP1 provision 5

This is done by re-writing the update URL in the client. You can simulate this by adding the &phase=update query argument to your browser URL. You will see then: 

# 'update' is last phase

Config File Re-Execution

In the last (that is, standard) phase, the config files will be executed only when changed. This is done using a check command : 

# skip remainder if there were no changes in config files
mod cmd UP1 check ser d41d8cd98f00b204e9800998ecf8427e

The serial d41d8cd98f00b204e9800998ecf8427e is a hash code computed from the content of all script files that are valid for the calling device. We can see how this works when we create an update script that is valid for the device. You create a file called all-all-all.txt in the update server directory and place something (e.g. a comment in it):

You will now see that the script retrieved changes: 

# skip remainder if there were no changes in config files
mod cmd UP1 check ser 1e906fd3737281ebd4edbdf60a196fb3

# 1 config files follow:
# { begin script 'all-all-all.txt' 
# my first update script

# end script 'all-all-all.txt' }

The new serial 1e906fd3737281ebd4edbdf60a196fb3 differs from the previous one. As a result, the devices will execute their update scripts whenever they change.

Debugging

To debug update script execution in your real devices, you can

  • open the devices debug.xml page (e.g. http://x.x.x.x/debug.xml)
  • check the Update/Polling, Update/Execution, HTTP/Client and HTTP/verbose check marks
  • force the update client in the device to run using the mod cmd UP1 poll command (e.g. http://x.x.x.x/!mod cmd UP1 poll)
  • look at the traces (best with wireshark)

You can easily see the current status of your update server and scripts by calling the URL http://your-update-server/update/update.php?show

Your own Update Scripts

As you have seen, the update server will do some meaningful things (updating the device firmware and creating backups) even with no user supplied update scripts at all.

Feel free now to add your own update scripts. All update script names must have the form phase-class-environment.txt (e.g. update-phone-intern.txt). If the word all is used for any of the components, the file is executed for all phases, classes or environments respectively.

When you retrieve the update script URL manually (e.g. using your browser), it will list all the potential update script files: 

# possible files (from C:\sources\wiki-src\sample\php-update-server\sample\sources):
# staging-phone-intern.txt (does not exist)
# staging-phone-all.txt (does not exist)
# staging-all-intern.txt (does not exist)
# staging-all-all.txt (does not exist)
# all-phone-intern.txt (does not exist)
# all-phone-all.txt (does not exist)
# all-all-intern.txt (does not exist)
# all-all-all.txt (does not exist)

Note that this list depends on

  • the device's class (inferred from the ?type=#t query arg in the update URL)
  • the device's environment (as specified by the optional &env=environment-name query arg in the update URL)

Complete Parameter Reference

Note: attributes are marked with an at (@) prefix. So master/@info refers to the info attribute in the master tag.

The following tags and attributes are available:

master

Defines the reference device where the firmware and boot code information is taken from.

/master/@info
the URL to the device info data. Set it to an URL like http://your-reference-device/CMD0/box_info.xml. Please note that this device must not have the Password protect all HTTP pages set in Services/HTTP/Server.
/master/@expire
the firmware info cache lifetime. The firmware information is cached in a file (to make sure this information is present even if the reference device is off-line). The cache will not be refreshed before the expire time (in seconds) is expired
/master/@cache
the cache file name. The web server must be able to write this file (see Installation above)

   <master 
       info="http://172.16.0.10/CMD0/box_info.xml"
       expire="3600"
       cache="cache/master-info.xml"/>

fwstorage

Defines from where the device will retrieve the firmware files for updates.

fwstorage/@url
defines the URL to retrieve the files from. In this URL, the following meta words will be replaced as follows:
  • {build} - the requested firmware or boot-code
  • {model} - the devices type (e.g. IP232). This is derived from the type query argument used in the update URL which is set to the value #t which in turn is expanded to the device type of the device requesting the update script
  • {filetype} - the type of the required file, either boot or prot

Note that if the url ends with a slash (/), the calling device will automatically append the name of the requested file. In this case, the file system the URL points to must therefore contain sub-directories for each firmware build which contain the corresponding firmware builds (e.g, fw/12345/ip232.bin). If url is not an URL (as is the case for the default value fw/{build}/), it is treated as a sub-directory underneath the script directory 

   <fwstorage url="fw/{build}/"/>

backup

Defines where backup files are kept.

backup/@dir
the directory underneath the script directory where backups are stored
backup/@nbackups
the number of different backup files kept
backup/@scfg
the target URL for the scfg command. If this is not an url, it is interpreted relative to the script directory URL. You can add more options for the scfg command, like in scfg="update.php?mode=backup&hwid=#h&sn=#m ser hourly /force 1" (this example will make sure backups are attempted only once per hour, so as to reduce load on the server).

   <backup 
       dir="backup"
       nbackups="10"
       scfg="update.php?mode=backup&hwid=#h&sn=#m"
   />

status

Defines details regarding the state kept for device requesting an update script.

status/@dir
the directory the status files are kept in (e.g. status). Used as the name for a sub-directory underneath your install directory on your web server. You must make sure that files in this directory cannot be read by anyone without proper authentication, as such files may contain sensitive information.
status/@expire
if set and not empty, devices which have not requested an update script will be removed from the inventory after n seconds. For example, expire="7776000" will forget about devices after 90 days with no contact
status/@missing
devices are shown in a different colour (indicating that they are missing) after n seconds. For example, missing="604800" will flag devices as missing after 7 days with no contact
status/@refresh
if set and not empty, the admin user interface showing the known devices will refresh the status of all shown devices each n seconds

   <status 
       dir="status" 
       expire="7776000" 
       missing="200" 
       refresh="10"/>

times

Defines details regarding the delivery of update scripts to requesting devices.

times/@dir
the directory the update scripts are stored in. For security reasons, you may want to make sure that files in this directory cannot be read without proper authentication.

These attributes define the arguments of the times command. If neither allow nor initial is set, no times command is used (that is, the update scripts will be processed at all times).

times/@allow
the hours to be used in the times command
times/@initial
the minutes to be used in the times command

Delivered update scripts can include a check command if the check attribute is set to true. . In this case, the devices will executed the scripts again only when you have edited/changed them.

times/@check
if set to true, update scripts will be executed once only (unless their contents change)

When editing a number of update scripts, it is often not desirable if one changed script is already executed before another is changed too. When the grace attribute is set to a positive value, no script at all is delivered to requesting devices unless n seconds have expired since the last edit. For example, setting grace to 90 gives you one and a half minute to finish all your edits.

times/@grace
defines the number of seconds that must expire before update script changes take effect

The update server works in either fast or normal mode. In fast mode, update scripts have to be requested more frequently, for example to step through multiple staging phases faster. This is enforced by modifying the calling device's Interval setting in [Reference11r1:Services/Update | Services/Update ]] and setting it to 1 in fast mode.

times/@interval
polling interval in minutes for normal operation (that is, when all staging s done)
times/@polling
when you are using multiple phases (e.g. staging and update), this defines the number of seconds to wait before the device reads the scripts for the next phase (see provision command) (please do not modify the default value)

In some installations it may be desired to deliver update scripts with HTTPS only or even only to calling devices which have identified themselves with a proper TLS certificate.

times/@forcehttps
if set to true, update scripts will only be delivered if the device call in with HTTPS. If the device is ready to receive an update script and still calls in with HTTP only, its Command File URL will be re-configured to use HTTPS automatically.
times/@httpsport
if you use a non-standard port for HTTPS access to update scripts, it can be defined here
times/@httpsurlmod
if your HTTPS URL differs from the HTTP URL, you can specifiy a modifier. It has the form /pattern/replacement where pattern is a PHP PCRE pattern. For example, /mtls// will remove the string mtls from the URL used by the device to create the HTTPS URL to use
times/@forcetrust
if set to true, update scripts will only be delivered if the device call in with HTTPS and a valid and trusted client certificate that identifies itself as the devices it claims to be

   <times 
       dir="scripts" 
       allow="" 
       initial="" 
       check="true" 
       polling="5" 
       interval="15" 
       grace="90" 
       forcehttps="true" 
       httpsport="444" 
       forcetrust="false" 
       httpsurlmod="/mtls//"/>

phases

Defines the scripting phases. In many installations, there are initializations which should be performed once only. This is known as the staging phase. Once this is done, the devices continue with the execution of the normal update scripts (this phase is known as update by default). In the (unlikely) event that you want more or less phases, you can add/remove phase tags.

phases/phase

phases/phase/@id
the name for the phase
phases/phase/@seq
the sequential number for the phase. Phases are stepped-through in the numeric order defined by their seq atribute.

By virtue if the seq attribute, you can insert phases in to the set of phases defined by default, which is 

   <phases>
       <phase id="staging" seq="100"/>
       <phase id="update" seq="200"/>
   </phases>

For example, 

   <phases>
       <phase id="phase" seq="150"></phase>
   </phases>

will insert a custom phase myphase as second phase.

environments

Defines the distinguished environments. Devices can have zero or more environments associated. The environments a device is considered to be in must be passed as ?env=name1,name2,.. URL query arg in the update url. Update scripts for all environments listed will be applied. The default config file defines the environments intern and homeoffice but you can define your own if you like.

environments/environment =

Each environment may have a number of sub-tags of type implies. It contains the id of another environment. If a device is in an environment with an implies sub-tag, it is assumed that it is in the implied environment also.

environments/environment/@id
the name for the environment

   <environments>
       <environment id='hq'>
           <implies>intranet</implies>
       </environment>
       <environment id='intranet'/>
       <environment id='homeoffice'/>
   </environments>

classes

Defines the available device classes. Devices of different classes (e.g. phones and gateways) can have different update scripts. The device class is derived from the #t (device type) query arg of the update script URL (which is why you must configure it in the update URL). By default, the classes phone, opus_phone, pre_opus_phone, phone_oldui, phone_newui, mobile and gw are recognized. A given device can be in multiple classes and will execute all update scripts available for these classes.

classes/class

Each class has a number of sub-tags of type model. It contains the value replaced for the #t query arg. The models listed belong to the class.

classes/class/@id
the name of the class

   <classes>
       <class id="gw">
           <model>ip0010</model>
           ...
       </class>
       <class id="phone">
           <model>ip110</model>
           <model>ip232</model>
           ...
       </class>
       <class id="phone_oldui">
           <model>ip110</model>
           ...
       </class>
       <class id="phone_newui">
           <model>ip232</model>
           ...
       </class>
   </classes>

stdargs

Better don't touch :-)

customcerts

innovaphone hardware devices come with a device specific, trust-able device certificate. However, in many situations it is desirable to roll-out customer-created custom certificates to all devices. The update server can do this. This will replace the shipped devices certificates both on hardware devices (where all such certificates are derived from innovaphone's certificate authority) as well as on soft devices which run with a self.-signed certificate by default (such as the software-phone or myPBX for Android/iOS).

customcerts/@dir
the name of the directory underneath your installation directory which stores device certificates. you should make sure that this directory cannot be accessed without proper authentication
customcerts/@CAkeys
a file name pattern (e.g. CAkey*). All files in customcerts/@dir which match the pattern mut contain PEM encoded public keys of all the CAs you trust for device uthentication (usually it is only one and the public key of your CA)
customcerts/@CAtype
must be manual currently
customcerts/@CAname
a comma-separated list of CA names. Devices presenting a certificate signed by one of these CAs will be considered as having a valid certificate. Otherwise, they wil be instructed to create a certificate signing request (CSR) for subsequent submission to your CA
customcerts/@CEnamesep
defines the name separator for customcerts/@CAname. Teh default is , and you must chaneg it to a different value if one of your CAs includes a comma in its name
customcerts/@renew
if set and not 0, certificates are replaced by new ones n days before they expire
customcerts/@storeky
must be set to true if the update server shall roll-out custom certificates to devices with pre-v12r2 firmware

When the update server determines that a device calls in with an un-trusted or expired certificate, it will have it create a signing-request for a new certificate. The requested certificate properties can be defined:

customcerts/@CSRkey
the key size, e.g. 2048-bi
customcerts/@CSRsignature
the signature algorithm, e.g. SH256
customcerts/@CSRdn-cn
the CN (common name) part used for the DN
customcerts/@CSRdn-ou
the OU (organizational unit) part used for the DN
customcerts/@CSRdn-o
the O (organization) part used for the DN
customcerts/@CSRdn-l
the L (locality) part used for the DN
customcerts/@CSRdn-st
the ST (state or province) part used for the DN
customcerts/@CSRdn-st
the C (country) part used for the DN (note: CSRdn for many CAs, must be a 2-letter ISO country code
customcerts/@CSRsan-dns-1
the first of up to three DNS names
customcerts/@CSRsan-ip-1
the first of up to two IP addresses

Within the set of CSR attributes, the following meta-words will be replaced:

  • {realip} : the real IP address of the device as reported in the ip=#i query argument
  • {ip} : the IP address of the device as seen by the update server
  • {proxy} : the IP address of the reverse proxy the device used to reach the update server
  • {sn} : the serial number of the device as reported in the sn=#m query argument (e.g. 00-90-33-03-0d-f0)
  • {hwid} : the hardware-id of the device as reported in the sn=#m query argument (e.g. IP1200-03-0d-f0)
  • {name} : the Device Name (set in General/Admin) of the device
  • {rdns} : the DNS name found in a reverse DNS lookup for the real-ip address reported by the device (see {realip}) above

Download

  • not yet available to the public

Related Articles

Retrieved from "https://wiki.innovaphone.com/index.php?title=Howto:PHP_based_Update_Server_V2&oldid=44898"