http_client

Olle E. Johansson

   Edvina AB
   <oej@edvina.net>

Juha Heinanen

   TutPro Inc.
   <jh@tutpro.com>

Carsten Bock

   ng-voice GmbH
   <carsten@ng-voice.com>

Hugh Waite

   Xura Inc
   <hugh.waite@xura.com>

   Copyright © 2008-2009 Juha Heinanen

   Copyright © 2013 Carsten Bock, ng-voice GmbH

   Copyright © 2015 Olle E. Johansson, Edvina AB

   Copyright © 2016 Hugh Waite, Xura Inc
     __________________________________________________________________

   Table of Contents

   1. Admin Guide

        1. Overview
        2. Dependencies

              2.1. Kamailio Modules
              2.2. External Libraries or Applications

        3. Parameters

              3.1. httpredirect (int)
              3.2. httpproxy (string)
              3.3. httpproxyport (string)
              3.4. useragent (string)
              3.5. maxdatasize (int)
              3.6. connection_timeout (int)
              3.7. client_cert (string)
              3.8. client_key (string)
              3.9. cacert (string)
              3.10. cipher_suites (string)
              3.11. verify_peer (int)
              3.12. verify_host (int)
              3.13. tlsversion (int)
              3.14. authmethod (int)
              3.15. keep_connections (int)
              3.16. httpcon (string)
              3.17. config_file (string)

        4. Functions

              4.1. http_connect(connection, url, [content_type, data,]
                      result)

              4.2. http_get_redirect(connection, result)
              4.3. http_client_query(url, [post-data], [hdrs], result)

        5. Pseudovariables

              5.1. $curlerror(error)

        6. RPC Commands

              6.1. httpclient.listcon

        7. Counters

              7.1. httpclient.connections
              7.2. httpclient.connok
              7.3. httpclient.connfail

        8. Remarks

   2. Developer Guide

        1.
        2. Available Functions

              2.1. int http_connect(msg, connection, url, result,
                      content_type, post)

              2.2. int http_connection_exists(str *connection)
              2.3. int http_query(msg, url, dest, post)
              2.4. http_get_content_type(str connection)

   List of Examples

   1.1. Set httpredirect parameter
   1.2. Set httpproxy parameter
   1.3. Set httpproxyport parameter
   1.4. Set useragent parameter
   1.5. Set maxdatasize parameter
   1.6. Set connection_timeout parameter
   1.7. Set client_cert parameter
   1.8. Set client_key parameter
   1.9. Set cacert parameter
   1.10. Set cipher_suites parameter
   1.11. Set verify_peer parameter
   1.12. Set verify_host parameter
   1.13. Set tlsversion parameter
   1.14. Set authmethod parameter
   1.15. Set keep_connections parameter
   1.16. Set httpcon parameter
   1.17. Set config_file parameter
   1.18. Short http_client config file
   1.19. http_connect() usage
   1.20. http_get_redirect() usage
   1.21. http_client_query() usage

Chapter 1. Admin Guide

   Table of Contents

   1. Overview
   2. Dependencies

        2.1. Kamailio Modules
        2.2. External Libraries or Applications

   3. Parameters

        3.1. httpredirect (int)
        3.2. httpproxy (string)
        3.3. httpproxyport (string)
        3.4. useragent (string)
        3.5. maxdatasize (int)
        3.6. connection_timeout (int)
        3.7. client_cert (string)
        3.8. client_key (string)
        3.9. cacert (string)
        3.10. cipher_suites (string)
        3.11. verify_peer (int)
        3.12. verify_host (int)
        3.13. tlsversion (int)
        3.14. authmethod (int)
        3.15. keep_connections (int)
        3.16. httpcon (string)
        3.17. config_file (string)

   4. Functions

        4.1. http_connect(connection, url, [content_type, data,] result)
        4.2. http_get_redirect(connection, result)
        4.3. http_client_query(url, [post-data], [hdrs], result)

   5. Pseudovariables

        5.1. $curlerror(error)

   6. RPC Commands

        6.1. httpclient.listcon

   7. Counters

        7.1. httpclient.connections
        7.2. httpclient.connok
        7.3. httpclient.connfail

   8. Remarks

1. Overview

   This module implements protocol functions that use the libcurl library
   to fetch data from external HTTP servers or post data to HTTP servers.
   The module is using a concept of "connections" to define properties of
   HTTP sessions in a simple way. A connection has one or multiple servers
   and a set of settings that apply to the specific connection.

   The http_client module has multiple settings, some of them applies to a
   defined connection. You can set timeouts, max data sizes for download
   and much more either using modparam settings or parameters to the
   connection definition.

   The connections can either be defined with the "httpcon" module
   parameter or in a separate configuration file, as specified by the
   "config_file" module parameter.

   Like in SIP, the HTTP URL may need encoding to be transported safely
   over the network. Check the string encoding functions in the
   Transformation Cookbook (as used in the http_client_query example
   below).

   The function http_client_query allows Kamailio to issue an HTTP GET
   request and get access to parts of the reply. This function has been
   ported from the utils module and now use the same libcurl functions. We
   recommend using the new functionality provided by this module.

   The http_client module use the CURL library setting up connections. The
   CURL library by default use the system configured DNS resolvers, not
   the Kamailio resolver.

   The module is limited to using HTTP and HTTPS protocols.

2. Dependencies

   2.1. Kamailio Modules
   2.2. External Libraries or Applications

2.1. Kamailio Modules

   The following modules must be loaded before this module:
     * TLS - if you use TLS connections (https) the tls module should be
       loaded first in order to initialize OpenSSL properly.

2.2. External Libraries or Applications

   The following libraries or applications must be installed before
   running Kamailio with this module loaded:
     * libcurl.

3. Parameters

   3.1. httpredirect (int)
   3.2. httpproxy (string)
   3.3. httpproxyport (string)
   3.4. useragent (string)
   3.5. maxdatasize (int)
   3.6. connection_timeout (int)
   3.7. client_cert (string)
   3.8. client_key (string)
   3.9. cacert (string)
   3.10. cipher_suites (string)
   3.11. verify_peer (int)
   3.12. verify_host (int)
   3.13. tlsversion (int)
   3.14. authmethod (int)
   3.15. keep_connections (int)
   3.16. httpcon (string)
   3.17. config_file (string)

3.1. httpredirect (int)

   If set to 1, enabled, http_client will follow HTTP 302 Redirects. If
   set to 0, http_client will not follow redirects. Default is 1, enabled.

   The latest redirect URL will be stored in the $curlredirect
   pseudovariable.

   Example 1.1. Set httpredirect parameter
...
modparam("http_client", "httpredirect", 0)
...

3.2. httpproxy (string)

   URL for a HTTP proxy to use as a default proxy for all connections.

   This setting is also available on a per connection basis in the
   http_client configuration file.

   Example 1.2. Set httpproxy parameter
...
modparam("http_client", "httpproxy", "https://superproxy.example.com")
...

3.3. httpproxyport (string)

   Port number for a HTTP proxy to use as a default proxy port for all
   connections.

   This setting is also available on a per connection basis in the
   http_client configuration file.

   Example 1.3. Set httpproxyport parameter
...
modparam("http_client", "httpproxyport", 8042)
...

3.4. useragent (string)

   Useragent to use in the HTTP protocol for requests. Defaults to the
   Kamailio SIP useragent string - including software version and
   platform.

   Example 1.4. Set useragent parameter
...
modparam("http_client", "useragent", "Secret HTTP REST grabber 0.42")
...

3.5. maxdatasize (int)

   Defines the maximum size in bytes for a response. Note that this is
   allocated from pkg memory (process memory) dynamically.

   Default value is zero, i.e., the limit on the datasize is disabled.

   Example 1.5. Set maxdatasize parameter
...
modparam("http_client", "maxdatasize", 2000)
...

3.6. connection_timeout (int)

   Defines in seconds how long Kamailio waits for response from servers.

   Default value is zero, i.e., the timeout function is disabled.

   Example 1.6. Set connection_timeout parameter
...
modparam("http_client", "connection_timeout", 2)
...

3.7. client_cert (string)

   File name for a TLS client certificate. The certificate needs to be
   encoded in PEM format.

   Default value is empty string, i.e. no client certificate used. Note
   that if you specify a client cert, you also need to specify the
   client_key.

   Example 1.7. Set client_cert parameter
...
modparam("http_client", "client_cert", "/var/certs/sollentuna.example.com.cert")
...

3.8. client_key (string)

   File name for a TLS client key. The key needs to be encoded in PEM
   format.

   Default value is empty string, i.e. no client certificate or key is
   used. Note that if you specify a client key, you also need to specify
   the client_cert.

   Example 1.8. Set client_key parameter
...
modparam("http_client", "client_key", "/var/certs/sollentuna.example.com.key")
...

3.9. cacert (string)

   File name for the trusted TLS CA cert used to verify servers. The
   certificates need to be encoded in PEM format.

   Default value is empty string, i.e. no CA certificate is used to verify
   the host. If tlsverifyhost is on, all TLS connections will fail without
   any CA certificate to validate with.

   Example 1.9. Set cacert parameter
...
modparam("http_client", "cacert", "/var/certs/ca/edvina-sip-ca.pem")
...

3.10. cipher_suites (string)

   List of allowed cipher suites. See
   http://curl.haxx.se/libcurl/c/CURLOPT_SSL_CIPHER_LIST.html for details
   of the cipher list curl option.

   Default value is empty string, i.e. the default list of ciphers in
   libcurl will be used.

   Example 1.10. Set cipher_suites parameter
...
modparam("http_client", "cipher_suites", "ecdhe_ecdsa_aes_128_gcm_sha_256,rsa_ae
s_128_gcm_sha_256")
...

3.11. verify_peer (int)

   If set to 0, TLS verification of the server certificate is disabled.
   This means that the connection will get encrypted, but there's no
   authentication. There's no proof that the transmission of data is to
   the host that is meant to receive data.

   If set to 1, default setting, and one or more CA certificates is
   configured, the server TLS certificate will be validated. If validation
   fails, the connection fails.

   See the curl documentation for more details.
   http://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYPEER.html

   Example 1.11. Set verify_peer parameter
...
modparam("http_client", "verify_peer", 1)
...

3.12. verify_host (int)

   If set to 0, domain verification of the server certificate is disabled.
   This means that the connection will get encrypted but there is no check
   that data will be sent to the host that is meant to receive it. Disable
   with caution.

   If set to 2, default setting, the hostname in the URL will be verified
   against the Common Name or Subject Alt Name in the certificate. If
   validation fails, the connection fails.

   See the curl documentation for more details.
   http://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYHOST.html

   Example 1.12. Set verify_host parameter
...
modparam("http_client", "verify_host", 2)
...

3.13. tlsversion (int)

   Sets the preferred TLS/SSL version.

   Valid values are:
     * 0 - Use libcurl default
     * 1 - "TLSv1"
     * 2 - "SSLv2"
     * 3 - "SSLv3"
     * 4 - "TLSv1.0"
     * 5 - "TLSv1.1"
     * 6 - "TLSv1.2"

   SSL versions are now disabled by default. See the curl documentation
   for more details. http://curl.haxx.se/libcurl/c/CURLOPT_SSLVERSION.html

   Example 1.13. Set tlsversion parameter
...
modparam("http_client", "tlsversion", 6)
...

3.14. authmethod (int)

   Sets the preferred authentication mode for HTTP/HTTPS requests. The
   value is a bitmap and multiple methods can be used. Note that in this
   case, the CURL library will make an extra request to discover
   server-supported authentication methods. You may want to use a specific
   value.

   Valid values are:
     * 1 - BASIC authentication
     * 2 - HTTP Digest authentication
     * 4 - GSS-Negotiate authentication
     * 8 - NTLM authentication
     * 16 - HTTP Digest with IE flavour

   Default value is 3 - BASIC and Digest authentication.

   This is also configurable per connection in the http_client
   configuration file.

   Example 1.14. Set authmethod parameter
...
# Use the best of BASIC and Digest authentication.
modparam("http_client", "authmethod", 3)
...

3.15. keep_connections (int)

   If an HTTP server is accessed multiple times keeping the connection
   open for reuse saves a significant amount of time, especially if TLS is
   used. If this function is enabled, the Curl library will try to reuse
   existing open connections. The HTTP server will have to support this
   feature and keep connections open for it to work properly.

   Valid values are:
     * 0 - Close connections after request (default)
     * 1 - Reuse connections

   This is also configurable per connection in the http_client
   configuration file.

   Example 1.15. Set keep_connections parameter
...
modparam("http_client", "keep_connections", 1)
...

3.16. httpcon (string)

   Defines a connection and credentials for the connection for use in a
   connection-oriented function call in this module.

   Syntax:
   <connection-name>=><schema>://[<username>:<password>@]<hostname/address
   >[;param=value]

   The address in the URL is the base for the URL in the http_connect()
   call. The address given in the function call will be appended to the
   base URL in the connection definition.

   The HTTP connection will be defined using default values in modparam's
   above the definition of the httpcon in the configuration file. Also
   note that connections can be defined in a separate text file if you
   have many parameters per connection, or want to use a per-connection
   setting that can be set in that file but not in the httpcon modparam,
   like authmethod.

   By default, no connections are defined.

   Parameters
     * useragent Useragent used for HTTP requests. Overrides useragent
       modparam.
     * verify_peer Set to 1 to enable or 0 to disable server certificate
       verification. Overrides verify_peer modparam.
     * verify_host Set to 2 to enable or 0 to disable server hostname
       verification. Overrides verify_host modparam.
     * client_cert Client certificate used for this connection. Overrides
       the default client_cert modparam.
     * client_key Client key used for this connection. Overrides the
       default client_key modparam.
     * cipher_suites Client certificate used for this connection.
       Overrides the default cipher_suite modparam.
     * timeout Timeout used for this connection. Overrides the default
       connection_timeout for the module.
     * tlsversion TLS version used for this connection. Overrides the
       default tlsversion for the module.
     * maxdatasize The maximum datasize for a response. Overrides the
       maxdatasize modparam setting.
     * httpredirect Set to 1 for following HTTP 302 redirect. 0 to
       disable. Overrides the default httpredirect modparam.
     * failover The name of another httpcon connection to use with the
       same arguments in case a connection with this http_con fails.
       Failure is either a connection failure or a response code of 500 or
       above.

   Example 1.16. Set httpcon parameter
...
modparam("http_client", "httpcon", "apione=>http://atlanta.example.com")
modparam("http_client", "httpcon", "apitwo=>http://atlanta.example.com/api/12")
modparam("http_client", "httpcon", "apithree=>http://annabella:mysecret@atlanta.
example.com/api/12")
modparam("http_client", "httpcon", "apifour=>http://stockholm.example.com/api/ge
tstuff;timeout=12;failover=apione")
...

3.17. config_file (string)

   The file name of a configuration file containing definitions of http
   connections. This is an alternative to the "httpcon" module parameter -
   especially when the number of options per line gets too big.

   If the file or directory name starts with a '.' the path will be
   relative to the working directory (at runtime). If it starts with a '/'
   it will be an absolute path and if it starts with anything else the
   path will be relative to the main config file directory (e.g.: for
   kamailio -f /etc/kamailio/kamailio.cfg it will be relative to
   /etc/kamailio/).

   The following parameters can be set in the config file, for each
   connection. If a parameter is not specified, the default values set by
   the modparams will be used.
     * url
     * username
     * password
     * authmethod
     * keep_connections
     * useragent
     * verify_peer
     * verify_host
     * client_cert
     * client_key
     * cipher_suites
     * tlsversion - Valid values are:
          + "DEFAULT"
          + "TLSv1"
          + "SSLv22
          + "SSLv3"
          + "TLSv1.0"
          + "TLSv1.1"
          + "TLSv1.2"
     * timeout
     * maxdatasize
     * http_follow_redirect
     * httpproxy
     * httpproxyport
     * failover

   See the "httpcon" module parameter for explanation of these settings.

   By default no config file is specified.

   All the parameters that take filenames as values will be resolved using
   the same rules as for the tls config filename itself: starting with a
   '.' means relative to the working directory, a '/' means an absolute
   path and anything else a path relative to the directory of the current
   Kamailio main config file.

   To set a string value to null, in order to override default settings,
   you can specify an value of "" - two quotation marks. In order to
   disable a http proxy setting you can set the port to zero.

   Example 1.17. Set config_file parameter
...
modparam("http_client", "config_file", "httpconnections.cfg)
...

   Example 1.18. Short http_client config file
[authapiserver]
url = https://api.runbo.example.com/v4.2/auth
timeout = 1
maxdatasize = 4
tlsversion = TLSv1.2
verify_peer = yes
client_key = default_key.pem
client_cert = default_cert.pem
http_follow_redirect = no

4. Functions

   4.1. http_connect(connection, url, [content_type, data,] result)
   4.2. http_get_redirect(connection, result)
   4.3. http_client_query(url, [post-data], [hdrs], result)

4.1.  http_connect(connection, url, [content_type, data,] result)

   Sends HTTP GET or POST request to a given connection. For a POST
   request, content-type can be specified.
     * connection - the name of an existing HTTP connection, defined by a
       httpcon modparam.
       url - the part of the URL to add to the predefined URL in the
       connection definition.
       content_type - Used only when posting data with HTTP POST. An
       Internet Media type, like "application/json" or "text/plain". Will
       be added to the HTTP request as a header.
       data - Data or a pseudo variable holding data to be posted.
       result - The name of a pseudo variable that will have the data of
       the response from the HTTP server.

   The return value is the HTTP return code (if >=100) or the CURL error
   code if below 100. See the $curlerror pseudovariable below for more
   information about CURL error codes.

   This function can be used from REQUEST_ROUTE, ONREPLY_ROUTE,
   FAILURE_ROUTE, and BRANCH_ROUTE.

   Example 1.19. http_connect() usage
...
modparam("http_client", "httpcon", "apiserver=>http://kamailio.org/api/");
...
# POST Request
$var(res) = http_connect("apiserver", "/mailbox", "application/json", "{ ok, {20
0, ok}}", "$avp(gurka)");
xlog("L_INFO", "API-server HTTP connection: $avp(gurka) Result code $var(res)\n"
);

$var(res) = http_connect("apiserver", "/callroute", "application/json", "$var(js
ondata)", "$avp(route)");
xlog("L_INFO", "API-server HTTP connection: $avp(route) Result code $var(res)\n"
);
...

4.2.  http_get_redirect(connection, result)

   When a http connection gets a redirect and the connection is configured
   to follow redirects (301,302) then the target URL, the result of the
   redirects can be retrieved with this function after a successful
   connection.
     * connection - the name of an existing HTTP connection, defined by a
       httpcon modparam.
       result - The name of a pseudo variable that will contain the last
       used URL.

   Example 1.20. http_get_redirect() usage
...
modparam("http_client", "httpredirect", 1);
...
http_get_redirect("apiserver", "$var(targeturl)");
...

4.3.  http_client_query(url, [post-data], [hdrs], result)

   Sends HTTP GET or POST request according to URL given in “url”
   parameter, which is a string that may contain pseudo variables.

   If you want to make a POST-Request, you have to define the “post”-data,
   that should be submitted in that request as the second parameter.

   Custom headers may be specified via “hdrs” parameter (e.g.,
   Content-Type).

   Either of “post-data” or “hdrs” can be also set to empty string in
   order to be ignored.

   If HTTP server returns a class 2xx, 3xx or 4xx reply, the first line of
   the reply's body (if any) is stored in “result” parameter, which must
   be a writable pseudo variable.

   Function returns reply code of HTTP reply or -1 if something went
   wrong.

   This function can be used from REQUEST_ROUTE, ONREPLY_ROUTE,
   FAILURE_ROUTE, and BRANCH_ROUTE.

   Note that this function is based on the http_query function in the
   utils module. It is changed to use the same base library and settings
   as the rest of the functions in this module.

   Example 1.21. http_client_query() usage
...
# GET-Request
http_client_query("http://api.com/index.php?r_uri=$(ru{s.escape.param})&f_uri=$(
fu{s.escape.param})",
           "$var(result)");
switch ($rc) {
    ...
}
...
# POST-Request
http_client_query("http://api.com/index.php",
    "r_uri=$(ru{s.escape.param})&f_uri=$(fu{s.escape.param})",
    "$var(result)");
}
...
# POST-Request
http_client_query("http://api.com/index.php", "src=$si",
    "Content-Type: text/plain", "$var(result)");
...

5. Pseudovariables

   5.1. $curlerror(error)

5.1.  $curlerror(error)

   The cURL library returns error codes from the protocol used. If an
   error happens, a cURL specific error code below 100 is returned. The
   $curlerror pv returns a text string representing the error. For more
   information on cURL error codes, please visit
   http://curl.haxx.se/libcurl/c/libcurl-errors.html

6. RPC Commands

   6.1. httpclient.listcon

6.1. httpclient.listcon

   Lists all defined httpcon connections

   Parameters:
     * No parameters

7. Counters

   7.1. httpclient.connections
   7.2. httpclient.connok
   7.3. httpclient.connfail

7.1.  httpclient.connections

   The number of connection definitions that are in-memory.

7.2.  httpclient.connok

   The number of successful connections since Kamailio start

7.3.  httpclient.connfail

   The number of failed connections since Kamailio start

8. Remarks

   Note: libcurl leak in CentOS 6 - this module uses libcurl library and
   in case if you are using CentOS 6, be aware that standard
   libcurl-7.19.7-52 has a memory leak. To fix this memory, install
   libcurl from city-fan repository. More details at:
   https://www.digitalocean.com/community/questions/how-to-upgrade-curl-in
   -centos6

Chapter 2. Developer Guide

   Table of Contents

   1.
   2. Available Functions

        2.1. int http_connect(msg, connection, url, result, content_type,
                post)

        2.2. int http_connection_exists(str *connection)
        2.3. int http_query(msg, url, dest, post)
        2.4. http_get_content_type(str connection)

   This module provides a set of API functions that other modules can use
   in order to integrate with HTTP services.

2. Available Functions

   2.1. int http_connect(msg, connection, url, result, content_type, post)

   2.2. int http_connection_exists(str *connection)
   2.3. int http_query(msg, url, dest, post)
   2.4. http_get_content_type(str connection)

2.1.  int http_connect(msg, connection, url, result, content_type, post)

   Sends HTTP GET or POST request to a given connection. If content_type
   and post are NULL GET will be used. If post is not null the data will
   be POSTed using the specified content_type.

   Returns the status code of the HTTP response (if >= 100), or a curl
   error code (if < 100)

   Meaning of the parameters is as follows:
     * struct sip_msg *msg
       The current sip message structure.
     * const str *connection
       The name of a preset http_con connection to use for this query.
     * const str *url
       A string that will be appended to the base URL specified in the
       connection. This parameter can be NULL, which means nothing will be
       appended to the base URL.
     * str *result
       A pointer to a string that will contain the response body. On
       success, the data is allocated in pkg memory by the http_client
       module and must be freed by the caller.
     * const char *content_type
       A null-terminated string specifying the content type to place in a
       Content-Type header. Use NULL when a message body is not required.
     * const str *post
       A string containing the message body to send. Use NULL when a
       message body is not required.

2.2.  int http_connection_exists(str *connection)

   Check if a connection definition exists. Connections are defined as
   modparam's in the http_client modules.

   Returns 1 if the connection exists, 0 if a connection with the given
   name can't be found.

2.3.  int http_query(msg, url, dest, post)

   Sends a HTTP GET or POST request to a given connection. If post data is
   defined, POST will be used, otherwise GET. The default settings defined
   as module params of the http_client module will be used for the
   connection.

   Meaning of the parameters is as follows:
     * struct sip_msg *msg
       The current SIP message structure.
     * const char *url
       A string that will be used as the URL specified in the connection.
     * str *dest
       A pointer to a string that will contain the first line of the
       response body. On success, the data is allocated in pkg memory by
       the http_client module and must be freed by the caller.
     * const char *post
       If not null, the data will be posted to the URL.

2.4.  http_get_content_type(str connection)

   Get the content-type of the last result for this connection. This will
   be something like "text/html" for html or "application/json" for JSON
   data.

   Result will be a pointer to a char string (char *). This is per
   process, so the connection will have to be done in the same process.
   Returns a NULL pointer if the connection does not exist or there's no
   previous connection data delivered.
