Thursday, August 5, 2010

CTS System - DTLS Call analysis

As of now CTS codecs supports 2 methods for securing Media traffic: SRTP and DTLS, in this section we are going to describe DTLS protocol and the way it works with CTS units. This article refers what is needed to enable DTLS it does not cover other security protocols used by CUCM, CTMS or CTS, to obtain more information consult TelePresence security guide in CCO.
http://www.cisco.com/en/US/docs/solutions/Enterprise/Video/telepresence.html#wp41946

DTLS
Datagram Transport Layer Security (DTLS) provides communication security for datagram protocols. DTLS is based on Transport Layer Security (TLS) protocol. This datagram-compatible version of the protocol is specifically designed to be similar to TLS with the minimal amount of changes needed to fix problems created by the reordering or loss of packets. There are two main areas that unreliability creates problems for TLS:
  • The traffic encryption layer does not allow individual packets to be decrypted, there are two inter-record dependencies:


    • Cryptographic context is chained between records
    • A Message Authentication Code (MAC) that includes a sequence number provides anti-replay and message reordering protection, but the sequence numbers are implicit in the records
  • The handshake layer breaks if messages are lost because it depends on them being transmitted reliably for these two reasons:


    • The handshake is a lockstep cryptographic handshake requiring messages to be transmitted and received in a defined order, causing a problem with potential reordering and message loss
    • Fragmentation can be a problem because the handshake messages are potentially larger then any given datagram
The first problem caused by the inter-packet dependencies can be solved by using a method employed in the Secure Internet Protocol (IPsec) by adding explicit state to each individual record.
To solve the issue of packet loss DTLS employs a simple retransmission timer. Figure 1 below illustrates the basic concept. The client is expecting to see the HelloVerifyRequest message from the server. If the timer expires then the client knows that either the ClientHello or the HelloVerifyRequest was lost and retransmits.


In TelePresence solution it does not matter the number of codecs in the system only primary codecs will perform dTLS handshake, IP Phones are not associated in the process. In every TelePresence call two dTLS handhsakes occur, one for each stream (video and audio)

DTLS reuses almost all the protocol elements of TLS with minor but important modifications for it to work properly with datagram transport,

A TLS client initiates the handshake by sending the ClientHello message. This message contains the TLS version, a list of algorithms and compression methods that the client will accept and a random nonce used for anti- replay.
The server responds with The Server- Hello contains the server’s choice of version and algorithms and a random nonce. The Certificate contains the server’s certificate chain. The ServerHelloDone is sim- ply a marker message to indicate that no other messages are forthcoming. In more complicated handshakes other messages would appear between the Certificate and the ServerHelloDone messages.
Then send the ChangeCipherSpec message to indicate that it is changing to the newly negotiated protection suite.





In order to debug DTLS for CTMS o CTS we need to obtain CTS logs and perform a packet capture at the beginning of the call. You can use utils network capture command from CLI. (You need CTS 1.7 and up CSCtf08998)

Here is the capture of a sample call between CTS and CTMS:


As mentioned before, in DTLS implementation for TelePresence for each call we will see two dTLS handhsakes, one for each stream (video and audio). (This description does not include Audio-add in)

Decode DTLS using Wireshark

1. Use "Decode as" option in Wireshark to decode UDP packets containing even and odd numbers to RTP and to RTCP respectively.
2. The DTLS negotiation occurs at the beginning of the call, so once you decode UDP packets as RTP you will be able to see the DTLS packets showing as RTP packets using RTP version 0. Those packets can be exported and then decoded as DTLS packets. DTLS will be seen on even port numbers.
3. Once you decode DTLS you can verify the proper handshake messages are seen. Check Fig above.
4. Verify keyexchange and cca logs in CTS /nv/log if at network level DTLS negotiation is succesful.
Image below shows decoded RTP stream as DTLS and you will be able to see DTLS negotiation for audio and video.


By looking closer DTLS packets you will obeserve MIC certificates being exchanged:
ServerHello packet (packet 3) you will see MIC Certificate being negotiated by CTS 
rdnSequence: 1 item (id-at-commonName=CTS-CODEC-PRI-G2-SEPXXXXXXXXXX)

ServerHello packet ( packet 10) you will see MIC Certificate being negotiated by CTMS

rdnSequence: 4 items (id-at-commonName=CTI-XX,id-at-organizationalUnitName=XXXXX,id-at-organizationName=Cisco Systems,id-at-countryName=US)

CTS Configuration in CUCM:

Verify CTS system configuration security is enabled, CUCM cluster must be configured for security in order to enable Media security.


CTMS Configuration
This image below show required configuration in CTMS to negotiate DTLS, As you can see only Media encryption is checked, by default this checkbox is disabled, you need to enable security before you can enable this checkbox in CTMS, this procedure is already well documented in CCO:




Troubleshooting

1. in CTS verify MIC certificate is loaded correctly
Use CLI command:

admin:show cert mic
admin:show security info
security information
====================
   version           : 1.0 [Apr  8 2010, 23:49:41]
   large buffers     : statically allocated
   memory functions  : using CNU malloc/free

   'root'    dir     : /nv/security
   CTL       dir     : /nv/security/ctl
   misc      dir     : /nv/security/misc
   temp      dir     : /usr/tmp/security
   ssl cache dir     : /nv/security/misc
   sec mode  dir     : /nv/security/misc
   dflt LSC  dir     : /nv/security/lsc0
   MIC cert (if any) : /nv/security/mic/certs/MIC-pem.crt
   MIC pkey (if any) : /nv/security/mic/private/priv-pem.key
  
LSC cert (if any) : /nv/security/lsc0/phone.crt (* TEST *)
   LSC pkey (if any) : /nv/security/lsc0/phoneKey.pvt (* TEST *)

   clnt proxy sock   : /tmp/sslClntProxySock
   req srvr sock     : /tmp/secSrvrSock
   req clnt sockets  : /tmp/secClnt_xxx (eg: /tmp/secClnt_mytest)
   command sock      : /tmp/secCmdSock
   max file path len : 128 chars
   max host name len : 64 chars
   max open fd's     : 40
   max sockets       : 30
   max timers        : 16
   min timer val     : 20 millisec
   max timer val     : 3600 seconds
   max SSL clnt      : 8 connections
   max SSL conn tmo  : 75 seconds
   max clnt cache    : 32 sessions
   max sec request   : 8 (concurrent)
   max sec req size  : 2500 bytes
   max sec req wait  : 120 seconds
   max CTL entries   : 32
   max SRST entries  : 5
   max CAPF entries  : 3

   timer type        : millisec resolution
   sock close grace  : 1000 millisec
   dflt SSL conn tmo : 15 seconds
   clnt SSL buf size : 2048 bytes
   SSL clnt cache    : /nv/security/misc/clntSessCache
   SRST list cache   : /usr/tmp/security/srstCache

   MIC               : yes
  
LSC               : NO
   device sec mode   : 1 (auth)
   num CTL entries   : 9
   timers in use     : 0 timers
   sockets in use    : 3 sockets
   clnt SSL in use   : 0 clnt SSL
   active reqs.      : 0 reqs.

   info    o/p on    : general packet socket timer cert ssl conn req ctl
   debug   o/p on    : general packet socket timer cert ssl conn req ctl
   verbose o/p on    : none

In case DTLS negotiation is not successfully executed DTLS Alert protocol messages may appear

In Sysop logs you will see errors if MIC is not loaded or generate errors during the process, CTS built in code has verification of MIC certs:

2010-04-27 11:07:28,130 - ******************************
2010-04-27 11:07:28,130 - WARNING: No valid Manufacturing Installed Certificate found
2010-04-27 11:07:28,131 - Secure mode operation may not be possible
2010-04-27 11:07:28,131 - ******************************

==

2010-10-14 10:51:10,877 - TLS: Error loading CA certificate file
/nv/security/mic/ca/root-pem.crt
TLS: Error loading CA certificate file /nv/security/mic/ca/root-pem.crt

In case MIC is not present please open a TAC case so they regenerate MIC via root or verify existing status of CTS system; there are reports in the past (~2 years) of MIC from manufacturing not loaded or CF STI flash corruption.

admin:show hardware system

     U-Boot Version      : 0.0.35
     Hardware Version    : 0500
     Serial Number       : FHHXXXXXXXXX
     Product ID          : CTS-CODEC-PRIM=
     OS Version          : CTS 1.6.3(4042)
     OS Date             : 2010-02-24 10:55:54
     Compact Flash Model : STI Flash 7.2.0 
     PoE Reset is not available

     Mfg Installed Cert       : INFO:No certificate found
     Locally Significant Cert : INFO:No certificate found
     Max Security Setting    : Encrypted
admin:show cert mic

There is no certificate to display


ERROR Unable to load Certificate Authority file
/nv/security/mic/ca/root-pem.crt"

Verify run control log rc.log when system is booting up

2. Verify DTLS negotiation is successful by looking DTLS messages, in case there is a problem at DTLS layer, Alert messages may be present: By checking RFC for TLS we can find in section 7.2 the Alert protocol:

7.2. Alert protocol

   One of the content types supported by the TLS Record layer is the
   alert type. Alert messages convey the severity of the message and a
   description of the alert. Alert messages with a level of fatal result
   in the immediate termination of the connection. In this case, other
   connections corresponding to the session may continue, but the
   session identifier must be invalidated, preventing the failed session
   from being used to establish new connections. Like other messages,
   alert messages are encrypted and compressed, as specified by the
   current connection state.

       enum { warning(1), fatal(2), (255) } AlertLevel;

       enum {
           close_notify(0),
           unexpected_message(10),
           bad_record_mac(20),
           decryption_failed(21),
           record_overflow(22)
decompression_failure(30),
           handshake_failure(40),
           bad_certificate(42),
           unsupported_certificate(43),
           certificate_revoked(44),
           certificate_expired(45),
           certificate_unknown(46),
           illegal_parameter(47),
           unknown_ca(48),
           access_denied(49),
           decode_error(50),
           decrypt_error(51),
           export_restriction(60),
           protocol_version(70),
           insufficient_security(71),
           internal_error(80),
           user_canceled(90),
           no_renegotiation(100),
           (255)
       } AlertDescription;

       struct {
           AlertLevel level;
           AlertDescription description;
       } Alert;
 
3. Verify security devices are not blocking RTP packets at the beginning which can cause negotiation to fail,
you will need packet captures in both sides or check CTS and CTMS logs to confirm. 
 
Reference
Cisco TelePresence Fundamentals Book
http://tools.ietf.org/html/rfc4347#page-2

No comments:

Post a Comment