Network Layer Security

Network layer security controls have been used frequently for securing communications, particularly over shared networks such as the Internet because they can provide protection for many applications at once without modifying them.

In the earlier chapters, we discussed that many real-time security protocols have evolved for network security ensuring basic tenets of security such as privacy, origin authentication, message integrity, and non-repudiation.

Most of these protocols remained focused at the higher layers of the OSI protocol stack, to compensate for inherent lack of security in standard Internet Protocol. Though valuable, these methods cannot be generalized easily for use with any application. For example, SSL is developed specifically to secure applications like HTTP or FTP. But there are several other applications which also need secure communications.

This need gave rise to develop a security solution at the IP layer so that all higher-layer protocols could take advantage of it. In 1992, the Internet Engineering Task Force (IETF) began to define a standard ‘IPsec’.

In this chapter, we will discuss how security is achieved at network layer using this very popular set of protocol IPsec.

Security in Network Layer

Any scheme that is developed for providing network security needs to be implemented at some layer in protocol stack as depicted in the diagram below −

The popular framework developed for ensuring security at network layer is Internet Protocol Security (IPsec).

Features of IPsec

• IPsec is not designed to work only with TCP as a transport protocol. It works with UDP as well as any other protocol above IP such as ICMP, OSPF etc.
• IPsec protects the entire packet presented to IP layer including higher layer headers.
• Since higher layer headers are hidden which carry port number, traffic analysis is more difficult.
• IPsec works from one network entity to another network entity, not from application process to application process. Hence, security can be adopted without requiring changes to individual user computers/applications.
• Tough widely used to provide secure communication between network entities, IPsec can provide host-to-host security as well.
• The most common use of IPsec is to provide a Virtual Private Network (VPN), either between two locations (gateway-to-gateway) or between a remote user and an enterprise network (host-to-gateway).

Security Functions

The important security functions provided by the IPsec are as follows −

• Confidentiality
  • Enables communicating nodes to encrypt messages.
  • Prevents eavesdropping by third parties.
• Origin authentication and data integrity.
  • Provides assurance that a received packet was actually transmitted by the party identified as the source in the packet header.
  • Confirms that the packet has not been altered or otherwise.
• Key management.
  • Allows secure exchange of keys.
  • Protection against certain types of security attacks, such as replay attacks.

Virtual Private Network

Ideally, any institution would want its own private network for communication to ensure security. However, it may be very costly to establish and maintain such private network over geographically dispersed area. It would require to manage complex infrastructure of communication links, routers, DNS, etc.

IPsec provides an easy mechanism for implementing Virtual Private Network (VPN) for such institutions. VPN technology allows institution’s inter-office traffic to be sent over public Internet by encrypting traffic before entering the public Internet and logically separating it from other traffic. The simplified working of VPN is shown in the following diagram −

Operations Within IPsec

The IPsec suite can be considered to have two separate operations, when performed in unison, providing a complete set of security services. These two operations are IPsec Communication and Internet Key Exchange.

• IPsec Communication
  • It is typically associated with standard IPsec functionality. It involves encapsulation, encryption, and hashing the IP datagrams and handling all packet processes.
  • It is responsible for managing the communication according to the available Security Associations (SAs) established between communicating parties.
  • It uses security protocols such as Authentication Header (AH) and Encapsulated SP (ESP).
  • IPsec communication is not involved in the creation of keys or their management.
  • IPsec communication operation itself is commonly referred to as IPsec.
• Internet Key Exchange (IKE)
  • IKE is the automatic key management protocol used for IPsec.
  • Technically, key management is not essential for IPsec communication and the keys can be manually managed. However, manual key management is not desirable for large networks.
  • IKE is responsible for creation of keys for IPsec and providing authentication during key establishment process. Though, IPsec can be used for any other key management protocols, IKE is used by default.
  • IKE defines two protocol (Oakley and SKEME) to be used with already defined key management framework Internet Security Association Key Management Protocol (ISAKMP).
  • ISAKMP is not IPsec specific, but provides the framework for creating SAs for any protocol.

IPsec Communication Modes

IPsec Communication has two modes of functioning; transport and tunnel modes. These modes can be used in combination or used individually depending upon the type of communication desired.

Transport Mode

• IPsec does not encapsulate a packet received from upper layer.
• The original IP header is maintained and the data is forwarded based on the original attributes set by the upper layer protocol.
• The following diagram shows the data flow in the protocol stack.

Tunnel Mode

• This mode of IPsec provides encapsulation services along with other security services.
• In tunnel mode operations, the entire packet from upper layer is encapsulated before applying security protocol. New IP header is added.
• The following diagram shows the data flow in the protocol stack.

• Tunnel mode is typically associated with gateway activities. The encapsulation provides the ability to send several sessions through a single gateway.
• The typical tunnel mode communication is as depicted in the following diagram.

• As far as the endpoints are concerned, they have a direct transport layer connection. The datagram from one system forwarded to the gateway is encapsulated and then forwarded to the remote gateway. The remote associated gateway de-encapsulates the data and forwards it to the destination endpoint on the internal network.
• Using IPsec, the tunneling mode can be established between the gateway and individual end system as well.