1 Configuration
- 1.1 Simple Example
  - 1.1.1 Required Configuration
  - 1.1.2 Optional Configuration
- 1.2 set resources group address-group <name>
- 1.3 set service nat pool <name>
- 1.4 set service nat cgnat policy <name>
- 1.5 set service cgnat session-timeout
- 1.6 Other configuration commands
2 Operational Commands
- 2.1 show cgnat summary
- 2.2 show cgnat error
- 2.3 show nat pool
- 2.4 show cgnat policy
- 2.5 show cgnat subscriber
- 2.6 show cgnat public
- 2.7 show cgnat session

Configuration

Simple Example

Required Configuration

Address-group to match subscriber (source) addresses

set resources group address-group AG_MATCH1
set resources group address-group AG_MATCH1 address 10.10.1.0/28
set resources group address-group AG_MATCH1 address-range 10.10.1.20 to 10.10.1.30
set resources group address-group AG_MATCH1 address 10.10.1.10

NAT Pool to specify Public Addresses and how they are Allocated

set service nat pool NAT_POOL1 entry RANGE1 ip-address range start 10.10.3.1
set service nat pool NAT_POOL1 entry RANGE1 ip-address range end 10.10.3.63
set service nat pool NAT_POOL1 entry RANGE2 ip-address prefix 10.10.3.64/28
set service nat pool NAT_POOL1 type CGNAT
set service nat pool NAT_POOL1 address-allocation round-robin
set service nat pool NAT_POOL1 address-pooling paired
set service nat pool NAT_POOL1 port dynamic-block-allocation block-size 128
set service nat pool NAT_POOL1 port dynamic-block-allocation max-blocks-per-subscriber 8
set service nat pool NAT_POOL1 port allocation sequential
set service nat pool NAT_POOL1 port range start 1024
set service nat pool NAT_POOL1 port range end 65535

CGNAT Policy pulls together the 'match' address-group and the NAT pool

set service nat cgnat policy POLICY1 match source address-group AG_MATCH1
set service nat cgnat policy POLICY1 priority 10
set service nat cgnat policy POLICY1 translation pool NAT_POOL1

Assign the CGNAT Policy to the interface which routes to the external network

Optional Configuration

Session Timeouts

Per-port established-state timeouts are available per port

Logging

Note that enabling session logging will switch CGNAT to record destination address and port in sub-sessions (2-tuple sessions in tables within 3-tuple sessions). This will affect performance.

set resources group address-group <name>

CGNAT uses an address-group to match inside subscriber source addresses.

Parameter	Format	Description	Details

Parameter	Format	Description	Details
address	<h:h:h:h:h:h:h:h/x>	IPv6 subnet to match	not applicable for CGNAT
	<h:h:h:h:h:h:h:h>	IPv6 address to match	not applicable for CGNAT
	<x.x.x.x/x>	IP subnet to match	The first and last addresses in the prefix will be matched. For example, 10.10.1.0/28 will match address range 10.10.1.0 to 10.10.1.15.
	<x.x.x.x>	IP address to match
address-range	<h:h:h:h:h:h:h:h> to <h:h:h:h:h:h:h:h>	IPv6 address	not applicable for CGNAT
	<x.x.x.x> to <x.x.x.x>	IPv4 address
description	<text>	Address-group description

set service nat pool <name>

A NAT pool is used to specify the public addresses that CGNAT will use to translate inside source addresses to.

Command	Format	Details

Command	Format	Details
entry <range-name> ip-address range start	<x.x.x.x>	IPv4 Address
entry <range-name> ip-address range end	<x.x.x.x>	IPv4 Address
entry <range-name> ip-address prefix	<x.x.x.x/x>	IPv4 address and mask. First and last addresses are used provided they are not '.0' or '.255'. For example, 10.10.3.0/28 would give range 10.10.3.1-10.10.3.15.
entry <range-name> ip-address subnet	<x.x.x.x/x>	IPv4 subnet and mask. First and last addresses are never used. For example, 10.10.3.0/28 would give range 10.10.3.1-10.10.3.14.
type	CGNAT	Only option. Required for future compatibility.
address-allocation	round-robin	Addresses are allocated from the NAT pool in a round-robin fashion. Only option. Required for future compatibility.
address-pooling	paired	Internal IP addresses are paired with external IP addresses. An internal IP address will never use more than one external IP address at any given time. Only option. Required for future compatibility.
port dynamic-block-allocation block-size	<64..4096>	The number of ports in a block (multiple of 64 only).
port dynamic-block-allocation max-blocks-per-subscriber	<1..32>	Maximum number of port-blocks per subscriber
port allocation	random	Allocate ports randomly from within the current 'active' port-block.
	sequential	Allocate ports sequentially from within the current 'active' port-block.
port range start	<0..65535>	Start of port rang. It is recommended that this is set no lower than 1024.
range end	<0..65535>	End of port range.
blacklist address-group	<addr-grp-name>	The address-group identifying the pool addresses being blacklisted. Adding external IP addresses or prefixes to a blacklist address group will cause sessions creating using that external address to be cleared, and for that address to no longer be used to create new sessions. It is useful on a live system where the administrator wants to prevent an address in the NAT pool form being used for CGNAT mappings. This has an advantage over reconfiguring the NAT pool range to exclude that address in that reconfiguring the NAT pool range will cause all sessions using that NAT pool to be torn down.

set service nat cgnat policy <name>

A CGNAT policy ties together an address-group for matching inside source addresses with a NAT pool for specifying the set of external public addresses to use. A CGNAT policy is the thing that is attached to an interface,

Command	Format	Description

Command	Format	Description
match source address-group	<addr-grp-name>	Address group name
translation pool	<nat-pool-name>	NAT pool name
priority	<1..9999>	The priority of CGNAT policy. Multiple CGNAT policies on an interface are evaluated in order of priority, with the lowest numerical priority first.

set service cgnat session-timeout

Established-state timeout values may be configured per port (a destination port in outbound traffic).


other	partially-open	<10..240>	Non-TCP/UDP 'partially-open' timeout in seconds
	established	<30..1800>	Non-TCP/UDP 'established' timeout in seconds
udp	partially-open	<10..240>	UDP 'partially-open' timeout in seconds
	established	<30..1800>	UDP 'established' timeout in seconds
tcp	partially-open	<10..240>	TCP 'partially-open' timeout in seconds
	partially-closed	<10..240>	TCP 'partially-closed' timeout in seconds
	established	<30..14400>	TCP 'established' timeout in seconds

Other configuration commands

set service cgnat ...


cpu-affinity event session	<0..65535>	Number of the core to process session events on
disable-hairpinning		Disable hairpinning in CGNAT device
export event ...		Used to control logging
max-dest-per-session	<1..64>	Max number of destination records that can be associated with a session
max-sessions	<1..33554432>	System-wide maximum number of sessions
select warning event resource-constraint ...
snat-alg-bypass		Enable SNAT ALG traffic to bypass CGNAT

Operational Commands

show cgnat summary

The error and summary statistics will count every packet that matches either of the following two criteria:

outbound packet whose source address matches a CGNAT policy, or
inbound packet who's destination address is a CGNAT public address

Counts marked with a tilde (~) are approximate. These are collated from each CGNAT policy every time the "show cgnat summary" command is run. The policy counts are only updated approximately every 20 secs from similar counts in the subscriber data structures.

Sessions

Sessions
Active sessions	Active sessions. An original global atomic counter. This is not simply 'sessions created less sessions destroyed'.
Sessions created	Sum of 'sessions created' counts on all CGNAT policies.
Sessions destroyed	Sum of 'sessions destroyed' counts on all CGNAT policies.
Active sub-sessions	Count of current sub-sessions. An original global atomic counter. This is not simply 'sub-sessions created less sub-sessions destroyed'.
Sub-sessions created	Sum of 'sub-sessions created' counts on all CGNAT policies.
Sub-sessions destroyed	Sum of 'sub-sessions destroyed' counts on all CGNAT policies.
Maximum table size	Configurable maximum session table size. (This value applies to sessions, not sessions plus sub-sessions)
Table full	Set when the 'active sessions' count exceeds 'Maximum table size'.
Public address mapping table
Used	Active public addresses in use by CGNAT. An original global atomic counter. There is no maximum value for this table. Its size is constrained by the number of addresses in the NAT pool used by CGNAT.
Subscriber address table
Used	Active CGNAT subscribers. An original global atomic counter.
Max	Maximum CGNAT subscribers allowed in the subscriber address table. Configurable. Default 64k.
Out
Translated packets	Count of outbound packets translated. Sum of 'Out packets' counts on all CGNAT policies.
Translated bytes	Total byte count of the outbound translated packets. (IP header and payload). Sum of 'Out bytes' counts on all CGNAT policies.
Did not match CGNAT policy	Subscriber address did not match any CGNAT policy on the outbound interface.
Untranslatable packets	Untranslatable packets. Sum of error counts 'Untranslatable IP protocol' and 'Untranslatable ICMP message'
Hairpinned packets	Count of hairpinned packets.
In
Translated packets	Count of inbound packets translated. Sum of 'In packets' counts on all CGNAT policies.
Translated bytes	Total byte count of the inbound translated packets. (IP header and payload). Sum of 'In bytes' counts on all CGNAT policies.
Unknown source addr or port	Sum of 'In, unknown source' counts on all CGNAT policies. Counts inbound packets whose dest address and port have matched a session, but whose source address and port have failed to match a sub-session. Only applies when sub-sessions are being recorded for this session.
Did not match CGNAT session	Destination address, port and protocol did not match a CGNAT session and the destination address does not match a CGNAT public address.
Did not match CGNAT pool	Destination address does not match a CGNAT public address.
Session Hash Tables
Created	A hash table may be created for each main session in order to store sub-sessions if: Destination addr and port are being recorded, and More than one destination address and port is seen for the same source address, port, and protocol. Creation and maintenance of these tables is an expensive operation. If many hash tables are being created and destroyed then performance may degrade somewhat. These counters are designed to provide some indication that this may be occurring. Typically multiple sub-sessions per main session will occur with peer-to-peer gaming.
Destroyed

PCP sessions created	Sessions created by the Port Control Protocol
PCP errors	Failures when trying to create a PCP session.
Memory allocation failures	Sum of all memory allocation failures. Use "show cgnat error" to see details.
Resource limitation failures	Sum of all resource limitation failures. Use "show cgnat error" to see details.
Thread contention errors	Sum of all thread contention errors. Use "show cgnat error" to see details.
Packet buffer errors	Sum of all packet buffer errors. Use "show cgnat error" to see details.

show cgnat error

N/A = Means that count is not applicable in this direction
OK = Means that count is applicable in this direction

Unable to translate packet:	In	Out	Comment

Unable to translate packet:	In	Out	Comment
Subscriber address did not match a CGNAT policy	N/A	OK	Subscriber address did not match a CGNAT policy
Packet did not match a CGNAT session	OK	N/A	Inbound pkts that do not match a CGNAT session and whose dest addr does match a CGNAT policy. These are dropped.
Destination address did not match CGNAT pool	OK	N/A	Inbound pkts that do not match a CGNAT session and whose dest addr does not match a CGNAT policy. These are forwarded untranslated.
CGNAT bypassed by SNAT-ALG packets	N/A	OK	Pkt do not match a CGNAT session and does not match a CGNAT policy and SNAT is cfgd and pkt matches an ALG session or tuple. Pkt is forwarded unchanged.
Untranslatable IP protocol	OK	OK	Protocol is not one of the following: TCP, UDP, UDP-lite, DCCP or ICMP
Untranslatable ICMP message	OK	OK	ICMP type is not one of: echo-req, echo-reply, dest unreach, redirect, time-exceeded, or parameter-problem
Resource limitations:
Subscriber port-block limit	N/A	OK	CGNAT policy config contains the max number of port-blocks that anyone subscriber may use at any one time. Once this is reached, further mapping requests will fail, and this count will increment.
No free port-blocks on the selected public address	N/A	OK	Counts mapping failures that occur when a subscriber is within his port-block limit but there are no free port-blocks on the paired public address. This can occur when 1. the max port-blocks per subscriber and port-block size equate to more than 64512 ports, or 2. More than one subscriber is using the same public address, which will occur if there are totally unused public addresses.
No free public addresses in NAT pool	N/A	OK	All port-blocks on all public addresses are in use. This only affects new subscribers, i.e. subscribers that do not already have a paired public address.
Subscriber table full	N/A	OK	Affects new subscribers. The size of the subscriber hash table is configurable. Default is 64k.
Session table full	N/A	OK	Max number of main sessions (tuple is: source/subscriber addr, source port, and protocol aka 3-tuple session). The size of the session table is configurable. Default is 32m.
Dest session table full	N/A	OK	Max number of destination records, or sub-sessions, per main session. Configurable. Default is 64. The tuple is a destination address and port.
Memory allocation failures:			Note that there can be a delay between a chunk of memory no longer being visible in a show output and the memory for that chunk actually being freed. Background garbage collectors and RCU callbacks mean this delay can be in the order of tens of seconds.
Failed to allocate session	N/A	OK
Failed to allocate destination session	N/A	OK
Failed to allocate port block	N/A	OK
Failed to allocate public address	N/A	OK
Failed to allocate subscriber address	N/A	OK
Thread contention errors:
Lost race to insert session into table	N/A	OK	This occurs if two forwarding threads both try and create an identical session (src addr, src port, and protocol) at the same time. The first to be added to the session table 'wins' the race. The losing thread will drop the packet and release the mapping it had previously obtained.
Lost race to insert destination session into table	N/A	OK	This occurs if two forwarding threads both try and create an identical sub-session (dest addr, dest port) on the same main session at the same time. The first to be added to the sub-session table 'wins' the race. The losing thread will drop the packet.
Public address destroyed while waiting for lock	N/A	N/A	No longer applicable.
Subscriber address destroyed while waiting for lock	N/A	OK	This may occur if two forwarding threads try and create new mappings for the same subscriber while at the same time the subscriber table is being cleared. This is unlikely ever to occur.
Packet buffer errors:
IP header not available in message buffer	OK	OK	Pkt does not contain all of the IP header. (Note that the system reassembles IP fragments before CGNAT sees the pkt).
L4 header not available in message buffer	OK	OK	Pkt does not contain all of the layer 4 header (e.g. TCP, UDP etc.). This can also apply to pkts embedded within ICMP error messages. i.e. we will attempt to translate any such embedded pkts if there is enough of the pkt to allow us to do so.
Prepare message buffer for header change failed	OK	OK	It is possible for the system to receive a pkt into multiple pkt buffers. CGNAT will detect this and attempt to coalesce these such that the l3 and l4 headers are in the same pkt buffer. If this fails then the pkt is dropped. This typically happens when there are no pkt buffers in the buffer pool.
Cannot advance beyond end of message buffer	N/A	N/A	No longer applicable.
PCP errors:			In/Out direction is meaningless for PCP. We store any errors in the 'Out' counter for convenience.
PCP invalid or missing argument	N/A	OK
PCP public address and port not available	N/A	OK
Other:
ICMP Echo Request for CGNAT public address	OK	N/A	An ICMP echo request has been sent to a CGNAT public address, and no CGNAT session exists for that flow. Reply with an ICMP echo-reply and drop the request pkt. This is not actually an error. No translation takes place.
Unknown			Should never occur.

show nat pool

Active	Yes/No. A NAT pool may become inactive momentarily as it is reconfigured or deleted
Type	Always 'cgnat'
User count	Number of CGNAT policies using this NAT pool
User addresses	Number of subscribers using this NAT pool. This is derived from CGNAT policy 'Match' config
Addresses:
Address pooling	Always 'paired'. (more options may be available in a future release)
Address allocation	Always 'round-robin'
Address count	Number of addresses in the pool
Contention ratio	Ratio of subscribers to pool addresses. 2.72 means there are 2.72 subscriber addresses for every pool address
Address Ranges:
name (type)	There are two types: 'prefix' and 'range'
Range	First and last address in the configured prefix or range
Address count	Number of addresses in the range
Prefix	If type is 'prefix' then this shows the configured prefix and mask value. The first and last addresses in the prefix are included provided they are not equivalent to .0 or .255, respectively.
Last Allocated Address:	Each eligible port of every pool address may be used separately for TCP, UDP and 'other' (where other is ICMP, UDP-lite and DCCP). So if port 1024 is used by subscriber 'A' for TCP then port 1024 will also be available to subscriber 'A' for UDP.
TCP
UDP
Other
Ports:
Port allocation	'sequential' or 'random'. Configurable. Determines how ports are allocated from the current port-block in-use my a subscriber.
Port range	Port range per pool address that may be used to create mappings. Max range is 1024 - 65535.
Port count	Number of ports in the port range.
Port-block size	Port-block size. Configurable.
Max port-blocks per user	Max number of port-blocks that anyone subscriber can use at any one time. Configurable.
Logging:
Log port-block allocation	No longer in-use.
Log all	No longer in-use.
Translation Mappings:
Active	Active mappings. The number of active mappings should be approximately equal to the number of CGNAT sessions.
Total requests	Total mapping requests
Ok	Total mapping request ok ('total reqs' - 'failures')
Failed	Total mapping request failures
Port Block Allocation:
Active	Active port-blocks
Total	Total port-block allocation requests
Total freed	Total port-blocks that have been released
Total Failures	Total port-block allocation request failures
Failures exceeding max	Port-block request failures caused by a subscriber reaching the max port-blocks per subscriber limit

show cgnat policy

Counts marked with a tilde (~) are approximate since they are only updated approximately every 20 secs from similar counts in the subscriber data structures.

Policy: POLICY1	Policy name is "POLICY1"
Match address-group	Name of address-group that is used to match subscriber addresses. The address-group internal ID is in brackets. The prefix, address, and address-range sub-sections show the specific config items for the group.
Prefix
Address
Address-range
Interface	Interface that the policy is applied to
Priority	The policy priority determines the order in which multiple policies on the same interface are examined. Lowest numerical value is highest priority.
Pool	The NAT pool of public addresses to use for CGNAT mappings
Log all sessions	Status of the per-policy log config. This has been deprecated. It is replaced by the "service nat cgnat log event session" global command
Log select sessions	Deprecated
Active subscribers	Count of currently active subscribers that have matched on this policy.
Active sessions	Approximate number of current active 3-tuple sessions created via this policy.
Sessions created	Total number of main sessions created by subscribers matching this policy.
Sessions destroyed	Total main sessions destroyed.
Active sub-sessions	Approximate number of current active sub-sessions created via this policy. (sub-sessions are the dest addr/port sessions that are per main session). Calculated from 'sub-sessions created' minus 'sub-sessions destroyed'.
Sub-sessions created	Total number of sub-sessions created by subscribers matching this policy.
Sub-sessions destroyed	Total sub-sessions destroyed.
Out, packets	Total outbound translated pkts for subscribers matching this policy
bytes	Total outbound bytes (IP header and payload) for subscribers matching this policy
In, packets	Total inbound translated pkts for subscribers matching this policy
bytes	Total inbound bytes (IP header and payload) for subscribers matching this policy
unknown source	Counts inbound packets whose dest address and port have matched a session, but whose source address and port have failed to match a sub-session. Only applies when sub-sessions are being recorded for this session.
Max Session Rates:	List of (up to) five subscribers with highest session creation rates. The rate and timestamp that rate occurred are shown.

show cgnat subscriber

A data record is maintained in CGNAT for each active subscriber. This persists as long as there are active sessions for that subscriber in the session table. Once all sessions have expired, and all port-blocks returned to their parent public addresses, then the subscriber data record will expire between 20-40 seconds later.

Once the last port-block has expired from a subscriber record then the link with the paired public address will be removed. The next time that subscriber requires a mapping then its likely that a different public address will be paired with the subscriber.

This shows two subscribers. 10.10.1.2 has one 3-tuple sessions only, and no sub-sessions. 10.10.1.5 has one 3-tuple session and four sub-sessions,

Paired address	The public address 'paired' to this subscriber (note that the same public address can be paired to more than one subscriber)
Duration	How long this subscriber has active sessions
Expired	The subscriber data structure is expired a short time after all sessions for that subscriber have closed. This field will display 'Yes' for a short time before the subscriber data structure is freed.
Active sessions	Current active sessions. Calculated from 'created minus destroyed'.
Sessions created	Total number of sessions created
Sessions destroyed	Total number of sessions destroyed
Sessions rate	Sessions creation rates. We record the number of sessions created every 20 sec period for the last 5 mins. The sessions rates for the three periods is determined from this.
20 secs
1 minute
5 minutes
Active sub-sessions	Current active sub-sessions. Calculated from 'created minus destroyed'.
Sub-sessions created	Total number of sub-sessions created
Sub-sessions destroyed	Total number of sub-sessions destroyed
Mapping requests	Total number of mapping requests made by this subscriber
Ok	Successful mapping requests
Failed	Unsuccessful mapping requests
Out, packets	Total outbound translated pkts for this subscriber
bytes
In, packets	Total inbound translated pkts for this subscriber
bytes
unknown source	Counts inbound packets whose dest address and port have matched a session, but whose source address and port have failed to match a sub-session. Only applies when sub-sessions are being recorded for this session.
Port blocks	Number of port-blocks in-use by this subscriber
Ports in-use	Ports in-use per-protocol out of total ports currently available. For example, '1/128' means that 1 port is in-use out of the 128 available to this subscriber.
Active Block number	Each port-block of a public-address has 'block number'. This field denotes from which port-block that ports are currently being allocated.

The following is displayed for each port-block assigned to the subscriber:

Block	Port block number
Public Address	Public addess the port-block was allocated from (currently these will be the same for all port-blocks on a subscriber)
Port Range	The range of ports this port-block contains
Total	Number of ports in the port-block
Used	How many ports are in-use per protocol.

If a port-block representing ports 1024-1151 is assigned to a subscriber then each port in that port-block is available for TCP, UDP and 'other'. In other words, a TCP session can use port 10124, a separate UDP session can also use 1024, and an ICMP echo-request could use ID 10124.

show cgnat public

This shows the CGNAT public addresses. These are assigned out of the NAT pool attached to the CGNAT policy.

Public Address	The public address obtained from the NAT pool configured in the CGNAT policy
Port Range	The total port range available on this address
#Prts	Number of ports in the above port range
Blks Used	Number of port blocks in-use and the total number of port blocks
Blk	Block number. This applies to the fields to the right in the table
Port range	Port range for this block
Proto	Each port block contains three mapping spaces, one for each derived protocol type
Ports	Number of ports being used by this protocol on this address
Bitmaps	Bitmaps of which ports are currently in use in the port-block. Once all ports for all protocols of any port-block are released, then the port-block is freed. For example, for port range 1024-1151 a bitmap of 0x0000000000000005 means that the 1st and 3rd ports of that range are in-use, i.e. ports 1024 and 1026.

show cgnat session

3-tuple session:

5-tuple session:

Session ID	This of the form 'x.y', where 'x' is the main session (3-tuple session) and 'y' is the sub-session (dest addr and port). If no sub-sessions are being recorded then 'x' will be 0.
State	One of: Closed, Opening, Established, or Transitory. TCP sessions have additional states: Client-FIN-Rcv, Server-FIN-Rcv, and CS-FIN-Rcv.
Expired	Expired state of session and sub-session
Interface	Interface the session was created on
Policy name	Parent CGNAT policy that created the session
NAT pool	NAT pool from which the public address mapping was obtained
Timeout	Seconds remaining before the session will timeout. An eligible packet will set this time back to the max value. Eligible packets are any outbound packet that is not one of: TCP reset, any ICMP packet except echo request.
Duration	How long the session has been in existence
Start time	Date and time the session was created
Initial dest port	This is the destination port of the pkt that created the session. (This is redundant information when destination address and port sub-sessions are being recorded. See 'Out' below)
Mapping	Format: 'subscriber addr and port' \| 'Public addr and port', e.g. '10.10.1.2/3001 \| 10.10.3.64/1024' shows that source 10.10.1.2 port 3001 was translated to 10.10.3.64 port 1024.
Out	Outbound source addr/port and destination addr/port before translation 3-tuple sessions will show 'any' for the dest addr/port.
packets	Outbound packet and byte counts
In	Inbound source addr/port and destination addr/port before translation. 3-tuple sessions will show 'any' for the source addr/port.
packets	Inbound packet and byte counts

DANOS

Carrier Grade NAT (CGNAT)

Analytics