Configuration

Simple Example

Required Configuration

Address-group to match subscriber (source) addresses

set resources group address-group AG_MATCH1
set resources group address-group AG_MATCH1 address 10.10.1.0/28
set resources group address-group AG_MATCH1 address-range 10.10.1.20 to 10.10.1.30
set resources group address-group AG_MATCH1 address 10.10.1.10

NAT Pool to specify Public Addresses and how they are Allocated

set service nat pool NAT_POOL1 entry RANGE1 ip-address range start 10.10.3.1
set service nat pool NAT_POOL1 entry RANGE1 ip-address range end 10.10.3.63
set service nat pool NAT_POOL1 entry RANGE2 ip-address prefix 10.10.3.64/28
set service nat pool NAT_POOL1 type CGNAT
set service nat pool NAT_POOL1 address-allocation round-robin
set service nat pool NAT_POOL1 address-pooling paired
set service nat pool NAT_POOL1 port dynamic-block-allocation block-size 128
set service nat pool NAT_POOL1 port dynamic-block-allocation max-blocks-per-subscriber 8
set service nat pool NAT_POOL1 port allocation sequential
set service nat pool NAT_POOL1 port range start 1024
set service nat pool NAT_POOL1 port range end 65535

CGNAT Policy pulls together the 'match' address-group and the NAT pool

set service nat cgnat policy POLICY1 match source address-group AG_MATCH1
set service nat cgnat policy POLICY1 priority 10
set service nat cgnat policy POLICY1 translation pool NAT_POOL1

Assign the CGNAT Policy to the interface which routes to the external network

set service nat cgnat interface dp0p1s2 policy POLICY1

Optional Configuration

Session Timeouts

set service nat cgnat session-timeout tcp partially-open 240
set service nat cgnat session-timeout udp partially-open 240
set service nat cgnat session-timeout other partially-open 20

set service nat cgnat session-timeout udp established 1800
set service nat cgnat session-timeout tcp established 1800
set service nat cgnat session-timeout other established 30

Per-port established-state timeouts are available per port

set service nat cgnat session-timeout tcp port 53 established 10

Logging

set service nat cgnat log event subscriber
set service nat cgnat log event session
set service nat cgnat log event resource-constraint
set service nat cgnat log event port-block-allocation

set service nat pool NAT_POOL1 select event port-block-allocation

set service nat cgnat policy POLICY1 select event session all-subscribers
set service nat cgnat policy POLICY1 select event session creation
set service nat cgnat policy POLICY1 select event session deletion
set service nat cgnat policy POLICY1 select event subscriber

Note that enabling session logging will switch CGNAT to record destination address and port in sub-sessions (2-tuple sessions in tables within 3-tuple sessions).  This will affect performance.

set resources group address-group <name>

CGNAT uses an address-group to match inside subscriber source addresses.   

Parameter

Format

Description

Details

address

<h:h:h:h:h:h:h:h/x>

IPv6 subnet to match

not applicable for CGNAT


<h:h:h:h:h:h:h:h>

IPv6 address to match

not applicable for CGNAT


<x.x.x.x/x>

IP subnet to match

The first and last addresses in the prefix will be matched.  For example,  10.10.1.0/28 will match address range 10.10.1.0 to 10.10.1.15.


<x.x.x.x>

IP address to match


address-range

<h:h:h:h:h:h:h:h> to <h:h:h:h:h:h:h:h>

IPv6 address

not applicable for CGNAT


<x.x.x.x> to <x.x.x.x>

IPv4 address


description

<text>

Address-group description


set service nat pool <name>

A NAT pool is used to specify the public addresses that CGNAT will use to translate inside source addresses to.

Command

Format

Details

entry <range-name> ip-address range start

<x.x.x.x>

IPv4 Address

entry <range-name> ip-address range end

<x.x.x.x>

IPv4 Address

entry <range-name> ip-address prefix

<x.x.x.x/x>

IPv4 address and mask.  First and last addresses are used provided they are not '.0' or '.255'.  For example, 10.10.3.0/28 would give range 10.10.3.1-10.10.3.15.

entry <range-name> ip-address subnet

<x.x.x.x/x>

IPv4 subnet and mask.  First and last addresses are never used.  For example, 10.10.3.0/28 would give range 10.10.3.1-10.10.3.14.

type

CGNAT

Only option.  Required for future compatibility.

address-allocation

round-robin

Addresses are allocated from the NAT pool in a round-robin fashion. Only option.  Required for future compatibility.

address-pooling

paired

Internal IP addresses are paired with external IP addresses.  An internal IP address will never use more than one external IP address at any given time.  Only option.  Required for future compatibility.

port dynamic-block-allocation block-size

<64..4096>

The number of ports in a block (multiple of 64 only).

port dynamic-block-allocation max-blocks-per-subscriber

<1..32>

Maximum number of port-blocks per subscriber

port allocation

random

Allocate ports randomly from within the current 'active' port-block.


sequential

Allocate ports sequentially from within the current 'active' port-block.

port range start

<0..65535>

Start of port rang.  It is recommended that this is set no lower than 1024.

range end

<0..65535>

End of port range.

blacklist address-group

<addr-grp-name>

The address-group identifying the pool addresses being blacklisted.  Adding external IP addresses or prefixes to a blacklist address group will cause sessions creating using that external address to be cleared, and for that address to no longer be used to create new sessions.   It is useful on a live system where the administrator wants to prevent an address in the NAT pool form being used for CGNAT mappings.

This has an advantage over reconfiguring the NAT pool range to exclude that address in that reconfiguring the NAT pool range will cause all sessions using that NAT pool to be torn down.

set service nat cgnat policy <name>

A CGNAT policy ties together an address-group for matching inside source addresses with a NAT pool for specifying the set of external public addresses to use.   A CGNAT policy is the thing that is attached to an interface,

Command

Format

Description

match source address-group

<addr-grp-name>

Address group name

translation pool

<nat-pool-name>

NAT pool name

priority

<1..9999>

The priority of CGNAT policy.  Multiple CGNAT policies on an interface are evaluated in order of priority, with the lowest numerical priority first.

set service cgnat session-timeout

Established-state timeout values may be configured per port (a destination port in outbound traffic).





other

partially-open

<10..240>

Non-TCP/UDP 'partially-open' timeout in seconds


established

<30..1800>

Non-TCP/UDP 'established' timeout in seconds

udp

partially-open

<10..240>

UDP 'partially-open' timeout in seconds


established

<30..1800>

UDP 'established' timeout in seconds

tcp

partially-open

<10..240>

TCP 'partially-open' timeout in seconds


partially-closed

<10..240>

TCP 'partially-closed' timeout in seconds


established

<30..14400>

TCP 'established' timeout in seconds

Other configuration commands

set service cgnat ...




cpu-affinity event session

<0..65535>

Number of the core to process session events on

disable-hairpinning


Disable hairpinning in CGNAT device

export event ... 


Used to control logging

max-dest-per-session

<1..64>

Max number of destination records that can be associated with a session

max-sessions

<1..33554432>

System-wide maximum number of sessions

select warning event resource-constraint ...



snat-alg-bypass


Enable SNAT ALG traffic to bypass CGNAT

Operational Commands

show cgnat summary

The error and summary statistics will count every packet that matches either of the following two criteria:

  1. outbound packet whose source address matches a CGNAT policy, or

  2. inbound packet who's destination address is a CGNAT public address

CGNAT Summary
  Sessions:                       
    Active sessions                                 2
      Sessions created                             ~2
      Sessions destroyed                           ~0
    Active sub-sessions                             4
      Sub-sessions created                         ~4
      Sub-sessions destroyed                       ~0
    Maximum table size                       33554432
    Table full                                     No
  Public address mapping table:   
    Used                                            2
  Subscriber address table:       
    Used                                            2
    Max                                         65536
  Out:                            
    Translated packets                            ~12
               bytes                             ~336
    Did not match CGNAT policy                      0
    Untranslatable packets                          0
    Hairpinned packets                              0
  In:                             
    Translated packets                             ~0
               bytes                               ~0
    Unknown source addr or port                    ~0
    Did not match CGNAT session                     0
    Did not match CGNAT pool                        0
  Session hash tables:            
    Created                                         1
    Destroyed                                       0
  PCP sessions created                              0
  PCP errors                                        0
  Memory allocation failures                        0
  Resource limitation failures                      0
  Thread contention errors                          0
  Packet buffer errors                              0

Counts marked with a tilde (~) are approximate.  These are collated from each CGNAT policy every time the "show cgnat summary" command is run.  The policy counts are only updated approximately every 20 secs from similar counts in the subscriber data structures.

Sessions


    Active sessions

Active sessions.  An original global atomic counter.  This is not  simply 'sessions created less sessions destroyed'.

        Sessions created

Sum of 'sessions created' counts on all CGNAT policies. 

        Sessions destroyed

Sum of 'sessions destroyed' counts on all CGNAT policies.

    Active sub-sessions

Count of current sub-sessions.  An original global atomic counter.  This is not  simply 'sub-sessions created less sub-sessions destroyed'.

        Sub-sessions created

Sum of 'sub-sessions created' counts on all CGNAT policies. 

        Sub-sessions destroyed

Sum of 'sub-sessions destroyed' counts on all CGNAT policies.

    Maximum table size

Configurable maximum session table size.  (This value applies to sessions, not  sessions plus sub-sessions)

    Table full

Set when the 'active sessions' count exceeds 'Maximum table size'.

Public address mapping table


    Used

Active public addresses in use by CGNAT.  An original global atomic counter.  There is no maximum value for this table.  Its size is constrained by the number of addresses in the NAT pool used by CGNAT.

Subscriber address table


    Used

Active CGNAT subscribers.  An original global atomic counter. 

    Max

Maximum CGNAT subscribers allowed in the subscriber address table.   Configurable.  Default 64k.

Out


    Translated packets

Count of outbound packets translated. Sum of 'Out packets' counts on all CGNAT policies. 

    Translated bytes

Total byte count of the outbound translated packets.  (IP header and payload). Sum of 'Out bytes' counts on all CGNAT policies.

    Did not match CGNAT policy

Subscriber address did not match any CGNAT policy on the outbound interface.

    Untranslatable packets

Untranslatable packets.  Sum of error counts 'Untranslatable IP protocol' and 'Untranslatable ICMP message'

    Hairpinned packets

Count of hairpinned packets. 

In


    Translated packets

Count of inbound packets translated. Sum of 'In packets' counts on all CGNAT policies. 

    Translated bytes

Total byte count of the inbound translated packets.  (IP header and payload). Sum of 'In bytes' counts on all CGNAT policies.

    Unknown source addr or port

Sum of 'In, unknown source' counts on all CGNAT policies.    Counts inbound packets whose dest address and port have matched a session, but whose source address and port have failed to match a sub-session.  Only applies when sub-sessions are being recorded for this session.

    Did not match CGNAT session

Destination address, port and protocol did not match a CGNAT session and the destination address does not match a CGNAT public address.

    Did not match CGNAT pool

Destination address does not match a CGNAT public address.

Session Hash Tables


    Created

A hash table may be created for each main session in order to store sub-sessions if:

  1. Destination addr and port are being recorded, and

  2. More than one destination address and port is seen for the same source address, port, and protocol. 

Creation and maintenance of these tables is an expensive operation.   If many hash tables are being created and destroyed then performance may degrade somewhat.   These counters are designed to provide some indication that this may be occurring.   Typically multiple sub-sessions per main session will occur with peer-to-peer gaming.

    Destroyed




PCP sessions created

Sessions created by the Port Control Protocol

PCP errors

Failures when trying to create a PCP session.

Memory allocation failures

Sum of all memory allocation failures.  Use "show cgnat error" to see details.

Resource limitation failures

Sum of all resource limitation failures.  Use "show cgnat error" to see details.

Thread contention errors

Sum of all thread contention errors.  Use "show cgnat error" to see details.

Packet buffer errors

Sum of all packet buffer errors.  Use "show cgnat error" to see details.

show cgnat error

$ show cgnat error
CGNAT Global Counts                                                  In          Out
  Unable to translate packet:
    Subscriber address did not match a CGNAT policy                   0            0
    Packet did not match a CGNAT session                              0            0
    Destination address did not match CGNAT pool                      0            0
    CGNAT bypassed by SNAT-ALG packets                                0            0
    Untranslatable IP protocol                                        0            0
    Untranslatable ICMP message                                       0            0
  Resource limitations:
    Subscriber port-block limit                                       0            0
    No free port-blocks on selected public address                    0            0
    No free public addresses in NAT pool                              0            0
    Subscriber table full                                             0            0
    Session table full                                                0            0
    Dest session table full                                           0            0
  Memory allocation failures:
    Failed to allocate session                                        0            0
    Failed to allocate destination session                            0            0
    Failed to allocate port block                                     0            0
    Failed to allocate public address                                 0            0
    Failed to allocate subscriber address                             0            0
  Thread contention errors:
    Lost race to insert session into table                            0            0
    Lost race to insert destination session into table                0            0
    Subscriber address destroyed while waiting for lock               0            0
  Packet buffer errors:
    IP header not available in message buffer                         0            0
    L4 header not available in message buffer                         0            0
    Prepare message buffer for header change failed                   0            0
  PCP errors:            .
    PCP invalid or missing argument                                   0            0
    PCP public address and port not available                         0            0
  Other:
    ICMP Echo Request for CGNAT public address                        0            0
    Unknown                                                           0            0

N/A = Means that count is not applicable in this direction
OK = Means that count is applicable in this direction

Unable to translate packet:

In

Out

Comment

  Subscriber address did not match a CGNAT policy

N/A

OK

Subscriber address did not match a CGNAT policy

  Packet did not match a CGNAT session

OK

N/A

Inbound pkts that do not match a CGNAT session and whose dest addr does match a CGNAT policy.  These are dropped.

  Destination address did not match CGNAT pool

OK

N/A

Inbound pkts that do  not  match a CGNAT session and whose dest addr does not  match a CGNAT policy.   These are forwarded untranslated.

  CGNAT bypassed by SNAT-ALG packets

N/A

OK

Pkt do not  match a CGNAT session and does not match a CGNAT policy and SNAT is cfgd and pkt matches an ALG session or tuple.  Pkt is forwarded unchanged.

  Untranslatable IP protocol

OK

OK

Protocol is not  one of the following: TCP, UDP, UDP-lite, DCCP or ICMP

  Untranslatable ICMP message

OK

OK

ICMP type is not  one of: echo-req, echo-reply, dest unreach, redirect, time-exceeded, or parameter-problem

Resource limitations:




  Subscriber port-block limit

N/A

OK

CGNAT policy config contains the max number of port-blocks that anyone subscriber may use at any one time.  Once this is reached, further mapping requests will fail, and this count will increment.

  No free port-blocks on the selected public address

N/A

OK

Counts mapping failures that occur when a subscriber is within his port-block limit but there are no free port-blocks on the paired public address.  This can occur when 1. the max port-blocks per subscriber and port-block size equate to more than 64512 ports, or 2. More than one subscriber is using the same public address, which will occur if there are totally unused public addresses.

  No free public addresses in NAT pool

N/A

OK

All port-blocks on all public addresses are in use.  This only affects new subscribers, i.e. subscribers that do not already have a paired public address.

  Subscriber table full

N/A

OK

Affects new subscribers.  The size of the subscriber hash table is configurable.  Default is 64k.

  Session table full

N/A

OK

Max number of main sessions (tuple is: source/subscriber addr, source port, and protocol aka 3-tuple session). The size of the session table is configurable.  Default is 32m.

  Dest session table full

N/A

OK

Max number of destination records, or sub-sessions, per main session.  Configurable.  Default is 64.    The tuple is a destination address and port.

Memory allocation failures:



Note that there can be a delay between a chunk of memory no longer being visible in a show output and the memory for that chunk actually being freed.  Background garbage collectors and RCU callbacks mean this delay can be in the order of tens of seconds.

  Failed to allocate session

N/A

OK


  Failed to allocate destination session

N/A

OK


  Failed to allocate port block

N/A

OK


  Failed to allocate public address

N/A

OK


  Failed to allocate subscriber address

N/A

OK


Thread contention errors:




  Lost race to insert session into table

N/A

OK

This occurs if two forwarding threads both try and create an identical session (src addr, src port, and protocol) at the same time.   The first to be added to the session table 'wins' the race.  The losing thread will drop the packet and release the mapping it had previously obtained.

  Lost race to insert destination session into table

N/A

OK

This occurs if two forwarding threads both try and create an identical sub-session (dest addr, dest port) on the same main session at the same time.   The first to be added to the sub-session table 'wins' the race.  The losing thread will drop the packet.

  Public address destroyed while waiting for lock

N/A

N/A

No longer applicable.

  Subscriber address destroyed while waiting for lock

N/A

OK

This may occur if two forwarding threads try and create new mappings for the same subscriber while at the same time the subscriber table is being cleared.   This is unlikely ever to occur.

Packet buffer errors:




  IP header not available in message buffer

OK

OK

Pkt does not contain all of the IP header.  (Note that the system reassembles IP fragments before CGNAT sees the pkt).

  L4 header not available in message buffer

OK

OK

Pkt does not contain all of the layer 4 header (e.g. TCP, UDP etc.).  This can also apply to pkts embedded within ICMP error messages.  i.e. we will attempt to translate any such embedded pkts if there is enough of the pkt to allow us to do so.

  Prepare message buffer for header change failed

OK

OK

It is possible for the system to receive a pkt into multiple pkt buffers.    CGNAT will detect this and attempt to coalesce these such that the l3 and l4 headers are in the same pkt buffer.    If this fails then the pkt is dropped.   This typically happens when there are no pkt buffers in the buffer pool.

  Cannot advance beyond end of message buffer

N/A

N/A

No longer applicable.

PCP errors:



In/Out direction is meaningless for PCP.  We store any errors in the 'Out' counter for convenience.

  PCP invalid or missing argument

N/A

OK


  PCP public address and port not available

N/A

OK


Other:




  ICMP Echo Request for CGNAT public address

OK

N/A

An ICMP echo request has been sent to a CGNAT public address, and no CGNAT session exists for that flow.   Reply with an ICMP echo-reply and drop the request pkt.   This is not actually an error.  No translation takes place.

  Unknown



Should never occur.

show nat pool

v@vm-rtr2-1:~$ show nat pool
NAT Pool POOL1
  Active                                          Yes
  Type                                          cgnat
  User count                                        1
    User addresses                                 29
  Addresses:                      
    Address pooling                            paired
    Address allocation                    round-robin
    Address count                                  79
    Contention ratio                           1:2.72
    Address Ranges:               
      RANGE2 (prefix)
        Range                 10.10.3.64 - 10.10.3.79
        Address count                              16
        Prefix:                         10.10.3.64/28
      RANGE1 (range)
        Range                  10.10.3.1 - 10.10.3.63
        Address count                              63
  Last Allocated Address:         
    TCP                                          none
    UDP                                    10.10.3.65
    Other                                        none
  Ports:                          
    Port allocation                        sequential
    Port range                             1024-65535
    Port count                                  64512
    Port-block size                               128
    Max port-blocks per user                        8
  Logging:                        
    Log port-block allocation                     Yes
    Log all                                        No
  Translation Mappings:           
    Active                                          2
    Total requests                                  2
      Ok                                            2
      Failed                                        0
  Port Block Allocation:          
    Active                                          2
    Total                                           2
    Total Freed                                     0
    Total Failures                                  0
    Failures exceeding max                          0

Active

Yes/No.  A NAT pool may become inactive momentarily as it is reconfigured or deleted

Type

Always 'cgnat'

User count

Number of CGNAT policies using this NAT pool

    User addresses

Number of subscribers using this NAT pool.   This is derived from CGNAT policy 'Match' config

Addresses:


    Address pooling

Always 'paired'.  (more options may be available in a future release)

    Address allocation

Always 'round-robin'

    Address count

Number of addresses in the pool

    Contention ratio

Ratio of subscribers to pool addresses.  2.72 means there are 2.72 subscriber addresses for every pool address

    Address Ranges:


        name (type)

There are two types: 'prefix' and 'range'

        Range

First and last address in the configured prefix or range

        Address count

Number of addresses in the range

        Prefix

If type is 'prefix' then this shows the configured prefix and mask value.   The first and last addresses in the prefix are included provided they are not equivalent to .0 or .255, respectively.

Last Allocated Address:

Each eligible port of every pool address may be used separately for TCP, UDP and 'other' (where other is ICMP, UDP-lite and DCCP).  So if port 1024 is used by subscriber 'A' for TCP then port 1024 will also be available to subscriber 'A' for UDP. 

    TCP


    UDP


    Other


Ports:


    Port allocation

'sequential' or 'random'.  Configurable.  Determines how ports are allocated from the current port-block in-use my a subscriber.

    Port range

Port range per pool address that may be used to create mappings.   Max range is 1024 - 65535.

    Port count

Number of ports in the port range.

    Port-block size

Port-block size.  Configurable.

    Max port-blocks per user

Max number of port-blocks that anyone subscriber can use at any one time.  Configurable.

Logging:


    Log port-block allocation

No longer in-use.

    Log all

No longer in-use.

Translation Mappings:


    Active

Active mappings.  The number of active mappings should be approximately equal to the number of CGNAT sessions.

    Total requests

Total mapping requests

        Ok

Total mapping request ok ('total reqs' - 'failures')

        Failed

Total mapping request failures

Port Block Allocation:


    Active

Active port-blocks

    Total

Total port-block allocation requests

    Total freed

Total port-blocks that have been released

    Total Failures

Total port-block allocation request failures

    Failures exceeding max

Port-block request failures caused by a subscriber reaching the max port-blocks per subscriber limit

show cgnat policy

v@vm-rtr2-1:~$ show cgnat policy
Policy: POLICY1
  Match address-group                  SRC_MATCH1 (1)
    Prefix                               10.10.1.0/28
    Address                                10.10.1.10
    Address                                10.10.1.11
    Address-range            10.10.1.20 to 10.10.1.30
  Interface                                   dp0p1s2
  Priority                                         10
  Pool                                          POOL1
  Log all sessions                                 No
  Log select sessions                    LOG_GRP1 (0)
      Address                               10.10.1.3
      Address                               10.10.1.5
    Log session start                             Yes
    Log session end                               Yes
    Log session periodically                       No
  Active subscribers                                2
  Active sessions                                  ~2
    Sessions created                               ~2
    Sessions destroyed                             ~0
  Active sub-sessions                              ~4
    Sub-sessions created                           ~4
    Sub-sessions destroyed                         ~0
  Out, packets                                    ~12
       bytes                                     ~336
  In,  packets                                     ~0
       bytes                                       ~0
       unknown source                              ~0
  Max Session Rates:
    Subscriber       Max Rate  Time    
    10.10.1.2               1  2020-03-30 09:39:07 +0000
    10.10.1.5               1  2020-03-30 09:39:07 +0000

Counts marked with a tilde (~) are approximate since they are only updated approximately every 20 secs from similar counts in the subscriber data structures.

Policy: POLICY1

Policy name is "POLICY1"

Match address-group

Name of address-group that is used to match subscriber addresses.  The address-group internal ID is in brackets.

The prefix, address, and address-range sub-sections show the specific config items for the group.

    Prefix


    Address


    Address-range


Interface

Interface that the policy is applied to

Priority

The policy priority determines the order in which multiple policies on the same interface are examined.   Lowest numerical value is highest priority.

Pool

The NAT pool of public addresses to use for CGNAT mappings

Log all sessions

Status of the per-policy log config.  This has been deprecated.  It is replaced by the "service nat cgnat log event session" global command

Log select sessions

Deprecated

Active subscribers

Count of currently active subscribers that have matched on this policy.

Active sessions

Approximate number of current active 3-tuple sessions created via this policy. 

    Sessions created

Total number of main sessions created by subscribers matching this policy.  

    Sessions destroyed

Total main sessions destroyed. 

Active sub-sessions

Approximate number of current active sub-sessions created via this policy.  (sub-sessions are the dest addr/port sessions that are per main session). Calculated from 'sub-sessions created' minus 'sub-sessions destroyed'.  

    Sub-sessions created

Total number of sub-sessions created by subscribers matching this policy.  

    Sub-sessions destroyed

Total sub-sessions destroyed. 

Out, packets

Total outbound translated pkts for subscribers matching this policy

    bytes

Total outbound bytes (IP header and payload) for subscribers matching this policy

In, packets

Total inbound translated pkts for subscribers matching this policy

    bytes

Total inbound bytes (IP header and payload) for subscribers matching this policy

    unknown source

Counts inbound packets whose dest address and port have matched a session, but whose source address and port have failed to match a sub-session.  Only applies when sub-sessions are being recorded for this session.

Max Session Rates:

List of (up to) five subscribers with highest session creation rates.   The rate and timestamp that rate occurred are shown.

show cgnat subscriber

A data record is maintained in CGNAT for each active subscriber.  This persists as long as there are active sessions for that subscriber in the session table.   Once all sessions have expired, and all port-blocks returned to their parent public addresses, then the subscriber data record will expire between 20-40 seconds later.

Once the last port-block has expired from a subscriber record then the link with the paired public address will be removed.   The next time that subscriber requires a mapping then its likely that a different public address will be paired with the subscriber.

v@vm-rtr-1:~$ show cgnat subscriber 
Possible completions:
  <Enter>                    Execute the current command
  <IPv4-address>             An IPv4 address
  <IPv4-address>/<mask-len>  An IPv4 address with mask length
  detail                     Show detailed CGNAT subscriber information

This shows two subscribers.  10.10.1.2 has one 3-tuple sessions only, and no sub-sessions.  10.10.1.5 has one 3-tuple session and four sub-sessions,

v@vm-rtr2-1:~$ show cgnat subscriber
     Subscriber      Paired Addr            Sessions  Blks  Ports    Map Reqs  Map Fails  Duration
      10.10.1.2       10.10.3.64                   1     1      1           1          0  00:01:37 (hh:mm:ss)
      10.10.1.5       10.10.3.65                 1/4     1      1           1          0  00:01:37 (hh:mm:ss)
v@vm-rtr2-1:~$ show cgnat subscriber 10.10.1.2 detail
Subscriber: 10.10.1.2
  Paired address                           10.10.3.64
  Duration                                   00:01:46 (hh:mm:ss)
  Expired                                          No
  Active sessions                                   1
    Sessions created                                1
    Sessions destroyed                              0
  Session rate:               Current         Maximum
    20 secs                         0               1 (2020-03-30 09:39:07 +0000)
    1 minute                        0               1 (2020-03-30 09:39:07 +0000)
    5 minutes                       1
  Active sub-sessions                               0
    Sub-sessions created                            0
    Sub- sessions destroyed                         0
  Mapping requests                                  1
    Ok                                              1
    Failed                                          0
  Out, packets                                     ~4
       bytes                                     ~112
  In,  packets                                     ~0
       bytes                                       ~0
       unknown source                              ~0
  Port blocks                                       1
  Ports in-use:               Active Block number:   
    TCP               0/128                         0
    UDP               1/128                         0
    Other             0/128                         0
  Block    Public Address    Port Range   Total    Used (tcp/udp/other)
      0        10.10.3.64     1024-1151     128       1 (0/1/0)

v@vm-rtr2-1:~$ show cgnat subscriber 10.10.1.5 detail
Subscriber: 10.10.1.5
  Paired address                           10.10.3.65
  Duration                                   00:01:51 (hh:mm:ss)
  Expired                                          No
  Active sessions                                   1
    Sessions created                                1
    Sessions destroyed                              0
  Session rate:               Current         Maximum
    20 secs                         0               1 (2020-03-30 09:39:07 +0000)
    1 minute                        0               1 (2020-03-30 09:39:07 +0000)
    5 minutes                       1
  Active sub-sessions                               4
    Sub-sessions created                            4
    Sub- sessions destroyed                         0
  Mapping requests                                  1
    Ok                                              1
    Failed                                          0
  Out, packets                                     ~8
       bytes                                     ~224
  In,  packets                                     ~0
       bytes                                       ~0
       unknown source                              ~0
  Port blocks                                       1
  Ports in-use:               Active Block number:   
    TCP               0/128                         0
    UDP               1/128                         0
    Other             0/128                         0
  Block    Public Address    Port Range   Total    Used (tcp/udp/other)
      0        10.10.3.65     1024-1151     128       1 (0/1/0)

Paired address

The public address 'paired' to this subscriber (note that the same public address can be paired to more than one subscriber)

Duration

How long this subscriber has active sessions

Expired

The subscriber data structure is expired a short time after all sessions for that subscriber have closed.  This field will display 'Yes' for a short time before the subscriber data structure is freed. 

Active sessions

Current active sessions.  Calculated from 'created minus destroyed'.

    Sessions created

Total number of sessions created

    Sessions destroyed

Total number of sessions destroyed

Sessions rate

Sessions creation rates.  We record the number of sessions created every 20 sec period for the last 5 mins.   The sessions rates for the three periods is determined from this.

    20 secs


    1 minute


    5 minutes


Active sub-sessions

Current active sub-sessions.  Calculated from 'created minus destroyed'.

    Sub-sessions created

Total number of sub-sessions created

    Sub-sessions destroyed

Total number of sub-sessions destroyed

Mapping requests

Total number of mapping requests made by this subscriber

    Ok

Successful mapping requests

    Failed

Unsuccessful mapping requests

Out, packets

Total outbound translated pkts for this subscriber

    bytes


In, packets

Total inbound translated pkts for this subscriber

    bytes


    unknown source

Counts inbound packets whose dest address and port have matched a session, but whose source address and port have failed to match a sub-session.  Only applies when sub-sessions are being recorded for this session.

Port blocks

Number of port-blocks in-use by this subscriber

Ports in-use

Ports in-use per-protocol out of total ports currently available.  For example, '1/128' means that 1 port is in-use out of the 128 available to this subscriber.

Active Block number

Each port-block of a public-address has 'block number'.  This field denotes from which port-block that ports are currently being allocated.

The following is displayed for each port-block assigned to the subscriber:

Block

Port block number

Public Address

Public addess the port-block was allocated from (currently these will be the same for all port-blocks on a subscriber)

Port Range

The range of ports this port-block contains

Total

Number of ports in the port-block

Used

How many ports are in-use per protocol.

If a port-block representing ports 1024-1151 is assigned to a subscriber then each port in that port-block is available for TCP, UDP and 'other'.   In other words, a TCP session can use port 10124, a separate UDP session can also use 1024, and an ICMP echo-request could use ID 10124.

show cgnat public

This shows the CGNAT public addresses.  These are assigned out of the NAT pool attached to the CGNAT policy.

v@vm-rtr-1:~$ show cgnat public 
Possible completions:
  <Enter>                    Execute the current command
  <IPv4-address>             An IPv4 address
  <IPv4-address>/<mask-len>  An IPv4 address with mask length
  detail                     Show detailed CGNAT public address information
v@vm-rtr-1:~$ show cgnat public
 Public Address  Port Range #Prts Blks Used  Blk  Port range   TCP   UDP Other
      10.10.3.1  1024-65535 64512     2/504    0   1024-1151     0   128     0
                                               1   1152-1279     0     1     0
v@vm-rtr-1:~$ show cgnat public detail
 Public Address  Port Range #Prts Blks Used  Blk  Port range Proto Ports            Bitmaps
      10.10.3.1  1024-65535 64512     2/504    0   1024-1151   tcp     0 0x0000000000000000
                                                                         0x0000000000000000
                                                               udp   128 0xffffffffffffffff
                                                                         0xffffffffffffffff
                                                             other     0 0x0000000000000000
                                                                         0x0000000000000000
                                               1   1152-1279   tcp     0 0x0000000000000000
                                                                         0x0000000000000000
                                                               udp     1 0x0000000000000001
                                                                         0x0000000000000000
                                                             other     0 0x0000000000000000
                                                                         0x0000000000000000

Public Address

The public address obtained from the NAT pool configured in the CGNAT policy

Port Range

The total port range available on this address

#Prts

Number of ports in the above port range

Blks Used

Number of port blocks in-use and the total number of port blocks

Blk

Block number.  This applies to the fields to the right in the table

Port range

Port range for this block

Proto

Each port block contains three mapping spaces, one for each derived protocol type

Ports

Number of ports being used by this protocol on this address

Bitmaps

Bitmaps of which ports are currently in use in the port-block.   Once all ports for all protocols of any port-block are released, then the port-block is freed.   

For example, for port range 1024-1151 a bitmap of 0x0000000000000005 means that the 1st and 3rd ports of that range are in-use, i.e. ports 1024 and 1026.

show cgnat session

v@vm-rtr-1:~$ show cgnat session 
Possible completions:
  <Enter>              Execute the current command
  count                The requested number of entries to return
  destination-address  Match destination IPv4 address with optional mask
  destination-port     Match destination port
  detail               Show detailed session information
  exclude-inner        Exclude inner per-destination sessions
  interface            Match interface name
  pool                 Match pool name
  protocol             Match protocol
  public-address       Match public IPv4 address with optional mask
  public-port          Match public port
  session-id           Match session identifier
  sub-session-id       Match sub-session identifier
  subscriber-address   Match subscriber IPv4 address with optional mask
  subscriber-port      Match subscriber port
  unordered            Return sessions unordered
v@vm-rtr2-1:~$ show cgnat session
State codes: CL - CLOSED, OP - OPENING, ES - ESTABLISHED, TR - TRANSITORY, CG - CLOSING

                                 Subscriber                Public                      Destination
ID     Proto State            Address  Port         Address  Port       Intf         Address  Port Timeout PktOut  PktIn
1.0    udp   OP             10.10.1.2  3001      10.10.3.64  1024    dp0p1s2               -    80      83      4      0
2.1    udp   OP             10.10.1.5  4001      10.10.3.65  1024    dp0p1s2       10.10.2.2     9      83      2      0
2.2    udp   OP             10.10.1.5  4001      10.10.3.65  1024    dp0p1s2       10.10.2.3    10      83      2      0
2.3    udp   OP             10.10.1.5  4001      10.10.3.65  1024    dp0p1s2       10.10.2.4    11      83      2      0
2.4    udp   OP             10.10.1.5  4001      10.10.3.65  1024    dp0p1s2       10.10.2.5    12      83      2      0

3-tuple session:

v@vm-rtr2-1:~$ show cgnat session subscriber-address 10.10.1.2 detail
Session ID: 1.0, State: Opening, Expired: No/-
  Interface:                      dp0p1s2
  Policy name:                    POLICY1
  NAT pool:                         POOL1
  Timeout:                             69 (max 240)
  Duration:                      00:03:00 (hh:mm:ss)
  Start time:         2020-03-30 09:38:53
  Initial dest port:                   80
  Mapping:   10.10.1.2/3001 | 10.10.3.64/1024
  Out:       10.10.1.2/3001 --> any, Proto: udp
    packets: 4, bytes: 112
  In:        any --> 10.10.3.64/1024, Proto: udp
    packets: 0, bytes: 0
    unknown source: 0

5-tuple session:

v@vm-rtr2-1:~$ show cgnat session subscriber-address 10.10.1.5 destination-port 9 detail
Session ID: 2.1, State: Opening, Expired: No/No
  Interface:                      dp0p1s2
  Policy name:                    POLICY1
  NAT pool:                         POOL1
  Timeout:                             55 (max 240)
  Duration:                      00:03:20 (hh:mm:ss)
  Start time:         2020-03-30 09:39:18
  Initial dest port:                    9
  Mapping:   10.10.1.5/4001 | 10.10.3.65/1024
  Out:       10.10.1.5/4001 --> 10.10.2.2/9, Proto: udp
    packets: ~2, bytes: ~56
  In:        10.10.2.2/9 --> 10.10.3.65/1024, Proto: udp
    packets: ~0, bytes: ~0

Session ID

This of the form 'x.y', where 'x' is the main session (3-tuple session) and 'y' is the sub-session (dest addr and port).   If no sub-sessions are being recorded then 'x' will be 0.

State

One of: Closed, Opening, Established, or Transitory.   TCP sessions have additional states: Client-FIN-Rcv, Server-FIN-Rcv, and CS-FIN-Rcv.

Expired

Expired state of session and sub-session

Interface

Interface the session was created on

Policy name

Parent CGNAT policy that created the session

NAT pool

NAT pool from which the public address mapping was obtained

Timeout

Seconds remaining before the session will timeout.  An eligible packet will set this time back to the max value.  Eligible packets are any outbound packet that is not one of: TCP reset, any ICMP packet except echo request.

Duration

How long the session has been in existence

Start time

Date and time the session was created

Initial dest port

This is the destination port of the pkt that created the session.   (This is redundant information when destination address and port sub-sessions are being recorded.  See 'Out' below)

Mapping

Format: 'subscriber addr and port' | 'Public addr and port',
e.g. '10.10.1.2/3001 | 10.10.3.64/1024' shows that source 10.10.1.2 port 3001 was translated to 10.10.3.64 port 1024.

Out

Outbound source addr/port and destination addr/port before translation
3-tuple sessions will show 'any' for the dest addr/port.

    packets

Outbound packet and byte counts

In

Inbound source addr/port and destination addr/port before translation.
3-tuple sessions will show 'any' for the source addr/port.

    packets

Inbound packet and byte counts