-
Notifications
You must be signed in to change notification settings - Fork 232
Functional Block: GK
The Gatekeeper (GK) block can be considered the main component of Gatekeeper servers, as its task is to accept incoming packets, lookup policy decisions on those packets, and queue requests and granted packets for transmission to Grantor. It is in the data plane and can scale across multiple lcores. It only runs when the Gatekeeper executable is being run as a Gatekeeper server (as opposed to a Grantor Server).
Each instance of the GK block listens on a separate queue and processes packets that arrive at that queue. We utilize RSS to distribute incoming packets among the GK instances. RSS also provides a guarantee that packets belonging to the same flow to the will be directed to the same queue. Therefore, each GK instance maintains a flow table without locks because each block is the only writer and reader of this table. However, all the GK instances look up policy decisions in a global LPM table.
The basic algorithm of the GK block is to repeatedly load a set of packets from the front interface, and for each packet in the set do the following:
- If the pair of source and destination addresses is in the flow table, proceed as the forwarding entry instructs (send request to Grantor, transmit packet according to some rate limit, or drop packet).
- Otherwise, look up the destination address in the global LPM table.
- If there is an entry for the destination, initialize an entry in the flow table. Proceed with the policy decision that the LPM entry instructs: send request to Grantor, transmit packet according to some rate limit, or drop packet.
- Otherwise, drop the packet.
To run the executable as a Gatekeeper server (as opposed to a Grantor server), then local gatekeeper_server in lua/main_config.lua should be set to true
To change the number of lcores assigned to the GK block (i.e. the number of GK instances), change the variable local n_lcores in lua/main_config.lua.
All other static configuration variables can be configured in lua/gk.lua.
These variables are likely to change from deployment-to-deployment based on the operator's preferences.
log_level
The log level for the GK block. Can be set to any one of the following values: RTE_LOG_EMERG, RTE_LOG_ALERT, RTE_LOG_CRIT, RTE_LOG_ERR, RTE_LOG_WARNING, RTE_LOG_NOTICE, RTE_LOG_INFO, RTE_LOG_DEBUG.
Since we typically use RTE_LOG_ERROR as the most severe log condition, we recommend not to set this value below RTE_LOG_ERROR.
It is not crucial to change these variables, and they only need to be changed to fine tune the performance of Gatekeeper. Otherwise, the default values are likely fine.
mailbox_max_entries_exp
The log (base 2) of the maximum size of the GK mailbox. For example, if the variable is set to 7, then room for 2^7 = 128 entries will be made in the mailbox.
Also used to determine how many entries will actually be available for use in the mailbox, which for efficiency reasons is one less than the maximum size of the mailbox (127 in the example above).
mailbox_mem_cache_size
Number of mailbox entries to keep in the cache for more efficient use of the mailbox. Set to 0 to disable the cache of the memory pool for the mailbox.
mailbox_burst_size
Maximum number of entries to receive in a burst every time the mailbox is checked.
log_ratelimit_interval_ms
The interval at which logs are rate limited (in milliseconds). For a given interval, only log_ratelimit_burst log entries are permitted. The count of entries is reset for each new interval.
log_ratelimit_burst
The number of entries per interval allowed to be logged. When the number of log entries exceeds this limit in a given interval, the entries will be dropped.
max_pkt_burst_front & max_pkt_burst_back
A suggestion for the maximum number of packets received in each burst on the front and back interfaces in the GK instances, respectively. For each interface, this parameter is only a suggestion since the true burst size is chosen as the maximum between this configured value and the number of ports that compose in the interface.
- flow_ht_size
- flow_table_full_scan_ms
Gatekeeper uses the DPDK Hash library for its flow table. Each GK instance maintains its own flow table for flows that it has handled.
flow_ht_size specifies the size of the flow hash table for each GK instance.
flow_table_full_scan_ms is the interval at which a complete scan of flow table occurs (in milliseconds). Each instance's flow table is pruned of expired entries periodically.
- max_num_ipv4_rules & max_num_ipv6_rules
- num_ipv4_tbl8s & num_ipv6_tbl8s
- max_num_ipv6_neighbors
- max_num_ipv4_fib_entries & max_num_ipv6_fib_entries
For its global LPM table shared among the GK instances, Gatekeeper uses the DPDK LPM library. This library implements the DIR-24-8 algorithm using two types of tables: (1) tbl24, a table with 2^24 entries; (2) tbl8, a table with 2^8 entries. To configure an LPM component instance, one needs to specify: (1) the maximum number of rules to support and (2) the number of tbl8 tables.
max_num_ipv4_rules and max_num_ipv6_rules represent the maximum number of rules to support for the IPv4 and IPv6 LPM tables, respectively.
num_ipv4_tbl8s and num_ipv6_tbl8s represent the number of tbl8 tables used in the DIR-24-8 algorithm for the IPv4 and IPv6 LPM tables, respectively.
max_num_ipv6_neighbors is the maximum number of neighbor entries for the IPv6 forwarding table. For IPv4, the address space is small enough that we can allocate enough room for all possible neighbors. For IPv6, the address space is too large; therefore, we specify the expected maximum size of this table in the configuration.
max_num_ipv4_fib_entries and max_num_ipv6_fib_entries represent the maximum number of entries in the IPv4 and IPv6 forwarding tables, respectively.
- front_icmp_msgs_per_sec & front_icmp_msgs_burst
- back_icmp_msgs_per_sec & back_icmp_msgs_burst
It is necessary to limit the number of ICMP replies Gatekeeper issues per second to avoid a scenario where Gatekeeper servers are the source of an attack. We limit the rate of ICMP messages using the token bucket algorithm, and the limit is set per network interface.
front_icmp_msgs_per_sec and front_icmp_msgs_burst represent the rate and burst size of the ICMP messages for the front interface, and back_icmp_msgs_per_sec and back_icmp_msgs_burst represent the rate and burst size of the ICMP messages for the back interface.
request_timeout_sec
The amount of time a flow's request to be granted can be considered before expiring. Unanswered requests that reach expiration can be pruned from a GK instance's flow table.
basic_measurement_logging_ms
The interval at which basic measurements of the GK instances are taken (in milliseconds). These measurements include the number of packets and bytes seen, requested, declined, granted, etc., and are written to the Gatekeeper log every interval.
None.
There are various ways to configure and inspect the GK block at runtime using Gatekeeper's dynamic configuration.
To add an entry to the forwarding table, a Lua script can be used with the dynamic configuration library to call:
int add_fib_entry(const char *prefix, const char *gt_ip, const char *gw_ip,
enum gk_fib_action action, struct gk_config *gk_conf)- prefix is an IP prefix specified in CIDR notation of the entry to add
- gt_ip is the IP address of the Grantor server to forward packets in this prefix to (if applicable)
- gw_ip is the IP address of the gateway through which packets in this prefix should be forwarded (if applicable)
- action is one of `GK_FWD_GRANTOR`, `GK_FWD_GATEWAY_FRONT_NET`, `GK_FWD_GATEWAY_BACK_NET`, or `GK_DROP` to indicate that flows from this prefix should be forwarded to a Grantor server, forwarded to a destination in the front or back network through a gateway, or dropped, respectively
- gk_conf is the context for the GK block currently running
int del_fib_entry(const char *ip_prefix, struct gk_config *gk_conf)- ip_prefix is an IP prefix specified in CIDR notation of the entry to delete
- gk_conf is the context for the GK block currently running
The IPv4 and IPv6 forwarding tables can be separately dumped to the console by calling either `list_gk_fib4()` or `list_gk_fib6()`. They accept as parameters a GK configuration structure, a callback function that is invoked once for every entry in the forwarding table, and a accumulator string.
For examples, see `lua/examples/example_of_dynamic_config_request.lua`.
Similar to the FIB, the IPv4 and IPv6 neighbor tables can be separately dumped to the console by calling either `list_gk_neighbor4()` or `list_gk_neighbor6()`. They accept as parameters a GK configuration structure, a callback function that is invoked once for every entry in the neighbor table, and a accumulator string.
For examples, see `lua/examples/example_of_dynamic_config_request.lua`.
The dynamic configuration enables the user to flush entries from the GK flow tables using the following C function:
int gk_flush_flow_table(const char *src_prefix,
const char *dst_prefix, struct gk_config *gk_conf)- src_prefix is an IP address range in CIDR notation or NULL
- dst_prefix is an IP address range in CIDR notation or NULL
- gk_conf is the context for the GK block currently running
For examples, see `lua/examples/example_of_dynamic_config_request.lua`.