Vplaned
- John Southworth
Vplaned is a daemon that manages interaction with the dataplane, it acts as a conduit between management and control plane services and the dataplane. Vplaned provides an intermediate processing stage for netlink state and configuration commands in order to support these fast restart and hotplug for the dataplane. Since both netlink state and configuration commands are stored in vplaned, the dataplane can request a dump of this cached information in order to restore its previously configured state if restarted. It is important to note that Vplaned does not understand the information it caches for the dataplane. It is simply an intermediate store for this information to aid in dataplane management; there are no special per feature handlers in Vplaned.
The format of configuration commands sent to vplaned is JSON with either a base64 protobuf blob or space delimited text over a ZMQ socket. This was done for backwards compatibility with existing features when creating the newer protobuf based configuration protocol. The format of messages sent to the dataplane is either netlink over ZMQ, protobuf over ZMQ, or configuration strings over ZMQ depending on use. The dataplane will properly dispatch each of these to its internal per-feature registered handlers.
Internals
The Vplaned process consists of a netlink message listener to relay state programmed into the Linux kernel and a configuration relay mechanism for configuration changes which are not mirrored in the kernel. Vplaned also has the ability to dispatch a cached version of this information to the dataplane if it restarts. ZMQ is used as a transport to send all of this information to the dataplane. Vplaned handles creation of shadow devices in the Linux kernel that correspond to dataplane managed interfaces. The dataplane then handles send and receive of packets to and from the control plane services via these shadow devices.
Vplaned has of a number of connections to perform its required actions:
NETLINK connections into the kernel in order to receive updates for the various kernel mirrored features (routing, IPsec, L2TP, TEAM, etc.). Note that only non-unicast routes are retrieved from the kernel, unicast routes are published directly to the dataplane via a route broker process.
A Command Proxy (ZMQ REQUEST-REPLY) connection used to request and receive additional configuration information and operational state from the dataplane.
A Request (ZMQ ROUTER-DEALER) connection to one or more dataplane processes. The connection is used by a dataplane to register itself and its associated local interfaces.
A Publication (ZMQ PUB-SUB) connection to one or more dataplane processes. The connection is used to publish configuration updates to dataplanes.
A Query (ZMQ REQUEST-REPLY) connection. The connection allows applications to request operational state (e.g. list of dataplane instances) and configuration information (e.g. current IP address) from vplaned.
Snapshot Database
Vplaned maintains a snapshot database that consists of the netlink messages received from the kernel. Each received netlink message is parsed in order to produce a "topic" (a string representation of the message contents). The combined object (topic string + netlink data) is inserted into the snapshot database and published directly to any/all connected dataplane(s).
Following initial contact with vplaned, the dataplane asks for the current system configuration - a "WHATSUP?" request message. Vplaned responds by "dumping" the contents of the database down to the dataplane. The leading word of the topic string is used by the dataplane to dispatch the object to the appropriate subsystem.
Configuration Database
Whilst the majority of the network configuration can be represented through netlink messages, there are elements of application state that require additional configuration. The configuration store (cstore) consists of a key-value database. Applications (YANG backend scripts) use a Perl, Python, C++, or Go API to inject a JSON or Protobuf encoded message into vplaned. The message consists of two parts, a key (or path) and a value. The value part is an arbitrary string or Protobuf binary that is passed down (published) to the dataplane for processing by the control module.
Other than certain parts of the path element, vplaned has no understanding of the message; the message is simply stored and the value part forwarded to the dataplane. Similar to netlink messages, the dataplane uses the leading word of the value part to dispatch the message to the appropriate subsystem. There's also an __INTERFACE__ hint for commands that will store but not forward commands where the interface does not yet exist. These commands are released when the interface is instantiated.
As with the Snapshot Database, in response to a "WHATSUP?" request, vplaned pushes down the contents of the cstore database to the dataplane.
Dataplane Database
Vplaned derives a core set of attributes from its configuration file. Among the attributes is the local IP address of the vplaned daemon together with a list of defined dataplane instances. For each instance the file defines the IP address of the dataplane together with its UUID.
There is a single defined dataplane. The IP address of both vplaned and the dataplane is the loopback address (127.0.0.1) and the UUID is simply zero (00000000-0000-0000-0000-000000000000). VPlaned and the dataplane are designed such that they can operate on separate hosts but this functionality is currently not being used.
The dataplane makes contact with vplaned via the Request connection.
Vplaned establishes its authenticity (UUID match together with any ZMQ socket authentication) - "CONNECT", "ACCEPT" and "CONFQUERY" messages.
The dataplane reports on each of its hardware interfaces - "NEWPORT" messages. The "NEWPORT" message is used to create the "local" interface representation, e.g. dp0p1s1 or dp2p1s3.
The dataplane requests the current system configuration - "WHATSUP?" message.
