Go VCI API

Owned by John Southworth
Nov 10, 2020
8 min read

Introduction

This document covers the specifics of writing a VCI component in the Go programming language. For general VCI concepts please refer to the VCI API overview documentation.

Go is the recommended language for writing VCI components. When creating a brand new implementation one gets to choose a language to build it in based on the needs of the feature and what you are comfortable with. VCI supports Go, C++ and Python 3 for implementing feature components. Like anything there are tradeoffs with each of language for the purposes of implementing a component below are the tradeoffs to consider when choosing Go.

Pros

  • Great concurrency support

  • Fast compiled code

  • Fastest interface to the VCI bus

  • Good library support for config file generation (text/template)

  • Good library support for the RFC7951 encoding we use

    • Generic immutable rfc7951 tree data-structure available.

  • Good for string handling

  • Garbage collected

  • Support for interfacing with vplaned and vyatta-dataplane.

Cons

  • It is a newish language (~10 years old) that you may not be familiar with

    • Different object model from other OO languages which takes some getting used to initially

Debian Packaging

DANOS uses the Debian packaging system for packages and we have created helpers to aid in creating VCI components. To use these, we have to setup the debian build environment to use the right helpers and create the component descriptor file (.component file) in the directory.

VCI Helpers

To enable the VCI helper one needs to pass "–with=vci" to the "dh" command in debian/rules. An example is included below.

#!/usr/bin/make -f # # This debian/rules uses debhelper. # export DEB_BUILD_HARDENING=1 # Uncomment this for verbose logging #export DH_VERBOSE=1 %: dh $@ --buildsystem=golang --with=golang,vci,yang

Component Descriptor

dh-vci uses a component descriptor file in the debian directory to generate system specific configurations. One must also supply this component descriptor file for dh-vci to work properly. A sample is included below.

[Vyatta Component] Name=com.example.myfeature Description=VCI component for My Feature ExecName=/lib/myfeature/vci-myfeature [Model com.example.myfeature.v1] Modules=myfeature-v1,myfeature-interfaces-dataplane-v1,my-feature-interfaces-vhost-v1 ModelSets=vyatta-v1

Component Structure

VCI components implement the logic for a given set of YANG modules. A component owns the data for the modules defined in the ".component" file. In the VCI API there are 6 important concepts, the component, the models the component implements, the features that a model supports (Configuration, Operational State and RPCs) and the client. Components are simply containers for models. Models are containers for the Config, State and RPC objects. The Config object implements the logic to validate and apply configuration data for the YANG modules that a component manages. The State object retrieves and formats operational state data for the YANG modules that a component manages. The RPC object implements RPC handlers for any RPC defined in the managed YANG modules.

The Go VCI API is defined in the "github.com/danos/vci" library. For the objects to be exposed on the bus, reflection is used to generate the interface. One implements an interface as described below and reflection is used to generate an appropriate generic interface that will be exposed to the bus. The Go API will automatically do the rfc7951 encoding and decoding for any type that supports it via these generic interfaces.

API Documentation

Component

Creating a component is a relatively straightforward process. One needs to create the objects for the desired functionality, attach them to a model and attach the model to a component object. Finally, one starts the component bus process, and waits for it to exit. All methods in a Go VCI component may be called in parallel and care should be taken to ensure proper ordering where appropriate.

Config interface

The VCI API uses reflection to build an appropriate generic object to expose on the bus. This generic object can be thought of as implementing the following two interfaces. The Config interface has an optional Get method that is not used by the implementation but is allowed to be exposed to the bus for compatibility with older components. This interface would be specified as follows if it were an actual Go interface:

type Config interface { Set(data string) error Check(data string) error Get() string }

The user supplied objects are dissected with reflection to build this generic object. This allows for more specialized objects that implement the following methods:

Where T is any type that can be marshaled via the "github.com/danos/encoding/rfc7951" library. The VCI API library will handle the marshaling of any input and output type of an object. If one wishes to implement a method that does not use the default encoding library then one may use a "string" or "[]byte" type as types. An example of a user object is supplied below:



State Object

Similarly to the Config object, the State Object supports automatic encode/decode of rfc7951 data so it supports a generic interface for objects implementing the proper method. The bus interface would be specified as below if it were a concrete Go interface.

The user supplied objects are dissected with reflection to build this generic object. This allows for more specialized objects that implement the following methods:

Where T is any type that can be marshaled via the "github.com/danos/encoding/rfc7951" library. The VCI API library will handle the marshaling of any input and output type of an object. If one wishes to implement a method that does not use the default encoding library then one may use a "string" or "[]byte" type as types. An example of a user object is supplied below:



RPCs

RPC functions must be of the form: "func(T) (T, error)" where T is any type that can be encoded/decoded using the RFC7951 library.

RPC definitions may be passed to the VCI API using 2 methods. One is an object implementing the RPC methods and the other is passing a map from string to function implementation. An algorithm is used to map from Go method naming convention to the YANG naming convention. The algorithm changes "GoCase" names  into "yang-case" names. Each time there is a transition from lower case to upper case or to a number a hyphen is inserted and the word is converted to lower case. This mechanism works in most situations but doesn't work in all situations so the alternative mapping mechanism is provided to provide exact names.  Some examples of the conversion mechanism are provided below.

Go name

YANG name

Go name

YANG name

FooB

foo-b

FooBar

foo-bar

FooBAr

foo-bar

FOOBAR

foobar

Foo1

foo1

Foo1B

foo-1b

Foo12b

foo-12b

FooB123

foo-b-123

FOO1B

foo-1b

FOO1BAR

foo-1bar

The below example will expose the RPC as rpc1 as specified in the YANG model.

The alternative case would look like the following.

Model

A Model is created by associating Config, State and RPC objects with the model name as defined in the ".component" file. When one defines a model, one may chain together the method invocations to implement the full thing declaratively as in the example below. Config and state object lifetimes are managed by the VCI library.

Component

The Component is the mechanism that starts the process that listens on the VCI Bus for messages and dispatches to the appropriate object in the program. One defines a component by creating the component and attaching the model to it. An example is included below.

Once defined one can start the component process using the "Run" method. The "Run" method returns immediately causing the bus listener to run in parallel with the current thread of execution this allows a component to perform other operations concurrently with the bus handler.

Since the "Run" method starts the bus process and returns immediately,  one may need to block until component to exit using the "Wait" method. One may use standard Go patterns to wait for multiple processes to complete.

One may tell the component to cleanup gracefully if required using the "stop" method. If the process exits without calling stop the VCI bus will cleanup automatically.

Client

When one creates a component, one may use the same connection to the VCI Bus as a VCI client to talk to other components. One uses the "client" method of the component to access this part of the connection.

The client can subscribe to notifications, request configuration and state data for a given model, and call RPCs via the VCI bus directly. 

One may also create a new client connection to the VCI Bus if one doesn't need a full component.

Notifications

One of the most useful operations for the client is to subscribe to and emit notifications on the VCI bus.

Emit

Emitting notifications is easy. One simply uses the "client.Emit" method providing the module name the notification is modeled in, the name of the notification and the body as an object that may be encoded to RFC7951.

The body of the notification will be validated against the data-model before emission on the bus occurs. This means that an invalid notification will be discarded. Any bus client may emit any notification and no access control is provided to ensure notifications come from a given source. This is a tradeoff was chosen to allow flexibility for integrating with existing open-source projects in the most efficient way possible (e.g. a script that is called when a given event occurs).

Subscribe

One may also subscribe to notifications via the VCI client object. Subscriptions have several buffering mechanisms which may be chosen for a given notification based on the desired semantics. By default all notifications are "infinitely buffered". One may choose to coalesce notifications such that only the last received notification is delivered dropping all previous messages, this can be useful when subscribing to notifications that act like state updates. One may choose to drop or block after some limit. One may also remove any previously instantiated limit.

Subscriptions are not active until one calls "run" on the subscription object and one may stop delivery of a subscription via the "cancel" operation. Subscriptions are delivered via registered functions of the type "func (body T)" where T is any type that may be decoded from RFC7951. Subscriptions may also export to a go channel directly by passing a "chan T" type where T is any type that may be decoded from RFC7951.

RPC

The client can call RPCs directly on the bus. This is only for use when calling RPCs between components. On should use CallRPC in the configd API for management interface applications.

State

The client can request the state of a given model via the bus directly. This is mostly for the interface from configd to the components.

Config

The client can request the configuration for a given model via the bus directly. This generally shouldn't be used as one component shouldn't depend on the configuration of another. This is mostly for the interface from configd to the components.