Python VCI API

Owned by John Southworth
Nov 10, 2020

Introduction

This document covers the specifics of writing a VCI component in the Python 3 programming language. For general VCI concepts please refer to the VCI API overview documentation.

When creating a brand new implementation one gets to choose a language to build it in based on the needs of the feature and what you are comfortable with. VCI supports Go, C++ and Python 3 for implementing feature components. Like anything there are tradeoffs with each of language for the purposes of implementing a component below are the tradeoffs to consider when choosing Python 3. Python 3 should only be used for relatively simple features which have a straight forward interaction mechanism due to the single threaded nature of the framework implementation.

Pros

  • Seems to be preferred currently for ease of use and familiar object model

  • Good support for text templates (string)

  • Good support for string handing

Cons

  • Slowest interface to the VCI Bus (20x slower than C++)

  • Concurrency support is lacking

    • The GIL gets in the way, each bus call will be serialized.

    • If you are expecting a lot of parallel calls to your feature avoid using Python 3 as the implementation.

  • Slower interpreted implementation

Debian Packaging

DANOS uses the Debian packaging system for packages and we have created helpers to aid in creating VCI components. To use these, we have to setup the debian build environment to use the right helpers and create the component descriptor file (.component file) in the directory.

VCI Helpers

To enable the VCI helper one needs to pass "–with=vci" to the "dh" command in debian/rules. An example is included below.

#!/usr/bin/make -f %: dh $@ --with=python3,vci,yang

Component Descriptor

dh-vci uses a component descriptor file in the debian directory to generate system specific configurations. One must also supply this component descriptor file for dh-vci to work properly. A sample is included below.

[Vyatta Component] Name=com.example.myfeature Description=VCI component for My Feature ExecName=/lib/myfeature/vci-myfeature ConfigFile=/etc/myfeature.conf [Model com.example.myfeature.v1] Modules=myfeature-v1,myfeature-interfaces-dataplane-v1,my-feature-interfaces-vhost-v1 ModelSets=vyatta-v1

Component Structure

VCI components implement the logic for a given set of YANG modules. A component owns the data for the modules defined in the ".component" file. In the VCI API there are 6 important concepts, the component, the models the component implements, the features that a model supports (Configuration, Operational State and RPCs) and the client. Components are simply containers for models. Models are containers for the Config, State and RPC objects. The Config object implements the logic to validate and apply configuration data for the YANG modules that a component manages. The State object retrieves and formats operational state data for the YANG modules that a component manages. The RPC object implements RPC handlers for any RPC defined in the managed YANG modules.

The Python 3 VCI API is defined in the "python3-vci" library. For the Config and State objects, base classes are defined for the required objects. One implements one of these base classes and passes the object to the VCI library so that the appropriate calls are made when the operation is requested on the bus. In Python 3 RPCs are implemented a function of a single parameter. In all cases in the Python 3 VCI API input and output parameters are python dictionaries representing the RFC7951 data. Due to interfacing with the C++ library and interaction of such with the GIL, all calls happen in a single threaded manner.

API Documentation

Component

Creating a component is a relatively straightforward process. One needs to create the objects for the desired functionality, attach them to a model and attach the model to a component object. Finally, one starts the component bus process, and waits for it to exit.

Config Object

The Config Object is as follows:

class Config(): def set(self, input): ... def check(self, input): ... def get(self): ...

State Object

The state object is defined as follows.

RPCs

RPCs are represented by a python function taking one input argument that will be a python dictionary containing the RPC input data as decoded from the RFC7951 encoded values and returning a dictionary containing the output data as to be encoded as RFC7951 data.

Model

A Model is created by associating Config, State and RPC objects with the model name as defined in the ".component" file. When one defines a model, one may chain together the method invocations to implement the full thing declaratively as in the example below. Config and state object lifetimes are managed by the VCI library.

Component

The Component is the mechanism that starts the process that listens on the VCI Bus for messages and dispatches to the appropriate object in the program. One defines a component by creating the component and attaching the model to it. An example is included below.

Once defined one can start the component process using the "run" method.

The "run" method starts the bus process and returns immediately. One may block until component to exit using the "wait" method.

One may tell the component to cleanup gracefully if required using the "stop" method. If the process exits without calling stop the VCI bus will cleanup automatically.

Client

When one creates a component, one may use the same connection to the VCI Bus as a VCI client to talk to other components. One uses the "client" method of the component to access this part of the connection.

The client can subscribe to notifications, request configuration and state data for a given model, and call RPCs via the VCI bus directly. 

One may also create a new client connection to the VCI Bus if one doesn't need a full component.

Notifications

One of the most useful operations for the client is to subscribe to and emit notifications on the VCI bus.

Emit

Emitting notifications is easy. One simply uses the "client.Emit" method providing the module name the notification is modeled in, the name of the notification and the body as a RFC7951 encoded string.

The body of the notification will be validated against the data-model before emission on the bus occurs. This means that an invalid notification will be discarded. Any bus client may emit any notification and no access control is provided to ensure notifications come from a given source. This is a tradeoff was chosen to allow flexibility for integrating with existing open-source projects in the most efficient way possible (e.g. a script that is called when a given event occurs).

Subscribe

One may also subscribe to notifications via the VCI client object. Subscriptions have several buffering mechanisms which may be chosen for a given notification based on the desired semantics. By default all notifications are "infinitely buffered". One may choose to coalesce notifications such that only the last received notification is delivered dropping all previous messages, this can be useful when subscribing to notifications that act like state updates. One may choose to drop or block after some limit. One may also remove any previously instantiated limit.

Subscriptions are not active until one calls "run" on the subscription object and one may stop delivery of a subscription via the "cancel" operation. Subscriptions are delivered via registered functions  taking one input argument that will be a python dictionary containing the RPC input data as decoded from the RFC7951 encoded values.

RPC

The client can call RPCs directly on the bus. This is only for use when calling RPCs between components. On should use CallRPC in the configd API for management interface applications.

State

The client can request the state of a given model via the bus directly. This is mostly for the interface from configd to the components.

Config

The client can request the configuration for a given model via the bus directly. This generally shouldn't be used as one component shouldn't depend on the configuration of another. This is mostly for the interface from configd to the components.