Introduction

The Vyatta Component Infrastructure (VCI) is a mechanism to interface data modeled by YANG with the actual model of system services. The DANOS system is made up of disparate open source projects, these projects do not have common interfaces, but we'd like to be able to interface with them all in a more dynamic way. Instead of forking every project we use and maintaining them solely ourselves in order to provide a homogeneous interface, DANOS takes the approach of creating management components for each one of these services. The responsibility of these management components is to translate between a homogeneous message bus based interface and the concrete interface for the service that is being managed.

A component is an autonomous agent responsible for realizing the user's desired configuration for a given feature as the system evolves with time. This means that the configuration data that a component receives may contain configuration that is impossible to apply to the system at the time the configuration is committed (ex: configuration that relies on a network interface that hasn't been added to the system yet). This configuration should be deferred until the operational state of the system evolves to a point where it is possible to apply it, and then applied by the component. In addition to configuration handling, components allow for retrieval of operational state data from the managed subsystem and are the endpoint for any modeled RPC.

Component responsibilities

Components are responsible for translating the configuration data, operational data and RPC calls. All inbound data for a component is an RFC7951 encoded string fulfilling the YANG modeled data and can then be converted to the native format for a service (ex: a configuration file). All data exiting the component must be encoded as a RFC7951 string conforming to the YANG data-model for that component.

• Configuration data

  • Committed configuration conforming to all models managed by the component comes in as a chunk.

    • The configuration chunk is translated to the configuration model for the service being modeled.

    • The configuration is applied to the service

• Operational state

  • Operational state data is any data nodes modeled as "config false" in the YANG data-model.

  • When requested, operational state, must be queried from the modeled service and then translated to a RFC7951 encoded string conforming to the YANG model by the component.

  • All returned operational data must contain context from the root of the data hierarchy.

• RPC calls

  • RPC calls have YANG modeled 'input' and 'output' data.

  • When an RPC call is made, the input data must be translated to the services native format, the request made and any returned data translated into a RFC7951 encoded string conforming to the YANG model.

• Notifications

  • When a notification is sent it must be a RFC7951 encoded string conforming to the YANG model for the notification.

Components are also responsible for modeling dependencies between each other. This information will be used during the commit process to order changes in configuration.

Component metadata

When defining a component a certain amount of metadata is required to get it integrated into the system. This information is defined in ".component" files. DANOS has a debian helper "dh-vci" which will handle generating the appropriate system integration files from this metadata. ".component" files belong in the debian directory of a project implementing a VCI component.

An example component file is shown:

This is for the 'exampled' component, which provides 2 different models.

'Vyatta Component' Fields

Name

Name of the component that will be used to register with the VCI infrastructure. Name must match the component name used in the call to vci.NewComponent().

Description

Free-text description

ExecName

Name of the executable to be run when this component

ConfigFile

Fully-qualified path for the component's config file. This field is DEPRECATED.

Before

Other components that this component should be started before, ie any component listed here is started after this component. Names here should match the names in the relevant component's 'Name' field, comma-separated.

After

Opposite of 'Before' - a list of components that must be started before this component. Names here should match the names in the 'Name' field, comma-separated.

DefaultComponent

This is used to provide support for one component to own all YANG modules that are not explicitly claimed by any VCI component. Obviously only one component may be the default. This is primarily to provide a way to assign ownership of legacy Vyatta-v1 modelset modules to provisiond.

The default component cannot list any modules explicitly.

'Model' fields

Each component may provide one or more models. These each represent a view (or interface) of the component that consists of a set of YANG modules.

Different models may contain the same YANG modules.

A model may belong to one or more Model sets, if it wishes to present the same interface for each model set. However, it may only provide a single model per model set.

model-name (part of Model header)

Unique name of the model. Must match name used in call to comp.RegisterModel()

Modules

Comma-separated list of YANG modules provided by this model. The component owns these models and receives configuration information for them. Only one component may own a given YANG module.

ModelSets

Comma-separated list of model sets supported by this model.

ImportsRequiredForCheck

Optional field used when a component needs candidate configuration from other components to be able to carry out the check() function. Content is a comma- separated list of YANG modules required.

Component interface

All components on the system will share a homogeneous interface for configuration data and operational data. RPC interfaces are component specific. All of this is accessed over the VCI message bus. The descriptions provided below are a high level overview that consists of the semantics of each method, language specific quirks are included in the language API documents linked at the end of this page.

• Configuration interface

  • Set(rfc7951-encoded-data-tree) -> warnings

  • Check(rfc7951-encoded-data-tree) -> errors

• Operational interface
  
  

  • Get() rfc7951-encoded-data-tree

• RPC

  • Each component will expose additional RPCs as modeled in their YANG model

Configuration

The configuration interface to a component consists of two methods.

Check

Pseudo-code prototype

Description

The "check" method allows a component to implement configuration validation that is inefficient or impossible to model in the YANG modeling language. This should only be used when something cannot be modeled in YANG. By modeling constraints in YANG one is communicating the semantics of the interface to a 3rd party without requiring communication with the component its self. The "check" method may be called multiple times, even outside of the context of a configuration change. Each time a user runs the "validate" command or the "commit" command, the "check" method is called on all components. Errors reported by the "check" method abort a commit immediately. Any configuration that passes the "check" method is expected to be accepted by the "set" method.

Input

The configuration tree for all nodes within the modules owned by the component including any context needed to uniquely identify augmented data. This tree will also include all configuration data for any modules listed in the "ImportsRequiredForCheck" option in the ".component" file.

Output

Any error to be returned to the user.

Set

Pseudo-code prototype

Description

The "set" method is called by the configuration infrastructure when a user commits a set of configuration changes. The "set" method is only called if a portion of the configuration modeled in one of the managed components has changed. The full new configuration for all modules managed by the component are provided to the "set" method. It is the components responsibility to diff this against either the running state of the system or the old configuration as appropriate for its use case as it applies the configuration. This configuration tree may be cached in volatile storage ("/run") if one needs to retain it in the event of a component crash. Any reported errors from the "set" method are treated as warnings to the user by the system, they do not cause the system to roll back to the previous configuration. The "set" method is expected to be able to accept and apply or defer configuration that is valid according to the YANG model and the "check" method. In the case of runtime errors such as being out of memory one must communicate the problem to the user by logging it as one would do had the problem occurred via some other runtime mechanism (such as receiving a route update that was too large). If one wishes to provide warnings to the user then one may return an error via the "set" method.

Input

The configuration tree for all nodes within the modules owned by the component including any context needed to uniquely identify augmented data.

Output

Any warnings to be returned to the user.

Operational State

The operational state interface to a component consists of one method.

Get

Pseudo-code prototype

Description

The "get" method is called anytime a user requests operational state for one of the YANG modules managed by a given component. If an error occurs while retrieving stats, one should log the error if it is something the user needs to resolve, otherwise, one should log it as a debug message that may be used when trying to find the problem. Other than this general guidance, what a given component does when an error occurs is up to the implementation. One may choose to return the portion of the state that was able to be retrieved or one may return no state information at all.

Output

The rfc7951 encoded representation of all config false nodes in the modules managed by the component including any required context.

RPC

The RPC interface to a component consists of "n" methods. Each method a combination of the YANG module and the modeled name of RPC. Registering RPCs consists of linking a module name, rpc name pair with the implementing code for the method.

Pseudo-code prototype

All RPCs are of the following form

Error handling recommendations

When returning errors to the user they should be succinct and related to the RPC's model. These messages shouldn't be opaque messages from the underlying implementation. Additional information for errors may be logged to the system logs using the appropriate level. Only messages which the user can have an impact on should be reported as failures.

Component design considerations

With the responsibilities of a component in mind, the following consideration should be taken when designing a component.

Don't rely on configuration data for modules a component doesn't own

Only one component is allowed to receive configuration changes for a given YANG module. Configuration change doesn't reflect a change to the operational state of the system, only the desire that the change occur, therefore configuration is a private interface for a given component. Ither components that need to know information in the component's configuration data should do so by other means such as reading operational state data, listing for notifications, calling RPCs, reading information from the underlying system or some combination of these options. Components should make "interesting" information available via these mechanisms for other components that need to know them. What is deemed "interesting" is expressed by modeling it in YANG and making it available via the appropriate mechanism. In this way a component's public interface (operational state, notifications, and RPCs), which is available to other components, is well documented as is its private interface (configuration), which is available only via the transactional interface via configd.

Cache the service's configuration to volatile storage

Each component is responsible for and manages the configuration for a service. If the service has no native configuration and is fully dynamic, then the component should cache the configuration data that so it can bring up the service when required. One should save the configuration to volatile storage (/run) incase the component crashes.

Model relationships between components, not YANG models

Since a YANG model is but one potential interface into a component, one should model relationships between the components themselves, not between their YANG models. This is done using 'Before' and 'After' clauses to the component configuration file.

Translate YANG modeled data in realtime

In order for changes made by other configuration services to be seen, YANG modeled data should be translated in realtime, as requests come in, do not try to cache the information in order to prematurely optimize what is likely to not be a problem. The VCI libraries will provide an abstraction for accessing this information so it can be made more efficient by tweaking the library implementation if it is required.

Don't over segment services

YANG models get broken into tiny modules as a side effect of the way we model some services. Components do not need to map 1-to-1 onto the YANG models, instead a component may implement many YANG models as one cohesive unit.

Use VCI for inter-component communication

All signaling between components should happen through the VCI tools. Data transfer may be done through a secondary channel negotiated through the signaling mechanism. The VCI Bus is not meant for high throughput information, that is best left for sockets or zmq.

Proxy notifications from other sources into VCI

Notifications coming from other parts of the system should be modeled in YANG and then proxied onto the bus via the VCI tools. There are obvious exceptions to this (routing netlink messages for example). An example of a particularly useful signal to put onto the VCI Bus is interface insertion and deletion from udev and netlink linkup/linkdown messages.

Component API Usage

One may create components in a number of programming languages. A full example is provided for each supported language in the pages linked below.