Wiki Navigation

WS-Eventing in Openwsman

Introduction & Overview

This document describes the High Level design of WS-Eventing feature in Openwsman. Openwsman is a project intended to provide an open-source implementation of the Web Services Management specification (WS-Management) and to expose system management information on the Linux operating system using the WS-Management protocol. WS-Management is based on a suite of web services specifications and usage requirements that exposes a set of operations focused on and covers all system management aspects. WS-Eventing is a feature supported by the WS-Management protocol.

There are three participants in WS-Eventing protocol: Subscriber, Event Source and Event Sink.

Subscriber: A client that subscribes to a certain event
Event Source: Accepts a subscription and produces a events
Event Sink: Receives events.

Event Source consists of two components:

Subscription Manager and
Notification Manager

The Subscription Manager deals with the subscription while the Notification Manager with the notification.

Processing Model of WS-Eventing in Openwsman

In our implementation openwsman acts as an Event Source, which is responsible for receiving a subscription from a subscriber and sending a notification to an event sink. As there are different events created by different applications in the real world, it is important to make this process as general as possible. The solution is to put the specific eventing production in the plug-in part and integrate shared WS-Eventing protocol process in the framework of openwsman. That is to say, the framework and plug-in act together to accomplish a process of WS-Eventing.

Subscription Manager

Data Structures

struct __WsSubscribeInfo {
	pthread_mutex_t notificationlock;
	unsigned long flags;
	long heartbeatCountdown; //countdown of heartbeat, when it is 0, a heartbeat generated
	char            subsId[EUIDLEN];
	char *	soapNs;
	char *	uri;
	char *	epr_notifyto; //A delivery destination for notification messages, using some delivery mode
	char * locale; // language code
	char * contentEncoding; //"UTF-8" or "UTF-16" or something else
	unsigned long expires;
	int	deliveryMode; /*The delivery mode to be used for notification messages sent in relation to this subscription.
	                                         Implied value is WS_EVENT_DELIVERY_MODE_PUSH,
	                                         which indicates that Push Mode delivery should be used. */
	unsigned int	connectionRetryCount; // count of connection retry
	unsigned long connectionRetryinterval; //how long to wait between retries while trying to connect
	unsigned long heartbeatInterval; //Interval to send a heartbeart
	unsigned char eventSentLastTime; //To indicate whether an event is sent since last heartbeat
	WsXmlDocH bookmarkDoc;
	unsigned char bookmarksFlag; // whether bookmark is needed
	filter_t	*filter;
	WsmanAuth       auth_data;
	WsEndPointEventPoll eventpoll; // plugin related poll process
	WsXmlDocH templateDoc; //template notificaiton document
	WsXmlDocH heartbeatDoc; //Fixed heartbeat document
};

The structure __WsSubscribeInfo contains necessary parameters defined in WS-Mangement, which determines how events are generated and how a notification is produced and sent. This structure is shared by “renew” and “unsubscribe” requests with the same “subsId”, which is generated when the service accepts a “subscribe” request. In addition, this structure contains a field “eventpoll”, which is used to poll the events if the plug-in doesn’t support independent events report mechanisms. Field “templateDoc” is a template message for usage when a real event message is produced. Field “heartbeatDoc” is heartbeat notification message produced when a subscription is dealt with successfully and a heartbeat is requested.

A new member “subscriptionList" is added to structure of __Soap. When a “subscribe” request is dealt with successfully, a corresponding “WsSubscribeInfo” structure will be generated and appended to this list and kept until an corresponding “unsubscribe” request comes or it expires.

struct __Soap {
        /* do not move this field */
	pthread_mutex_t lockData;
	void           *parserData;
	unsigned long   uniqueIdCounter;

	list_t         *inboundFilterList;
	list_t         *outboundFilterList;

	list_t         *dispatchList;
	list_t         *responseList;
	list_t         *processedMsgIdList;
	pthread_mutex_t lockSubs; //lock for Subscription Repository
	list_t	         *subscriptionMemList; //memory Repository of Subscriptions
	char 	*uri_subsRepository; //URI of repository
	SubsRepositoryOpSetH subscriptionOpSet; //Functions talbe of Subscription Repository
	WsContextH      cntx;
	              //TBD claen up and initilaize it;

	unsigned long   lastResponseListScanTicks;

	              //TBD:? ? ? Make it thread and pass as parameters
	int             resendCount;
	unsigned long   resentTimeout[SOAP_MAX_RESENT_COUNT];

	list_t         *WsSerializerAllocList;

	void           *dispatcherData;
	DispatcherCallback dispatcherProc;
	void *listener;
};
typedef struct __WsEventThreadContext * WsEventThreadContextH;
void * wse_notification_manager(void * thrdcntx); //Thread to be created to handle event report

Three stubs are added to “wsman-soap.h”

int     wse_subscribe_stub(SoapOpH op, void *appData, void *opaqueData);
int 	wse_unsubscribe_stub(SoapOpH op, void *appData, void *opaqueData);
int 	wse_renew_stub(SoapOpH op, void *appData, void *opaqueData);

These stubs deal with common operation in WS-Evening. And any specific operations are to be handled in the plug-in endpoints.

Notes: Other related macros and definitions are omitted.

Working Mechanism

Response to subscribe request

When a subscription request is received, if it is a valid request, it will be saved in the subscription repository and in the memory at the same time and a response message will be returned with expiration time(if any) and corresponding subscription identifier. If it is not, an error message will be returned with causes.

Response to renew request

Renew request messages should include subscription identifier which is returned by the subscription manager when subscription is made.

Response to unsubscribe request

Unsubscribe request should include subscription identifier which is returned by the subscription manager when subscription is made.

Subscription Repository

Why is Subscription Repository needed

There are mainly two reasons:

In many occasions, WS-Management working environment is distributed. When the Subscription Manager receives a subscribe request, it has to save this subscription to a Subscription Repository in another physical location, which can also be accessed by the Notification Manager.
If there is a subscription repository, the service will recover after restarting without loss of subscribtion data.

Goal of implementation

Although we have to decide for one type of repository in our implementation, the implementation should be designed extensible. That is, it should be changed easily if another kind of repository is considered.

Related Data structures

To achieve this goal, callback functions are used. Function table of Subscription Repository operations will be registered in the __Soap object when the service finishes loading all plug-ins. The function table should include operations “load all subscriptions”, “get/update/save/delete a specific subscription”. Once the function table is registered, it can be used in the runtime late. If you want to implement a different repository, just change related subscriptions functions.

Related data structures are shown below:

typedef int (*SubscriptionOpInit) (char *, void *);
typedef int (*SubscriptionOpFinalize) (char *, void *);
typedef int (*SubscriptionOpSave) (char *, char *, char *);
typedef int (*SubscriptionOpDelete) (char *, char *);
typedef int (*SubscriptionOpGet)(char *, char *, char *);
typedef int (*SubscriptionOpSearch) (char *, char *);
typedef int (*SubscriptionOpUpdate)(char *, char *, char *);
typedef int (*SubscriptionOpLoad) (char *, list_t *);

struct __SubsRepositoryEntry {
	char *strdoc;
	char *uuid;
};
typedef struct __SubsRepositoryEntry *SubsRepositoryEntryH;

struct __SubsRepositoryOpSet{
	SubscriptionOpLoad load_subscription; 
	SubscriptionOpGet get_subscription;
	SubscriptionOpSave save_subscritption;
	SubscriptionOpUpdate update_subscription;
	SubscriptionOpDelete delete_subscription;	
};

And we also need to add two fields in the `Soap structure:

char 	*uri_subsRepository;	//URI of repository
SubsRepositoryOpSetH subscriptionOpSet; //Function talbe of Subscription Repository

Implementation of Local Subscription Repository

Local Subscription Repository is implemented in openwsman. If another repository type is needed, just change related subscription operations. In this implementation a subscription is saved in a file with the name of the UUID returned by the Notification Manager.

Definitions of local Subscription Repository operations:

int LocalSubscriptionOpInit (char * uri_repository, void *opaqueData);
int LocalSubscriptionOpFinalize(char * uri_repository, void *opaqueData);
int LocalSubscriptionOpGet(char * uri_repository, char * uuid, char **subscriptionDoc);
int LocalSubscriptionOpSearch(char * uri_repository, char * uuid);
int LocalSubscriptionOpLoad (char * uri_repository, list_t * subscription_list);
int LocalSubscriptionOpSave (char * uri_repository, char * uuid, char *subscriptionDoc);
int LocalSubscriptionOpUpdate(char * uri_repository, char * uuid, char *expire);
int LocalSubscriptionOpDelete (char * uri_repository, char * uuid);

Function table of local Subscription Repository operations:

struct __SubsRepositoryOpSet subscription_repository_op_set = {LocalSubscriptionOpInit, LocalSubscriptionOpFinalize, LocalSubscriptionOpLoad, LocalSubscriptionOpGet, LocalSubscriptionOpSearch, LocalSubscriptionOpSave, LocalSubscriptionOpUpdate, LocalSubscriptionOpDelete};

Implementation of Event Pool

Event Pool is responsible for events storage. All events except heartbeats generated by the service are put here before they are sent eventually. Notification Manager gets events from Event Pool, while Plug-in adds events to it.

Data Structures for describing events

To describe different kinds of events, a unified data structure should be defined to represent different events.

struct __WsNotificationInfo {
	WsXmlDocH headerOpaqueData; //header conten
	char *EventAction; //event action URL
	WsXmlDocH EventContent; //event content
};
typedef struct __WsNotificationInfo * WsNotificationInfoH;

Event Pool Data Structures

Events are grouped by subscription ID which they belong to. In other words, events for different subscription are put in different lists. The definition of event source will be like this:

/*to store events for a subscription*/
struct __event_entry {
   char subscription_id[EUIDLEN]; //to identify the event entry
   list_t *event_content_list; //a list holds __WsNotificationInfo
};
typedef struct __event_entry *event_entryH;

typedef void (*clearproc) (WsNotificationInfoH);

typedef int (*EventPoolInit) (void *);
typedef int (*EventPoolFinalize) (void *);
typedef int (*EventPoolCount) (char *);
typedef int (*EventPoolAddEvent) (char *, WsNotificationInfoH);
typedef int (*EventPoolAddPullEvent) (char *, WsNotificationInfoH);
typedef int (*EventPoolGetAndDeleteEvent) (char *, WsNotificationInfoH*);
typedef int (*EventPoolClearEvent) (char *, clearproc);

/*Event Source Function Table*/
struct __EventPoolOpSet {
	EventPoolInit init;
	EventPoolFinalize finalize;
	EventPoolCount count;
	EventPoolAddEvent add;
	EventPoolAddPullEvent addpull;
	EventPoolGetAndDeleteEvent delete;
	EventPoolClearEvent clear;
};
typedef struct __EventPoolOpSet *EventPoolOpSetH;

Implementation of Notification Manager

Notification defined in WS-Eventing protocol can be a real event produced by the event source or a heartbeat event just to send in a fixed interval. It is an obvious and reasonable idea to put the two functions in two separate threads. So the notification manager can be designed to consist of the notification manager thread and the heartbeat thread.

To decrease the memory footprint, we design a single notification manager thread for all notifications. The thread is created when the service is started. The notification manager thread works for all notifications. It loops for ever until the service is stopped and detect whether there is an event. If an event is detected, a notification sender thread will be created to send the event to the event sink. The notification sender will quit after the send operation is completed. The notification manager is also responsible for subscriptions operations. If there is a subscription that is unsubscribed or expired, the notification manager will delete it from the memory. If the subscription is unsubscribed, the subscription data will also be deleted from the repository.

If heartbeat is asked for in the “subscribe” message, a heartbeat event will be sent in a fixed interval if there is no event from the event source. Heartbeat generator is also an independent loop thread strarted when the service starts. It generates all heartbeat events in the system. Like the notification manager thread, if there is a heartbeat event is created, the heartbeat generator will also create a notification sender to send the event to the event sink.

So the notification manager consists of a notification manager thread, a notification sender and the shared heartbeat generator.

Notification Manager Thread

When the service is started, the notification manager thread is created. The thread loops and inspects the state of every subscription in the memory. If a subscription is unsubscribed or expired, it will be removed from the memory. If it is unsubscribed, it will also be deleted from the repository. If the subscription is still alive(it hasn’t expired yet), the thread will query there is an event available for a subscription. If there is, a notification message will be produced according to the delivery mode and a notification sender will be created to send it.

Heartbeat Generator

Heartbeat generator inspects the states of all alive subscriptions to generate heartbeats. It is also started when the service is started. It checks all subscriptions’ states to know whether there is an event generated by the event source during the heartbeat interval. If there is no event for a subscription in such a period, a heartbeat event will be generated and a corresponding notification sender will be created to send it.

Notification Sender

Notification sender is created by the notification manager thread when there is an event for a subscription or when the heartbeat generator decides to send a heartbeat for a subscription. It quits when the task is finished.

Pull Events

WS-Management adds a new event delivery mode --“pull”, which is useful when the event source cannot invoke communication with the event sink on its own. Pull delivery mode borrows the idea of Pull enumeration and works in the same way. A Pull event request message should carry an enumeration context which is returned when a subscription with pull delivery request is made.

CIM Plug-in Implementation

Overview

In this implementation, all things defined in WS-Eventing protocol are dealt with by Subscription Manager, Notification Manager. Different kinds of WS-Management bindings are done in plug-ins. So it is natural that logic about CIM event stuff are also added in the CIM Plug-in.

In the latest CIM Schema 2.15, subscriptions are represented by a trio of classes:

CIM_IndicationFilter - Captures the query or filter identifying the subnet of indications of interest.
CIM_IndicationHandler(CIM_IndicationHandlerCIMXML) – Captures information about where/how the indications are to be delivered.
CIM_IndicationSubscription – Associates an instance of CIM_IndicationFilter with CIM_IndicationHandler

Subscribe Request

According to Specification of WS-Management – CIM Binding, there are three kinds of Subscribe Request.

Subscribing to the CIM Server

In this case, a new CIM_IndicationFilter instance, CIM_IndicationHandlerCIMXML instance and CIM_IndicaitonSubscription should be created.

Subscribing to an Indication Class

In this case, firstly we should construct a query string from the request indication class. Consequently, things are done like the case above.

Subscribing to an Existing Filter

In this case, the existing filter should be checked whether it exists in the system. If it does exist, CIM_IndicationHandlerCIMXML instance and CIM_IndicaitonSubscription, which associates the existing filter and new created handler, will be created.

Event Delivery

WS-Management defines two delivery modes (PushWithAck and Events) whose status of delivery will affect the subscription.If the notification isn’t acknowledged by the event sink, the subscription may be cacelled or terminated. So in order to be aware of the delivery status, notifications will be sent to Openwsman firstly and redirected to the event sink finally. That is to say, there should be a listener in Openwsman, which is responsible for accepting events from CIMOM. And to distinguish different subscriptions, the events for different subscriptions should be sent to different destinations. So we can arrange different http service root as an event destination for different subscription. Here we suggest using Subscription ID as a part of http service root to distinguish different paths. For example, we can use the format like wsman/cimindicationlistener/uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.

CIM Schema 2.15, instances of CIM_IndicationHandler or its subclass are indicated by the eventing infrastructure. We should create an instance of CIM_IndicationHandler for every subscription with the destination property set to the corresponding URL.

CIM Subscription expression

The CIM_IndicationSubscription class captures association between the indication filter and the endpoint for event delivery. The instance of this class represents the subscription created by the subscribe request. The instance and associated instances will be deleted when the subscription is unsubscribed or expired.

CIM Indication Listener Implementation

In the above discussion we have talked about the idea of designing a CIM Indication Listener in OpenWSMAN. CIM Indication is delivered by CIM-XML export operation, so it is a straight idea to extend our current http server to support this.

When a CIM subscription request is accepted successfully, a unique service path is created and stored in a list. Afterwards, when a CIM-XML indication arrives, HTTP server will inspect the posted path. If the path is registered, the indication will be accepted. Or else, it will be discarded. Accepted indications will be analyzed according to the CIM-XML schema and related stuff we want is added in the event pool. By now CIM specific processing is completed.

If a subscription delivery mode is not Pull, when there are events related to it, notification Manager will pick them off, assemble WS-Management event packages and sent them out.

Testing

Set up CIM indication environment

We use sblim sfcbd as background CIMOM. So for detailed sfcbd setup information, visit http://sblim.wiki.sourceforge.net/Sfcb.

In order to produce an indication, there should be an indication provider for sfcbd. Here we use a simple indication provider which produces an indication every 5 seconds.

Once sfcb is set up correctly, you can add indication provider.

Copy indication provider image to sfcb provider directories(Default are /usr/local/lib /usr/local/lib/cmpi).
Register indication provider by sfcbstage.
1. Prepare a corresponding sfcb provider registry file. A standard registry file would be like the following:
```
[CIM_ProcessIndication]
   provider: CPUUsage
   location: processIndication
   type: indication
   namespace: root/interop
```
2. Registered to sfcb by sfcbstage Execute sfcbstage to copy the registry file and related mof file to sfcb specified location. An example execution is like “sfcbstage -n root/interop -r processIndication.reg”
Rebuild mof repository
Execute sfcbrepos -f to rebuild mof repository. And a CIM indication testing environment is ready.