From OpenFlow Wiki

Contents 1 Rate Limiter Proposal 1.1 Introduction 1.2 Rate Limiters 1.3 Use cases 1.4 DiffServ operation 1.5 Implementations 1.6 Description

Rate Limiter Proposal

Jean Tourrilhes - HP-Labs

Introduction

This proposal describe an extension to the OpenFlow protocol to support Rate Limiters.

The vast majority of QoS frameworks are based on queues and rate limiters. For example, DiffServ is based on a combination of queues and rate limiters. OpenFlow 1.0 already support queues (Slicing feature), it is therefore natural to extend the QoS support of OpenFlow to rate limiters.

This proposal is designed to be simple yet cover various implementations and usages of rate limiters, and it is designed to be orthogonal to other OpenFlow features, to avoid adverse feature interactions. This proposal has already been implemented in hardware and software switches for OpenFlow 0.8.9 and OpenFlow 1.0.

This proposal is part of the list of OpenFlow 1.2 proposals.

Rate Limiters

A rate limiter control the rate of packets passed through it. It is one of the common basic block of various QoS frameworks, such as DiffServ, can be combined with queues and used for multiple QoS operations.

The main drawback of rate limiters is that they are usually not work conserving like queues, in other words they may leave unused capacity in the network. DiffServ style rate limiters that only remark packets can be work conserving when properly used.

There is many advantage to rate limiters.

First, they offer a very simple conceptual model, their behaviour is simple to predict, and are widely used in many QoS frameworks.

Second, the implementation of rate limiter is simple and efficient, and most hardware can offer a large number of rate limiters, whereas most hardware have a limited number of queues.

Third, rate limiters are not tied to a port, so they can usually be deployed much more flexibly and process any arbitrary set of packets.

Use cases

• Flow limiting. Limit a specific misbehaving flow to a specific rate target.
• Ingress limiting. Limit the amount of traffic that a single port or set of ports can send to the switch or network.
• Slice limiting. Limit a whole FlowVisor slice to a specific rate target.
• default rule limiting. Limit rate of unmatched packets. HP ProCurve and Open vSwitch already implement such rate limiter.
• OFPP_CONTROLLER limiting. Limit the packets sent to controller.
• Feature protection. Limit all the packets sent to a particular features that is resource constrained, for example limit all packets needing header rewriting, or limit all packets that need processing in software, etc...
• DiffServ policing. There is a vast amount of expertise in DiffServ, so we should enable DiffServ style of operation on top of OpenFlow.

Note that none of these can be implemented with the slicing support part of OpenFlow 1.0.

DiffServ operation

The goal of this proposal was to enable DiffServ operation and compatibility, with the simplest possible API, and avoid to duplicate other OpenFlow functionality. The consequence is that the rate limiter can not in itself implement DiffServ, and need to be combined with other existing OpenFlow features.

The main operation used for DiffServ is OFPLC_MARK. Packets that exceed "mark_rate" have their drop precedence increased, while packets that don't exceed that rate are untouched. For example, if a packet in class AF22 (DSCP 20) exceed the "mark_rate", its ToS would be rewritten to the one of class AF23 (DSCP 22).

If ToS need to be set to a specific value, a OFPAT_SET_NW_TOS need to be performed prior to the rate limiting. The rate limiting does not set the packet priority, this needs to be done in a subsequent lookup (i.e. in another table) based on the ToS value. A full DiffServ policying operation could look like :

Table 1 : in_port == 34 => OFPAT_SET_NW_TOS(20) ; OFPLC_MARK(500kb/s) ; goto 2
Table 2 : nw_tos == 20 => priority(5);
Table 2 : nw_tos == 22 => priority(2);

Implementations

This proposal is partly implemented as a vendor extension in the following OpenFlow switches :

• Procurve 1.09 firmware (OpenFlow 0.8.9)
• Procurve 2.02 firmware (OpenFlow 1.0)
• Open vSwitch 0.90.6 (OpenFlow 0.8.9)
• Open vSwitch 1.0.1 (OpenFlow 1.0)

Description

This is a quick description of our proposal. More details can be found in our implementation.

Rate limiters are addressed by a limiter_id, which is an arbitrary 32 bit number in its own private space. There is one "special" rate limiter for the hidden default rule sending unmatched packets to the controller.

/* Maximum of flexible rate limiters */
#define OFPL_MAX		0xffff0000;

/* Rate limiter associated with the default rule - can only be modified */
#define OFPL_DEFAULT_RULE	0xfffffffe;

Capabilities describe what the various implementations can do in struct ofp_switch_features.

enum ofp_limiter_capabilities {
  OFPLC_KBPS	= 1 << 0,	/* Rate value in kb/s (kilo-bit per second) */
  OFPLC_PKTPS	= 1 << 1,	/* Rate value in packet/sec */
  OFPLC_DROP	= 1 << 2,	/* Drop packet if rate exceeded */
  OFPLC_MARK	= 1 << 3,	/* Do DiffServ codepoint (DSCP) remarking */
  OFPLC_BURST	= 1 << 4,	/* Do burst size */
  OFPLC_STATS	= 1 << 5,	/* Collect statistics */
};

Configuration of rate limiters is done through a simple message which can add, modify or remove a rate limiter. Rates can be expressed in packets per seconds or kb/s depending on the flags set. Values are valid when the corresponding flag is set.

/* Limiter configuration. OFPT_LIMIT_REQUEST. */
struct ofp_limiter_mod {
  struct ofp_header	header;
  uint32_t		limiter_id;	/* Rate Limiter instance. */
  uint32_t		flags;		/* One of OFPLC_*. */
  uint32_t		drop_rate;	/* Rate at which we drop packets */
  uint32_t		mark_rate;	/* Rate for DSCP remarking */
  uint32_t		burst_size;	/* Size of bursts */
  uint16_t		command;	/* One of OFPFC_*. */
  uint8_t		pad[2];
};

Flows are directed to rate limiters through an action. Multiple flows can be associated with the same rate limiter. A flow can specify multiple rate limiters (although I doubt many implementations will support that).

/* Action structure for OFPAT_LIMIT. */
struct ofp_action_limit {
    uint16_t	type;		/* OFPAT_LIMIT. */
    uint16_t	len;		/* Length is 8. */
    uint32_t	limiter_id;	/* Rate Limiter instance. */
};

Statistics can be read for each rate limiter. Actual statistics are optional.

/* Limiter statistics. OFPST_LIMIT. */
struct ofp_limiter_stats {
  /* Config */
  uint32_t		limiter_id;	/* Rate Limiter instance. */
  uint32_t		flags;		/* All OFPLC_* that apply. */
  uint32_t		drop_rate;	/* Rate at which we drop packets */
  uint32_t		mark_rate;	/* Rate for DSCP remarking */
  uint32_t		burst_size;	/* Size of bursts */
  /* Statistics */
  uint32_t		flow_count;	/* Number of flows bound to it */
  uint64_t		packet_count;	/* Number of packets in input. */
  uint64_t		byte_count;	/* Number of bytes in input. */
  uint64_t		packet_drop;	/* Number of packets dropped. */
  uint64_t		byte_drop;	/* Number of bytes dropped. */
  uint64_t		packet_mark;	/* Number of packets marked. */
  uint64_t		byte_mark;	/* Number of bytes marked. */
};