Run Time Buffers Configuration Update design
Run-time Buffers Configuration feature allows to adjust buffer parameters on port when port speed changes.
Current buffers configuration implementation performs only initial configuration on switch start.
An update suggested in this design document enables dynamic buffer profile selection based on current port parameters (speed and attached cable length) which allows to guarantee lossless traffic flows after port speed change.
PG profile update on port speed change includes:
- Get cable length for this port
- Get speed for this port (speed )
- Using speed&cable-to-pg_profile look-up table determine new profile.
Port speed and cable length are the two parameters which uniquely define the profile to be used on the port. - If profile is different from the old one – replace profile configuration on port
This also includes:- Check whether old profile is still used on other ports – if not – delete it
- Check whether new profile is already used. If not – create it first
Three Databases are involved:
- CONFIG_DB: Stores buffers information expected to be applied on the switch and some information used by Buffer manager Daemon.
- ASIC_DB: Stores information actually applied on the switch. Can be used in tests for validation.
The template used for buffers config generation can implement static or variable configuration.
Static part contains information about cables length, buffer pools description and port profiles which should not change on port speed change.
Optionally there could be variable part of buffers configuration. This assumes having a look-up table for pg buffer profiles.
Buffer Configuration Daemon(BCD) reacts to port speed change and updates PG profiles on port. This includes the following:
- subscribe to CONFIG_DB:* to access buffers related tables and catch port update event
- create/delete PG profiles in CONFIG_DB
- update PG profile for port which speed was changed
Existing PG creation code needs to be updated to to remove unused PG profiles.
The following tables to be added to the
{
"BUFFER_POOL": {
"egress_lossless_pool0": {
"size": "3637248",
"type": "egress",
"mode": "static"
},
"egress_lossless_pool1": {
"size": "3637248",
"type": "egress",
"mode": "static"
},
"egress_lossy_pool": {
"size": "5491712",
"type": "egress",
"mode": "dynamic"
}
},
"BUFFER_PROFILE": {
"ingress_lossless_profile0": {
"pool": "[BUFFER_POOL:ingress_lossless_pool0]",
"xon": "18432",
"xoff": "76800",
"size": "113664",
"static_th": "0"
},
"ingress_lossless_profile1": {
"pool":"[BUFFER_POOL:ingress_lossless_pool1]",
"xon":"18432",
"xoff":"76800",
"size":"113664",
"static_th": "0"
}
}
}
-
cables length table
{ "CABLE_LENGTH": { "AZURE": { "Ethernet0": "5m", "Ethernet4": "40m", "Ethernet8": "300m", ... } } }
-
look-up table for mapping speed and cable length to pg buffer profiles
# PG lossless profiles. # speed cable sizexon xoff threshold 10000 5m 34816 18432 16384 0 25000 5m 34816 18432 16384 0 40000 5m 34816 18432 16384 0 50000 5m 34816 18432 16384 0 100000 5m 36864 18432 18432 0 10000 40m 36864 18432 18432 0 25000 40m 39936 18432 21504 0 40000 40m 41984 18432 23552 0 50000 40m 41984 18432 23552 0 100000 40m 54272 18432 35840 0 10000 300m 49152 18432 30720 0 25000 300m 71680 18432 53248 0 40000 300m 94208 18432 75776 0 50000 300m 94208 18432 75776 0 100000 300m 184320 18432 165888 0 -
Profile and port to profile mapping for each port (created by Buffer Manager)
"BUFFER_PG": { "Ethernet0:3-4": { "profile" : "[BUFFER_PROFILE:pg_lossless_100G_300m_profile]" } } "BUFFER_PROFILE": { "pg_lossless_100G_300m_profile": { "pool":"[BUFFER_POOL:ingress_lossless_pool]", "xon":"18432", "xoff":"165888", "size":"184320", "dynamic_th":"1" } }
To make the buffers configuration more flexible and adjustable to different configurations it will be partially implemented as a template. The following parts is implemented in the template:
-
Cable length table
Generated from template to support different number of ports and distinguish cable length from the switch and neighbor roles. -
port names
Some static configurations (like queue profiles, egress or lossy PG profiles) should be applied for all ports. To avoid hard coding interface names they can be generated in the template.
Besides the table declaring ports cable length, Buffer Configuration Daemon needs a lookup table described in 3.2.1 (variable part of the config). It will be passed to the utility as a command-line argument.
Buffer Configuration Daemon will be implemented as a stand alone utility. It will load inside swss docker container. So the appropriate record to the sonic-buildimage/dockers/docker-orchagent/supervisord.conf need to be added:
[program:buffermgrd]
command=/usr/bin/buffermgrd -l /usr/share/sonic/hwsku/pg_profile_lookup.ini
priority=10
autostart=false
autorestart=false
stdout_logfile=syslog
stderr_logfile=syslog
Every time when speed change event arrives, Buffer Configuration Daemon determines which profile need to be used for the port and updates appropriate BUFFER_PG table. Before updating the BUFFER_PG the utility checks whether needed profile is already created (it maybe used by some other port). If not - it needs to be created first (table BUFFER_PROFILE).
Initial buffers configuration is validated on compile stage by the unit test.
Validations to be implemented:
- no unused pools defined
- all profiles referenced in the look-up table are declared
- all pools referenced by profiles are declared
Test for VS.