The DENT project has been promoting the use of an Open Network Install Environment (ONIE) within Linux-based networking products. Recently, a number of useful features have been added to the upstream Linux kernel to help with the adoption of this environment. Deep dive into the story behind these contributions.
ONIE hardware requirements
The project has little-to-no hardware limitation, at least regarding the underlying platform. Manufacturers can use almost any common architecture supported by the Linux kernel. One requirement however, is to include a non-volatile storage, whatever its type, to store ONIE specific product data. The range of information the manufacturer can provide is wide, ranging from the serial number of the device to the MAC addresses for the network interfaces, along with various version numbers, manufacturing date and place, as well as manufacturer specific information.
To be really useful, this data must be parsable in a generic way. This is the purpose of ONIE’s TlVInfo format. The format is named after the data layout within the storage: TLV or Type Length and Value fields. Basically, we expect each entry to be made of an encoded value giving the type of field, concatenated with the size of the payload and the data itself.
All TLV entries follow each other in a table which shall not exceed 2048 bytes. The ONIE format specifies a number of mandatory fields which define the overall structure, such as an easy to spot 8-bytes ID (“TlvInfo”), a version number for the entire table, its length, and at the end, a CRC-32 to check data integrity. In between, all possible fields are listed here, aside from vendor-specific extensions of course:
https://opencomputeproject.github.io/onie/design-spec/hw_requirements.html#type-code-values
Such information can be used by the system administrators to know the origin of the product, its serial number, etc. But most importantly, one could expect the Linux kernel drivers to extract information from this table in order to configure the hardware. On network switches, a common requirement is to use a consistent and unique range of MAC addresses for the network interfaces, which could typically be specified in the ONIE TlvInfo table. This however requires a good integration in the system, deeper than just offering access to the raw content of the storage medium.
The NVMEM subsystem
Due to too much code duplication between various misc/ EEPROM drivers, and seeing the growing need for a standard consumer API to access their content, an NVMEM framework was contributed to the Linux kernel in 2015. Besides the needed factorization between all drivers, it introduced a standard device-tree API to both expose small amounts of data through “NVMEM cells”, as well as a consumer API which can be used by device drivers to retrieve the content of these cells. Over the years, the NVMEM subsystem was extended to support any kind of storage like NAND or SPI-NOR flashes. As an example of use, the NVMEM abstraction quickly became the right way to expose MAC addresses to network controller drivers. ONIE TlvInfo tables can be stored in almost any kind of non-volatile storage medium and need to expose their content to drivers, making the fit with the NVMEM subsystem quite obvious. Unfortunately, the subsystem had to evolve a little bit in order to allow this synergy.
Until recently, the framework only supported static device-tree bindings, only describing the precise location of each piece of data inside the storage. Said otherwise, one would need to know precisely the location and size of each field in advance, which does not match ONIE’s TlvInfo concept where flexibility is key. The DENT project hence decided to fund additional kernel development in order to be able to support these tables.
The NVMEM layout interface and the ONIE layout driver
The entire design of the NVMEM subsystem predating 2015, no support whatsoever for any advanced parsing mechanism was considered. The only binding existing was using fixed offsets and sizes and would definitely not fit the ONIE TlvInfo purpose. Instead, such a design would need to target a storage area from which useful information could be extracted, and tell the Linux kernel what logic to follow in order to parse its content and expose the cells.
Supporting such a new logic in the NVMEM subsystem became the target and in order to tackle this, the DENT project contracted Bootlin, an engineering company which made Linux kernel developments and upstreaming its expertise. As the company was already deeply involved in the kernel community, Miquèl Raynal, who drove these developments, quickly found the most relevant and hopefully efficient approach to address the DENT’s needs. Indeed, sharing with the community has been key in this quest, as Michael Walle, an active member of the community, had already tried to do something similar. His first approach, while technically relevant, unfortunately did not convince the maintainers. Exchanging with Michael and building a common solution to overcome the known difficulties lead to a satisfying cooperation which ended into a common series which, after several additional tries, would become the adopted solution. Despite having received early validation from many members of the community as there were actual expectations regarding the whole NVMEM layout logic, the upstreaming process took time, meticulous testing and many reviews, further certifying the commitment of the DENT project to act as a leader and a contributor of the Linux kernel community.
The final proposal added a truly new parser alongside the fixed cells parser, named “NVMEM layouts”. The needed infrastructure would rely on an underlying driver to perform the parsing, the “layout driver”, and would expose NVMEM cells based on what has been dynamically discovered. The exact layout in the storage medium is still unknown when the platform starts, but still, system administrators can describe the cells they need in the NVMEM device-tree node and still point to them from the consumer devices, waiting for them at run-time to be filled. Within this new scope, the DENT project additionally funded the writing of an ONIE layout driver to parse the TlvInfo table, while Walle complemented the work with an SL28 VPD layout driver, exposing MAC addresses with the same constraint as ONIE tables. Both layout drivers would dynamically extract data from the underlying devices following their own rules and expose them through the generic NVMEM abstraction.
Since the initial merge, an additional layout driver has been proposed, showing even more interest for this work beyond the single use of the layouts for the ONIE purpose, further validating the relevance of this contribution pushed by the DENT project.
ONIE TlvInfo tables support being upstream since version 6.4 of the Linux kernel, hopes are to see all userspace parsers disappear in favor of this united solution, like the common Marvell mvpp2 network controller driver which is now capable of getting its MAC addresses automatically. We all hope that the lessons learned with this project as well as the positive outcome of this venture laid solid foundations for further upstream contributions, working with all people involved in this powerful community.