Overlaybd is recommended as the backend of rootfs and template

[lihuiba]

Rootfs and its template is currently implemented as a layered nbd block device. Actually a similar component has already been implemented in the container community and used extensively in production for years. It is overlaybd, a sub-project of containerd.

I would recommend using overlaybd as the backend of rootfs and template, because:

• It is highly performant, featuring
  • ublk or TCMU as the block device frontend (both are much better than nbd)
  • io_uring or epoll asynchronous API
  • AVX-512 vectorization and linearized B+ tree for index query (millions of QPS on a single CPU core)
• It is feature-rich, including all possible operations on the layers.
• It connects naturally to container registry ecology, which fits perfectly for template management.
• It has a big user base, including Alibaba, Azure, DataBricks, Boss直聘, Meituan, fly.io, hocus.dev, etc.

[linear]

ENG-3206

[BigVan]

Currently, overlaybd relies on OCI registry authentication to access backend storage (such as S3, Azure Blob Storage, and OSS).
It would also be beneficial to add a mode that directly accesses various backends through their official SDKs or APIs.