Excerpt from https://docs.docker.com/storage/storagedriver/overlayfs-driver/
Page Caching. OverlayFS supports page cache sharing. Multiple containers accessing the same file share a single page cache entry for that file.
This is in the context of layered docker image, where multiple containers access the same image. In my deployment, however, I can see very stable (in time) difference in page cache utilization by the same image running on different but similarly configured nodes of Kubernetes cluster. Neither node has cache pressure that could lead to different reclaiming rates.
So, I was wondering if the same file in the above excerpt could refer to the "sameness" verified by a hash, and the file could be actually a part of various docker images?
The difference in question is on the order of 30-60MB, and that's consistent with python/libgc libraries my container uses. It would all make sense if common shared libraries were "deduplicated" in page cache node-wide at overlayfs level. The page cache would be counted to a cgroup on first-touch basis as per para 2.3 of
https://www.kernel.org/doc/Documentation/cgroup-v1/memory.txt
Therefore, the image running on the node, where same python libraries were used by other docker image(s), would show less utilization of the page cache by comparison with the node, where those libraries were used by my container only.
I am aware of abundant thought along deduplication, such as https://lwn.net/Articles/636943/ Y2015:
Chinner spoke up to describe the problem, which is that there might be a hundred containers running on a system all based on a snapshot of a single root filesystem. That means there will be a hundred copies of glibc in the page cache because they come from different namespaces with different inodes, so there is no sharing of the data.
And no, I am not using KSM, so no need to mention that. I would appreciate having some references to the source code shedding light on this behavior.