Who Ate My Memory « damnever's blog

It’s important to take a break and get out more. I’ve been working for over two years and nobody has shared with me anything beyond technology. I was at home pondering life when my server strangely reported a high memory usage rate. There were no abnormal processes on the monitor, but the overall memory usage rate was very high. Checking on the server, the monitor seemed to be working fine. After a careful study of cat /proc/meminfo, I found that none of the important indicators were particularly exaggerated, adding up to no more than 15G. However..:

PageTables:     113739060 kB

This is mainly related to Linux’s memory mapping. It’s clear that it’s not caused by using too much physical memory. On a whim, I wanted to curse. Originally, in order to reduce a few system calls, I used mmap when writing files. Now, the occupied virtual memory is already very large. The VmPTE in /proc/<pid>/status can confirm that our process is correct. According to the current usage, it’s about this number. But in theory, in the worst case, it can directly blow up the memory. I’m not sure what the effect would be, but it’s exciting to think about it ^_^

The simplest solution is to use mmap.MAP_HUGETLB, but this requires changing the system configuration. Another method is to stop using mmap. Upon testing, mmap did not actually significantly improve performance. The write performance of mmap will become slightly slower over time, and it’s inseparable from pagetables/mmu/tlb. Actually, I refuse to write code. Thinking about having to do my own operations and maintenance makes me even more frustrated. It’s a good opportunity to refactor, so..

As for the huge page configuration, it really needs to be reconsidered. It may improve performance, but it may also have pitfalls. Having seen Shanghai at four in the morning, it’s about time to lie down..