amd64 process sizes

[Ruslan Ermilov]

On Wed, Sep 05, 2007 at 07:50:49PM +1000, Peter Jeremy wrote:
> I've been comparing VSZ reported for similar processes on amd64 and
> i386 and found that amd64 processes report as vastly larger than I
> expected:
> 
> i386  amd64
> 1476   3572  getty
> 6272  29184  xclock
> 4784  28020  xdm
> 6420  30000  xterm
> 3828   9444  sendmail
> 
> I did some further digging into the procfs map for xterm and found
> that each of the amd64 shared objects has an additional mapping that
> is 255 or 256 pages.  Once you remove those mappings, the sizes are
> reasonably similar.  A typical set of mappings is:
> 
> 0x800f60000 0x800fba000 61 0 0xffffff0027ac2540 r-x 36 18 0x0 COW NC vnode /usr/local/lib/libXaw8.so.8
> 0x800fba000 0x800fbb000 1 0 0xffffff002765ed20 r-x 1 0 0x2180 COW NC vnode /usr/local/lib/libXaw8.so.8
> 0x800fbb000 0x8010bb000 18 0 0xffffff0027ac2540 r-x 36 18 0x0 COW NC vnode /usr/local/lib/libXaw8.so.8 [*]
> 0x8010bb000 0x8010cd000 18 0 0xffffff00274d2c40 rw- 1 0 0x2180 COW NC vnode /usr/local/lib/libXaw8.so.8
> 
> 0x28284000 0x282d7000 15 0 0xc360ad68 r-x 9 6 0x0 COW NC vnode /usr/local/lib/libXaw8.so.8
> 0x282d7000 0x282d8000 0 0 0xc3a4b210 r-x 1 0 0x2180 COW NC vnode /usr/local/lib/libXaw8.so.8
> 0x282d8000 0x282df000 3 0 0xc3aef39c rwx 1 0 0x2180 COW NC vnode /usr/local/lib/libXaw8.so.8
> 
> Could someone please explain the purpose of the mapping [*] and what
> system resources they occupy.
> 
These mappings are produced by rtld(1) when it maps shared objects.
I took cat(1) here, and examined its libc.so's mappings on i386 and
amd64.  What rtld(1) basically does is mmap's the PT_LOAD segments
(see the Program Header in the "objdump -x /lib/libc.so.*" output).
Since the size of the loadable segment is not necessaily a multiple
of the page size, it zeroes out the rest of the last page of each
loadable segment.  If the segment is read-only (like is the case
for the "text" segment), it temporarily write-unprotects the last
page, then zeroes its tail, then makes it read-only again.

The first mapping you see is created by mapping a "text" segment
of the library.  The last mapping is for the "data" segment.
The second mapping is created by toggling the write protection
of the page, and this mapping is 4096 bytes in size.

The mysterious third mapping on amd64 is due to a gap between the
virtual addresses of the two loadable segments (text and data) on
amd64:

Program Header:
   LOAD off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**20
        filesz 0x00000000000d7051 memsz 0x00000000000d7051 flags r-x
   LOAD off    0x00000000000d7060 vaddr 0x00000000001d7060 paddr 0x00000000001d7060 align 2**20
        filesz 0x000000000001a010 memsz 0x0000000000032df8 flags rw-

vend0 = rounddown(vaddr0, pagesize) + roundup(memsz0, pagesize) = 0x0 + 0xd8000 = 0xd8000
gap = rounddown(vaddr1, pagesize) - vend0 = 0x1d7000 - 0xd8000 = 255 pages

On i386 there's no gap:

Program Header:
LOAD off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**12
     filesz 0x000ca5d2 memsz 0x000ca5d2 flags r-x
LOAD off    0x000ca5e0 vaddr 0x000cb5e0 paddr 0x000cb5e0 align 2**12
     filesz 0x000053f0 memsz 0x0001b404 flags rw-

vend0 = rounddown(vaddr0, pagesize) + roundup(memsz0, pagesize) = 0x0 + 0xcb000 = 0xcb000
gap = rounddown(vaddr1, pagesize) - vend0 = 0xcb000 - 0xcb000 = 0 pages

Why amd64 linker does so, I don't know.  :-)


Cheers,
-- 
Ruslan Ermilov
ru at FreeBSD.org
FreeBSD committer