Robert,
Also look at the Postgres usage. Some of the caching is with the pg process. Restarting nsd would not clear the pg caches; restarting pg would..
For reference, I admin sites using aolserver 4.5.1, pg 8.2 and freebsd 7.3 (OpenACS 5.5+head). Here are some RSS sizes.
a site using caching with ecommerce:
397968 with pg RSS ranging from 1152 to 126736 (5 near 126k size, 6 averaging about 3k)
a site using multiple host node mappings:
424968 with pg RSS ranging from 7708 to 47420 (6 near 40k, 2 averaging 8k)
an unpopular site:
89944 with pg RSS ranging from 0 to 6936 (2 near 5k, 4 near 0 )
an artist website:
215152 with pg RSS ranging from 55144 to 68656 (fairly evenly spread)
cheers,
Torben