I get real memory 137 MB, and VM 1.03 GB.

Okay I narrowed the problem! I get 137MB too unless I switch file type to "Plain text", handled by "Text" bundle.

Can anybody tell that has 137 MB together with "Plain text"? Now switching back the file to "Plain text" sucks my memory back to 564 MB.

Cheers,

On 13 May 2008, at 11:10, Adam Strzelecki wrote:

...

I get real memory 137 MB, and VM 1.03 GB.

Okay I narrowed the problem! I get 137MB too unless I switch file type to "Plain text", handled by "Text" bundle.

Can anybody tell that has 137 MB together with "Plain text"? Now switching back the file to "Plain text" sucks my memory back to 564 MB.

Yes.

RSIZE: 137 MB VM: 1GB

in "Plain Text".

To be sure, I upgraded the bundles to latest: it's the same.

-- FredB

On 16 May 2008, at 22:59, Till Backhaus wrote:

checked it too. memory usage is the same here (~560MB) for /tmp/tm.s. Unfortunately after closing the file memory consumption sinks only by 40MB to ~520MB.

See http://kevin.sb.org/2007/07/05/malloc-free-space-not-reclaimed/#comment-247

As for the high memory usage, it is likely related to the number of lines in the file. It amounts to around one kilo byte per line. TM stores the data as an array of lines, each line node has meta data such as the scope info, misspelled words, the actual line buffer (which is in UTF-16 and rounded up to the nearest power of two), etc. With each dynamic memory allocation tagged with size and rounded up to cache line sizes, and maybe a dozen of these per line, it is probably not unrealistic that there is a 1 KB overhead per line.

There are certainly some pathetic cases in TM I want to address, but generally my view on this is that this is just the price for properly abstracted and maintainable code -- at least as long as this is more of a theoretical than practical concern.

So it seems tose 400MB that are freed are still kept by the OSX, which is IMHO pretty lame behavior. I just wonder it there's any kernel function to trigger VM flush/cleanup. Alternative solution I can see is writing own memory pool that uses vm_allocate. Is this feasible Allan?

BTW. It is all about default OSX/libc allocator which keeps in the memory all blocks < 16KB, so they are not reclaimed at all (given back to the system).

But there's quite easy solution, alternative allocator ptmalloc (used by Linux glibc): http://www.malloc.de/en/index.html

I've tested it with simple program:

#include "stdio.h" #include "stdlib.h" #include "string.h"

int main(int argc, char const *argv[]) { char b[4*1024], c = 0; void *p[4*10*1024]; size_t i, len = sizeof(b)/sizeof(b[0]); printf("Preparing buffer...\n"); for(i = 0; i < sizeof(b)/sizeof(b[0]); ++i) { b[i] = c + 'a'; c = (c + 1) % 30; }

printf("Allocating...\n"); for(i = 0; i < sizeof(p)/sizeof(p[0]); ++i) { p[i] = malloc(len); memcpy(p[i], b, len); } printf("Press any key...\n"); getchar();

printf("Freing...\n"); for(i = 0; i < sizeof(p)/sizeof(p[0]); ++i) free(p[i]); printf("Press any key...\n"); getchar();

return 0; }

Without using libptmalloc3.a after freeing, it still uses 161 MB real memory with MALLOC_SMALL blocks. With libptmalloc3.a after freeing, memory drops to 512KB! So this may be the solution for TextMate to give back small allocated memory blocks back to system.

Cheers,