[TxMt] problems with large text files

[Allan Odgaard]

On 24/1/2006, at 15:01, Christof Janssen wrote:

> [...] According to the terminal command wc (wordcount), the file  
> that I want to open contains 280 983 978 characters [...] TextMate,  
> however, takes about 10 s on my 1.5 GHz G4 PB (2 GB Ram), then it  
> unexpectedly quits without actually displaying the file. Can/ 
> Shouldn't I expect TextMate to work with files this large ? There  
> appears to be a problem with memory allocation.

Currently TM uses utf-16 internally, so your file will take up 536 MB  
of RAM. But since I use an STL vector that grows exponentially  
(iirc), the last allocation will be for the lowest power of 2 which  
is higher than 536 MB (which is 1 GB), and that allocation (judging  
from your console log) fails.

So in short, it doesn't work.

I do have plans about using utf-8 internally and switch to disk- 
mapping for huge files, but this is currently just some far away pipe- 
dreams, as the focus currently is on editing “experience” more than  
(border case) performance improvements.