[TxMt] TM tokenizer is taking 100% CPU for long while using 100-200KB text files with single line

Sven Axelsson sven.axelsson at gmail.com
Wed Feb 27 13:53:16 UTC 2008 

On 27/02/2008, Adam Strzelecki <ono at java.pl> wrote:
>  When I use:
>  $ perl -e '{print "<html></html>\n" x 20000}' > test.html
>  (Note the \n)
>  File loads immediately together with syntax highlight, no 100% CPU.
>
>  So I think there's definitely something wrong with syntax highlight
>  (tokenizer).
>  I remember compiler & parser construction lessons on my university,
>  and the lexer & parser performance shouldn't matter of line breaks.

But in TextMate the syntax highlighter (and more) is line-based and
works with regular expressions and not a precompiled lexer/parser, so,
yes, the line length does matter.

-- 
Sven Axelsson
++++++++++[>++++++++++>+++++++++++>++++++++++>++++++
>++++<<<<<-]>++++.+.++++.>+++++.>+.<<-.>>+.>++++.<<.
+++.>-.<<++.>>----.<++.>>>++++++.<<<<.>>++++.<----.

More information about the textmate mailing list