From sven.axelsson@gmail.com Wed Feb 27 13:53:19 2008 From: Sven Axelsson To: textmate@lists.macromates.com Subject: Re: [TxMt] TM tokenizer is taking 100% CPU for long while using 100-200KB text files with single line Date: Wed, 27 Feb 2008 14:53:16 +0100 Message-ID: In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8045323690032962267==" --===============8045323690032962267== Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit On 27/02/2008, Adam Strzelecki wrote: > When I use: > $ perl -e '{print "\n" x 20000}' > test.html > (Note the \n) > File loads immediately together with syntax highlight, no 100% CPU. > > So I think there's definitely something wrong with syntax highlight > (tokenizer). > I remember compiler & parser construction lessons on my university, > and the lexer & parser performance shouldn't matter of line breaks. But in TextMate the syntax highlighter (and more) is line-based and works with regular expressions and not a precompiled lexer/parser, so, yes, the line length does matter. -- Sven Axelsson ++++++++++[>++++++++++>+++++++++++>++++++++++>++++++ >++++<<<<<-]>++++.+.++++.>+++++.>+.<<-.>>+.>++++.<<. +++.>-.<<++.>>----.<++.>>>++++++.<<<<.>>++++.<----. --===============8045323690032962267==--