[TxMt] Latex Bundle grammar

[Charilaos Skiadas]

On Sep 11, 2007, at 1:00 PM, Alain Matthes wrote:

>
> Le 11 sept. 07 à 13:23, Charilaos Skiadas a écrit :
>
>> 3) The LaTeX grammar is the basic grammar. It includes the TeX  
>> grammar and adds all the LaTeX specific stuff. In particular, it  
>> adds the \[ ... \] and \( ... \) math modes, and hence needs  
>> direct access to the TeX Math bundle, which it includes at those  
>> points. (This is why we could not simply have TeX Math as a  
>> repository in the TeX grammar, because then the LaTeX grammar  
>> couldn't call it.)
>
> Firstly, Thanks for this answer. You understand exactly my problem
>
> I need some precisions about patterns and include:
>
> include = 'text.tex'; i understand this one but the next are not  
> very clear for me
>
>  include = '$self';
>
> include = '$base';

$self and $base are very subtle. The idea is the following. Let's  
consider a rule in the TeX grammar, for instance:

		{	name = 'meta.group.braces.tex';
			begin = '\{';
			end = '\}';
			captures = { 0 = { name = 'punctuation.section.group.tex'; }; };
			patterns = ( { include = '$base'; } );
		},


Now this rule is supposed to match pairs of braces. Now, inside the  
braces everything TeX would go. So with pattern = ( {include=  
'$base'; } ); we tell TM that between the begin and end part it  
should use all the rules in the TeX bundle for it.

Strictly speaking, I just lied. This is what $self would do. To  
understand the difference, let's move one step further. We are now in  
the LaTeX grammar, and we include the TeX grammar in it. Now the TM  
parser encounters the open brace. It looks it the LaTeX grammar  
rules, finds nothing, and then moves on to the TeX grammar, where it  
finds the above rule. It start matching the rule, and now needs to  
know what is allowed to be used between the braces:

1) $base means everything from the original, LaTeX, bundle (which  
includes the TeX bundle).
2) $self means only stuff from the TeX bundle (where the rule occured).

In most cases in fact $base is what we want, I think most occurences  
of $self in the LaTeX grammars should be changed to $base.

>> 4) For specialized document classes, at this point in time, we  
>> simply create a new grammar that includes the LaTeX grammar and  
>> adds its own specific stuff. (This might become simpler/better in  
>> TM 2.0). Examples of this are the Beamer and Memoir grammars.
>
> Perhaps I need to wait for TM 2 !
>
> Questions about Beamer
>
> 1) In text.tex.latex we have : firstLineMatch = '^\\documentclass 
> (?!.*\{beamer\})';
>
> and in 'text.tex.latex.beamer : firstLineMatch = '^\\documentclass(\ 
> [.*\])?\{beamer\}';

Right, Beamer files are distinguished by the occurence of  
"\documentclass{beamer}" in the first line (ideally it should be the  
first 20 lines or something like that, but we can't do this atm). So  
we explicitly rule out beamer as a class name from the plain  
text.tex.latex, so that it gets picked up by the Beamer grammar. We  
should probably add memoir there.

>>
>> 5) For commands in particular packages, we typically would add  
>> those to the LaTeX grammar.
>
> I'm not sure that's a good way because i don't use listings (pb  
> with utf8) and I prefer pgf/tikz to pstricks.
> I would like to know if it's possible to enable or disable some  
> parts. I'm not sure but I think that's a big and long grammar is  
> not good for the speed to match a long text. That is why , I made a  
> remark about "listings" because it's not "pure" latex.
>
> There is now an excellent way for the preferences with a panel, we  
> can imagine a panel for the grammar package to add :
> beamer, memoir, xcolor, listings, pstricks, tikz, amsmath etc...   
> and for the author : calc ifthen etex multido

> Or a file in the preferences

There is at the moment no easy way to dynamically include grammars.  
The speed hit from a large grammar is actually minimal, I believe.  
Also we should separate between classes and packages. Packages are  
just like libraries in other languages, the commands they provide  
should just be part of the language (The equivalent in Ruby would be  
requiring a special language just so that we can match the methods of  
the String class, or something like that). Also, some packages/ 
classes automatically load other packages, and it would be hard to  
detect that and "switch mode".

Do you really want, whenever you have a document and you add a new  
\usepackage to it, to have to go to a preferences setting and tell TM  
that for this file you want it to recognize the keywords from this  
particular package? TM should be just recognizing the keywords right  
away.

>> Just make the additions you want to the grammar and send it to us,  
>> and we would add them. (Or consult with us on what changes you  
>> think are needed.). So for adding specific package commands, this  
>> is the option you should aim for.
>
> for text.tex.latex
>
> I've make somme additions :
>
> {name = 'meta.preamble.latex';
> contentName = 'support.class.latex';
> begin = '((\\)(?:usepackage|documentclass|RequirePackage| 
> usetikzlibrary))(?:(\[)([^\]]*)(\]))?(\{)';
>
> and some modifications
>
> 		{name = 'constant.numeric.math.latex';
> 		match = '((\+|-)?)(([0-9]*[\.][0-9]+)|[0-9]+)';
>
> because There is constant.numeric.math.tex and no  
> onstant.numeric.math.latex and i want the sign + or - in the same  
> color !
>
> for text.tex
>
> With keyword.control.tex, I've a problem with some terms like  
> \if at tkz@visible or  \ifTKZ at tkzInit@NO . The symbol @ is a problem  
> and in a package author there a lot of terms with @.
>
> I add name = 'keyword.operator.tex'
>
> with begingroup, endgroup, global def edef xdef gdef expandafter  
> newbox newdimen newcount advance multiply divide etc...
>
> but i'm not sure like keyword.control.tex ( with if else fi) that's  
> necessary to a classic user of TeX.

Thanks for these additions, I'll get them in when I get a chance.

> When you include text.tex in text.tex.latex, it's not a good thing.  
> there are good package to avoid the use of if etc...
> for example ifthen

I am not following you here, why is it not a good thing?

> A latex user must use \ifthenelse  \newboolean \setboolean \isodd  
> \whiledo \equal \lengthtest all these macros are in Latex and not  
> in tex.
>
Oh these are not tex? Ok, we can fix those, is that the complete list?

> Important keywords in LaTex are newcommand, renewcommand,  
> newcounter, setcounter, newlength, setlength, addtolength and some  
> keywords tex are also necessary : advance multiply divide etc...
>

Basically I don't have a complete list of the commands in LaTeX, so  
we just need to keep on adding on our existing list.

>  I'm sorry because a lot of ideas are not in a good order.
>
> Regards Alain

Haris Skiadas
Department of Mathematics and Computer Science
Hanover College