11 Jul
2014
11 Jul
'14
9:27 a.m.
From my - admittedly somewhat limited - experience with bundle grammars
I got the feeling that it’s often very hard to improve or fix a grammar
without accidentally breaking some other parts of it.
Has anyone already experimented with regression tests for bundle grammars?
I think to make this possible it would be necessary to somehow access
the “scope structure” of a parsed file, maybe yielding a data structure
like this:
[
{ start: 0, end: 123, scope: "source.sass" },
{ start: 7, end: 12, scope: "string.quoted.double.sass" },
…
]
Then a grammar could include both a “test source file” and its
known-good “scope structure”. After changing the grammar one could
(semi-)automatically regenerate the scope structure data and compare it
to the original to (hopefully) catch any regressions.
Any thoughts?
Stefan
14 Jul
14 Jul
8:03 a.m.
On Jul 11, 2014, at 4:27 AM, Stefan Daschek <stefan(a)daschek.net> wrote:
From my - admittedly somewhat limited - experience
with bundle grammars I got the feeling that it’s often very hard to improve or fix a
grammar without accidentally breaking some other parts of it.
Has anyone already experimented with regression tests for bundle grammars?
The biggest issue with tests for grammars is simply writing tests that are worthwhile. The
errors you typically see are the exceptional cases you wouldn’t have thought to test for.
But it can prevent the same error from happening twice.
The second issue is that there is no ‘perfect’ way to scope a portion of code and thusly
the common manner of writing tests would break every time you made even the slightest
change to the grammar. To be something people would use it needs to be relatively
painless.
I’ve put a lot of thought into how to solve this for grammars and my current best thoughts
are:
A two part test with a string of text to be matched and then a series of assertions at
various points in the string. An example:
"foo\"bar"
1:6 – B:constant.character.escape
1:8 – string
The string would be parsed and then the matches provide a caret position and a scope to
match against. Here we use a B: scope to ensure we are inside an escape and then after
that point ensure the string didn’t end early. The scopes are very intentionally as vague
as possible to prevent minor changes in scopes from invalidating the test. Here we are
only worried about the string and the escape so it’s all we test for.
15 Jul
15 Jul
5:29 p.m.
Hello all,
I love this and it could be used also to validate/test theme coverage.
What do we have to do to get something like this created so bundle authors
can use it?
Take care,
Jeremy Whitlock (@whitlockjc on Twitter)
http://www.thoughtspark.org
On Mon, Jul 14, 2014 at 2:03 AM, Michael Sheets <mummer(a)whitefalls.org>
wrote:
On Jul 11, 2014, at 4:27 AM, Stefan Daschek
<stefan(a)daschek.net> wrote:
From my - admittedly somewhat limited -
experience with bundle grammars
I got the feeling that it’s often very hard to
improve or fix a grammar
without accidentally breaking some other parts of it.
Has anyone already experimented with regression tests for bundle
grammars?
The biggest issue with tests for grammars is simply writing tests that are
worthwhile. The errors you typically see are the exceptional cases you
wouldn’t have thought to test for. But it can prevent the same error from
happening twice.
The second issue is that there is no ‘perfect’ way to scope a portion of
code and thusly the common manner of writing tests would break every time
you made even the slightest change to the grammar. To be something people
would use it needs to be relatively painless.
I’ve put a lot of thought into how to solve this for grammars and my
current best thoughts are:
A two part test with a string of text to be matched and then a series of
assertions at various points in the string. An example:
"foo\"bar"
1:6 – B:constant.character.escape
1:8 – string
The string would be parsed and then the matches provide a caret position
and a scope to match against. Here we use a B: scope to ensure we are
inside an escape and then after that point ensure the string didn’t end
early. The scopes are very intentionally as vague as possible to prevent
minor changes in scopes from invalidating the test. Here we are only
worried about the string and the escape so it’s all we test for.
_______________________________________________
textmate mailing list
textmate(a)lists.macromates.com
http://lists.macromates.com/listinfo/textmate
2472
days inactive
2476
days old
2 comments
3 participants
participants (3)
-
Jeremy Whitlock -
Michael Sheets -
Stefan Daschek