Added support for Left Recursive Grammars, using the algorithm from the Ometa paper

[joachimm]

Also added some stuff in the TextMate bundle

[mjackson]

Wow! This is awesome. Thanks for this work. I'll be reviewing these commits as soon as I get the chance.

[davidkellis]

Is http://www.vpri.org/pdf/tr2007002_packrat.pdf [1] the Ometa paper? I didn't know what ometa was until I started searching for it. The reason I ask is because I'm reading through http://tratt.net/laurie/research/publications/papers/tratt__direct_left_recursive_parsing_expression_grammars.pdf [2] and the author, Tratt, makes the observation that Warth's Ometa tool, which implements the technique described in [1], incorrectly parses the expression 1-2-3, as (1-(2-(3))), given the grammar:
expr <- expr "-" expr / num
num <- [0-9]+
joachimm, does your implementation of Warth's technique produce the same right-associative parse as Warth's Ometa tool? I'm thinking that Tratt's alteration to Warth's algorithm might parse a broader class of languages. On the other hand, this might not be an issue at all. I'm just bringing it up because I haven't tinkered around enough with your implementation to know, and reading through Tratt's paper brought the issue to my attention.

Thanks!

[mjackson]

David, Thanks for researching this. As I mentioned above, I'd love to be able to support left recursion in Citrus, but I haven't had the time to thoroughly test the algorithm myself yet. It's an important change, and it deserves careful consideration.

[joachimm]

David, yes [1] is the Ometa paper. According to this test I believe left associtivity works as intended https://github.com/joachimm/citrus/blob/master/test/indirect_left_recursion_test.rb

[davidkellis]

I just tried implementing the special case (a grammar production is both left-recursive and right-recursive) that Tratt documented in his paper [2].

The grammar is:
expr <- expr "-" expr / num
num <- [0-9]+

In left_recursion_test.rb, I slightly modified your LR grammar:

  grammar :LR do
    rule :expr do
      any(all(label(:expr, 'lhs'), '-', label(:expr, 'rhs')){lhs.value - rhs.value},
          :num)
    end

    rule :num do
      ext(/[0-9]+/){to_i}
    end
  end

  def test_lr
    match = LR.parse("3-4-5", {:leftrecursive=>true})
    assert(match)
    assert_equal("3-4-5", match)
    assert_equal(-6, match.value)
  end

Unless I've done something wrong, which is not unlikely, this test fails because 3-4-5 is parsed as (3 - (4 - 5)) and produces the result 4.

Can you verify that this is in fact an issue. I'm not 100% sure that I've set up a proper test to verify whether or not Tratt's observation is an issue in this implementation.

Thanks.

[joachimm]

You are right. Making rules both left-recursive and right-recursive, seems to give unwanted results. Thanks for the link to the Tratt's paper. I will see if I can fit his idea into the algorithm.

[CMCDragonkai]

Hey @joachimm have you made any progress on this? I'm interested in whether Warth's implementation is correct?

[davidkellis]

Yesterday I ran across ohmjs/ohm#55, which addresses the same "incorrect parse" in another PEG parser library, and @alexwarth's opinion - ohmjs/ohm#55 (comment) - is that Tratt's conclusion is just one interpretation, but nothing definitive.

Something to consider. It may boil down to preference.