如何使用解析器组合器处理'续行'

Question

我正在尝试使用Sprache解析器组合器库编写一个小解析器。解析器应该能够解析以单个\结尾的行作为不重要的空白。

问题

如何创建一个解析器，可以解析可能包含续行字符\的=符号后的值？ 例如

a = b\e,\ 
    c,\ 
    d 

应被解析为(KeyValuePair (Key, 'a'), (Value, 'b\e, c, d'))。

我是一般使用这个库和解析器组合器的新手。所以任何正确的方向指针都非常感谢。

我已经试过

测试

public class ConfigurationFileGrammerTest 
{ 
    [Theory] 
    [InlineData("x\\\n y", @"x y")] 
    public void ValueIsAnyStringMayContinuedAccrossLinesWithLineContinuation(
     string input, 
     string expectedKey) 
    { 
     var key = ConfigurationFileGrammer.Value.Parse(input); 
     Assert.Equal(expectedKey, key); 
    } 
} 

生产

尝试一个

public static readonly Parser<string> Value = 
     from leading in Parse.WhiteSpace.Many() 
     from rest in Parse.AnyChar.Except(Parse.Char('\\')).Many() 
      .Or(Parse.String("\\\n") 
      .Then(chs => Parse.Return(chs))).Or(Parse.AnyChar.Except(Parse.LineEnd).Many()) 
     select new string(rest.ToArray()).TrimEnd(); 

测试输出

Xunit.Sdk.EqualException: Assert.Equal() Failure 
      ↓ (pos 1) 
Expected: x y 
Actual: x\ 
      ↑ (pos 1) 

尝试2

public static readonly Parser<string> SingleLineValue = 
     from leading in Parse.WhiteSpace.Many() 
     from rest in Parse.AnyChar.Many().Where(chs => chs.Count() < 2 || !(string.Join(string.Empty, chs.Reverse().Take(2)).Equals("\\\n"))) 
     select new string(rest.ToArray()).TrimEnd(); 

    public static readonly Parser<string> ContinuedValueLines = 
     from firsts in ContinuedValueLine.AtLeastOnce() 
     from last in SingleLineValue 
     select string.Join(" ", firsts) + " " + last; 

    public static readonly Parser<string> Value = SingleLineValue.Once().XOr(ContinuedValueLines.Once()).Select(s => string.Join(" ", s)); 

测试输出

Xunit.Sdk.EqualException: Assert.Equal() Failure 
      ↓ (pos 1) 
Expected: x y 
Actual: x\\n y 
      ↑ (pos 1) 

Answer 1

您不得在输出线延续。这是最后一次单元测试的唯一问题。当你解析延续\\\n时，你必须从输出结果中删除它并返回空字符串。对不起，我不知道如何使用C＃sprache来做到这一点。也许有类似的东西：

Parse.String("\\\n").Then(chs => Parse.Return('')) 

我解决了使用combinatorix Python库的问题。这是一个解析器组合器库。 API使用函数而不是使用链接的方法，但想法是相同的。

以下是完整的代码注释：

# `apply` return a parser that doesn't consume the input stream. It 
# applies a function (or lambda) to the output result of a parser. 
# The following parser, will remove whitespace from the beginning 
# and the end of what is parsed. 
strip = apply(lambda x: x.strip()) 

# parse a single equal character 
equal = char('=') 

# parse the key part of a configuration line. Since the API is 
# functional it reads "inside-out". Note, the use of the special 
# `unless(predicate, parser)` parser. It is sometime missing from 
# parser combinator libraries. What it does is use `parser` on the 
# input stream if the `predicate` parser fails. It allows to execute 
# under some conditions. It's similar in spirit to negation in prolog. 
# It does parse *anything until an equal sign*, "joins" the characters 
# into a string and strips any space starting or ending the string. 
key = strip(join(one_or_more(unless(equal, anything)))) 

# parse a single carriage return character 
eol = char('\n') 

# returns a parser that return the empty string, this is a constant 
# parser (aka. it always output the same thing). 
return_empty_space = apply(lambda x: '') 
# This will parse a full continuation (ie. including the space 
# starting the new line. It does parse *the continuation string then 
# zero or more spaces* and return the empty string 
continuation = return_empty_space(sequence(string('\\\n'), zero_or_more(char(' ')))) 

# `value` is the parser for the value part. Unless the current char 
# is a `eol` (aka. \n) it tries to parse a continuation, otherwise it 
# parse anything. It does that at least once, ie. the value can not be 
# empty. Then, it "joins" all the chars into a single string and 
# "strip" from any space that start or end the value. 
value = strip(join(one_or_more(unless(eol, either(continuation, anything))))) 

# this basically, remove the element at index 1 and only keep the 
# elements at 0 and 2 in the result. See below. 
kv_apply = apply(lambda x: (x[0], x[2])) 

# This is the final parser for a given kv pair. A kv pair is: 
# 
# - a key part (see key parser) 
# - an equal part (see equal parser) 
# - a value part (see value parser) 
# 
# Those are used to parse the input stream in sequence (one after the 
# other). It will return three values: key, a '=' char and a value. 
# `kv_apply` will only keep the key and value part. 
kv = kv_apply(sequence(key, equal, value)) 


# This is sugar syntax, which turns the string into a stream of chars 
# and execute `kv` parser on it. 
parser = lambda string: combinatorix(string, kv) 


input = 'a = b\\e,\\\n c,\\\n d' 
assert parser(input) == ('a', 'b\\e,c,d')