Clarify `LANGMATCHES` specification

[kasei]

The current definition of LANGMATCHES reads:

xsd:boolean langMatches (xsd:string language-tag, xsd:string language-range)
Returns true if the language tag (first argument) matches language-range (second argument). Otherwise, the function returns false. Matching is performed according to the basic filtering scheme defined in [RFC4647] section 3.3.1. language-range is a basic language range per Matching of Language Tags [RFC4647] section 2.1. A language-range of "*" matches any non-empty language-tag string. Otherwise, the function returns false.

I think there's an implicit assumption here that the language-range is in fact a valid language range per the RFC (similarly for language-tag being a valid language tag). In most cases, if it weren't I think it would necessarily not match any valid language-tag. However, in the case of language-range being an empty string (which is not a syntactically valid range), I think there's some ambiguity here. A generous reading would just fall through to the "otherwise return false" text, but I think a better handling here would raise an error if the input variables don't conform to the expected form (which in many cases would likely have the same overall behavior as returning false). Either way, I think improving the text to explicitly define what happens in the case of invalid language tags and ranges would be good.

[afs]

Please can we have some concrete examples? (and then some test cases - we can use test cases to clarify, or even capture preferred behaviour)

The "" case is different from the invalid, non-empty language-tag case.

The RDF data and SPARQL grammars allow invalid but non-empty strings (+ see below). I think we have be cautious because there is public data with improper language tags and we have to live with that.

if it weren't I think it would necessarily not match any valid language-tag.

How can an invalid language-range basic-match a valid language tag? or a RDF/SPARQL grammar-compliant language tag?

RFC 4647 -->

A language range matches a
particular language tag if, in a case-insensitive comparison, it
exactly equals the tag, or if it exactly equals a prefix of the tag
such that the first character following the prefix is "-"

The following character rule makes a range a subtag sequence prefix, not a string prefix, even if implemented by string matching.

A string of "" does not match a valid language-tag because the language tag can not start with "-".

Aside for the grammar, the other place in SPARQL is term-constructors:

17.4.2.16 STRLANG
17.4.2.17 STRLANGDIR

The argument langTag MUST not be an empty string and SHOULD be a a valid language tag.

[kasei]

Adding language similar to the term constructors to LANGMATCHES would be an improvement, IMO.

Understood about user data with invalid langtags. I don't want to venture into fully validating these things, as SPARQL has historically pretty hands-off about that. But the case of an empty range stood out to me as being sufficiently different that it might deserve special handling (and either way deserves clarification).

So here are some examples to consider:

LANGMATCHES("", "")
LANGMATCHES("en", "")
LANGMATCHES("-GB", "")
LANGMATCHES("😱", "😑")
LANGMATCHES("😆", "😆")
LANGMATCHES("😆", "😆-qq")
LANGMATCHES("😆", "😆-🤷‍")

I'd be happy to write some test cases, but I'm honestly not sure what the expectation is here.
I could write the tests to assert either false or type error (unbound), but assume early choice here may be changed in discussion.
None of these are valid pairs of tags and ranges, so I'm not sure the current spec text makes sense applied to these.

[kasei]

A string of "" does not match a valid language-tag because the language tag can not start with "-".

Agreed, but my concern here is that there are no current restrictions on the arguments to LANGMATCHES that ensure they are "valid language-tag [and -range]". If we ignore the need for such restrictions, I think the RFC text would suggest that LANGMATCHES("-GB", "") would be true.

[afs]

One non-solution is to change the signature to use xsd:language. That's impractical.

Therefore, is the proposal to note language tags can't be "" (which is the more important point IMO), and that arguments SHOULD comply with the requirements of RFC 4647?

[kasei]

Yeah, I think xsd:language is a non-starter.

I wasn't proposing anything specific at this point. What I would like is for the spec to be clear about what the result of calls to LANGMATCHES should be when the input is not of the expected form. I think there are some obvious cases here: empty strings (which are by definition and trivially not valid tags/ranges), values that match SPARQL's grammar or * (which may be valid, and SPARQL has always been defined to accept these), and non-empty values that do not match SPARQL's grammar.

I think I'd be OK with any solution that improves clarity of what results should be. That could be just improving the text to make false the fallback value for both non-matching and invalid inputs. But it could also be carving off things like the empty string case as invalid input and defining that as an error case (in that case, I think it might be good to pair that with SHOULD language about tags/ranges that happen to match the SPARQL grammar but which are invalid per the RFC).

I could just choose an option and submit a PR, but I was hoping to see if anyone had preferences one way or the other before proceeding.

[afs]

PR!

That could be just improving the text to make false the fallback value for both non-matching and invalid inputs.

false means is known to be not true. For invalid inputs it would be an error - like xsd:integer("word").

[kasei]

@afs – possible clarification in #316.

The risky part about an error is that (unlike your xsd:integer example), the SPARQL grammar is less strict than the RFC grammar. So there's some danger if we went that path and there were existing systems that implemented and depended on matching behavior that works with the more lenient SPARQL-allowed tags which would suddenly become errors.