Regression in Schematron Validation After Upgrading to ph-schematron-pure 8.0.3

[magdakw]

Description
After upgrading ph-schematron-pure from 7.1.3 to 8.0.3, we encountered issues with specific Schematron definitions. The behavior of certain validation rules has changed unexpectedly.

Minimal Reproducible Example
Consider the following Schematron definition:

<?xml version="1.0" encoding="UTF-8"?>
<schema xmlns="http://purl.oclc.org/dsdl/schematron"
        xmlns:cbc="urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2"
        queryBinding="xslt2">
    <title>EN16931 model bound to UBL</title>
    <ns prefix="ext" uri="urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2"/>
    <ns prefix="cbc" uri="urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2"/>
    <ns prefix="ubl" uri="urn:oasis:names:specification:ubl:schema:xsd:Invoice-2"/>
    <ns prefix="xs" uri="http://www.w3.org/2001/XMLSchema"/>

    <pattern xmlns="http://purl.oclc.org/dsdl/schematron" id="Check-Country">
        <let name="ISO-3166-RO-CODES" value="('AA','BB','BT','RO-CT','RO-BC')"/>
        <rule context="/ubl:Invoice">
            <report test="not(normalize-space(cbc:Country) = ('AA','BB','BT','RO-CT','RO-BC'))"
                    flag="fatal"
                    id="REG1">
                Error Entity1
            </report>
            <report test="not(normalize-space(cbc:Country) = $ISO-3166-RO-CODES)"
                    flag="fatal"
                    id="REG2">
                Error entity2
            </report>
        </rule>
    </pattern>
</schema>

And the following XML document:

<?xml version="1.0" encoding="UTF-8"?>
<Invoice xmlns="urn:oasis:names:specification:ubl:schema:xsd:Invoice-2"
         xmlns:cbc="urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2">
                <cbc:Country>BB</cbc:Country>
</Invoice>

Observed Behavior

• In ph-schematron-pure 7.1.3:
  • Both REG1 and REG2 passed successfully.
• In ph-schematron-pure 8.0.3:
  • REG1 still passes.
  • REG2 fails when <cbc:Country>BB</cbc:Country>.
  • However, if we change <cbc:Country>BB</cbc:Country> to <cbc:Country>AA</cbc:Country>, both REG1 and REG2 pass again.

Possible Cause
It seems that the handling of variables in Schematron expressions has changed in version 8.0.3. Specifically:

The variable $ISO-3166-RO-CODES is defined as a list:

  <let name="ISO-3166-RO-CODES" value="('AA','BB','BT','RO-CT','RO-BC')"/>

but may not be interpreted as an XPath sequence in the new version.

The expression

normalize-space(cbc:Country) = $ISO-3166-RO-CODES

might be evaluated incorrectly due to changes in how lists are processed in XPath/XSLT2 within ph-schematron-pure.

Constraints
We cannot modify the Schematron file because it is officially published by a government authority.

Expected Behavior
The validation logic should remain consistent between versions, or at least clear documentation should be provided regarding changes in variable handling.

Request for Help
Is there a known change in how variables containing lists are evaluated in version 8.0.3?
Is there a recommended workaround for handling such cases without modifying the original Schematron definition?
Any guidance or clarification would be greatly appreciated!

[phax]

Well, you found an interesting case. Indeed the pure variable resolution was changed for v8.0 as you can also see in https://github.com/phax/ph-schematron/wiki/News-and-noteworthy

Previously variables were only used as string and the string was replaced before the variable was evaluated. However, this was not 100% spec compliant and #164 provided the necessary fix for this. The major change is now, that variables are evaluated at the point of declaration, and the evaluated object is used instead of the String replacement. Unfortunately the "sequence of string" that you are using cannot be properly evaluated, so I need to dig a bit into this. Eventually I need to raise a ticket at Saxon.

Sorry for the inconvenience caused - the next version will have a test case :)

[phax]

Created https://saxonica.plan.io/issues/6703 for this

[phax]

Okay, the problem is in the API used in ph-schematron. I need to switch to Saxons s9api to make it work with Xpath 2 expressions

[blutorange]

Just for reference, I'm trying to update BPMNspector, which also uses Schematron. It also uses variables this way, which fails with the current schematron version. And I guess it's unrelated to Saxon -- I set a custom XPathConfig to use the default XPathFactory (http://java.sun.com/jaxp/xpath/dom)

https://github.com/uniba-dsg/BPMNspector/blob/55f1ce8d7c46fc247f2c55df51f80d297615128e/src/main/resources/EXT_descriptive.sch#L305-L315

    <let name="eventDefinitions" value="bpmn:eventDefinitionRef | bpmn:messageEventDefinition | bpmn:timerEventDefinition | bpmn:escalationEventDefinition | bpmn:conditionalEventDefinition | bpmn:linkEventDefinition | bpmn:errorEventDefinition | bpmn:cancelEventDefinition | bpmn:compensateEventDefinition | bpmn:signalEventDefinition | bpmn:terminateEventDefinition" />

    <iso:pattern name="EXT.100">
        <iso:rule context="bpmn:subProcess[@triggeredByEvent = 'false']/bpmn:startEvent">
            <iso:assert test="not(./$eventDefinitions)" diagnostics="id">EXT.100|No EventDefinition is allowed for Start Events in Sub-Process definitions</iso:assert>
        </iso:rule>
    </iso:pattern>