Peppol invoice regression testing
Which of your invoices does the next rule set break?
Peppol publishes new validation rules several times a year. The release notes tell you what changed. They cannot tell you whether it affects you — that depends on what your invoices actually contain.
No password. Metadata-only retention available from the first upload.
What a release note looks like
“Changed rule PEPPOL-COMMON-R052 and PEPPOL-COMMON-R053 to be errors instead of warnings.”
Those two rules are named “Danish chamber of commerce number” and “Danish ERSTORG number”. Read that note and you check your Danish invoices, if you send any.
3.0.20 → 3.0.21
Published565
41
PEPPOL-COMMON-R05337
PEPPOL-EN16931-R0073
PEPPOL-COMMON-R0521
- Corpus
- ubl · 565 docs
- Engine
- Saxon-HE
- Source
- docs.peppol.eu
Same invoices. Two rule sets. Every change traced.
37
of the 41 broken invoices were German.
Documents carrying a nine-digit company identifier tagged with the Danish ERSTORG scheme, where the rule wants DK followed by eight digits. A warning under 3.0.20. Fatal under 3.0.21.
A rule set is not a document you read.
It is a program that runs against your data. The only honest way to know what a new release does to your invoices is to run it against your invoices — which is the entire product, and the reason there is no clever shortcut in it.
The case no diff can see
A rule broke invoices without changing at all.
PEPPOL-EN16931-R007Same invoice, both releases. Nothing about R007 is different between them — but the variable it reads was rewritten, so the same document now resolves to a value the rule rejects.
3.0.20
accepted- Invoice carries
ProfileID = billing:02:1.0
- $profile is defined as
matches(ProfileID, 'billing:([0-9]{2}):1.0')any two-digit profile - so $profile resolves to
'02'
- R007 asserts
$profile != 'Unknown'
3.0.21
rejected- Invoice carries
ProfileID = billing:02:1.0
- $profile is defined as
'billing:01:1.0' → '01' 'billing_with_response' → '02' else → 'Unknown'
exact matches only - so $profile resolves to
'Unknown'
- R007 asserts
$profile != 'Unknown'
Only the marked rows differ. R007 itself is identical in both releases — same id, same flag="fatal", same test expression. Twelve <let> variables changed in 3.0.21, two of them regular expressions.
This is why the product replays documents instead of reading changelogs. A tool comparing rule identifiers and severities between releases reports no change here — not from carelessness, but because there genuinely is none to find in the rule. The full report carries every rule responsible and the invoices each one broke.
Three ways to answer the question
What each method can actually tell you.
| Can it tell you… | Release notes | Rule-id diff | Replaying the corpus |
|---|---|---|---|
| A rule changed | yes | yes | yes |
| A rule changed severity | sometimes | yes | yes |
| A rule changed behaviour without changing | no | no | yes |
| Whether any of it touches your invoices | no | no | yes |
| Which documents, by filename | no | no | yes |
| Answered on the day of release | no | sometimes | yes |
● yes · ◐ sometimes · ○ no — methods, not products. The third column is the only one that has seen your documents.
How it works
Four steps, no modelling.
No heuristic and no prediction anywhere in it. The rule set is compiled and executed exactly as an access point would execute it.
- 01
Pull
Schematron for each rule set, from docs.peppol.eu — the authority, not a mirror and not a git tag.
- 02
Compile
Three ISO skeleton stages turn the .sch into SVRL-producing XSLT, with Saxon-HE.
- 03
Run twice
Your corpus through each rule set. 565 invoices take about eight seconds per rule set.
- 04
Diff
Every invoice whose fatal outcome changed, grouped by the rule responsible.
Limits
What it does not do.
Stated here rather than discovered later.
- Only fatal transitions count as breakage
An invoice that passed and now fails is a regression. A new warning is reported separately and is not counted, because an access point will still accept the document.
- A removed rule is never a regression
When a rule disappears, invoices it used to reject start passing. That is reported as newly passing, never as something to act on.
- It needs your invoices
There is no useful answer without a corpus. The whole claim is that the question cannot be answered from a changelog, and that applies to us too.
- It tells you what broke, not how to fix it
You get the rule, the mechanism, and the exact documents. Correcting them is a change in the system that produced them.
Early access
Find out before your counterparty does.
Upload a corpus and run it against the last two releases — that alone usually answers whether this is a problem you have. Revalidate is in early access while the first customers shape it, so pricing is agreed directly rather than picked off a page.