Expected language-tagged values disappear when the field is present without the languageID

[schivmeister]

Problem

Language-tag-related mappings such as for BT-99-Lot apply a function on the languageID attribute of the field in question, making the attribute mandatory. Therefore, not having a languageID will result in an unmapped value, which is not expected.

Mapped

<cbc:Description languageID="SWE">Publikationsbyråns lokaler i Luxemburg.</cbc:Description>

Unmapped

<cbc:Description>Premises of the Publications Office in LUXEMBOURG.</cbc:Description>

Originally posted by @csnyulas in #96 (comment)

I understand that this change is because changes to the SDK example XML files in the patch versions, such as this.
However, it seems really dubious that, after previously having multiple values, in multiple languages, now we have no values at all. Why don't we have the one value present in the latest XML version, without the language tag, present in the output? This looks like a bug in our mapping. Same applies to the epo:ReviewTerm instance below.

Cause

At one time we had to change how we write mappings for data with language tags. For that we commented out the original mapping and created a new one with the iterator directly at the XPath of the data itself (in most cases a description), for e.g. BT-99-Lot:

tedm:MG-langString-hasReviewDeadlineInformation-ReviewTerm-isSubjectToLotSpecificTerm-Lot_ND-ReviewPresentationPeriod a rr:TriplesMap ;
  rml:logicalSource [
    rml:source "data/source.xml" ;
    rml:iterator "/*/cac:ProcurementProjectLot[cbc:ID/@schemeName='Lot']/cac:TenderingTerms/cac:AppealTerms/cac:PresentationPeriod/cbc:Description" ;
    rml:referenceFormulation ql:XPath ;
  ] ;

This mapping also includes a function that acts on the attribute and has the RML reference @languageID. It therefore only maps data with the languageID attribute and skips those without, leaving such fields (where a language tag is optional) unmapped.

Coincidentally, this issue also triggers #121, where an RMLMapper error is thrown because of the function exception. However, even if RMLMapper did not throw an error, fields with an optional language tag (languageID attribute) will remain unmapped.

SDK Versions

While we detected this issue when an SDK 1.3 example was updated to the latest patch version (later retracted along with the fix so as to not infringe upon pre-existing production mappings), this issue applies to all eForms SDK versions.

Potential Fix

The original mapping of these language-related mappings can be restored, but with a specific condition on the absence of the languageID attribute. There will then be two parallel mappings for the same field.

Data Impact

Both this and #121 may currently only manifest in and be demonstrable using SDK example data. There is no known live sample or production data that exhibits the conditions required for this issue to be triggered.

[schivmeister]

This is indeed a bug, as the dedicate TMap for the languageMap fails to apply, and there is no alternative mapping for a non-language tagged one. We would have to bring back the old mappings and add a condition there to be activated only when there is no languageID.