D-Net

expecting null affiliations instead of empty array

adding missing affiliations field in input data, removing duplicates from outut

adding missing affiliations field in integration test expected output

#1315 propagating confidenceLevel to DocumentToConceptIds. Updating PIG transformer script by introducing concept identifiers deduplication UDF function picking record with the highest confidence level, introducing unit and integration tests. Propagating changes in document to concepts exporter module.

removing obsolete test resources

#1329 adding affiliations field in ExtractedDocumentMetadata PMC schema. Metadata extraction code refactoring by extracting code responsible for building Affiliation avro records to AffiliationBuilder class and sharing it with pmc ingestion. Implementing affiliations ingestion functionality in PmcXmlHandler covered with unit tests. Adding affiliations field support in ingest pmc metadata transformer.

#1301 skipping transformation when input set to $UNDEFINED$ value

#1301 removing redundant schema parameter

#1301 introducing generic avro to json transformer

bugfix: adding missing start element