D-Net

Initial commit - settings and project files added to svn:ignore

#1422 fixing Java Heap Space error while executing checksum postprocessing worfklow on pmc plaintexts

merging trunk changes with IIS-CDH-5.3.0 branch

#1381 porting pmc citations ingestion from cascading framework to pig. Moving code from icm-iis-ingest-pmc to icm-iis-transformers including itegration tests, removing obsolete scala code along with unneded dependencies. Switching subworkflow in primary workflow.

expecting null affiliations instead of empty array

adding missing affiliations field in input data, removing duplicates from outut

adding missing affiliations field in integration test expected output

#1315 propagating confidenceLevel to DocumentToConceptIds. Updating PIG transformer script by introducing concept identifiers deduplication UDF function picking record with the highest confidence level, introducing unit and integration tests. Propagating changes in document to concepts exporter module.

removing obsolete test resources

#1329 adding affiliations field in ExtractedDocumentMetadata PMC schema. Metadata extraction code refactoring by extracting code responsible for building Affiliation avro records to AffiliationBuilder class and sharing it with pmc ingestion. Implementing affiliations ingestion functionality in PmcXmlHandler covered with unit tests. Adding affiliations field support in ingest pmc metadata transformer.