D-Net

merging trunk changes with 20150518_new_funding_model branch

#1304 updating both project importer modules, reading from rdb and hbase, by supporting new XML funding tree representation.

#1302 introducing 20150518_new_funding_model branch

#1302 introducing support for updated project model containing funding tree defined as XML instead of JSON, not enabled yet.

Removing usage of working_dir from Java workflow node.

#1195 removing obsolete ports docreation and datasetid from hbase mapred import, removing references to those ports in workflow.xml files, updating transformer by removing filtering by datasetid due to decisions made in #1072

#1109 fixing building excluded acronym values

#1109 utilizing isAcronymValid() method in relational db importer. skipping project grant id whenever code is empty

#1109 making isAcronymValid() method public so it could be utilized by relational db importer as well

#1109 making isAcronymValid() method public so it could be utilized by relational db importer as well

#1109 introducing support for multiple acronym values to be skipped, currently set to 'unknown' and 'undefined' values.

#1070 introducing support for multiple context identifiers, replacing import_project_concepts_context_id IIS input parameter with import_project_concepts_context_ids_csv

#968 aligning IIS importer with ObjectStore#deliverObjects() API method changes

#919 supporting multiple profiles in concept importer, logging error instead of throwing exception when profile not found

removing obsolete package

#919 introducing Concept schema and importer module producing avro datastore based on XML profile

#963 propagating dataset -> mdstore from import to exporting phase: importer produces DocumentToMDStore datasetore utilized by exporter module. Updating transformer definition to handle DocumentToMDStore instead of Identifier schema

#118 introducing piwik logs importer module

#913 renaming DocumentContentUrl#contentSize to DocumentContentUrl#contentSizeKB changing field type from int to long, importing content size from ObjectStoreFile#fileSizeKB, updating dnet-objectstore-rmi dependency from 1.0.0 to 2.0.1-SNAPSHOT

#913 temporarily setting contentSize to -1 in ObjectStore DocumentContentURL importer module until ObjectStore exposes proper size value

removing redundant logging

logging added: imported -> processed, not all of them were imported

logging added: presenting total number of imported records

logging added when id type of id value is null and record is not written

#883 introducing support for blacklisting object store identifiers

bugfixing citations converter by prefixing identifier with 50| prefix which was removed when exporting destination document id in BLOB exporter

introducing support for handling update column qualifiers holding inferenced data, disabled by default

fixing NullPointerException in citations exporter

introducing regex support in result approver to support iis::* kind of provenance, updating workflow definitions with proper regex values

#840 moving IdentifierMapping from importer to common package

#840 renaming DeduplicationMapping to more generic IdentifierMapping

imports cleanup

#637 treemap->hashmap, order is not preserved anyway

#637 introducing ISLookup based vocabulary importer

#433 introducing natural citations ordering

#799 updating header name from header to oai:header. Introducing additional check verifying empty id.

moving ACM importer to icm-iis-mainworkflows due to extending dependances with cermine, introducing performance tests

checking whether trust level is empty before comparing to predefined threshold

introducing trust level threshold support when importing information space data

introducing shared citation ExtraData XML model in icm-iis-common, implementing citation importer in mapred_import workflow, implementing exporter module

supporting $UNDEFINED$ value in IMPORT_INFERENCE_PROVENANCE_BLACKLIST

#527 introducing ACM XML dump importer module importing bibliographic references for further citation-matching analysis

extending progress log interval from 10 000 to 100 000

fixing importing abstract after introducing fieldApprover for all Result fields

introducing fieldApprover for all Result fields