added more detailed counter about entity sub-type
several improvements
Increment counter in case of no rows to keep track of records without body.
updated version to 0.0.6.3.1
including changes to catch and fail for any exception of r35769 of trunk
branch for code before the re-implementation of context and fundingpaths
raised version
commenting test with big doaj dataset
different escaping
trying to catch any kind of exception
Testing DOAj for #1222#note-4
added DedupSimilarityToActionsMapper and relative dependency
increased version in scripts
updated the version of a dependency
fundingtree is an escaped xml, not a json anymore.
increased a minor virsion
sample records
reimplemented the fundingpath and context generation
updated packages
updated packages, codestyle
codestyle
OafMerger moved to mapping utils
temporary commit
offline dedup
added protobuf-java-format dependency
renamed test
added json size test
saving disk space, less logging
Updated configuration for testing
extended entities join configuration, added more tests
scripts using updated version 0.0.6.3
test record took from HDFS
discard persons in OAI feeding (#1107)
do not alter inferenceprovenance; codestyle
Using released hadoop parent.
added FCT fundings as contexts
merged branch ProtoMapping
imlemented retries
Added oaf:identifiers to record sample.
ignored iml file
updated tests
updated scripts
[maven-release-plugin] prepare for next development iteration
[maven-release-plugin] copy for tag dnet-mapreduce-jobs-0.0.5
[maven-release-plugin] prepare release dnet-mapreduce-jobs-0.0.5
removed extra scm tag
added scm
bumped version, updated parent: let's start to depend on releases
cleanup & tests
added default bestlicense value. Used when the records doesn't provide any
added more fields in test record
Moved counters from entity body to header.
- provenance information parsed from element "about" - namespace aware datacite mapping for oaf:language and oaf:dateaccepted - dedupBuildRoot doesn't write to WAL- removed unused claim_2_hbase.xsl- overall cleanup
added relationship/children counters
revised tests
Avoiding set '___' generated when we have "strange" set names such as those in cyrillic/ukrain. In those cases records are assigned to a default set, currently named "OTHER".
expanding provenanceaction classid
merge from branch newIndexFeed
fixing #783 (note-18)
extraInfo removed from CDATA block, expanding provenance action in inferred elements
fixed dependencies
created tag folder for release
removed CDATA from extraInfo payloads
using CloudSolrServer for parallel index feeding
added branch name
updated branch version and build scripts
update branch with contributes from trunk and other branches
changed properties passed to index feed m/r job
fixed pom and scripts
updated index feed job to make use of the new shared solr lib
branch to test the new index feeding libs
fixed blacklist type
more logging. fixed entity type check
more logging
defined limit to the maximum number of counters
added serialization, tests
instantiate one SAXReader for each call
fixed format-layout-interpretation concatenation,doesn't fail when the fieldExtractor returns a null result
added json serialization, builds the matching key one time only
do not upsert sets here in the mapper: we shall delegate to a separate workflow to be run after the OAI feeding is completed.
early implementation of jar upload script
Always new records to test how faster we go
Refactored class that extracts fields from records. When we can't find an expected index from the configuration to check its repeatability, the field is indexed as repeatable and a counter is updated.
idScheme and idNamespace defined as part of the OAI configuration profile
proto to pace mapping parses the whole entity
Removed dependency to dnet-oai-utils to avoid inheritance of unwanted jars such as cnr-rmi-api, cnr-service-common, spring, etc., which should not appear when running a job on the cluster. Needed classes have been copied and adapted so they do not use spring anymore.
branch to adapt the proto to pace mapping
extended dedup configuration, including now blacklists and algorithm parameters
Format, layout and interpretation are obtained from the collection name rather than being fixed.
namespace cleanup
removed unused field <dri:repositoryId/>
removed protocolbuffers dependency from dnet-pace-core, Builders and Proto specific tests moved in dnet-openaireplus-mapping-utils, adapted dnet-mapreduce-jobs