For incremental harvesting: return the date with date granularity level.
Test the logics of mdbuilder -- just in case
migration to spark
Added snapshot repository from FORTH, just in case
more testing for Ariadne and 3M mappings
removed unused test file
removed mapping from profiles section, mapping is taken only from url; x3m transformation node moved to dnet-ariadneplus module
aligned to aggregator instance
x3ml-engine version reset to 1.9.3
testing usage of 1.9.2-SNAPSHOT for troubleshooting issue with ariadne mapping
reverted to 57117
using version 1.9.2-SNAPSHOT of x3ml to troublesheet issue in ariadne
align svn folder info
mapping url downloaded from remote url set in workflow parameters
reverted for solr5
fixed dependencies and deploy for solr772
fixed dependencies and deply info for solr772
branch for solr772
x3ml-engine 1.9.1 does not help, so let's go back to 1.9.3
trying with x3ml-engine v1.9.1
using x3ml-engine version 1.9.3
[maven-release-plugin] prepare for next development iteration
[maven-release-plugin] copy for tag dnet-msro-service-6.0.1-SAXONHE
[maven-release-plugin] prepare release dnet-msro-service-6.0.1-SAXONHE
Updated URL for FORTH repository to avoid the "peer not authenticated" error when compiling
Default source should be the api, otherwise the delete of an api of one ds will trigger the deletion of all records for the data source in the OAI-PMH
Sticking to release version 3.9.0: using synchronised is not helpful, maybe the 1.9.0 release helps
trying to use x3ml-engine inside a synchronised method as suggested in https://support.d4science.org/issues/12919#change-77131
Log input file that cannot be transformed
Parthenos mappings now require new x3ml-engine version and field <thesaurus_info/> is now mandatory.
Updated default mail template with start and end dates in human readable version
Human readable start and end date
BBActorCallback now can pass parameters back to the workflow for logging purposes.
Added CrossRef plugin
fixes a xquery
fixed compilation error
merged r52660: email dispatching for wf template based on parent workflow notification configuration
Sending mails also to addresses in parentProfile
fixed dependency to x3ml-engine
Testing URI shortener
x3ml-engine upgraded to spring4: removed exclusion and enlarged the dependency version range
ensure we do not get 1.8.5-SNAPSHOT
Opened range for x3ml-engine dependency: up to 1.9.0 excluded
Have the x3ml engine generate real UUIDs instead of test values.
added log
implemented incremental transfrom and resolving
cleanup
MERGE: branch updated with trunk changes through r50565
Trying to exclude slf4j deps from x3ml to see if it helps in properly logging.
We can use x3ml version 1.8.2 but not later version (compilation problems due to some spring conflicts, apparently)
Job nodes related to Parthenos now extends AsyncJobNode
[maven-release-plugin] copy for tag dnet-msro-service-6.0.0-SAXONHE
[maven-release-plugin] prepare release dnet-msro-service-6.0.0-SAXONHE
removed ennoying log
logs
Do not log the content of mapping files
Tests moved from msro to dnet-parthenos, where they should have belonged in the first place
New parameter to decide to pass the full record or only the metadata content to x3m
fix find Date range bug
removed until
Using RDF_XML_PLAIN serialization.
Updated mapping for culturaItalia and ignored its test until they fix the mapping.
Add dri:datasourceapi in the header
Updated Clarin 365 mapping for tests
Upgraded incremental harvesting and other stuff
Updated tests and mappings
Fixed logs
More tests fro x3m
Ignore CLARIn test that fails for duplicate namespace prefix
Testing #8664#note-5 for EHRI and namespace
More tests for EHRI and CulturaItalia
Testing Parthenos 3M mappings
Logging
Returning more detailed counters for mdstore feeding.Fixed bug when trying to store too large XML: now all the page of XMLs is skipped, so we can go on with the other records (MongoBulkWritesManager)
Logging the extracted metadata for debugging
Added tests with real mappings
X3M Transformation node must be able to handle multiple 3M mappings
Handling workflow parameters of type 'list'
New type of workflow parameter that enables multiple selection of values from a list obtained via a function: "list"
Now we can tune the X3ML engine logs: https://support.d4science.org/issues/8117
fixed bug
Moved class in more proper package
Interpretation is not a solr-only property
Handle date range requests
New jobnode to get the last start date of a workflow of the same type of the current. The old job node is still needed to get the date as formatted string
Just one job node for refresh/incremental harvesting
UpdateIndex must consider also the interpretation, not only format and layout
Execution time monitored in debug mode on the jobNode, as the preparation for the mapping application does not impact on the total transformation time (from 0 to 3 ms, while transformation takes 100-200 ms)
logs for monitoring transofrmation execution time
Cleanup
calling the right deliver methods based on the noCursorTimeout param
Using new method to disable cursor timeout when reading from mdstore
Test
checking how long it takes to transform
I do not want to see dri:repositoryIdentifier anymore
removed some DEBUG log
DEBUGGING THE TIMEOUT OF THE MONGO CURSOR: is there a metadata record that it is really slow to transform?