Project

General

Profile

Statistics
| Revision:

# Date Author Comment
37980 26/06/2015 07:46 PM Marek Horst

#1395 WorkflowRuntimeParameters static fields cleanup, moving parameters to dedicated modules to prevent excessing icm-iis-common module modifications

37343 20/05/2015 06:49 PM Marek Horst

#1329 adding affiliations field in ExtractedDocumentMetadata PMC schema. Metadata extraction code refactoring by extracting code responsible for building Affiliation avro records to AffiliationBuilder class and sharing it with pmc ingestion. Implementing affiliations ingestion functionality in PmcXmlHandler covered with unit tests. Adding affiliations field support in ingest pmc metadata transformer.

36394 15/04/2015 05:17 PM Marek Horst

#1240 raising mapred.task.timeout to 3600000 (1h) just in case any extremely complex PDF document appear. All time consuming documents will be registered in failure sink.

36289 09/04/2015 07:10 PM Marek Horst

#1257 dropping schema generation related hacks in all map-reduce modules, switching to literal schema parameters

35986 03/04/2015 01:30 PM Marek Horst

#1248 persisting content url in supplementaryData to make it easier to find content causing failure

35937 02/04/2015 04:15 PM Marek Horst

#1248 bugfix renaming inputEntityId to inputObjectId after schema changes

35936 02/04/2015 04:01 PM Marek Horst

#1248 introducing failures sink datastore support in metadata extraction module

35832 30/03/2015 07:17 PM Marek Horst

#1240 extending mapred.task.timeout for metadata extraction to 30 minutes

35701 27/03/2015 06:18 AM Mateusz Kobos

Removing usage of working_dir from Java workflow node.

33369 12/12/2014 04:08 PM Marek Horst

#953 extending maximum heap size from 2048 to 4096 after Dominika introduced iText dependency upgrade to 5.5.3 in cermine. This combination should minimize the possibility of failures caused by fatal OOMErr.

31845 28/10/2014 03:31 PM Marek Horst

#913 renaming DocumentContentUrl#contentSize to DocumentContentUrl#contentSizeKB changing field type from int to long, importing content size from ObjectStoreFile#fileSizeKB, updating dnet-objectstore-rmi dependency from 1.0.0 to 2.0.1-SNAPSHOT

31782 28/10/2014 11:35 AM Marek Horst

#913 reading content size from newly introduced DocumentContentUrl#contentSize field not from URLConnection where size is not available when Transfer-Encoding=chunked, setting to 0 when size is not available

31757 27/10/2014 06:11 PM Marek Horst

#913 introducing support for max file size parameter, currently checked against Content-Lenght header

30887 25/09/2014 09:50 PM Dominika Tkaczyk

affiliation's address and country code passed from Cermine to Avro

30878 25/09/2014 05:07 PM Marek Horst

fixing converter code after recent Affiliation.avdl refactoring and adding countryCode field, renaming contry to countryName

29839 25/08/2014 02:14 PM Dominika Tkaczyk

convertBibEntry() made public

28805 02/07/2014 11:48 AM Marek Horst

setting excluded_ids to undefined value