Revision 52985
Added by Claudio Atzori over 5 years ago
| ConnectedComponentsReducer.java | ||
|---|---|---|
| 50 | 50 |
final byte[] root = DedupUtils.newIdBytes(ByteBuffer.wrap(Bytes.toBytes(key.toString())), dedupConf.getWf().getDedupRun()); |
| 51 | 51 |
|
| 52 | 52 |
StreamUtils.toStream(values.iterator()) |
| 53 |
// .limit(1000) // might cause timeouts in case of large number of items
|
|
| 53 |
// .limit(1000) // might cause timeouts in case of large number of items |
|
| 54 | 54 |
.flatMap(v -> v.getEdges().stream()) |
| 55 | 55 |
.forEach(q -> {
|
| 56 | 56 |
final byte[] qb = Bytes.toBytes(q.toString()); |
Also available in: Unified diff
do not skip processing datasets in DedupBuildRootsMapper, improved error reporting in DedupBuildRootsReducer