1 |
26600
|
sandro.lab
|
<?xml version="1.0" encoding="UTF-8"?>
|
2 |
|
|
<?xml-model href="http://www.oasis-open.org/docbook/xml/5.0/rng/docbook.rng" schematypens="http://relaxng.org/ns/structure/1.0"?>
|
3 |
|
|
<?xml-model href="http://www.oasis-open.org/docbook/xml/5.0/rng/docbook.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
|
4 |
|
|
<book xmlns="http://docbook.org/ns/docbook"
|
5 |
|
|
xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0">
|
6 |
|
|
<info>
|
7 |
|
|
<title>DNet Object Store Module</title>
|
8 |
|
|
<author>
|
9 |
|
|
<orgname>ISTI CNR</orgname>
|
10 |
|
|
<email>sandro.labruzzo@isti.cnr.it</email>
|
11 |
|
|
</author>
|
12 |
|
|
</info>
|
13 |
|
|
<part>
|
14 |
|
|
<title>ObjectStore API</title>
|
15 |
|
|
|
16 |
|
|
<chapter>
|
17 |
|
|
<title>Introduction</title>
|
18 |
|
|
<sect1>
|
19 |
|
|
<title>What is an Object Store</title>
|
20 |
|
|
<para>Object Store Service is a DNet service that provides functions for perstistent
|
21 |
|
|
storage of files, and provides a REST API for fetching them back. </para>
|
22 |
|
|
<para>It is composed by three different sub-modules:</para>
|
23 |
|
|
<para>
|
24 |
|
|
<itemizedlist>
|
25 |
|
|
<listitem>
|
26 |
|
|
<para><link
|
27 |
|
|
xlink:href="https://svn.driver.research-infrastructures.eu/driver/dnet11/modules/dnet-objectstore-rmi/trunk/"
|
28 |
|
|
>dnet-objectstore -rmi</link>: it contains the interface of the
|
29 |
|
|
Object Store Service and includes some utility classes defining
|
30 |
|
|
metadata information for each file (i.e., ObjectStoreFile). These
|
31 |
|
|
metadata allow the Object Store Service to store and fetch the
|
32 |
|
|
associated files in a consistent way.</para>
|
33 |
|
|
</listitem>
|
34 |
|
|
<listitem>
|
35 |
|
|
<para><link
|
36 |
|
|
xlink:href="https://svn.driver.research-infrastructures.eu/driver/dnet11/modules/dnet-modular-objectstore-service/trunk/"
|
37 |
|
|
>dnet-modular-objectstore-service</link>: it contains the high
|
38 |
|
|
level implementation of the service. It manages blackboard messages,
|
39 |
|
|
the creation of profiles in dnet, and defines the REST API.</para>
|
40 |
|
|
</listitem>
|
41 |
|
|
<listitem>
|
42 |
|
|
<para><link
|
43 |
|
|
xlink:href="https://svn.driver.research-infrastructures.eu/driver/dnet11/modules/dnet-gridfs-objectstore/trunks/"
|
44 |
|
|
>dnet-gridfs-objectstore</link>: low level implementation for
|
45 |
|
|
storing and retreiving files using mongo GRIDFS. </para>
|
46 |
|
|
</listitem>
|
47 |
|
|
</itemizedlist>
|
48 |
|
|
</para>
|
49 |
|
|
</sect1>
|
50 |
|
|
<sect1>
|
51 |
|
|
<title>The ObjectStoreFile Metadata</title>
|
52 |
|
|
<para>To enable the object store to store a file, it is necessary give some
|
53 |
|
|
information about it such as:<itemizedlist>
|
54 |
|
|
<listitem>
|
55 |
|
|
<para>ObjectID (Required to identify the file)</para>
|
56 |
|
|
</listitem>
|
57 |
|
|
<listitem>
|
58 |
|
|
<para>Mime type</para>
|
59 |
|
|
</listitem>
|
60 |
|
|
<listitem>
|
61 |
|
|
<para>Access protocol (required). At the moment the only protocols
|
62 |
|
|
available are: http, ftp, file system.</para>
|
63 |
|
|
</listitem>
|
64 |
|
|
<listitem>
|
65 |
|
|
<para>URI (required)</para>
|
66 |
|
|
</listitem>
|
67 |
|
|
<listitem>
|
68 |
|
|
<para>username, password for connection (optional, if needed).</para>
|
69 |
|
|
</listitem>
|
70 |
|
|
</itemizedlist></para>
|
71 |
|
|
</sect1>
|
72 |
|
|
</chapter>
|
73 |
|
|
<chapter>
|
74 |
|
|
<title>Feeding Object Store</title>
|
75 |
|
|
<sect1>
|
76 |
|
|
<title></title>
|
77 |
|
|
<para>Feeding Object Store Service can be done in two different ways:</para>
|
78 |
|
|
<para><itemizedlist>
|
79 |
|
|
<listitem>
|
80 |
|
|
<para>Feed of a single file: performed by invoking the following web
|
81 |
|
|
service method
|
82 |
|
|
<programlisting>feedObject( String obsId, String objectMetadata)</programlisting>where
|
83 |
|
|
objectMetadata is a JSON serialization of an ObjectStoreFile
|
84 |
|
|
instance defined in the objectStore-RMI.</para>
|
85 |
|
|
</listitem>
|
86 |
|
|
<listitem>
|
87 |
|
|
<para>Feed of multiple files in bulk: performed via blackboard message
|
88 |
|
|
on the profile of the Object Store Service. The blackboard message
|
89 |
|
|
has to specify an EPR of ObjectStoreFile result set and the targeted
|
90 |
|
|
the ObjectstoreID.</para>
|
91 |
|
|
</listitem>
|
92 |
|
|
</itemizedlist> Please find two example of feeding techniques below.</para>
|
93 |
|
|
</sect1>
|
94 |
|
|
<sect1>
|
95 |
|
|
<title>Feeding of single object</title>
|
96 |
|
|
<para>The code below shows how to upload a single file into an objectStore</para>
|
97 |
|
|
<para>
|
98 |
|
|
<programlisting>
|
99 |
|
|
private void feedTest(ServiceLocator <ObjectStoreService> ObjectStoreServicelocator,
|
100 |
|
|
String objectStoreID) throws ObjectStoreServiceException {
|
101 |
|
|
|
102 |
|
|
//Get a reference of the Service
|
103 |
|
|
ObjectStoreService obsService = ObjectStoreServicelocator.getService();
|
104 |
|
|
|
105 |
|
|
//Create the FILE
|
106 |
|
|
ObjectStoreFile metadataFile = new ObjectStoreFile();
|
107 |
|
|
metadataFile.setObjectID("MyID");
|
108 |
|
|
metadataFile.setMimeType("pdf");
|
109 |
|
|
metadataFile.setURI("ftp://myURL/FTP");
|
110 |
|
|
metadataFile.setAccessProtocol(Protocols.FTP);
|
111 |
|
|
//Could be unnecessary
|
112 |
|
|
metadataFile.setUsernameAuth("usr");
|
113 |
|
|
metadataFile.setPasswordAuth("pwd");
|
114 |
|
|
|
115 |
|
|
//Feed
|
116 |
|
|
obsService.feedObject(objectStoreID, metadataFile.toJSON());
|
117 |
|
|
} </programlisting>
|
118 |
|
|
|
119 |
|
|
</para>
|
120 |
|
|
</sect1>
|
121 |
|
|
<sect1>
|
122 |
|
|
<title>Feeding of multiple objects</title>
|
123 |
|
|
<para>The feeding of multiple objects can be performed by the mechanism of
|
124 |
|
|
bloackboard messaging within the profile service of the objectStore. The code
|
125 |
|
|
Below shows an example of feeding inside an example BlackBoardJobNode belonging
|
126 |
|
|
to a workflow.
|
127 |
|
|
<programlisting>
|
128 |
|
|
protected void prepareJob(final BlackboardJob job, final NodeToken token) {
|
129 |
|
|
|
130 |
|
|
final String epr = token.getEnv().getAttribute("epr");
|
131 |
|
|
final String objectStoreID = token.getEnv().getAttribute("objectStoreID");
|
132 |
|
|
super.prepareJob(job, token);
|
133 |
|
|
|
134 |
|
|
job.setAction("FEED");
|
135 |
|
|
|
136 |
|
|
try {
|
137 |
|
|
job.getParameters().put("epr",new String(Base64.encodeBase64(epr.getBytes("US-ASCII")),
|
138 |
|
|
"US-ASCII"));
|
139 |
|
|
}
|
140 |
|
|
catch (UnsupportedEncodingException e) {
|
141 |
|
|
throw new RuntimeException("ERROR " + e.getMessage());
|
142 |
|
|
}
|
143 |
|
|
job.getParameters().put("obsID",objectStoreID);
|
144 |
|
|
}
|
145 |
|
|
</programlisting>
|
146 |
|
|
</para>
|
147 |
|
|
</sect1>
|
148 |
|
|
</chapter>
|
149 |
|
|
<chapter>
|
150 |
|
|
<title>Delivery from Object Store</title>
|
151 |
|
|
<sect1>
|
152 |
|
|
<title/>
|
153 |
|
|
<para>
|
154 |
|
|
<programlisting> /**
|
155 |
|
|
* Returns ResultSet EPR for delivered ObjectStore records in a particular
|
156 |
|
|
* range date.
|
157 |
|
|
* <p>
|
158 |
|
|
* Please check service implementations for details on the expected format
|
159 |
|
|
* of the records in the result set epr.
|
160 |
|
|
* </p>
|
161 |
|
|
* <p>
|
162 |
|
|
* This method could be used for a bulk deliver of all objects in the store
|
163 |
|
|
* </p>
|
164 |
|
|
*
|
165 |
|
|
* @param obsId
|
166 |
|
|
* identifier of the ObjectStore
|
167 |
|
|
* @param from
|
168 |
|
|
* the minimum date of the object
|
169 |
|
|
* @param until
|
170 |
|
|
* the maximum date of the object
|
171 |
|
|
* @return a ResultSet EPR. Each element of the result set contains the
|
172 |
|
|
* objIdentifier of a record and its URL for retrieve the
|
173 |
|
|
* inputstream of the file.
|
174 |
|
|
* @throws ObjectStoreServiceException
|
175 |
|
|
* the object store service exception
|
176 |
|
|
*/
|
177 |
|
|
@WebMethod(operationName = "deliverObjects", action = "deliverObjects")
|
178 |
|
|
public W3CEndpointReference deliverObjects(
|
179 |
|
|
@WebParam(name = "obsId") String obsId,
|
180 |
|
|
@WebParam(name = "from") Double from,
|
181 |
|
|
@WebParam(name = "until") Double until)
|
182 |
|
|
throws ObjectStoreServiceException;
|
183 |
|
|
|
184 |
|
|
/**
|
185 |
|
|
* Returns ResultSet EPR for delivered ObjectStore records.
|
186 |
|
|
* <p>
|
187 |
|
|
* Please check service implementations for details on the expected format
|
188 |
|
|
* of the records in the result set epr.
|
189 |
|
|
* </p>
|
190 |
|
|
*
|
191 |
|
|
* @param obsId
|
192 |
|
|
* identifier of the ObjectStore
|
193 |
|
|
* @param eprId
|
194 |
|
|
* id of a ResultSet EPR with the identifiers of the interesting
|
195 |
|
|
* objects. Each element of the result set contains the
|
196 |
|
|
* objIdentifier of a record
|
197 |
|
|
* @return a ResultSet EPR. Each element of the result set contains the
|
198 |
|
|
* objIdentifier of a record and its URL for retrieve the
|
199 |
|
|
* inputstream of the file.
|
200 |
|
|
* @throws ObjectStoreServiceException
|
201 |
|
|
* the object store service exception
|
202 |
|
|
*/
|
203 |
|
|
@WebMethod(operationName = "deliverObjectsByIds", action = "deliverObjectsByIds")
|
204 |
|
|
public W3CEndpointReference deliverObjectsByIds(
|
205 |
|
|
@WebParam(name = "obsId") String obsId,
|
206 |
|
|
@WebParam(name = "eprId") W3CEndpointReference eprId)
|
207 |
|
|
throws ObjectStoreServiceException;
|
208 |
|
|
|
209 |
|
|
/**
|
210 |
|
|
* Returns an URL to retrieve the ObjectStore record.
|
211 |
|
|
*
|
212 |
|
|
* @param obsId
|
213 |
|
|
* identifier of the ObjectStore
|
214 |
|
|
* @param objectId
|
215 |
|
|
* the id of the object
|
216 |
|
|
* @return the URL for retrieve the record
|
217 |
|
|
* @throws ObjectStoreServiceException
|
218 |
|
|
* the object store service exception
|
219 |
|
|
*/
|
220 |
|
|
@WebMethod(operationName = "deliverObject", action = "deliverObject")
|
221 |
|
|
public String deliverRecord(@WebParam(name = "obsId") String obsId,
|
222 |
|
|
@WebParam(name = "objectId") String objectId)
|
223 |
|
|
throws ObjectStoreServiceException;</programlisting>
|
224 |
|
|
</para>
|
225 |
|
|
<para>The delivery of file can be done by a call of the ObjectStore Service method,
|
226 |
|
|
that allow the delivery in three different ways:<itemizedlist>
|
227 |
|
|
<listitem>
|
228 |
|
|
<para>DeliverObjects stored in an interval of time (from-to) </para>
|
229 |
|
|
</listitem>
|
230 |
|
|
<listitem>
|
231 |
|
|
<para>DeliveryObjects by a List of IDs of Objects</para>
|
232 |
|
|
</listitem>
|
233 |
|
|
<listitem>
|
234 |
|
|
<para>DeliveryObject by the ID</para>
|
235 |
|
|
</listitem>
|
236 |
|
|
</itemizedlist></para>
|
237 |
|
|
</sect1>
|
238 |
|
|
</chapter>
|
239 |
|
|
</part>
|
240 |
|
|
</book>
|