===================
Storlets Invocation
===================

Once the storlet and its dependencies are deployed the storlet is ready for invocation.
Storlets can be invoked in 2 ways:

1. Invocation upon GET. In this case the user gets a transformation of the object residing in the store (as opposed to the actual object). Typical use case for GET is anonimization, where the user might not have access to a certain data unless it is being anonymized by some storlet.
2. Invocation upon PUT. In this case the data kept in the store is a transformation of the object uploaded by the user (as opposed to the actual uploaded data or metadata). A typical use case is metadata enrichment, where a Storlet extracts format specific metadata from the uploaded data and adds it as Swift metadata.

Invocation involves adding an extra header to the Swift original
PUT/GET requests. Following our Identity Storlet example given in <https://github.com/openstack/storlets/blob/master/doc/source/writing_and_deploying_storlets.rst>, here are invocation examples. This time the examples make use of the python swift client.

Invocation on GET
=================

The code below shows the invocation. Some notes:

#. There are invocations with and without a parameter controlling whether the
   get42 binary dependency is to be called. Note the difference in the response
   headers where one shows the execution result and the other does not.
#. Note the X-Run-Storlet header. being added to the call.
#. Note the X-Generate-Log storlet that causes a log file to be created. 
   The execution results below show the log retrieval.

::

	from swiftclient import client as c
	
	def get_processed_object(url, token, storlet_name, container_name, object_name, invoke_get42 = False):
	    headers = {'X-Run-Storlet': storlet_name,
	               'X-Storlet-Generate-Log' : 'True'}
	    if (invoke_get42 == True):
	        querystring = 'execute=true'
	    else:
	        querystring = None
	
	    response_headers, object_content = c.get_object(url,
	                                          token,
	                                          container_name,
	                                          object_name,
	                                          query_string = querystring,
	                                          response_dict=dict(),
	                                          headers = headers)
	    print response_headers
	    print object_content
	
	AUTH_IP = '127.0.0.1'
	AUTH_PORT = '5000'
	ACCOUNT = 'service'
	USER_NAME = 'swift'
	PASSWORD = 'passw0rd'
	os_options = {'tenant_name': ACCOUNT}
	
	url, token = c.get_auth("http://" + AUTH_IP + ":" + AUTH_PORT + "/v2.0", ACCOUNT +":"+USER_NAME, PASSWORD, os_options = os_options, auth_version="2.0")
	print 'Identity Storlet invocation without calling get42'
	get_processed_object(url, token, 'identitystorlet-1.0.jar', 'myobjects', 'source.txt')
	print 'Identity Storlet invocation instructing to call get42'
	get_processed_object(url, token, 'identitystorlet-1.0.jar', 'myobjects', 'source.txt', True)

 

Here is the result of the running the above python script:

::

	eranr@lnx-ccs8:/tmp$ python get_object_with_storlet.py
	Identity Storlet invocation without calling get42
	{
	    'x-object-meta-x-object-meta-testkey': 'tester', 
	    'transfer-encoding': 'chunked', 
	    'accept-ranges': 'bytes', 
	    'x-object-meta-testkey': 'tester', 
	    'last-modified': 'Tue, 30 Sep 2014 22:07:42 GMT', 
	    'etag': '8ca2a24dbd9779d462c66866c0fb90c3', 
	    'x-timestamp': '1412114861.90504', 
	    'x-trans-id': 'tx464a488a618e44b5b763d-00542baa25', 
	    'date': 'Wed, 01 Oct 2014 07:15:50 GMT', 
	    'x-object-meta-type': 'SBUS_FD_INPUT_OBJECT', 
	    'content-type': 'application/octet-stream'
	}
	Some content to copy
	Identity Storlet invocation instructing to call get42
	{
	    'x-object-meta-execution result': '42', 
	    'x-object-meta-x-object-meta-testkey': 'tester', 
	    'transfer-encoding': 'chunked', 
	    'accept-ranges': 'bytes', 
	    'x-object-meta-testkey': 'tester', 
	    'last-modified': 'Tue, 30 Sep 2014 22:07:42 GMT', 
	    'etag': '8ca2a24dbd9779d462c66866c0fb90c3', 
	    'x-timestamp': '1412114861.90504', 
	    'x-trans-id': 'tx12a4f2a168804dcabf8fc-00542baa26', 
	    'date': 'Wed, 01 Oct 2014 07:15:50 GMT', 
	    'x-object-meta-type': 'SBUS_FD_INPUT_OBJECT', 
	    'content-type': 'application/octet-stream'
	}
	Some content to copy

We now show a download of the log file generated per the X-Storlet-Generate-Log header.
Again, we use the swift client assuming we have the appropriate environment variables in place.

Note that the log reflects the two invocations done above.

::

	eranr@lnx-ccs8:/tmp$ swift download storletlog identitystorlet.log
	identitystorlet.log [headers 0.243s, total 0.243s, 0.001 MB/s]
	eranr@lnx-ccs8:/tmp$ cat identitystorlet.log
	About to invoke storlet
	IdentityStorlet Invoked
	Storlet invocation done
	About to invoke storlet
	IdentityStorlet Invoked
	Exec = /home/swift/identitystorlet/get42
	Exit code = 42
	Storlet invocation done

Invocation on PUT
=================

the code below shows the invocation. Some notes:

#. As with the GET example there are invocations with and without a parameter controlling whether the get42 binary dependency is to be called. After each put we do a GET and print the response headers to show the difference between the invocations. See below.
#. As with the GET example we add the X-Run-Storlet header.
#. This time we do not add the X-Generate-Log header, which is the recommended way, as it saves a creation of an object.

::

	import random
	import string
	from swiftclient import client as c
	
	def put_processed_object(url, token, storlet_name, container_name, object_name, file_name_to_upload, invoke_get42 = False):
	    headers = {'X-Run-Storlet': storlet_name,
	               'X-Storlet-Generate-Log' : 'True'}
	    if (invoke_get42 == True):
	        querystring = 'execute=true'
	    else:
	        querystring = None
	
	    fileobj = open(file_name_to_upload,'r')
	    random_md = ''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(32))
	    headers = {'X-Run-Storlet': 'identitystorlet-1.0.jar', 'X-Object-Meta-Testkey' : random_md}
	    c.put_object(url,
	                 token,
	                 container_name,
	                 object_name,
	                 fileobj,
	                 headers = headers,
	                 query_string = querystring,
	                 response_dict=dict())
	
	    resp_headers, saved_content = c.get_object(
	                url,
	                token,
	                container_name,
	                object_name,
	                response_dict=dict())
	    print resp_headers
	
	AUTH_IP = '127.0.0.1'
	AUTH_PORT = '5000'
	ACCOUNT = 'service'
	USER_NAME = 'swift'
	PASSWORD = 'passw0rd'
	os_options = {'tenant_name': ACCOUNT}
	
	url, token = c.get_auth("http://" + AUTH_IP + ":" + AUTH_PORT + "/v2.0", ACCOUNT +":"+USER_NAME, PASSWORD, os_options = os_options, auth_version="2.0")
	print 'Identity Storlet invocation without calling get42'
	put_processed_object(url, token, 'identitystorlet-1.0.jar', 'myobjects', 'source.txt', '/tmp/source.txt')
	print 'Identity Storlet invocation instructing to call get42'
	put_processed_object(url, token, 'identitystorlet-1.0.jar', 'myobjects', 'source.txt', '/tmp/source.txt' , True)

 

Here is the result of the running the above python script:

::

	eranr@lnx-ccs8:/tmp$ python put_object_with_storlet.py
	Identity Storlet invocation without calling get42
	{
	    'content-length': '1024', 
	    'x-object-meta-x-object-meta-testkey': '1185FZ5FPQ1WXS9IDT4TZZB6GYAQQ0WL', 
	    'accept-ranges': 'bytes', 
	    'x-object-meta-testkey': '1185FZ5FPQ1WXS9IDT4TZZB6GYAQQ0WL', 
	    'last-modified': 'Wed, 01 Oct 2014 07:48:56 GMT', 
	    'etag': '7575c5b098f45ccabce1c3f7fc906eb9', 
	    'x-timestamp': '1412149735.87168', 
	    'x-trans-id': 'tx9a27ba91bee34a8ca9f0c-00542bb1e7', 
	    'date': 'Wed, 01 Oct 2014 07:48:55 GMT', 
	    'x-object-meta-type': 'SBUS_FD_INPUT_OBJECT', 
	    'content-type': 'text/plain'
	}
	Identity Storlet invocation instructing to call get42
	{
	    'x-object-meta-execution result': '42', 
	    'content-length': '1024', 
	    'x-object-meta-x-object-meta-testkey': '54YA1EDTTODMBUJOYCHEGSOQQPV0180L', // This looks like a bug
	    'accept-ranges': 'bytes', 
	    'x-object-meta-testkey': '54YA1EDTTODMBUJOYCHEGSOQQPV0180L', 
	    'last-modified': 'Wed, 01 Oct 2014 07:48:56 GMT', 
	    'etag': '7575c5b098f45ccabce1c3f7fc906eb9', 
	    'x-timestamp': '1412149735.97100', 
	    'x-trans-id': 'txde8619a966c14b0c99d97-00542bb1e8', 
	    'date': 'Wed, 01 Oct 2014 07:48:56 GMT', 
	    'x-object-meta-type': 'SBUS_FD_INPUT_OBJECT', 
	    'content-type': 'text/plain'
	}