Say you have a method that uploads a file via POST (-F) to a specific endpoint in a REST API (here, using Flask). Now, you want to add some logic to this process before POSTing the file, but without modifying the original API. So you create an intermediate REST API. How to allow POSTING the file through this intermediate API?
The intermediate API to be developed will be called “rest1”, while the original API accepting the POST of a file will be called “rest2”. The file to be POSTed will be a generic, binary file. Let us imagine that “rest1” will perform validation procedures on such file, before sending it towards “rest2” API. Finally, both interfaces will run in localhost (could be located anywhere as long as each point in the chain has connection to each other); where the server for the 1st interface runs in port 8000, and the server for the 2nd runs in port 8001.
1
2
3
4
5
6
7
8
9
10
client rest1 (intermediate) rest2 (original)
------------------------------------------------------------------
port=C1 port=8000
| |
+-----------------------+
port=I1 port=8001
| |
+-------------------------------+
Intermediate API
The intermediate or proxy API incorporates the new logic for validating the POSTed file. The endpoint at this interface can be defined in several ways, as best fit, for instance accepting the POSTed file:
- As a value in a form, acting as a simple proxy (example in cURL:
-F "somefile=@/path/to/file") - As a dictionary value, with different kind of paths (example in cURL:
-H "Content-Type: application/json" -X POST -d '{"somefile": "/path/to/file"}')
Simple proxy
The first example acts as a simple proxy, sending the binary file received in “rest1” towards “rest2” just as it was received. Note that this file must be present locally from where “rest1” is running (e.g. cURL does not allow remote paths after the “@”).
1
2
3
4
5
6
7
8
9
10
11
12
13
from flask import request
from werkzeug.datastructures import ImmutableMultiDict
import requests
@app.route("/upload", methods=["POST"])
def upload():
data = request.files
bin_file = data.get("somefile")
data_file = ImmutableMultiDict([("somefile", bin_file)])
resp = requests.post("://url/to/rest2:8001/upload",
files=data_file,
verify=False)
return resp.content
Local and remote
The second example would either read a local or a remote file, construct an object with a specific data structure (FileStorage, from werkzeug library) and send it towards the “rest2” API.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
from flask import request
from mimetypes import MimeTypes
from werkzeug.datastructures import FileStorage
from werkzeug.datastructures import ImmutableMultiDict
import os
import requests
import shutil
@app.route("/upload", methods=["POST"])
def upload():
file_path = request.json.get("somefile")
remove_after = False
if not os.path.isfile(file_path):
remove_after = True
file_path = fetch_content(file_path)
fp = open(file_path, "rb")
# Signature: FileStorage(stream=None, filename=None, name=None, content_type=None, content_length=None, headers=None)
filename = os.path.basename(file_path)
mime = MimeTypes()
content_type = mime.guess_type(file_path)
bin_file = FileStorage(fp, filename, "somefile", content_type)
data_file = ImmutableMultiDict([("somefile", bin_file)])
resp = requests.post("://url/to/rest2:8001/upload",
files=data_file,
verify=False)
fp.close()
if remove_after:
file_dir = os.path.dirname(file_path)
shutil.rmtree(file_dir)
return resp.content
Altogether
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
from flask import Flask
from flask import request
from mimetypes import MimeTypes
from tempfile import mkdtemp
from werkzeug import serving
from werkzeug.datastructures import FileStorage
from werkzeug.datastructures import ImmutableMultiDict
import os
import requests
import shutil
import ssl
app = Flask(__name__)
rest2_ep = "https://127.0.0.1:8001/upload"
def fetch_content(url):
tmp_folder = mkdtemp()
tmp_file = url.split("/")[-1]
tmp_path = os.path.join(tmp_folder, tmp_file)
try:
import urllib
data = urllib.urlretrieve(url, tmp_path)
except:
import urllib.request
data = urllib.request.urlopen(url).read()
f = open(tmp_path, "wb")
f.write(data)
f.close()
return tmp_path
def post_content(bin_file):
data_file = ImmutableMultiDict([("somefile", bin_file)])
resp = requests.post(rest2_ep,
files=data_file,
verify=False)
return resp.content
@app.route("/upload1", methods=["POST"])
def upload():
data = request.files
bin_file = data.get("somefile")
return post_content(bin_file)
@app.route("/upload2", methods=["POST"])
def upload2():
file_path = request.json.get("somefile")
remove_after = False
if not os.path.isfile(file_path):
remove_after = True
file_path = fetch_content(file_path)
fp = open(file_path, "rb")
# Signature: FileStorage(stream=None, filename=None, name=None, content_type=None, content_length=None, headers=None)
filename = os.path.basename(file_path)
mime = MimeTypes()
content_type = mime.guess_type(file_path)
bin_file = FileStorage(fp, filename, "somefile", content_type)
resp_content = post_content(bin_file)
fp.close()
if remove_after:
file_dir = os.path.dirname(file_path)
shutil.rmtree(file_dir)
return resp_content
context = "adhoc"
serving.run_simple("0.0.0.0", 8000, app, ssl_context=context)
Original API
For the sake of completeness or even testing purposes, the code of the original (“rest2”) API is here provided; although you will be most probably restricted by its original implementation.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
from flask import Flask
from flask import request
from tempfile import mkdtemp
from werkzeug import serving
import os
import requests
import ssl
app = Flask(__name__)
@app.route("/upload", methods=["POST"])
def upload():
data = request.files
stream = data.get("somefile").stream
tmp_folder = mkdtemp()
tmp_file = data.get("somefile").filename
tmp_path = os.path.join(tmp_folder, tmp_file)
with open(tmp_path, "wb") as out:
out.write(stream.read())
return '{"status": "done", "path": "%s"}' % tmp_path
context = "adhoc"
serving.run_simple("0.0.0.0", 8001, app, ssl_context=context)
Testing
Pointing cURL to the exposed endpoints will return an output similar to the one below (in these examples, it indicates the path where the POSTed file is stored in the server where “rest2” is running):
1
2
3
4
5
6
7
8
9
10
11
12
13
$ rest1_ip="https://127.0.0.1:8000"
# In case of self-signed certificates, force the insecure "-k" flag
$ curl -ik ${rest1_ip}/upload1 -X POST -F "somefile=@/opt/test/file.tar.gz"
$ curl -ik ${rest1_ip}/upload2 -H "Content-Type: application/json" -X POST -d '{"somefile": "/opt/test/file.tar.gz"}'
$ curl -ik ${rest1_ip}/upload2 -H "Content-Type: application/json" -X POST -d '{"somefile": "://path/to/remote/file.tar.gz"}'
HTTP/1.0 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 97
Server: Werkzeug/0.12.2 Python/3.6.3
Date: ...
{"status": "done", "path": "/var/folders/d8/mszhkck10glcmxjm6h93zm4r0000gn/T/tmp1vxwv9b1/somefile"}
