Async invocation callback follows 30x redirects but loses data on the way

[tmiklas]

When asynchronous function execution completes the data is sent back to callback url. However if that callback URL issues 30x redirect (from http to https or similar), the data seems to be posted to HTTP url, server responds 30x redirect to HTTPS, another POST requests is done to HTTPS url however function output data is missing.

Expected Behaviour

1. Do not follow 30x redirects - it is user's responsibility to provide correct data collector URL that is not redirecting elsewhere... OR
2. Follow 30x redirects and make sure the function output is posted to redirect destination as well

Current Behaviour

Client posts back results, gets 30x redirect from server and follows redirect passing all relevant headers, but does not include data (body) in that second request

Possible Solution

Same as expected behaviour above

Steps to Reproduce (for bugs)

1. Deploy something like requestbin on HTTPS with redirect from HTTP to HTTPS
2. Invoke function asynchronously providing HTTP url to requestbin as callback URL
3. Requestbin will see post with all headers but no body (function output)
4. If callback URL is switched to HTTPS to avoid redirect, all data gets captured correctly and available in requestbin

Context

Took a long time to debug and figure out what happens there and why async functions don't return data correctly.
This was caused literally by requestbin offering URL with HTTP when server running requestbin was forcing redirect to HTTPS - found by accident.

Your Environment

faasd v0.9.6 - default install and default docker-compose.yml

[alexellis]

Hi @tmiklas

Thank you for logging this issue.

If you can provide step by step instructions and an example of when you ran into this, I'm sure someone could look into it. If we have to spend 1-2 hours getting up to speed and configuring a new function it will take longer and might fall of the radar.

Alex

[tmiklas]

1. Set up requestbin on https://requestbin.lan, with http://requestbin.lan redirecting 301 to https://requestbin.lan - this can be done using docker and is default behaviour if you use LE for TLS certificates; if you visit HTTP URL browser will get redirected to HTTPS
2. Set up faasd v0.9.6 and deploy nodeinfo function
3. Start tcpdump to capture traffic on faasd server tcpdump -i eth0 -A tcp port 80 - this will prove what is posted back to callback URL
4. Create requestbin to track responses
5. curl -v -XPOST -H "X-Callback-Url: http://requestbin.lan/1krmt0o1" https://openfaas-gateway/async-function/nodeinfo to trigger function; you should get HTTP status 202 if async worked
6. tcpdump returns the following data sent by openfaas back to callback URL

Host: requestbin.lan
User-Agent: Go-http-client/1.1
Content-Length: 87
Content-Type: text/plain; charset=utf-8
Date: Mon, 26 Oct 2020 13:19:45 GMT
Etag: W/"57-EkC1toFvohU/eHz0smSVfLV0faw"
X-Call-Id: 5b559697-4f28-4586-bd98-d272f5d2bbc7
X-Duration-Seconds: 0.454151
X-Function-Name: nodeinfo
X-Function-Status: 200
Accept-Encoding: gzip
Connection: close

Hostname: localhost

Arch: x64
CPUs: 2
Total mem: 986MB
Platform: linux
Uptime: 357228

7. callback URL responds with redirect from HTTP to HTTPS:

HTTP/1.1 301 Moved Permanently
Server: nginx/1.14.1
Date: Mon, 26 Oct 2020 13:19:46 GMT
Content-Type: text/html
Content-Length: 185
Connection: close
Location: https://requestbin.lan/1krmt0o1

<title>301 Moved Permanently</title>

301 Moved Permanently

---

nginx/1.14.1

8. Requestbin confirms data received from the POST request:

Method: GET <-- this may be another issue or even THE issue - after redirect this should still be POST, not GET!

Headers:

X-Call-Id: 5b559697-4f28-4586-bd98-d272f5d2bbc7
Accept-Encoding: gzip
X-Function-Name: nodeinfo
X-Function-Status: 200
Connection: close
User-Agent: Go-http-client/1.1
Host: requestbin.lan
Etag: W/"57-EkC1toFvohU/eHz0smSVfLV0faw"
Referer: http://requestbin.lan/1krmt0o1
Date: Mon, 26 Oct 2020 13:19:45 GMT
X-Duration-Seconds: 0.454151
Content-Type: text/plain; charset=utf-8
X-Forwarded-Ssl: on

Form/POST parameters: none

Raw Body: none

[alexellis]

Thank you for the report. Just to clarify 30x means 301 and 302, not 30 redirects.