network.download fails only on Android, not on iOS or in Simulator on URLs that used to work

Solar2D Daily Builds and Public Releases
#41

I inspected the headers and it seems the final URL is this one; the one we’re having issues with:

http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

I used this code to take a look at the headers:
 

function dump(t, indent)

        local notindent = (indent == nil)

        if (notindent) then print(’-----dump-----’); indent=’{}’; end

        if (t and type(t) == ‘table’) then

                for k, v in pairs(t) do

                        if (type(k) ~= ‘number’) then

                                print(indent … ‘.’ … k … ’ = ’ … tostring(v))

                                if (indent) then

                                        dump(v, indent…’.’…k)

                                end

                        end

                end

                for i=1, #t do

                        print(indent … ‘[’ … i … '] = ’ … tostring(t[i]))

                        dump(t[i], indent … ‘[’ … i … ‘]’)

                end

        end

        if (notindent) then print(’-----dump-----’); end

end

– load needed modules

– load the http module

http = require(“socket.http”)

– Requests information about a document, without downloading it.

– Useful, for example, if you want to display a download gauge and need

– to know the size of the document in advance

r, c, h = http.request {

  method = “HEAD”,

  url = “http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png”

}

– r is 1, c is 200, and h would return the following headers:

– h = {

–   date = “Tue, 18 Sep 2001 20:42:21 GMT”,

–   server = “Apache/1.3.12 (Unix)  (Red Hat/Linux)”,

–   [“last-modified”] = “Wed, 05 Sep 2001 06:11:20 GMT”,

–   [“content-length”] = 15652,

–   [“connection”] = “close”,

–   [“content-Type”] = “text/html”

– }

 print("R is "…r)

 print("C is "…c)

 print_r(h)

 dump(h)

#42

I just received this response from Tom, Corona Support:

Hello,

I looked at your issue and it does fail in Android and not on the Mac simulator. We isolated the problem to an Android restriction with the openStream call we are using. The call doesn’t accept underscore characters in the subdomain field. See http://stackoverflow.com/questions/5446578/using-openstream-with-a-url-that-has-a-subdomain-with-an-underscore-in-it-return

I tried escaping the “_” character (using %5f) but that is not supported and returns 404 status.

Since we don’t see any way around this on our end, the only solution we can recommend is not using the underscore character in the subdomain field.

Regards,
Tom

Corona Labs - Support
support@coronalabs.com

#43

Well shoot if that is the case then the “temp fix” is simple…

Below is a full example just start new project and drop it in main.lau, just tested and verified on my Nexus7

It isn’t ideal as it makes two calls but is pretty fast at it…

So require lua sockets.

local socket = require(“socket”)

then

just use luasockets dns call like a so,

local awsIP = socket.dns.toip(“dbh_cache.s3.amazonaws.com”)

that will return the ip address of the server you are looking for then just simply swap out the dns name with the ip address and enter your download like normal.

in your existing code you should be able to just simply call socket.dns.toip right before you call network.download, and since it is blocking you shouldn’t need to call it and keep checking for the value just call it and continue as normal.

local socket = require("socket") timer.performWithDelay(1000, function() local yPos = 0 local function WriteDebug(text, x, y) local myText = display.newText(text, x, y, native.systemFont, 16 ) myText.anchorX, myText.anchorY = 0, 0 myText.x, myText.y = x, y myText:setFillColor( 1, 1, 1 ) end local function DownloadFile(url, saveAs) local params = {} params.progress = "download" params.response = {filename = saveAs, baseDirectory = system.CachesDirectory} network.request(url, "GET", function(event) if ( event.isError ) then WriteDebug( "Problem downloading file::" .. event.isError, 20, yPos) yPos = yPos + 20 elseif ( event.phase == "began" ) then if event.bytesEstimated \<= 0 then WriteDebug( "Start Download", 20, yPos) yPos = yPos + 20 else local estSize = math.floor((event.bytesEstimated \* 0.000001) \* 1000) \* 0.001 WriteDebug("Download starting, estimated size: " .. tostring(estSize) .. "mb", 20, yPos) yPos = yPos + 20 end elseif ( event.phase == "progress" ) then local bytesIn = event.bytesTransferred local totalBytes = event.bytesEstimated local percentage = tonumber(bytesIn / totalBytes \* 100) WriteDebug("Percent Complete: " .. tostring(percentage) .. "%", 20, yPos) yPos = yPos + 20 elseif ( event.phase == "ended" ) then WriteDebug( "Finished Downloading", 20, yPos) yPos = yPos + 20 local debugImage = display.newImage(saveAs, system.CachesDirectory) debugImage.anchorX, debugImage.anchorY = 0, 0 debugImage.width, debugImage.height = 200, 400 debugImage.x, debugImage.y = 0, yPos end end, params ) end local awsIP = socket.dns.toip("dbh\_cache.s3.amazonaws.com") DownloadFile("http://" .. tostring(awsIP) .. "/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png", "34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png") end, 1)
#44

This is an interesting idea, but I’m unable to get the above code working on my Nexus 7, that you’ve said you did get working on your Nexus 7.  I wonder how you were able to do it, but I am unable to? I tried the code on both the Nexus 7, Android 4.4.2 and the Corona Windows Simulator and the Corona Safari Simulator, and they all fail with the exact same result.

The result I’m getting is this:

The download is “ended”

I print  event.url and it is:

http://205.251.243.193/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

It does not download the PNG file.  I inspected the header, and it says this:

<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Message><BucketName>19445</BucketName><RequestId>6B4DC55C0562E57F</RequestId><HostId>904gu3G6u0YnY/4V6kILaqkXZmfEjrDWu1g9S2exzU9bjxYJk1VrNucueLs01I4X</HostId></Error>

#45

You’re going to need to put the original domainname in the “Host:” header field for that to work right.

#46

I wanted to confirm that “_” is not allowed in Amazon S3 bucket name.

I have exact problem and finally be able to fix it by removing “_” in all bucket names.

Using any network.request would easily reproduce the issue, not on iOS device or Corona simulator, but only on Android device.

However http.request works fine. I am not sure if it’s a bug or a common error.

Again, never use “_” in Amazon S3 bucket name!

#47

I wanted to confirm that “_” is not allowed in Amazon S3 bucket name.

I have exact problem and finally be able to fix it by removing “_” in all bucket names.

Using any network.request would easily reproduce the issue, not on iOS device or Corona simulator, but only on Android device.

However http.request works fine. I am not sure if it’s a bug or a common error.

Again, never use “_” in Amazon S3 bucket name!

#48

I have the same problem with ANdroid on new daily builds (2015.26X - 2015.27X) working with client server. Everything was fine with file name but I didn’t have access for server configuration files. Solution? Detect platform and use good old sockets for Android

 

 http.request{ url = "http://translate.google.com/translate\_tts?tl=".. language .."&q=" .. self.urlencode(t[i]) .."&ie=UTF-8&total=1&idx=0&client=t", sink = ltn12.sink.file(myFile), }&nbsp;
#49

What is most likely happening is that Google is redirecting to an https:// server and you’re getting a 301 redirect. But the underlying problem is that http.request does not support https://.  The network.request() API does support https, but on Android, 301 redirect’s are fragile.

Rob

#50

I have the same problem with ANdroid on new daily builds (2015.26X - 2015.27X) working with client server. Everything was fine with file name but I didn’t have access for server configuration files. Solution? Detect platform and use good old sockets for Android

 

 http.request{ url = "http://translate.google.com/translate\_tts?tl=".. language .."&q=" .. self.urlencode(t[i]) .."&ie=UTF-8&total=1&idx=0&client=t", sink = ltn12.sink.file(myFile), }&nbsp;
#51

What is most likely happening is that Google is redirecting to an https:// server and you’re getting a 301 redirect. But the underlying problem is that http.request does not support https://.  The network.request() API does support https, but on Android, 301 redirect’s are fragile.

Rob