network.download fails only on Android, not on iOS or in Simulator on URLs that used to work

Solar2D Daily Builds and Public Releases
#1

I’m suddenly having issues downloading PNG files only on Android devices from URLs that used to work.  I’m not exactly sure at what point this issue started, but it definitely worked before.

One of the URLs that fails only on Android is: http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

I’ve created a test case.  Here’s the code.  I’ve attached the results of the android log and I’ve attached a zip file with the project and the android log read to be tested by someone.  Please help!  I have no idea what to do!

        local function listener(event)

            if event.isError then

                print ( “It Failed - Only happens on Android Device” )

            else

                print ( “It Worked - Happens in iOS or Simulator” )

            end

        end

    local params = {}

    params.headers = {

        [“Content-Type”] = “application/x-www-form-urlencoded”,

        [“Accept”] = “*/*”,

        [“Accept-Charset”] = “*”,

        [“Accept-Language”] = “en-us”,

        [“Accept-Encoding”] = “gzip, deflate”,

        [“Date”] = os.date("!%a, %d %b %Y %H:%M:%S GMT") 

    }

    

    local urlname = “http://cdn.designbyhumans.com/product_images/4bf5r3s0t9jek1f53.png”

    network.download(urlname, “GET”, listener, params, “test.png” ,system.CachesDirectory )

    

    –

    – Try it again with the re-directed URL name in case there is any difference

    –

    urlname = “http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png”        

    network.download(urlname, “GET”, listener, params, “test1.png” ,system.CachesDirectory )

#2

I know the headers might look weird.  The same problem happens with or without the headers; I was just playing around with them to see if it would solve the problem.  I hope somebody has some ideas what I might do.

#3

Can you dump the values of the event.table when it fails?  You can look up a table printing function (I like the print_r() function in the community code), perhaps there is more information to be found in the responses that come back.

Rob

#4

Hey Rob, OK, here’s the output

ERROR: http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

table: 0x686d63d8 {

[name] => “networkRequest”

[bytesTransferred] => 0

[isError] => true

[status] => -1

[phase] => “ended”

[url] => “http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png”  

[bytesEstimated] => 0

[requestid] = false

}

It Failed - Only happens on Android Device

ERROR: http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

table: 0x68772b00 {

[name] => “networkRequest”

[bytesTransferred] => 0

[isError] => true

[status] => -1

[phase] => “ended”

[url] => “http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png”  

[bytesEstimated] => 0

[requestid] = false

}

It Failed - Only happens on Android Device

#5

This is the same error as in forum item:
http://forums.coronalabs.com/topic/39124-what-does-network-error-1-mean/

I tried the re-try outlined in that post, but it didn’t work, it fails everytime, only on Android devices, and works fine every single time on iOS, and MAC & Windows simulators.

#6

An error -1 is generally a timeout of the server just closing the connection on you before any communications happens. 

Before I have you file a bug report, can you do it with out passing any of those parameters in your params table? They really shouldn’t be needed for a GET request.  You’re not sending a POST form that’s URL encoded data.  You may not want to have your png file zipped up and such.  I could see any of those creating problems trying to fetch an image.

Rob

#7

Rob, I took the params out and the exact same issue still happens.  Can you try it to see if it happens on your end.  I’ll also go ahead and submit a bug report.

    local errorcount = 1

    

       function print_r ( t ) 

            local print_r_cache={}

            local function sub_print_r(t,indent)

                    if (print_r_cache[tostring(t)]) then

                            print(indent…"*"…tostring(t))

                    else

                            print_r_cache[tostring(t)]=true

                            if (type(t)==“table”) then

                                    for pos,val in pairs(t) do

                                            if (type(val)==“table”) then

                                                    print(indent…"["…pos…"] => “…tostring(t)…” {")

                                                    sub_print_r(val,indent…string.rep(" ",string.len(pos)+8))

                                                    print(indent…string.rep(" “,string.len(pos)+6)…”}")

                                            elseif (type(val)==“string”) then

                                                    print(indent…"["…pos…’] => “’…val…’”’)

                                            else

                                                    print(indent…"["…pos…"] => "…tostring(val))

                                            end

                                    end

                            else

                                    print(indent…tostring(t))

                            end

                    end

            end

            if (type(t)==“table”) then

                    print(tostring(t)…" {")

                    sub_print_r(t,"  ")

                    print("}")

            else

                    sub_print_r(t,"  ")

            end

            print()

        end

    

        local function listener(event)

            print_r(event) 

            if event.isError then

                print ( errorcount … " It Failed - Only happens on Android Device" )

                if errorcount < 4 then

                    errorcount = errorcount + 1

                    network.download(event.url, “GET”, listener, “test.png” ,system.CachesDirectory )

                end

            else

                print ( “It Worked - Happens in iOS or Simulator” )

            end

        end

    local urlname = “http://cdn.designbyhumans.com/product_images/4bf5r3s0t9jek1f53.png”

    network.download(urlname, “GET”, listener, “test.png” ,system.CachesDirectory )

    

    –

    – Try it again with the re-directed URL name in case there is any difference

    –

    urlname = “http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png”        

    network.download(urlname, “GET”, listener, “test1.png” ,system.CachesDirectory )

#8

Can you file a bug report for this please?  When you get the acknowledgement email, please post the bug report number back here.

Thanks

Rob

#9

Change your permissions on AWS to public, when you upload a new item to AWS you “must” set the permision to public, I had the exact same problem :slight_smile:

#10

Good point, can you check the permissions before you file the bug report?

Thanks

Rob

#11

also remember when using AWS the permissions set back to private (or user that is logged in) on “every” upload

#12

“Change your permissions on AWS to public, when you upload a new item to AWS you “must” set the permision to public, I had the exact same problem  :)”

“also remember when using AWS the permissions set back to private (or user that is logged in) on “every” upload”

Can you explain this better.  I’m not uploading anything to AWS.  I’m not “designbyhumans” – this information is already on AWS and I’m simply trying to download the image.  Are you saying there is something in the code I’m supposed to add in order to download an image from AWS?

If you take a look at this: http://cdn.designbyhumans.com/product_images/4bf5r3s0t9jek1f53.png

no permission is required.

I’m simply trying to copy this png file from the server to the cache directory so I can display the image.  Is there some other code that Amazon requires ONLY ON ANDROID DEVICES just to download the image?

The sample code works fine on iOS.  The sample code works fine on the Windows simulator and the Safari simulator.

Why would you need to set a permission for AWS only on an Android device for an item that works without any permissions on all the other platforms? And how to do you add an AWS permission when you’re only doing a GET and you aren’t the creator of the AWS data?

#13

@greglosang

This is 100% on your side not on the corona side, I copied your image from your URL and sent it up to my S3 service, then started a new project and added this code to the main.lua (for a test) and pushed it to a couple of android devices like nexus 7 etc. and yours fails 100% of the time but using the exact same code on my S3 service download just fine…

timer.performWithDelay(1000, function() local yPos = 0 local function WriteDebug(text, x, y) local myText = display.newText(text, x, y, native.systemFont, 16 ) myText.anchorX, myText.anchorY = 0, 0 myText.x, myText.y = x, y myText:setFillColor( 1, 1, 1 ) end local function DownloadFile(url, saveAs) local params = {} params.progress = "download" params.response = {filename = saveAs, baseDirectory = system.CachesDirectory} network.request(url, "GET", function(event) if ( event.isError ) then WriteDebug( "Problem downloading file::" .. event.isError, 20, yPos) yPos = yPos + 20 elseif ( event.phase == "began" ) then if event.bytesEstimated \<= 0 then WriteDebug( "Start Download", 20, yPos) yPos = yPos + 20 else local estSize = math.floor((event.bytesEstimated \* 0.000001) \* 1000) \* 0.001 WriteDebug("Download starting, estimated size: " .. tostring(estSize) .. "mb", 20, yPos) yPos = yPos + 20 end elseif ( event.phase == "progress" ) then local bytesIn = event.bytesTransferred local totalBytes = event.bytesEstimated local percentage = tonumber(bytesIn / totalBytes \* 100) WriteDebug("Percent Complete: " .. tostring(percentage) .. "%", 20, yPos) yPos = yPos + 20 elseif ( event.phase == "ended" ) then WriteDebug( "Finished Downloading", 20, yPos) yPos = yPos + 20 local debugImage = display.newImage(saveAs, system.CachesDirectory) debugImage.anchorX, debugImage.anchorY = 0, 0 debugImage.width, debugImage.height = 200, 400 debugImage.x, debugImage.y = 0, yPos end end, params ) end --YOURS --DownloadFile("http://dbh\_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png", "34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png") --MINE DownloadFile("https://s3-us-west-1.amazonaws.com/purplebean/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png", "34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png") end, 1)

if you comment out --YOURS and comment --MINE you will see what i am talking about…

As to why yours is not working the only thing I can see that is different is that you are using the caching of S3 where I am not, so just off the top of my head I would say either your S3 is configure weird and is adding headers that android doesn’t like and/or something inside the network.request code it has an issue with gzip or network header redirects.

Also this is a no brainer but make sure you have 

 androidPermissions = { "android.permission.INTERNET", "android.permission.WRITE\_EXTERNAL\_STORAGE", "android.permission.READ\_EXTERNAL\_STORAGE", },

in your build.settings file.

#14

Thank you so much for your help, but again, the S3 service is not my account; I need to access these files as they are and I don’t have the ability to change the S3 settings because it’s not my account, it belongs to Design By Humans and I’m trying to access their images on S3 exactly as the URL is now, and this always worked before and it still works fine on iOS devices or in the simulators.

But it seems you’ve determined that there is an issue with Corona where this particular S3 configuration works fine on iOS and both the MAC and WINDOWS simulators and it fails ONLY on ANDROID devices.  Plus, this DID NOT used to be an issue, it has only suddenly become an issue.  And it seems your S3 configuration does work, so this bug is specific to the configuration used on this particular file: http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png
 

(And every other PNG file in the Design By Humans catalog)

If there is not a way to modify my code to handle loading in this exact URL:http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

which can be accessed just fine from any browser, or using network.download in Corona on any iOS device, or from the MAC or WINDOWS simulator, then it seems like it might be something to do with Corona only on ANDROID devices.

So your speculation about this could be the issue: " something inside the network.request code it has an issue with gzip or network header redirects."

 

#15

Rob, here’s the bug report number: 

Case 29402

I’m unable to change anything on the Amazon S3 servers because it’s not my Amazon S3 account, and I need to access these PNG images as they are currently configured on Amazon S3 since I don’t have the ability to change anything on the server side.

#16

I just tried gzip compression and that works fine so my guess is that it is not handling the redirect, it has been awhile since i have messed with http headers directly but in a cdn/cache setup like the one you are using the server responds back with a location (i want to say 301/302) but don’t have time to look it up for sure at the moment.

Then what happens is your browser (or socket connection) normally grabs this location and makes another request to the new location and so on and so forth until a valid response header for what it is you are looking for like a 200 etc. shows up.

I am not 100% sure this is what is happening but I could run some tests tomorrow to see if that is the case, i can just put a image up on my load balanced servers and see what happens.

If that is the case though, you could very easily just use lua sockets to get the raw headers and grab the location that is coming back then make your network.download request on that url instead of the primary (not ideal but if you are under a time crunch it would work (if that is the problem that is)).

Just a thought :slight_smile:

#17

I’m not sure I’m understanding exactly what you are saying, but the 2nd URL that it fails on doesn’t seem to be a re-direct.

The first URL: “http://cdn.designbyhumans.com/product_images/4bf5r3s0t9jek1f53.png”

redirects to: the second url in my test: http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

So the URL you’ve been testing is the final resolved URL, not the redirect.  I didn’t the the 2nd URL was a redirect, so it didn’t seem like redirect had anything to do with this issue.  Or are you implying that 2nd URL is also redirecting?  It didn’t appear to me that it was.

#18

They both rediect, your .designbyhumans does a dns pointer redirect where the S3 is a server or hardware level redirect to the best location for your request.

The nature of a cdn/load balanced servers or in your case a S3 service with redundancy turned on is that you have a single location where you request information from (for example www.mywebsite.com/myimage.png) in turn the server does a check for the best location (ie: server) to feed your request up and tells you where that location is so you can go get it. (this all happens at the server level not at the dns naming level)

The only way you could tell is by inspecting the headers and/or just knowing what happens when you hit a cdn/load balanced servers.

#19

I inspected the headers and it seems the final URL is this one; the one we’re having issues with:

http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

I used this code to take a look at the headers:
 

function dump(t, indent)

        local notindent = (indent == nil)

        if (notindent) then print(’-----dump-----’); indent=’{}’; end

        if (t and type(t) == ‘table’) then

                for k, v in pairs(t) do

                        if (type(k) ~= ‘number’) then

                                print(indent … ‘.’ … k … ’ = ’ … tostring(v))

                                if (indent) then

                                        dump(v, indent…’.’…k)

                                end

                        end

                end

                for i=1, #t do

                        print(indent … ‘[’ … i … '] = ’ … tostring(t[i]))

                        dump(t[i], indent … ‘[’ … i … ‘]’)

                end

        end

        if (notindent) then print(’-----dump-----’); end

end

– load needed modules

– load the http module

http = require(“socket.http”)

– Requests information about a document, without downloading it.

– Useful, for example, if you want to display a download gauge and need

– to know the size of the document in advance

r, c, h = http.request {

  method = “HEAD”,

  url = “http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png”

}

– r is 1, c is 200, and h would return the following headers:

– h = {

–   date = “Tue, 18 Sep 2001 20:42:21 GMT”,

–   server = “Apache/1.3.12 (Unix)  (Red Hat/Linux)”,

–   [“last-modified”] = “Wed, 05 Sep 2001 06:11:20 GMT”,

–   [“content-length”] = 15652,

–   [“connection”] = “close”,

–   [“content-Type”] = “text/html”

– }

 print("R is "…r)

 print("C is "…c)

 print_r(h)

 dump(h)

#20

I just received this response from Tom, Corona Support:

Hello,

I looked at your issue and it does fail in Android and not on the Mac simulator. We isolated the problem to an Android restriction with the openStream call we are using. The call doesn’t accept underscore characters in the subdomain field. See http://stackoverflow.com/questions/5446578/using-openstream-with-a-url-that-has-a-subdomain-with-an-underscore-in-it-return

I tried escaping the “_” character (using %5f) but that is not supported and returns 404 status.

Since we don’t see any way around this on our end, the only solution we can recommend is not using the underscore character in the subdomain field.

Regards,
Tom

Corona Labs - Support
support@coronalabs.com