network.download fails only on Android, not on iOS or in Simulator on URLs that used to work

Solar2D Daily Builds and Public Releases
#21

Well shoot if that is the case then the “temp fix” is simple…

Below is a full example just start new project and drop it in main.lau, just tested and verified on my Nexus7

It isn’t ideal as it makes two calls but is pretty fast at it…

So require lua sockets.

local socket = require(“socket”)

then

just use luasockets dns call like a so,

local awsIP = socket.dns.toip(“dbh_cache.s3.amazonaws.com”)

that will return the ip address of the server you are looking for then just simply swap out the dns name with the ip address and enter your download like normal.

in your existing code you should be able to just simply call socket.dns.toip right before you call network.download, and since it is blocking you shouldn’t need to call it and keep checking for the value just call it and continue as normal.

local socket = require("socket") timer.performWithDelay(1000, function() local yPos = 0 local function WriteDebug(text, x, y) local myText = display.newText(text, x, y, native.systemFont, 16 ) myText.anchorX, myText.anchorY = 0, 0 myText.x, myText.y = x, y myText:setFillColor( 1, 1, 1 ) end local function DownloadFile(url, saveAs) local params = {} params.progress = "download" params.response = {filename = saveAs, baseDirectory = system.CachesDirectory} network.request(url, "GET", function(event) if ( event.isError ) then WriteDebug( "Problem downloading file::" .. event.isError, 20, yPos) yPos = yPos + 20 elseif ( event.phase == "began" ) then if event.bytesEstimated \<= 0 then WriteDebug( "Start Download", 20, yPos) yPos = yPos + 20 else local estSize = math.floor((event.bytesEstimated \* 0.000001) \* 1000) \* 0.001 WriteDebug("Download starting, estimated size: " .. tostring(estSize) .. "mb", 20, yPos) yPos = yPos + 20 end elseif ( event.phase == "progress" ) then local bytesIn = event.bytesTransferred local totalBytes = event.bytesEstimated local percentage = tonumber(bytesIn / totalBytes \* 100) WriteDebug("Percent Complete: " .. tostring(percentage) .. "%", 20, yPos) yPos = yPos + 20 elseif ( event.phase == "ended" ) then WriteDebug( "Finished Downloading", 20, yPos) yPos = yPos + 20 local debugImage = display.newImage(saveAs, system.CachesDirectory) debugImage.anchorX, debugImage.anchorY = 0, 0 debugImage.width, debugImage.height = 200, 400 debugImage.x, debugImage.y = 0, yPos end end, params ) end local awsIP = socket.dns.toip("dbh\_cache.s3.amazonaws.com") DownloadFile("http://" .. tostring(awsIP) .. "/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png", "34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png") end, 1)
#22

This is an interesting idea, but I’m unable to get the above code working on my Nexus 7, that you’ve said you did get working on your Nexus 7.  I wonder how you were able to do it, but I am unable to? I tried the code on both the Nexus 7, Android 4.4.2 and the Corona Windows Simulator and the Corona Safari Simulator, and they all fail with the exact same result.

The result I’m getting is this:

The download is “ended”

I print  event.url and it is:

http://205.251.243.193/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

It does not download the PNG file.  I inspected the header, and it says this:

<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Message><BucketName>19445</BucketName><RequestId>6B4DC55C0562E57F</RequestId><HostId>904gu3G6u0YnY/4V6kILaqkXZmfEjrDWu1g9S2exzU9bjxYJk1VrNucueLs01I4X</HostId></Error>

#23

You’re going to need to put the original domainname in the “Host:” header field for that to work right.

#24

I know the headers might look weird.  The same problem happens with or without the headers; I was just playing around with them to see if it would solve the problem.  I hope somebody has some ideas what I might do.

#25

Can you dump the values of the event.table when it fails?  You can look up a table printing function (I like the print_r() function in the community code), perhaps there is more information to be found in the responses that come back.

Rob

#26

Hey Rob, OK, here’s the output

ERROR: http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

table: 0x686d63d8 {

[name] => “networkRequest”

[bytesTransferred] => 0

[isError] => true

[status] => -1

[phase] => “ended”

[url] => “http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png”  

[bytesEstimated] => 0

[requestid] = false

}

It Failed - Only happens on Android Device

ERROR: http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

table: 0x68772b00 {

[name] => “networkRequest”

[bytesTransferred] => 0

[isError] => true

[status] => -1

[phase] => “ended”

[url] => “http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png”  

[bytesEstimated] => 0

[requestid] = false

}

It Failed - Only happens on Android Device

#27

This is the same error as in forum item:
http://forums.coronalabs.com/topic/39124-what-does-network-error-1-mean/

I tried the re-try outlined in that post, but it didn’t work, it fails everytime, only on Android devices, and works fine every single time on iOS, and MAC & Windows simulators.

#28

An error -1 is generally a timeout of the server just closing the connection on you before any communications happens. 

Before I have you file a bug report, can you do it with out passing any of those parameters in your params table? They really shouldn’t be needed for a GET request.  You’re not sending a POST form that’s URL encoded data.  You may not want to have your png file zipped up and such.  I could see any of those creating problems trying to fetch an image.

Rob

#29

Rob, I took the params out and the exact same issue still happens.  Can you try it to see if it happens on your end.  I’ll also go ahead and submit a bug report.

    local errorcount = 1

    

       function print_r ( t ) 

            local print_r_cache={}

            local function sub_print_r(t,indent)

                    if (print_r_cache[tostring(t)]) then

                            print(indent…"*"…tostring(t))

                    else

                            print_r_cache[tostring(t)]=true

                            if (type(t)==“table”) then

                                    for pos,val in pairs(t) do

                                            if (type(val)==“table”) then

                                                    print(indent…"["…pos…"] => “…tostring(t)…” {")

                                                    sub_print_r(val,indent…string.rep(" ",string.len(pos)+8))

                                                    print(indent…string.rep(" “,string.len(pos)+6)…”}")

                                            elseif (type(val)==“string”) then

                                                    print(indent…"["…pos…’] => “’…val…’”’)

                                            else

                                                    print(indent…"["…pos…"] => "…tostring(val))

                                            end

                                    end

                            else

                                    print(indent…tostring(t))

                            end

                    end

            end

            if (type(t)==“table”) then

                    print(tostring(t)…" {")

                    sub_print_r(t,"  ")

                    print("}")

            else

                    sub_print_r(t,"  ")

            end

            print()

        end

    

        local function listener(event)

            print_r(event) 

            if event.isError then

                print ( errorcount … " It Failed - Only happens on Android Device" )

                if errorcount < 4 then

                    errorcount = errorcount + 1

                    network.download(event.url, “GET”, listener, “test.png” ,system.CachesDirectory )

                end

            else

                print ( “It Worked - Happens in iOS or Simulator” )

            end

        end

    local urlname = “http://cdn.designbyhumans.com/product_images/4bf5r3s0t9jek1f53.png”

    network.download(urlname, “GET”, listener, “test.png” ,system.CachesDirectory )

    

    –

    – Try it again with the re-directed URL name in case there is any difference

    –

    urlname = “http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png”        

    network.download(urlname, “GET”, listener, “test1.png” ,system.CachesDirectory )

#30

Can you file a bug report for this please?  When you get the acknowledgement email, please post the bug report number back here.

Thanks

Rob

#31

Change your permissions on AWS to public, when you upload a new item to AWS you “must” set the permision to public, I had the exact same problem :slight_smile:

#32

Good point, can you check the permissions before you file the bug report?

Thanks

Rob

#33

also remember when using AWS the permissions set back to private (or user that is logged in) on “every” upload

#34

“Change your permissions on AWS to public, when you upload a new item to AWS you “must” set the permision to public, I had the exact same problem  :)”

“also remember when using AWS the permissions set back to private (or user that is logged in) on “every” upload”

Can you explain this better.  I’m not uploading anything to AWS.  I’m not “designbyhumans” – this information is already on AWS and I’m simply trying to download the image.  Are you saying there is something in the code I’m supposed to add in order to download an image from AWS?

If you take a look at this: http://cdn.designbyhumans.com/product_images/4bf5r3s0t9jek1f53.png

no permission is required.

I’m simply trying to copy this png file from the server to the cache directory so I can display the image.  Is there some other code that Amazon requires ONLY ON ANDROID DEVICES just to download the image?

The sample code works fine on iOS.  The sample code works fine on the Windows simulator and the Safari simulator.

Why would you need to set a permission for AWS only on an Android device for an item that works without any permissions on all the other platforms? And how to do you add an AWS permission when you’re only doing a GET and you aren’t the creator of the AWS data?

#35

@greglosang

This is 100% on your side not on the corona side, I copied your image from your URL and sent it up to my S3 service, then started a new project and added this code to the main.lua (for a test) and pushed it to a couple of android devices like nexus 7 etc. and yours fails 100% of the time but using the exact same code on my S3 service download just fine…

timer.performWithDelay(1000, function() local yPos = 0 local function WriteDebug(text, x, y) local myText = display.newText(text, x, y, native.systemFont, 16 ) myText.anchorX, myText.anchorY = 0, 0 myText.x, myText.y = x, y myText:setFillColor( 1, 1, 1 ) end local function DownloadFile(url, saveAs) local params = {} params.progress = "download" params.response = {filename = saveAs, baseDirectory = system.CachesDirectory} network.request(url, "GET", function(event) if ( event.isError ) then WriteDebug( "Problem downloading file::" .. event.isError, 20, yPos) yPos = yPos + 20 elseif ( event.phase == "began" ) then if event.bytesEstimated \<= 0 then WriteDebug( "Start Download", 20, yPos) yPos = yPos + 20 else local estSize = math.floor((event.bytesEstimated \* 0.000001) \* 1000) \* 0.001 WriteDebug("Download starting, estimated size: " .. tostring(estSize) .. "mb", 20, yPos) yPos = yPos + 20 end elseif ( event.phase == "progress" ) then local bytesIn = event.bytesTransferred local totalBytes = event.bytesEstimated local percentage = tonumber(bytesIn / totalBytes \* 100) WriteDebug("Percent Complete: " .. tostring(percentage) .. "%", 20, yPos) yPos = yPos + 20 elseif ( event.phase == "ended" ) then WriteDebug( "Finished Downloading", 20, yPos) yPos = yPos + 20 local debugImage = display.newImage(saveAs, system.CachesDirectory) debugImage.anchorX, debugImage.anchorY = 0, 0 debugImage.width, debugImage.height = 200, 400 debugImage.x, debugImage.y = 0, yPos end end, params ) end --YOURS --DownloadFile("http://dbh\_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png", "34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png") --MINE DownloadFile("https://s3-us-west-1.amazonaws.com/purplebean/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png", "34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png") end, 1)

if you comment out --YOURS and comment --MINE you will see what i am talking about…

As to why yours is not working the only thing I can see that is different is that you are using the caching of S3 where I am not, so just off the top of my head I would say either your S3 is configure weird and is adding headers that android doesn’t like and/or something inside the network.request code it has an issue with gzip or network header redirects.

Also this is a no brainer but make sure you have 

 androidPermissions = { "android.permission.INTERNET", "android.permission.WRITE\_EXTERNAL\_STORAGE", "android.permission.READ\_EXTERNAL\_STORAGE", },

in your build.settings file.

#36

Thank you so much for your help, but again, the S3 service is not my account; I need to access these files as they are and I don’t have the ability to change the S3 settings because it’s not my account, it belongs to Design By Humans and I’m trying to access their images on S3 exactly as the URL is now, and this always worked before and it still works fine on iOS devices or in the simulators.

But it seems you’ve determined that there is an issue with Corona where this particular S3 configuration works fine on iOS and both the MAC and WINDOWS simulators and it fails ONLY on ANDROID devices.  Plus, this DID NOT used to be an issue, it has only suddenly become an issue.  And it seems your S3 configuration does work, so this bug is specific to the configuration used on this particular file: http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png
 

(And every other PNG file in the Design By Humans catalog)

If there is not a way to modify my code to handle loading in this exact URL:http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

which can be accessed just fine from any browser, or using network.download in Corona on any iOS device, or from the MAC or WINDOWS simulator, then it seems like it might be something to do with Corona only on ANDROID devices.

So your speculation about this could be the issue: " something inside the network.request code it has an issue with gzip or network header redirects."

 

#37

Rob, here’s the bug report number: 

Case 29402

I’m unable to change anything on the Amazon S3 servers because it’s not my Amazon S3 account, and I need to access these PNG images as they are currently configured on Amazon S3 since I don’t have the ability to change anything on the server side.

#38

I just tried gzip compression and that works fine so my guess is that it is not handling the redirect, it has been awhile since i have messed with http headers directly but in a cdn/cache setup like the one you are using the server responds back with a location (i want to say 301/302) but don’t have time to look it up for sure at the moment.

Then what happens is your browser (or socket connection) normally grabs this location and makes another request to the new location and so on and so forth until a valid response header for what it is you are looking for like a 200 etc. shows up.

I am not 100% sure this is what is happening but I could run some tests tomorrow to see if that is the case, i can just put a image up on my load balanced servers and see what happens.

If that is the case though, you could very easily just use lua sockets to get the raw headers and grab the location that is coming back then make your network.download request on that url instead of the primary (not ideal but if you are under a time crunch it would work (if that is the problem that is)).

Just a thought :slight_smile:

#39

I’m not sure I’m understanding exactly what you are saying, but the 2nd URL that it fails on doesn’t seem to be a re-direct.

The first URL: “http://cdn.designbyhumans.com/product_images/4bf5r3s0t9jek1f53.png”

redirects to: the second url in my test: http://dbh_cache.s3.amazonaws.com/19445/34173cb38f07f89ddbebc2ac9128303f-33b64a2ed0f1ff4750f183b4f2a161b8.png

So the URL you’ve been testing is the final resolved URL, not the redirect.  I didn’t the the 2nd URL was a redirect, so it didn’t seem like redirect had anything to do with this issue.  Or are you implying that 2nd URL is also redirecting?  It didn’t appear to me that it was.

#40

They both rediect, your .designbyhumans does a dns pointer redirect where the S3 is a server or hardware level redirect to the best location for your request.

The nature of a cdn/load balanced servers or in your case a S3 service with redundancy turned on is that you have a single location where you request information from (for example www.mywebsite.com/myimage.png) in turn the server does a check for the best location (ie: server) to feed your request up and tells you where that location is so you can go get it. (this all happens at the server level not at the dns naming level)

The only way you could tell is by inspecting the headers and/or just knowing what happens when you hit a cdn/load balanced servers.