Android Networking - Losing my mind - Does this work reliably?

Solar2D Android
#1

Has anyone gotten any network upload or download to work reliably on an non-IOS device, specifically an Android? 

Here is some sample code that works in the simulator but will not show an image on my Samsung S3 (AT&T).

The time updates so I know the app is not hanging but otherwise, HELP!!!

I cannot get the network library or lau socket calls to work reliably.

---------------- example  ---------------

local http = require(“socket.http”)
local ltn12 = require(“ltn12”)

– Create local file for saving data
local path = system.pathForFile( “rs.png”, system.DocumentsDirectory )
myFile = io.open( path, “w+b” )

– Request remote file and save data to local file
http.request{
    url = “http://wpi.imodules.com/s/648/images/editor/red_sox_logo.gif”,
    sink = ltn12.sink.file(myFile),
}

– Display local file
testImage = display.newImage(“rs.png”,system.DocumentsDirectory,60,50);
local t = "Time is : "…os.time(  )
x = display.newText( t, 120, 400, native.systemFontBold, 18, “center” )

#2

So I tried adding a loop and DNS translation…this wont event run in the simulator (and yes, I checked the IP, 72.52.5.200) but it also has a weird side effect.  The “recursive” retry look takes tremendously long to run…almost 30 seconds per loop.  Not sure why as this should be a simple stack push.

Ideas anyone?

local socket = require(“socket”)
local errorcount = 0
local IP = socket.dns.toip(“wpi.imodules.com”)
local url = “http://”…tostring(IP)…"/s/648/images/editor/red_sox_logo.gif"
print(url)

local function networkListener (event)
    print(“here”)
        if event.isError then
                print ( errorcount … " It Failed " )
                if errorcount < 4 then
                    errorcount = errorcount + 1
                    network.download(url, “GET”, networkListener, “red_sox_logo.gif” ,system.CachesDirectory )
                end
            else
                print ( “It Worked” )
            end
end

#3

Is there a reason you’re doing it this way rather than using our network.request() APIs?

#4

Rob,

Honestly? I have tried numerous permutations.  Just now I decided to make sure I was not making a mistake so I took the AsyncImageDownload sample, ran it with no problems in the emulator AND on my phone (Samsung S3 running 4.3).

I change the path to point to my server and it stops working.  I have tried it against several servers. I have a website hosted at Godaddy and an other at Amazon (EC2 micro) using Linux with Node.js.  I have also tried publicly accessible websites.  In all cases I am able to access the image I want to download via a standard browser (IE, FF or Chrome).

When I plug any of these addresses into a copy of the AsyncFileDownload sample the file is never returned.  Not in the simulator or on the device.  Mind Boggling!

Now obviously this leads me to believe it might be a server configuration issue but I am wondering if you are able to get this AsynFileDownload sample working against a public website such as cnn.com or any accessible image.  Have you tried a non-coronalabs web address in that sample?

I am more than happy to make any necessary permission or web hosting changes but I am stumped as to what to try first.

#5

So I grabbed a copy of Wireshark and captured attempts to access the same image file on the same server. One via FireFox and another via the simulator.  The Firefox worked, the simulator failed due to a supposed priv error 403.  The kick is I had full privs open - 777 and obviously the browser had no problem on port 80, only the simulator.  Below are the HTTP protocol captures and there are some differences.

I even tried renaming the image file form .jpg to .htm - no change

I tried moving the image file between the web root and images folder - no change

Firefox (successful image retrieval)

Hypertext Transfer Protocol
GET /images/license.jpg HTTP/1.1\r\n
[Expert Info (Chat/Sequence): GET /images/license.jpg HTTP/1.1\r\n]
[Message: GET /images/license.jpg HTTP/1.1\r\n]
[Severity level: Chat]
[Group: Sequence]
Request Method: GET
Request URI: /images/license.jpg
Request Version: HTTP/1.1
Host: www.kensteinberg.com\r\n
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n
Accept-Language: en-US,en;q=0.5\r\n
Accept-Encoding: gzip, deflate\r\n
DNT: 1\r\n
Connection: keep-alive\r\n
\r\n
Full request URI: [http://www.kensteinberg.com/images/license.jpg]
[HTTP request 1/1]

Hypertext Transfer Protocol
GET /images/license.jpg HTTP/1.1\r\n
[Expert Info (Chat/Sequence): GET /images/license.jpg HTTP/1.1\r\n]
[Message: GET /images/license.jpg HTTP/1.1\r\n]
[Severity level: Chat]
[Group: Sequence]
Request Method: GET
Request URI: /images/license.jpg
Request Version: HTTP/1.1
Connection: Keep-Alive\r\n
Host: www.kensteinberg.com\r\n
\r\n
Full request URI: [http://www.kensteinberg.com/images/license.jpg]
[HTTP request 1/1]
[Response in frame: 11]

Simulator (erroneous retrieval)

Interestingly when I use the coronalab url from the AsyncImageDownload sample in Firefox, I get an immediate 304 after the HTTP GET

Hypertext Transfer Protocol
GET /demo/hello.png HTTP/1.1\r\n
[Expert Info (Chat/Sequence): GET /demo/hello.png HTTP/1.1\r\n]
Request Method: GET
Request URI: /demo/hello.png
Request Version: HTTP/1.1
Host: developer.coronalabs.com\r\n
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n
Accept-Language: en-US,en;q=0.5\r\n
Accept-Encoding: gzip, deflate\r\n
DNT: 1\r\n
[truncated] Cookie: optimizelySegments=%7B%7D; optimizelyEndUserId=oeu1394367114444r0.3002637
7671530735; optimizelyBuckets=%7B%7D; __utma=102512659.222183461.1394367115.1398189649.1398204069
.83; __utmz=102512659.1397993086.68.43.utmcsr=goo
Connection: keep-alive\r\n
If-Modified-Since: Fri, 01 Apr 2011 05:33:04 GMT\r\n
If-None-Match: “6e03a-aec-49fd4be740400”\r\n
Cache-Control: max-age=0\r\n
\r\n
Full request URI: [http://developer.coronalabs.com/demo/hello.png]
[HTTP request 1/1]
[Response in frame: 6]
No. Time Source Destination Protocol Length Info
6 0.147423000 96.126.119.88 192.168.1.11 HTTP 225 HTTP/1.1 304 N
ot Modified
Frame 6: 225 bytes on wire (1800 bits), 225 bytes captured (1800 bits) on interface 0
Ethernet II, Src: Netgear_22:66:ff (04:a1:51:22:66:ff), Dst: IntelCor_59:da:14 (00:23:15:59:da:14
)
Internet Protocol Version 4, Src: 96.126.119.88 (96.126.119.88), Dst: 192.168.1.11 (192.168.1.11)
Transmission Control Protocol, Src Port: http (80), Dst Port: 52371 (52371), Seq: 1, Ack: 1037, L
en: 171
Hypertext Transfer Protocol
HTTP/1.1 304 Not Modified\r\n
[Expert Info (Chat/Sequence): HTTP/1.1 304 Not Modified\r\n]
Request Version: HTTP/1.1
Status Code: 304
Response Phrase: Not Modified
Date: Tue, 22 Apr 2014 22:07:57 GMT\r\n
Server: Apache\r\n
Connection: Keep-Alive\r\n
Keep-Alive: timeout=5, max=100\r\n
ETag: “6e03a-aec-49fd4be740400”\r\n
\r\n
[HTTP response 1/1]
[Time since request: 0.075544000 seconds]
[Request in frame: 5]

Conclusion, To be honest, I’m not sure.  Could it be the server looking for an identified UserAgent?  Both servers (mine and coronalab) are Apache.  Of course the configs could be different.

I am stumped that this simple sample app blows up when a new server is used, regardless of server or host.

#6

Look at the network.download() API call:   http://docs.coronalabs.com/api/library/network/download.html

It’s pretty straight forward, should handle the redirect and https.  The socket based method you’re using doesn’t handle https: which you might be getting redirected too.   Also make sure you have the latest build of Corona SDK (2189) because prior to that we had to put in some fixes because Android didn’t handle redirects very well.

Rob

#7

I just solved it. It has to do with the HTTP headers missing.

This worked in the simulator and on the phone (samsung s3).  Without the headers, no luck.  With them, works repeatedly

–local socket = require(“socket”)
local errorcount = 0
local url = “http://www.kensteinberg.com/license.jpg”
print(url)
    local params = {}
    params.headers = {}
params.headers[“Content-Type”] = “text/plain”
params.headers[“User-Agent”] = “Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0”
params.headers[“Accept-Language”]= “en-US,en;q=0.5”
params.headers[“Accept-Encoding”] = “gzip, deflate”

local function networkListener (event)
        if event.isError then
                print (" It Failed " )
            else
                print ( “It Worked” )
            end
end

myImage2 = display.loadRemoteImage(
    url,
    “GET”,
    networkListener,
    params,
    “x2.jpg”,
    system.TemporaryDirectory,
    centerX, 360 )
 

#8

Update ***

I have also verified that the same is required to use the network.download method.  You may or may not require the Accept-Encoding parameter.

I was able to download a json file from my server using this method of explicitly forming the HTTP headers to emulate a web client.

Ken

#9

So I tried adding a loop and DNS translation…this wont event run in the simulator (and yes, I checked the IP, 72.52.5.200) but it also has a weird side effect.  The “recursive” retry look takes tremendously long to run…almost 30 seconds per loop.  Not sure why as this should be a simple stack push.

Ideas anyone?

local socket = require(“socket”)
local errorcount = 0
local IP = socket.dns.toip(“wpi.imodules.com”)
local url = “http://”…tostring(IP)…"/s/648/images/editor/red_sox_logo.gif"
print(url)

local function networkListener (event)
    print(“here”)
        if event.isError then
                print ( errorcount … " It Failed " )
                if errorcount < 4 then
                    errorcount = errorcount + 1
                    network.download(url, “GET”, networkListener, “red_sox_logo.gif” ,system.CachesDirectory )
                end
            else
                print ( “It Worked” )
            end
end

#10

Is there a reason you’re doing it this way rather than using our network.request() APIs?

#11

Rob,

Honestly? I have tried numerous permutations.  Just now I decided to make sure I was not making a mistake so I took the AsyncImageDownload sample, ran it with no problems in the emulator AND on my phone (Samsung S3 running 4.3).

I change the path to point to my server and it stops working.  I have tried it against several servers. I have a website hosted at Godaddy and an other at Amazon (EC2 micro) using Linux with Node.js.  I have also tried publicly accessible websites.  In all cases I am able to access the image I want to download via a standard browser (IE, FF or Chrome).

When I plug any of these addresses into a copy of the AsyncFileDownload sample the file is never returned.  Not in the simulator or on the device.  Mind Boggling!

Now obviously this leads me to believe it might be a server configuration issue but I am wondering if you are able to get this AsynFileDownload sample working against a public website such as cnn.com or any accessible image.  Have you tried a non-coronalabs web address in that sample?

I am more than happy to make any necessary permission or web hosting changes but I am stumped as to what to try first.

#12

So I grabbed a copy of Wireshark and captured attempts to access the same image file on the same server. One via FireFox and another via the simulator.  The Firefox worked, the simulator failed due to a supposed priv error 403.  The kick is I had full privs open - 777 and obviously the browser had no problem on port 80, only the simulator.  Below are the HTTP protocol captures and there are some differences.

I even tried renaming the image file form .jpg to .htm - no change

I tried moving the image file between the web root and images folder - no change

Firefox (successful image retrieval)

Hypertext Transfer Protocol
GET /images/license.jpg HTTP/1.1\r\n
[Expert Info (Chat/Sequence): GET /images/license.jpg HTTP/1.1\r\n]
[Message: GET /images/license.jpg HTTP/1.1\r\n]
[Severity level: Chat]
[Group: Sequence]
Request Method: GET
Request URI: /images/license.jpg
Request Version: HTTP/1.1
Host: www.kensteinberg.com\r\n
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n
Accept-Language: en-US,en;q=0.5\r\n
Accept-Encoding: gzip, deflate\r\n
DNT: 1\r\n
Connection: keep-alive\r\n
\r\n
Full request URI: [http://www.kensteinberg.com/images/license.jpg]
[HTTP request 1/1]

Hypertext Transfer Protocol
GET /images/license.jpg HTTP/1.1\r\n
[Expert Info (Chat/Sequence): GET /images/license.jpg HTTP/1.1\r\n]
[Message: GET /images/license.jpg HTTP/1.1\r\n]
[Severity level: Chat]
[Group: Sequence]
Request Method: GET
Request URI: /images/license.jpg
Request Version: HTTP/1.1
Connection: Keep-Alive\r\n
Host: www.kensteinberg.com\r\n
\r\n
Full request URI: [http://www.kensteinberg.com/images/license.jpg]
[HTTP request 1/1]
[Response in frame: 11]

Simulator (erroneous retrieval)

Interestingly when I use the coronalab url from the AsyncImageDownload sample in Firefox, I get an immediate 304 after the HTTP GET

Hypertext Transfer Protocol
GET /demo/hello.png HTTP/1.1\r\n
[Expert Info (Chat/Sequence): GET /demo/hello.png HTTP/1.1\r\n]
Request Method: GET
Request URI: /demo/hello.png
Request Version: HTTP/1.1
Host: developer.coronalabs.com\r\n
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0\r\n
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n
Accept-Language: en-US,en;q=0.5\r\n
Accept-Encoding: gzip, deflate\r\n
DNT: 1\r\n
[truncated] Cookie: optimizelySegments=%7B%7D; optimizelyEndUserId=oeu1394367114444r0.3002637
7671530735; optimizelyBuckets=%7B%7D; __utma=102512659.222183461.1394367115.1398189649.1398204069
.83; __utmz=102512659.1397993086.68.43.utmcsr=goo
Connection: keep-alive\r\n
If-Modified-Since: Fri, 01 Apr 2011 05:33:04 GMT\r\n
If-None-Match: “6e03a-aec-49fd4be740400”\r\n
Cache-Control: max-age=0\r\n
\r\n
Full request URI: [http://developer.coronalabs.com/demo/hello.png]
[HTTP request 1/1]
[Response in frame: 6]
No. Time Source Destination Protocol Length Info
6 0.147423000 96.126.119.88 192.168.1.11 HTTP 225 HTTP/1.1 304 N
ot Modified
Frame 6: 225 bytes on wire (1800 bits), 225 bytes captured (1800 bits) on interface 0
Ethernet II, Src: Netgear_22:66:ff (04:a1:51:22:66:ff), Dst: IntelCor_59:da:14 (00:23:15:59:da:14
)
Internet Protocol Version 4, Src: 96.126.119.88 (96.126.119.88), Dst: 192.168.1.11 (192.168.1.11)
Transmission Control Protocol, Src Port: http (80), Dst Port: 52371 (52371), Seq: 1, Ack: 1037, L
en: 171
Hypertext Transfer Protocol
HTTP/1.1 304 Not Modified\r\n
[Expert Info (Chat/Sequence): HTTP/1.1 304 Not Modified\r\n]
Request Version: HTTP/1.1
Status Code: 304
Response Phrase: Not Modified
Date: Tue, 22 Apr 2014 22:07:57 GMT\r\n
Server: Apache\r\n
Connection: Keep-Alive\r\n
Keep-Alive: timeout=5, max=100\r\n
ETag: “6e03a-aec-49fd4be740400”\r\n
\r\n
[HTTP response 1/1]
[Time since request: 0.075544000 seconds]
[Request in frame: 5]

Conclusion, To be honest, I’m not sure.  Could it be the server looking for an identified UserAgent?  Both servers (mine and coronalab) are Apache.  Of course the configs could be different.

I am stumped that this simple sample app blows up when a new server is used, regardless of server or host.

#13

Look at the network.download() API call:   http://docs.coronalabs.com/api/library/network/download.html

It’s pretty straight forward, should handle the redirect and https.  The socket based method you’re using doesn’t handle https: which you might be getting redirected too.   Also make sure you have the latest build of Corona SDK (2189) because prior to that we had to put in some fixes because Android didn’t handle redirects very well.

Rob

#14

I just solved it. It has to do with the HTTP headers missing.

This worked in the simulator and on the phone (samsung s3).  Without the headers, no luck.  With them, works repeatedly

–local socket = require(“socket”)
local errorcount = 0
local url = “http://www.kensteinberg.com/license.jpg”
print(url)
    local params = {}
    params.headers = {}
params.headers[“Content-Type”] = “text/plain”
params.headers[“User-Agent”] = “Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0”
params.headers[“Accept-Language”]= “en-US,en;q=0.5”
params.headers[“Accept-Encoding”] = “gzip, deflate”

local function networkListener (event)
        if event.isError then
                print (" It Failed " )
            else
                print ( “It Worked” )
            end
end

myImage2 = display.loadRemoteImage(
    url,
    “GET”,
    networkListener,
    params,
    “x2.jpg”,
    system.TemporaryDirectory,
    centerX, 360 )
 

#15

Update ***

I have also verified that the same is required to use the network.download method.  You may or may not require the Accept-Encoding parameter.

I was able to download a json file from my server using this method of explicitly forming the HTTP headers to emulate a web client.

Ken