Network request: works in simulator and Nexus but not on Samsung or HTC

Solar2D Android
#1

Dear All, 

I just released my first app (in alpha) and have run into a major roadblock. The app logs into a website using HTTP(S) POST. 

The code is as follows:

 local headers = {} headers["Accept"] = "\*/\*" headers["Accept-Encoding"] = "" headers["Accept-Language"] = "en-US" headers["Content-Type"] = "application/x-www-form-urlencoded" local body = "POST body with username and password" local params = {} params.headers = headers params.body = body params.timeout = 30 print("Calling networkListenerLogin") network.request( "targetURL", "POST", networkListenerLogin, params)

There is 1 important thing of note: 

If the entered password is incorrect, the targetURL returns HTTP 200 and “login failed” HTML text. 

If the entered password is correct, the targetURL returns 302 Found and forwards to logged in welcome page. 

Problem: The “Calling networkListenerLogin” gets printed in adb logcat, but the control does not reach networkListenerLogin when correct username / password are used. Test cases are as follows: 

  1. works on Windows simulator

  2. works on MacOS simulator

  3. works on Nexus 4 (stock android)

  4. does not**  work** on Samsung Galaxy S3

  5. does not**  work** on HTC One

  6. works on iOS simulator

  7. works on actual ipad device

It reaches networkListenerLogin when wrong password is used for all devices. 

Question: is it the HTTP 302 which is messing it up? 

Kindly help as considerable effort has gone into App Dev and this is really a nasty shocker after releasing it in alpha.

Many thanks,

Anshuman

#2

It’s a little unclear from your description but when you say “the control does not reach networkListenerLogin when correct username / password are used” you actually mean it fails in cases 4 and 5, but the other 3 work?

The 302 response should be transparent to you and you should see the response that you get when the location you are redirected to is loaded.  For completeness, what is that response when the process succeeds?

Is this a REST API you are using or are you trying to login via a URL usually intended for interactive use in a browser?  Scripting interactive logins isn’t impossible but using an API intended for the purpose is way easier (I ask because a web service that responds with HTTP 200 on error and uses 302s for success sounds a little odd).  Interfaces intended for browsers frequently change their responses for different user agents or operating systems whereas purpose built APIs are much more predictable.  Is this a public service you are trying to access? If so, which one?

The way I would approach a problem like this is to set up a proxy on your desktop computer and make the phone connect to the internet through that.  This isn’t necessarily easy but it can reveal subtleties of the protocol that are hard to see otherwise. There’s an article about doing that here: http://goo.gl/ywsWaz

Lastly, can you post your  networkListenerLogin code? (or a simplification of it)

#3

Hi Perry, 

Many thanks for replying. Please find details below:

 

It’s a little unclear from your description but when you say “the control does not reach networkListenerLogin when correct username / password are used” you actually mean it fails in cases 4 and 5, but the other 3 work?

  • Correct, the control does not reach the network request listener function (networkListenerLogin) at all for Samsung and HTC phones but works on Nexus phones and simulators. I can also confirm its working on iphone and ipads, I have managed to deploy them there as well.

 

 

The 302 response should be transparent to you and you should see the response that you get when the location you are redirected to is loaded.  For completeness, what is that response when the process succeeds?

  • You are right it should be transparent to me, and it is so in simulator and Nexus phones. When using the correct username/pwd, the order is as follows:

  1. Login page gives “302 Found”

  2. It then loads a 2nd page, which itself gives “302 Moved Temporarily”

  3. This finally lands in the logged in page

 

 

Is this a REST API you are using or are you trying to login via a URL usually intended for interactive use in a browser?  

  • I am logging in via a POST to a URL which is usually intended for web browser logins. I have no control on the website as they are another system

 

 

Scripting interactive logins isn’t impossible but using an API intended for the purpose is way easier

  • I wish there was an API but there isnt. Well it does work on the simulator, Nexus devices, on iphone and ipad.

 

 

(I ask because a web service that responds with HTTP 200 on error and uses 302s for success sounds a little odd).

  • I think their final page gives the HTTP 200 (which is the final page found), in case of login failure they immediately give http 200 and show “login has failed”. But in case of success, I am assuming they have to fetch user account details and this logic sits in another page, so the login page redirects to that page.

 

 

Thank you for the debugging link, will try it out. 

 

Lastly, can you post your  networkListenerLogin  code? (or a simplification of it)

- sure, its as simple as this: 

 

– network listener for login page

function networkListenerLogin( event )

   print “Entering: networkListenerLogin”

    – process result

    if ( event.isError ) then

        local alert = native.showAlert( “Error”, “Network Error!”, { “OK”})

        print (event.bytesTransferred …","… event.status)

    else

       print (“something something”)

    end

end

In the logcat log, <print “Entering: networkListenerLogin”> is not getting printed on Galaxy and HTC phones, which implies the control is not even entering the network request listener function. Works like a charm on nexus and iOS.

This is the only thing stopping the app from going live, this is really a sticking point. 

Kindly help. 

Thanks and regards,

Anshuman

#4

Thanks for all the detail, it really makes working on issues like this a whole lot easier.

I need to schedule some time to set up a test environment but I hope to get a chance to look at it in the next couple of days.

#5

Many thanks, much appreciated. 

Cheers,

Anshuman

#6

Your recommendation for CharlesProxy was fab, I now have data from both - a working handset (Nexus) and from non working handset (Galaxy). 

On Nexus

  1. POST method for login to site

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegTm9QR3Z6Yy14ZzQ/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegVER4Smd3RUhEZHc/edit?usp=sharing

  1. The above step returns “302 found” - this then forwards to:

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegLXktaHB1QkJJXzA/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegR3hmM2J5VzN3d3M/edit?usp=sharing

  1. Finally it moves to the successful login welcome page:

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegTF9RSnhpNEhGbzA/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegNzJLV2x2WWd0eFU/edit?usp=sharing

On Galaxy

  1. POST method for login to site

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegaVVpWUUzWER0cEE/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegLUQ1SmdUaTlucE0/edit?usp=sharing

  1. The above step returns “302 found” - this then forwards (and this is where it times out)

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegN3JIbTVkZ0ZweGM/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegNEF3ZHJHY0NMUzQ/edit?usp=sharing (as we can see, no response was received)

Between 1 and 2, the control does not come to lua code (which is inline with the expectation that 302 will auto forward). What I am totally stumped with is that this works on Nexus but not on Galaxy or HTC :-/

PS: I do see 1 difference between the request headers on Nexus and Galaxy: in galaxy request header, there is an extra Content-Length param - is that something creating an issue? Would it lead to the timeout I am experiencing?

Any pointers will be very appreciated!

Thanks and regards,

Anshuman

#7

You have spotted the subtle mistake in the Galaxy GET request headers, the extraneous Content-Length header which is being sent with the same value as on the original POST.  This seems to be a problem with some Android implementations.  To be fair, redirecting from a POST to a GET in response to a 302 is theoretically illegal (the app is supposed to confirm with the user before redirecting according to the HTTP spec) so the fact that the implementations differ is not that surprising.  The timeout you see seems to one of several possible responses servers have.  For example, my test server rejects the GET with a  400 response (Bad Request).

I’ve rewritten some of our network code so that we handle the 302s ourselves instead of relying on whatever the individual device thinks is the best way to handle 302s.  This bug seems to be associated with some problems around cookies and redirects so I’m working on a fix for that too.

I’m in the testing phase of the fix which should show up in a Daily Build next week if all goes well.

Once again, many thanks for the tremendous detail you gave, it makes finding the bug and confirming that we’re seeing the same thing so much easier.

#8

Hi Perry,

Many thanks for your time and efforts on this, your swift response is much appreciated. I can empathise with the situation when not enough details are provided (as I am often in the same boat at work).

Any pointers when the fix would make it to the Public build? 

I am glad that my investigation details were helpful, a complimentary Pro account for me please?  ;)  :slight_smile:

Thanks and regards,

Anshuman

#9

Since the work you put in to reproduce the problem and identify the scenarios in which it was reproducible saved us a bunch of time, we’ll comp you a Pro account until the next Public Release so you can download a Daily Build with the fix in it.  PM me the email address you use to login to the Simulator.

Thanks for all the help!

#10

Hi Perry, that sounds awesome! I will drop you a PM. 

It’s a pleasure to be of help. Many thanks. 

#11

It’s a little unclear from your description but when you say “the control does not reach networkListenerLogin when correct username / password are used” you actually mean it fails in cases 4 and 5, but the other 3 work?

The 302 response should be transparent to you and you should see the response that you get when the location you are redirected to is loaded.  For completeness, what is that response when the process succeeds?

Is this a REST API you are using or are you trying to login via a URL usually intended for interactive use in a browser?  Scripting interactive logins isn’t impossible but using an API intended for the purpose is way easier (I ask because a web service that responds with HTTP 200 on error and uses 302s for success sounds a little odd).  Interfaces intended for browsers frequently change their responses for different user agents or operating systems whereas purpose built APIs are much more predictable.  Is this a public service you are trying to access? If so, which one?

The way I would approach a problem like this is to set up a proxy on your desktop computer and make the phone connect to the internet through that.  This isn’t necessarily easy but it can reveal subtleties of the protocol that are hard to see otherwise. There’s an article about doing that here: http://goo.gl/ywsWaz

Lastly, can you post your  networkListenerLogin code? (or a simplification of it)

#12

Hi Perry, 

Many thanks for replying. Please find details below:

 

It’s a little unclear from your description but when you say “the control does not reach networkListenerLogin when correct username / password are used” you actually mean it fails in cases 4 and 5, but the other 3 work?

  • Correct, the control does not reach the network request listener function (networkListenerLogin) at all for Samsung and HTC phones but works on Nexus phones and simulators. I can also confirm its working on iphone and ipads, I have managed to deploy them there as well.

 

 

The 302 response should be transparent to you and you should see the response that you get when the location you are redirected to is loaded.  For completeness, what is that response when the process succeeds?

  • You are right it should be transparent to me, and it is so in simulator and Nexus phones. When using the correct username/pwd, the order is as follows:

  1. Login page gives “302 Found”

  2. It then loads a 2nd page, which itself gives “302 Moved Temporarily”

  3. This finally lands in the logged in page

 

 

Is this a REST API you are using or are you trying to login via a URL usually intended for interactive use in a browser?  

  • I am logging in via a POST to a URL which is usually intended for web browser logins. I have no control on the website as they are another system

 

 

Scripting interactive logins isn’t impossible but using an API intended for the purpose is way easier

  • I wish there was an API but there isnt. Well it does work on the simulator, Nexus devices, on iphone and ipad.

 

 

(I ask because a web service that responds with HTTP 200 on error and uses 302s for success sounds a little odd).

  • I think their final page gives the HTTP 200 (which is the final page found), in case of login failure they immediately give http 200 and show “login has failed”. But in case of success, I am assuming they have to fetch user account details and this logic sits in another page, so the login page redirects to that page.

 

 

Thank you for the debugging link, will try it out. 

 

Lastly, can you post your  networkListenerLogin  code? (or a simplification of it)

- sure, its as simple as this: 

 

– network listener for login page

function networkListenerLogin( event )

   print “Entering: networkListenerLogin”

    – process result

    if ( event.isError ) then

        local alert = native.showAlert( “Error”, “Network Error!”, { “OK”})

        print (event.bytesTransferred …","… event.status)

    else

       print (“something something”)

    end

end

In the logcat log, <print “Entering: networkListenerLogin”> is not getting printed on Galaxy and HTC phones, which implies the control is not even entering the network request listener function. Works like a charm on nexus and iOS.

This is the only thing stopping the app from going live, this is really a sticking point. 

Kindly help. 

Thanks and regards,

Anshuman

#13

Thanks for all the detail, it really makes working on issues like this a whole lot easier.

I need to schedule some time to set up a test environment but I hope to get a chance to look at it in the next couple of days.

#14

Many thanks, much appreciated. 

Cheers,

Anshuman

#15

Your recommendation for CharlesProxy was fab, I now have data from both - a working handset (Nexus) and from non working handset (Galaxy). 

On Nexus

  1. POST method for login to site

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegTm9QR3Z6Yy14ZzQ/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegVER4Smd3RUhEZHc/edit?usp=sharing

  1. The above step returns “302 found” - this then forwards to:

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegLXktaHB1QkJJXzA/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegR3hmM2J5VzN3d3M/edit?usp=sharing

  1. Finally it moves to the successful login welcome page:

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegTF9RSnhpNEhGbzA/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegNzJLV2x2WWd0eFU/edit?usp=sharing

On Galaxy

  1. POST method for login to site

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegaVVpWUUzWER0cEE/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegLUQ1SmdUaTlucE0/edit?usp=sharing

  1. The above step returns “302 found” - this then forwards (and this is where it times out)

Request header: https://drive.google.com/file/d/0Bwv3_lBcyIegN3JIbTVkZ0ZweGM/edit?usp=sharing

Response header: https://drive.google.com/file/d/0Bwv3_lBcyIegNEF3ZHJHY0NMUzQ/edit?usp=sharing (as we can see, no response was received)

Between 1 and 2, the control does not come to lua code (which is inline with the expectation that 302 will auto forward). What I am totally stumped with is that this works on Nexus but not on Galaxy or HTC :-/

PS: I do see 1 difference between the request headers on Nexus and Galaxy: in galaxy request header, there is an extra Content-Length param - is that something creating an issue? Would it lead to the timeout I am experiencing?

Any pointers will be very appreciated!

Thanks and regards,

Anshuman

#16

You have spotted the subtle mistake in the Galaxy GET request headers, the extraneous Content-Length header which is being sent with the same value as on the original POST.  This seems to be a problem with some Android implementations.  To be fair, redirecting from a POST to a GET in response to a 302 is theoretically illegal (the app is supposed to confirm with the user before redirecting according to the HTTP spec) so the fact that the implementations differ is not that surprising.  The timeout you see seems to one of several possible responses servers have.  For example, my test server rejects the GET with a  400 response (Bad Request).

I’ve rewritten some of our network code so that we handle the 302s ourselves instead of relying on whatever the individual device thinks is the best way to handle 302s.  This bug seems to be associated with some problems around cookies and redirects so I’m working on a fix for that too.

I’m in the testing phase of the fix which should show up in a Daily Build next week if all goes well.

Once again, many thanks for the tremendous detail you gave, it makes finding the bug and confirming that we’re seeing the same thing so much easier.

#17

Hi Perry,

Many thanks for your time and efforts on this, your swift response is much appreciated. I can empathise with the situation when not enough details are provided (as I am often in the same boat at work).

Any pointers when the fix would make it to the Public build? 

I am glad that my investigation details were helpful, a complimentary Pro account for me please?  ;)  :slight_smile:

Thanks and regards,

Anshuman

#18

Since the work you put in to reproduce the problem and identify the scenarios in which it was reproducible saved us a bunch of time, we’ll comp you a Pro account until the next Public Release so you can download a Daily Build with the fix in it.  PM me the email address you use to login to the Simulator.

Thanks for all the help!

#19

Hi Perry, that sounds awesome! I will drop you a PM. 

It’s a pleasure to be of help. Many thanks.