Permitting cleartext on specific domains

Solar2D Android
#1

Is there presently any way to specify domains on which cleartext traffic is permitted on Android? We use app42 to download resources into our apps and this constitutes an important feature for most of our applications but we’re now unable to do this because app42 supplies us http URLs and https doesn’t work.

A network security config is the way to go as far as I’ve understood through other forums etc but there doesn’t seem to be a way to edit the xml file in Solar2D and without specifying the permission on app42’s domain, we will not be able to make HTTP calls.

I know this probably sounds a tad selfish because I don’t know how many other users will need this but I sincerely hope that @vlads and co can add some functionality which can allow editing the security config. Without the ability to do this, we will have to sacrifice some major features of our android apps.

Thanks

1 Like
Application without internet access
#2

Absolutely!
You can totally add cleartext exceptions. To do that, create AndroidResources/res/xml/network_security_config.xml file in your project with contents like:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <domain-config cleartextTrafficPermitted="true">
        <domain includeSubdomains="true">some.domain.com</domain>
        <domain includeSubdomains="true">other.domain.com</domain>
    </domain-config>
</network-security-config>
3 Likes
How to modify build.settings
Urgent Major Bugs In Versions After Solar2D-Windows-2021.3660
#3

Thanks for such a quick response! I’ll give this a go and post back.

#4

Thanks @vlads the approach worked for us. Really appreciate the support.

#5

This worked for me too, just set “cleartextTrafficPermitted” to false. Thank you very much Vlad!

#6

Hello.

I was hoping this would work for me, but it didn’t!
I’ve been beating my head against this one for awhile and hope that @vlads or someone can help. Here’s the facts:

(1) I have an app that uses “network.download”.
(2) The code works in the simulator and on my iPhone
(3) The code does NOT work on Android
(4) I can load the URL in a browser window on my Mac and using a browser on my Android phone
(5) The URL is:

https://graphical.weather.gov/xml/sample_products/browser_interface/ndfdBrowserClientByDay.php?whichClient=NDFDgenByDayLatLonList&listLatLon=44.112733%2C-73.923725&format=24+hourly&startDate=2022-06-03&numDays=4&Unit=e&Submit=Submit

(6) The file that returns from this is an XML file that I store locally in the “system.DocumentsDirectory”
(7) The file is returned, stored, read, and parsed correctly in the simulator and on my iPhone.

(8) On Android, when I look at “event.response” I get back an HTML formatted response that says the following:

<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
You don't have permission to access "http&#58;&#47;&#47;graphical&#46;weather&#46;gov&#47;xml&#47;sample&#95;products&#47;browser&#95;interface&#47;ndfdBrowserClientByDay&#46;php&#63;" on this server.<P>
Reference&#32;&#35;18&#46;97ee2117&#46;1654282553&#46;75ff6ce
</BODY>
</HTML>

(9) I added an “xml” folder to my “AndroidResources / res” directory with a new file - “network_security_config.xml” with the following:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <domain-config cleartextTrafficPermitted="false">
        <domain includeSubdomains="true">graphical.weather.gov</domain>
    </domain-config>
</network-security-config>

(10) The version of Solar2d that I’m using is: 2022.3673
(11) I have the following in my build.settings:
“android.permission.INTERNET”,
“android.permission.WRITE_EXTERNAL_STORAGE”,
“android.permission.READ_EXTERNAL_STORAGE”,

Help! Please! And thank you in advance. ~Keith

#7

Here’s the full output of the event response:

Jun 03 15:28:47.978 moto g(7): table: 0x7ca2cdffc0 
Jun 03 15:28:47.979 {
Jun 03 15:28:47.980 moto g(7):  [responseHeaders] => table: 0x7ca2cdffc0
                    moto g(7):  {
                    moto g(7):     [Connection] => "close"
                    moto g(7):     [Content-Length] => "364"
                    moto g(7):     [X-Android-Response-Source] => "NETWORK 403"
                    moto g(7):     [Date] => "Fri, 03 Jun 2022 19:28:47 GMT"
                    moto g(7):     [X-Android-Sent-Millis] => "1654284532650"
                    moto g(7):     [Expires] => "Fri, 03 Jun 2022 19:28:47 GMT"
Jun 03 15:28:47.990 moto g(7):     [X-Android-Selected-Protocol] => "http/1.1"
                    moto g(7):     [Strict-Transport-Security] => "max-age=31536000 ; includeSubDomains ; preload"
                    moto g(7):     [X-Android-Received-Millis] => "1654284532690"
                    moto g(7):     [Content-Type] => "text/html"
                    moto g(7):     [Mime-Version] => "1.0"
                    moto g(7):     [HTTP-STATUS-LINE] => "HTTP/1.1 403 Forbidden"
                    moto g(7):     [Server] => "AkamaiGHost"
                    moto g(7):  }
                    moto g(7):  [responseType] => "text"
                    moto g(7):  [phase] => "ended"
Jun 03 15:28:47.991 moto g(7):  [bytesEstimated] => 364
                    moto g(7):  [response] => "<HTML><HEAD>
                    moto g(7): <TITLE>Access Denied</TITLE>
                    moto g(7): </HEAD><BODY>
                    moto g(7): <H1>Access Denied</H1>
                    moto g(7):  
                    moto g(7): You don't have permission to access 
"http&#58;&#47;&#47;graphical&#46;weather&#46;gov&#47;xml&#47;sample&#95;products&#47;browser&#95;interface&#47;ndfdBrowserClientByDay&#46;php&#63;" on this server.<P>
                    moto g(7): Reference&#32;&#35;18&#46;7fee2117&#46;1654284527&#46;4b29c139
                    moto g(7): </BODY>
                    moto g(7): </HTML>
                    moto g(7): "
                    moto g(7):  [name] => "networkRequest"
                    moto g(7):  [bytesTransferred] => 364
                    moto g(7):  [status] => 403
                    moto g(7):  [url] => "https://graphical.weather.gov/xml/sample_products/browser_interface/ndfdBrowserClientByDay.php?whichClient=NDFDgenByDayLatLonList&listLatLon=44.112733%2C-73.923725&format=24+hourly&startDate=2022-06-03&numDays=4&Unit=e&Submit=Submit"
                    moto g(7):  [isError] => false
                    moto g(7):  [requestId] => false
                    moto g(7): }
#8

A little more information @vlads that may help:
The URL: I’m calling is:

https://graphical.weather.gov/xml/SOAP_server/ndfdSOAPclientByDay.php?whichClient=NDFDgenByDay&lat=44.11&lon=-73.923725&format=24+hourly&startDate=2022-06-04&numDays=3&Unit=e&Submit=Submit

I also replaced the URL in the sample app code in “AsyncHTTP” with the URL above and I get the same thing… a 403 access denied.

Again, the URL works in the simulator. It works on an iOS device. And it works if I enter it in a browser on my Android device. But it doesn’t work in an app on an Android device.

Help, please and thank you in advance. I have a couple of apps that use this service and that are now broken. ~Keith

#9

May be you’re missing the android internet permission from build.settings? It is weird, because 403 is a valid error code you’re getting from the server, so probably something is going on with the request. Like, it doesn’t like default http client header or something.
Also, 403 still can have a body explaining what went wrong. Check console or output body for details.
Btw, it is irrelevant to topic because you got encrypted traffic - URL is https.

Reading previous post - yup. Problem is probably in https client, because you are getting error from the server. Try to pretending to be a browser by setting User-Agent header.

#10

Thank you for replying:

I have this in my build settings:

	-- 
	-- Android section
	--	
	android =
   {
      versionCode = "260", -- app's actual version (release use 1 = 1.0; 12 = 1.2 - decimals not allowed)
      versionName = "2.6.0", -- shown to user
      usesPermissions =
         {
			"android.permission.CAMERA",
         "android.permission.INTERNET",
         "android.permission.WRITE_EXTERNAL_STORAGE",
         "android.permission.READ_EXTERNAL_STORAGE",
         "android.permission.ACCESS_FINE_LOCATION",
         "android.permission.ACCESS_COURSE_LOCATION",
         },

And I have this in the “AndroidResources / res / xml / newtowrk_security_config.xml” file:

<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
    <domain-config cleartextTrafficPermitted="true">
        <domain includeSubdomains="true">graphical.weather.gov</domain>
    </domain-config>
</network-security-config>

And I’ve been trying a bunch of header options including the most recent set, which is:

   local headers = {}
   headers["Connection"] = "keep-alive"
   headers["Content-Type"] = "text/xml"
   headers["Strict-Transport-Security"] = "max-age=31536000 ; includeSubDomains ; preload"
   headers["Vary"] = "Accept-Encoding"
   headers["Accept-Encoding"] = "deflate"
   headers["Accept-Language"] = "en-US"
   headers["Accept"] = "*/*"

   local params = {}
   params.headers = headers
   params.progress = "download"
      
   network.download( weatherparser.weatherPathFileName, "GET", XMLNetworkListener, weatherFileName, system.DocumentsDirectory )

All that being equal. The URL doesn’t work in the Solar2d sample code on an Android device, but it does everywhere else… Any help with this is appreciated.
Thank you. ~Keith

#11

Hey. Your issue is unrelated to the topic. You are getting response from server containing error. Try setting some User-Agent string or something like that so you won’t get bounced back. Error you getting is from server, it doesnt like the request.

#12

I apologize if this seems unrelated to the topic. I’ve been searching everywhere for an answer to this issue and this is the closest relevant issue was this one and it seemed like the Android manifest solution might fix it. If you could show me how to set the “User-Agent” in my call (which I can’t find in the Solar2d documentation) I’d appreciate it.

#13

Try adding


headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.63 Safari/537.36”
#14

Thank you. Also, sorry I hadn’t seen your previous comment only the one at the end of the list, so initially I missed your suggestion to try setting the user-agent.
Anyway, the above setting produces the same error.
I also tried “Mozilla/5.0 (Linux; Android 10; moto g(7)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.79 Mobile Safari/537.36” for the moto g(7) that the app is running on. Also, didn’t work.

#15

@Vlads,
Is there any chance that the network.download command on Android is ignoring my “user-agent” setting, or changing it?

"https://graphical.weather.gov/xml/sample_products/browser_interface/ndfdBrowserClientByDay.php?listLatLon=44.11,-73.923725&format=24+hourly&numDays=3"

The reason I ask is that I’ve copied the header information from a successful call via a browser and have used it in the app and again, it only fails on Android. Here’s the settings I’m using:

   headers["Accept"]          = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
   headers["Host"]          = "graphical.weather.gov"
   headers["User-Agent"]      = "ADK46erNow/v2.6.0 (http://onebadant.com; ant@onebadant.com)"
   headers["Accept-Language"] = "en-US,en;q=0.9"
   headers["Accept-Encoding"] = "gzip, deflate, br"
   headers["Connection"] = "keep-alive"

Thank you in advance. ~Keith

P.S. This just in… I’ve been on the gov weather forum and they were able to see that my user-agent header was being set back to the default Android user agent. Meaning that it’s ignoring the user-agent I’m specifically setting. This sounds like a bug or something that needs to be fixed in Solar2d’s code, unless there’s a setting I can make that specifically tells the app to use my user-agent setting and not change or ignore it.

#16

huh. I’ll try debugging it then. This is very curious. Thanks.

#17

Hey. I figured it out. It seems that params table wasn’t passed anywhere.
Here is fully working code. Btw, I would recommend to drop other headers, since they do not do much.

local json = require 'json'

function XMLNetworkListener(e)
	print(json.prettify(e))
end
weatherFileName = 'weather.xml'
weatherparser = { weatherPathFileName = 'https://graphical.weather.gov/xml/sample_products/browser_interface/ndfdBrowserClientByDay.php?listLatLon=44.11,-73.923725&format=24+hourly&numDays=3'}


local params = { 
	headers = {
		["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.63 Safari/537.36",
	},
}
  
network.download( weatherparser.weatherPathFileName, "GET", XMLNetworkListener, params, weatherFileName, system.DocumentsDirectory )
#18

Absolutely Beautiful!
Thank you, thank you, thank you!
Sincerely, ~Keith