Creating graphs with d3

Solar2D General Questions/Discussion
#1

Hi all,

I’m attempting to use the javascript graphing library d3 (via a webview) to create graphs in a Corona app. I have tested d3 successfully but am unsure of how to store and then access data that a user has created. For example, I’d like to create a line graph showing test results over several dates based on data generated by the app. Is there a way to store these data (generated in the app) and then retrieve them for use by d3 in my webview?

In other words, I need to dynamically create and load something like a json file that the webview can access.

Many thanks for any ideas,

Jeff

#2

Hi Jeff

If you put your webview html file in your Documents folder then write your JSON file there you can access it via Javascript in the webview using AJAX calls. You can also pass URL parameters to the webview when you open it and decode them in Javascript. It can be a bit clunky but you can certainly transfer data to and fro between Corona/Lua and Javascript in the webview by these means.

Stefan

#3

Hi Stefan,

Thanks very much for your reply. I haven’t used Ajax before with Corona so I was wondering if you could show me a simple code example of how that would work. 

Much appreciated!

Jeff

#4

You will need to make the Ajax calls in Javascript code in the HTML file used for the webview, so it is not Corona or Lua specific. Lot’s of different ways of doing that, try here for starters …

http://www.w3schools.com/json/json_http.asp

You could also use a Javascript library or framework like JQuery (or something more lightweight) to encapsulate the functionality for you.

You can create and read your local JSON data files from Lua tables using these functions …

local json = require(“json”)

function saveTable(t, filename)

    local path = system.pathForFile( filename, system.DocumentsDirectory)

    local file = io.open(path, “w”)

    if file then

        local contents = json.encode(t)

        file:write( contents )

        io.close( file )

        return true

    else

        return false

    end

end

function loadTable(filename)

    local path = system.pathForFile( filename, system.DocumentsDirectory)

    local contents = “”

    local myTable = {}

    local file = io.open( path, “r” )

    if file then

        – read all contents of file into a string

        local contents = file:read( “*a” )

        myTable = json.decode(contents);

        io.close( file )

        return myTable

    end

    return nil

end

Hope that helps get you started.

#5

Thanks spjnowak, that certainly helps on the lua side. But how would the html page load the json file?

Currently, I have the following to load the webpage:

webView:request( “graph.html”, system.ResourceDirectory )

What would the js code look like in graph.html to load the json file (say, “myData.json”) that is located in the document dir?

#6

Try something like this in your graph.html file (I just did a Goggle search to find the code so I haven’t tested it). Note that you should put both your HTML file and your JSON file in the same folder - as you’ll be writing to the JSON file that will need to be Documents not Resources.

<javascript>

function loadJSON(file, callback) {   

    var xobj = new XMLHttpRequest();

    xobj.overrideMimeType(“application/json”);

    xobj.open(‘GET’, file, true); // Replace ‘my_data’ with the path to your file

    xobj.onreadystatechange = function () {

          if (xobj.readyState == 4 && xobj.status == “200”) {

            // Required use of an anonymous callback as .open will NOT return a value but simply returns undefined in asynchronous mode

            callback(xobj.responseText);

          }

    };

    xobj.send(null);  

 }

function load() {

    

    loadJSON(“data.json”, function(response) {

  

        var actual_JSON = JSON.parse(response);

        console.log(actual_JSON);

    });

    

    

}

</javascript>

#7

Just a point of clarification about my question: The html file is in the same directory as my lua files, but the json file, created dynamically, would be in system.ResourceDirectory. So the question is what would be the path info for using something like $.getJSON(“my_data.json”) from within the html file? Hope that helps make my question clearer!

Jeff

#8

Wherever your JSON file is your HTML file should be in the same directory then you can just use the file name on its own. It’s been a few years since I’ve done this but I recall that it only worked if both files were in Documents, particularly on Android. Resources files are read-only and as the HTML file is being modified - you’re injecting data into the DOM with the ajax call - it needs to be in a writable directory (documents or Temporary). 

It’s also possible that the iOS9 security changes referred to elsewhere may have an impact even though you’re using local files rather than HTTP. 

#9

Hmmm… That leaves me kind of stuck because I have js, jquery, and a css file, in my main lua app directory. It would seem a bit over the top (if even possible) to write all of these via my app into the documents directory. 

Has anyone solved this issue elsewhere? What about passing the documents path to the webview as a parameter?

Seems like loading webview HTML pages with content that is dynamically generated in the app would be a useful/common thing.

#10

This is all possible. You can specify a folder name as part of the the file string when opening the webview if you want to. You can pretty much run an entire website within a webview if you want. All I am saying is that unless all the content is read-only you may run into problems having all your files in the system Resources folder. An easy way of achieving what you need is to zip all of your HTML, Javascript and CSS files into a zip file then unzip them into the Temporary folder when your app loads using the zip plugin. If you unzip into the Documents folder you may also need to set the file attribute telling iOS not to back up the files to iCloud otherwise you may get rejected by Apple.

#11

Wow. All this to use a chart? What type of chart do you need? I’ve just started created a fairly okay barChart module if you need it…

#12

Hey thomas, I’d love to see it if you can share. I actually need a fairly simple line chart with interactive nodes (to highlight data points). But d3 is pretty awesome for creating graphs of many kinds. However, it does seem painful!

For now, I’m working on spjnowak’s suggestion about zipping the files (I’ll just have the jquery and html file (will put all other js and css directly in the html file for convenience). I though I was going to be good to go using $.getJSON but it all breaks if the html is in the ResourceDirectory dir and the json file is in the DocumentsDirectory.

Cheers,

Jeff

#13

I’m working on the lineChart as we speak. Should have a simple lineChart up tomorrow. Keep you posted!

#14

Well, here’s something unexpected… I decided to just try passing some data as a comma delimited string (myString=“1234,1234,1234,1234,…”) via this call:

webView:request( "graph.html?myData=" .. myString, system.ResourceDirectory )

I didn’t try this before because I had figured that the maximum length of myString as a parameter would be quite short. However, doesn’t seem to be the case. I tried as much as 15,000 bytes (that’s more than enough data for my graph) and it handled it without any problems. Meaning that I passed the string and retrieved it in the webpage with this:

\<script type="text/javascript"\> var GET = {}; var query = window.location.search.substring(1).split("&"); for (var i = 0, max = query.length; i \< max; i++) { if (query[i] === "") // check for trailing & with no param continue; var param = query[i].split("="); GET[decodeURIComponent(param[0])] = decodeURIComponent(param[1] || ""); } var theData = GET.myData; jQuery(document).ready(function(event){ $('#myDiv').html(theData); }); \</script\>

Now I just need to make a string of real data and create a js function to parse it into the format that d3 requires. I’ll try that tomorrow and report back. But this is looking very promising.

Does anyone know what the limit actually is on passing strings as webview params?

#15

Just wanted to confirm that the technique above, passing a long string of data to the webView as a parameter works as a method to create a graph with d3. In the webpage, I use a js script to parse the string and put it into the format I need for the graph. The graph is a variation of this type of thing:

http://bl.ocks.org/d3noob/b3ff6ae1c120eea654b5

I would still be interested to know what the length limit is though, for passing a string as a webView parameter. 

#16

Hi Jeff,
nice examples!

I am wondering: Did you run into some difficulties to use d3 library in your HTML? I am experimenting with this, plain JavaScript works fine, but jQuery deosn’t work, placed locally in Resources dir, Documents dir, even embedded in the HTML file. When everything is online on a server, it works, but locally library doesn’t work. What’s your experience, pls?

#17

Hi Petrsvar,

Sorry for the delay in getting back to you. I didn’t use jQuery in my code although I would have thought that it should work since other js works.

Sorry, I have used jQuery (just checked!) and it seems to be fine. I use it just to check when the page is loaded ( jQuery(document).ready(function(event) ).

The biggest issue for me in using the web view is that now I can’t save the graph as an image to the camera roll (since its a native element). The workaround, which isn’t great, is to tell users to capture the image with their device’s native methods (e.g. side + home button on the iPhone). 

Too bad there isn’t a handy charting library for Corona.

#18

Hi Thomas,

Did you finish your lineChart code? I’d love to see it!

Jeff

#19

Hi Thomas,

Any news on your charting code? I am having more grief with my webView attempts because passing a GET var on Android devices doesn’t seem to work. Sigh.

#20

Hi Jeff

If you put your webview html file in your Documents folder then write your JSON file there you can access it via Javascript in the webview using AJAX calls. You can also pass URL parameters to the webview when you open it and decode them in Javascript. It can be a bit clunky but you can certainly transfer data to and fro between Corona/Lua and Javascript in the webview by these means.

Stefan