LuaSockets Data Corruption

Solar2D General Questions/Discussion
#1

Hi, ive just finished adding LAN network support for my game using luaSocket. Everything works perfectly on the simulator, however on the device i lose the ability to dispatch my receive events in the app after about an hour. Basically i have written a network library that dispatches runtime events when new tcp packets are received. After about an hour i notice i no longer get any events, originally i though the network connection was dropping out or something but after connecting the device up to xcode organiser to watch the console, i found that its actually failing when attempting to displatch a runtime event.

clientReceive - dispatching clientreceive event
CommCenter[61] : No more assertions for PDP context 0. Returning it back to normal.
CommCenter[61] : Client [com.apple.persistentconnection[apsd,73]] is telling PDP context 0 to go active.
CommCenter[61] : Scheduling PDP tear down timer for (376838182.514994) (current time == 376837882.515000)

So i can see that i am calling my dispatch event code, then i get this strange error in the log and i dont receive my event.

Any help/suggestions would be greatly appreciated
Thanks,
Craig [import]uid: 110197 topic_id: 33837 reply_id: 333837[/import]

#2

oh FYI im using Corona 2012.971, problem is occuring on both my devices, iPhone5 ios 6.0 and iPad3 ios 5.1.1 [import]uid: 110197 topic_id: 33837 reply_id: 134493[/import]

#3

Are you saying the object:dispatchEvent is not working after an hour? I’ve never heard of that problem before and would need to see the code along with a short project that demonstrates the problem. Can you file a bug report with project files?

Have you tried disabling the LAN code and dispatching events based on a timer to try and isolate what in the code is not working? [import]uid: 7559 topic_id: 33837 reply_id: 134624[/import]

#4

oh FYI im using Corona 2012.971, problem is occuring on both my devices, iPhone5 ios 6.0 and iPad3 ios 5.1.1 [import]uid: 110197 topic_id: 33837 reply_id: 134493[/import]

#5

Yep thats what im seeing.

I have a function that uses the lua socket tcp receive() function, looks at the result and then does

Runtime:dispatchEvent({name = “TCP_ClientReceive”, client = clients[i], ip = clientIP, data = msg})
print(“clientReceive - dispatching clientreceive event”)

The print line above comes up in the log right before the error. Those are the last two lines in that function.

The main game starts a listener
Runtime:addEventListener( “TCP_ClientReceive”, onTCPClientReceive )

which receives the notification of new TCP messages. The tcp messages are fired between turns. The code that executes for the end of each turn is the same. Im not sure how many turns i did in my testing, but the failures ive had have all been roughly after about an hour. Basically the call to onTCPClientReceive just stops occuring and that iOS error message was in console. The dispatched event is normally received immediately (when its working) which also generates an additional console output to say it received the event.

Im pretty busy coming up to xmas, but i will try and make up a test app to reproduce the problem when i get some time. In the meantime i will try removing the dispatching of events from the networking code and switch to a system of polling the network class for new messages instead of waiting for an event to see if that resolves the issue. [import]uid: 110197 topic_id: 33837 reply_id: 134792[/import]

#6

i made some changes to do the bulk of the network processing on request rather than via events and have found something strange is still going on. The network library is the new component to a well tested app, so i presume it must be something in that causing the issue. It was as if it stopped allocating memory for tables after a while, which may explain why the events stopped as well, so seems like it could be being triggered by a memory issue. Ill add some memory monitoring and see what else i can find out, by the looks of it chances are its something ive done wrong with the network manager. [import]uid: 110197 topic_id: 33837 reply_id: 134810[/import]

#7

Based on the time and what’s happening it could be a subtle memory leak. Generally you get will come low memory warnings in the Console plus you could implement the memoryWarning event to see if that triggers. This usually occurs when Texture Memory leaks. It could be a leak on the Lua side (instead of Texture Memory) so monitoring GC would be good too. [import]uid: 7559 topic_id: 33837 reply_id: 134842[/import]

#8

Are you saying the object:dispatchEvent is not working after an hour? I’ve never heard of that problem before and would need to see the code along with a short project that demonstrates the problem. Can you file a bug report with project files?

Have you tried disabling the LAN code and dispatching events based on a timer to try and isolate what in the code is not working? [import]uid: 7559 topic_id: 33837 reply_id: 134624[/import]

#9

Yep thats what im seeing.

I have a function that uses the lua socket tcp receive() function, looks at the result and then does

Runtime:dispatchEvent({name = “TCP_ClientReceive”, client = clients[i], ip = clientIP, data = msg})
print(“clientReceive - dispatching clientreceive event”)

The print line above comes up in the log right before the error. Those are the last two lines in that function.

The main game starts a listener
Runtime:addEventListener( “TCP_ClientReceive”, onTCPClientReceive )

which receives the notification of new TCP messages. The tcp messages are fired between turns. The code that executes for the end of each turn is the same. Im not sure how many turns i did in my testing, but the failures ive had have all been roughly after about an hour. Basically the call to onTCPClientReceive just stops occuring and that iOS error message was in console. The dispatched event is normally received immediately (when its working) which also generates an additional console output to say it received the event.

Im pretty busy coming up to xmas, but i will try and make up a test app to reproduce the problem when i get some time. In the meantime i will try removing the dispatching of events from the networking code and switch to a system of polling the network class for new messages instead of waiting for an event to see if that resolves the issue. [import]uid: 110197 topic_id: 33837 reply_id: 134792[/import]

#10

i made some changes to do the bulk of the network processing on request rather than via events and have found something strange is still going on. The network library is the new component to a well tested app, so i presume it must be something in that causing the issue. It was as if it stopped allocating memory for tables after a while, which may explain why the events stopped as well, so seems like it could be being triggered by a memory issue. Ill add some memory monitoring and see what else i can find out, by the looks of it chances are its something ive done wrong with the network manager. [import]uid: 110197 topic_id: 33837 reply_id: 134810[/import]

#11

Based on the time and what’s happening it could be a subtle memory leak. Generally you get will come low memory warnings in the Console plus you could implement the memoryWarning event to see if that triggers. This usually occurs when Texture Memory leaks. It could be a leak on the Lua side (instead of Texture Memory) so monitoring GC would be good too. [import]uid: 7559 topic_id: 33837 reply_id: 134842[/import]

#12

Thanks Tom, i have now tried a heap of different memory monitoring techniques and as far as i can see there isnt a memory leak. Ive been using Corona Profiler (by myDevelopers) and also added realtime memory stats to the screen to see whats happening. The memory is sitting around 664Kb and the texture memory doesnt move from 303Mb. (iPad 3 high res textures).

Im checking memory usage using collectgarbage( “count” ) and texture memory usage by using system.getInfo( “textureMemoryUsed” ).

There are absolutely no fluctations in the texture memory at all, and only quite small in the memory side of things (ie 10kb). I also dont get any memory warnings in the simulator (however i never have any problems with the simulator, always with the device).

Do you have any suggestions as to what else I can check or try?
[import]uid: 110197 topic_id: 33837 reply_id: 135217[/import]

#13

Actually i have now made a sample application which reproduces the problem, seems to be related to sending large packets over luasockets. ie a couple of kb. Sending small messages seems to work all the time, large ones dont. The packet data is corrupt, or the memory for it is. The test repeats sending the same block of data ~6kb back and forth between client and server. Im using simulator as server and iPad as client. It works correctly for a random number of times and then either returns a partial packet (only the end of the packet, the beginning is always missing) or it (client:eeceive) returns data but the pkt data is not a table, it is just the number 14… wtf? :slight_smile: Always one or the other with this samlpe app.

I will submit it to the bug reports tomorrow after some more messing around to see if i can figure anything else out first. [import]uid: 110197 topic_id: 33837 reply_id: 135237[/import]

#14

I have now submitted a bug report. My findings are as follows

Transmitting large blocks of data over tcp using luasockets (eg ~6k) intermittantly corrupts when received. The client/server receive() function is doing one of two things (randomly)

  1. It returns an incomplete packet (the beginning is missing, not the end). The receive function is not returning any errors
  2. It returns a non-table data packet. It is just the number 14.

I made a test program that demonstrates this. The test program pings 6k messages back and forth between a server and client. It will occur randomly (anywhere from 1 ping to a few hundred pings before it happens.)

NOTE: The test MUST be performed across a proper network. Running two simulators on the same computer does NOT produce the problem. I have reproduced using two simulators on different computers, 1 simulator and one device (iPad3) and also 2 devices (iPad4 and iPhone5).
[import]uid: 110197 topic_id: 33837 reply_id: 135296[/import]

#15

Thanks for creating a test case and filing the bug. We can look at this but if the problem is in LuaSockets, we may not be able to provide a solution since LuaSockets is a third party library that we include in Corona. There is an unofficial updated library available but if we included it in Corona, we don’t know what other issues it may cause. We need to wait for a tested and official release of the library but that may not happen anytime soon (if ever). We know the updated library fixes the existing base64 bug, but we not comfortable updating the library.

With that said, we can look at your test case and see if there anything we can do.

[import]uid: 7559 topic_id: 33837 reply_id: 135300[/import]

#16

Ok thanks Tom, I look forward to seeing what you find.

If it is a bug with the luasocket library, would there be a possibility of having a new option in the build.settings file to enable compiling with the new library? That way it would only be used by people who wanted to test the latest library and wouldnt affect anyone else. (not sure if this is possible with your compiler setup).

Alternatively are there any other networking libraries that you work with Corona? I would have thought this sort of networking would be quite heavily used for multiplayer games.

Cheers
Craig [import]uid: 110197 topic_id: 33837 reply_id: 135420[/import]

#17

On an unrelated note, is there a way to change the name of the thread?
“Help, dispatch events failing” doesnt really seem to apply anymore, may be more useful to other people with a better name [import]uid: 110197 topic_id: 33837 reply_id: 135421[/import]

#18

Thanks Tom, i have now tried a heap of different memory monitoring techniques and as far as i can see there isnt a memory leak. Ive been using Corona Profiler (by myDevelopers) and also added realtime memory stats to the screen to see whats happening. The memory is sitting around 664Kb and the texture memory doesnt move from 303Mb. (iPad 3 high res textures).

Im checking memory usage using collectgarbage( “count” ) and texture memory usage by using system.getInfo( “textureMemoryUsed” ).

There are absolutely no fluctations in the texture memory at all, and only quite small in the memory side of things (ie 10kb). I also dont get any memory warnings in the simulator (however i never have any problems with the simulator, always with the device).

Do you have any suggestions as to what else I can check or try?
[import]uid: 110197 topic_id: 33837 reply_id: 135217[/import]

#19

I changed the thread name but left the page URL the same. [import]uid: 7559 topic_id: 33837 reply_id: 135528[/import]

#20

Actually i have now made a sample application which reproduces the problem, seems to be related to sending large packets over luasockets. ie a couple of kb. Sending small messages seems to work all the time, large ones dont. The packet data is corrupt, or the memory for it is. The test repeats sending the same block of data ~6kb back and forth between client and server. Im using simulator as server and iPad as client. It works correctly for a random number of times and then either returns a partial packet (only the end of the packet, the beginning is always missing) or it (client:eeceive) returns data but the pkt data is not a table, it is just the number 14… wtf? :slight_smile: Always one or the other with this samlpe app.

I will submit it to the bug reports tomorrow after some more messing around to see if i can figure anything else out first. [import]uid: 110197 topic_id: 33837 reply_id: 135237[/import]