How to sync game progress across multiple devices

Solar2D General Questions/Discussion
#1

Skip to the bottom for a TL;DR

This is a pretty broad question but I’m hoping someone can give some insight on this topic. Most mainstream games have the feature of storing your game progress in the cloud so that you can pick up where you left off on other devices. On the surface this sounds pretty easy to implement but I’d like to do it the smart way. I also don’t want to rely on platform specific tools like iCloud or whatever the Android equivalent is.

My naive approach was to send the entire json file to and from the server when it’s time to sync. But if you consider a game like Candy Crush, which has hundreds of levels, if I send the entire file just to update one score then I’m wasting a lot of bandwidth.

My next idea was to duplicate any changes in a separate file that is specifically made for syncing. If the user is playing offline then any new progress will be saved there, and once the user connects only the data in that file will be sent to the server. Afterwards, the file will be cleared to indicate there is nothing new to send. But then, if I pick up my tablet and want to continue the game, how will the server know to only send the newly updated information? And what if the user has more than two devices? How would I make sure that all devices get properly updated?

I guess the main question is: What is the best way to store user progress on my server so that I can easily and cheaply sync that progress across multiple devices (regardless of device platform)?

Does anyone out there have experience with this stuff?

#2

I would think they are probably using an online database, perhaps something like Parse.com or your own MySQL database.  You can also check out Coronimum (http://coronium.io) which is built by our own Chris Byerley.

The idea is you only store and retrieve the bits you need for a given level.

Rob

#3

What I do is normally have a sequence ID (SID) which the server keeps track of and the device can persist. Every time there is a change, the server increments the SID. Every time the device sends or retrieves information, it receives a SID. So:

  • When the device sends information to the server, the server increments the SID and sends the new SID to the device. The device saves the SID locally.

  • When the device requests information, it includes the SID in the request. The server generates a list of “changes since SID” and sends a response with changes + new SID. The device saves the changes and the new SID locally.

  • When a device doesn’t have a SID (fresh app install), it asks with SID = 0. The server will generate “all changes since the beginning of time” + new SID. Again, the device saves the changes and the SID locally.

Hope that made sense.

#4

@Rob, thanks I’ve actually been looking at both platforms for my back-end.

@memo

This actually makes a lot of sense. The concept I’m still struggling with though is how do I generate the list of changes since the last sync? The only way I can think of is that each row in my database will have the playerid, data for one level, and the SID associated with that sync. Then when the device requests a sync it’s just a simple WHERE clause on the SID where each row would be the data for each level that needs to be synced.

SELECT \* FROM progress WHERE SID \> deviceSID AND playerID = userID

The only drawback I see with my approach above is that if my game achieves fairly moderate success of 1 million players, and they all get to level 100 or above, then there would be at least 100 million rows in my database. I guess that would be a nice problem to have haha.

Is that how you do it, or is there another way?

#5

That’s pretty much the idea; you have to store every change in the database, and add a SID (which can be an autoincrement column). You can choose either to store the whole level data in each row, or store it incrementally to save storage space. Either way you’ll have quite a few rows in your DB. However, if you do get to millions of players, I think your bottleneck is more likely to be your webserver breaking down from thousands of requests per second. When dealing with millions of clients, things become a lot more complex. You would require multiple server nodes, load balancing, possibly database sharding, etc. Or you would have to buy a cloud solution somewhere that would handle all of that for you. I would deal with that problem when you come to it – if you get millions of users, you’ll hopefully also be able to invest the money necessary to handle them :slight_smile:

#6

Sometimes I get ahead of myself and think too far into the future haha.

Thanks I really appreciate the insight!

#7

I would think they are probably using an online database, perhaps something like Parse.com or your own MySQL database.  You can also check out Coronimum (http://coronium.io) which is built by our own Chris Byerley.

The idea is you only store and retrieve the bits you need for a given level.

Rob

#8

What I do is normally have a sequence ID (SID) which the server keeps track of and the device can persist. Every time there is a change, the server increments the SID. Every time the device sends or retrieves information, it receives a SID. So:

  • When the device sends information to the server, the server increments the SID and sends the new SID to the device. The device saves the SID locally.

  • When the device requests information, it includes the SID in the request. The server generates a list of “changes since SID” and sends a response with changes + new SID. The device saves the changes and the new SID locally.

  • When a device doesn’t have a SID (fresh app install), it asks with SID = 0. The server will generate “all changes since the beginning of time” + new SID. Again, the device saves the changes and the SID locally.

Hope that made sense.

#9

@Rob, thanks I’ve actually been looking at both platforms for my back-end.

@memo

This actually makes a lot of sense. The concept I’m still struggling with though is how do I generate the list of changes since the last sync? The only way I can think of is that each row in my database will have the playerid, data for one level, and the SID associated with that sync. Then when the device requests a sync it’s just a simple WHERE clause on the SID where each row would be the data for each level that needs to be synced.

SELECT \* FROM progress WHERE SID \> deviceSID AND playerID = userID

The only drawback I see with my approach above is that if my game achieves fairly moderate success of 1 million players, and they all get to level 100 or above, then there would be at least 100 million rows in my database. I guess that would be a nice problem to have haha.

Is that how you do it, or is there another way?

#10

That’s pretty much the idea; you have to store every change in the database, and add a SID (which can be an autoincrement column). You can choose either to store the whole level data in each row, or store it incrementally to save storage space. Either way you’ll have quite a few rows in your DB. However, if you do get to millions of players, I think your bottleneck is more likely to be your webserver breaking down from thousands of requests per second. When dealing with millions of clients, things become a lot more complex. You would require multiple server nodes, load balancing, possibly database sharding, etc. Or you would have to buy a cloud solution somewhere that would handle all of that for you. I would deal with that problem when you come to it – if you get millions of users, you’ll hopefully also be able to invest the money necessary to handle them :slight_smile:

#11

Sometimes I get ahead of myself and think too far into the future haha.

Thanks I really appreciate the insight!