This project was to make single player city building game to Playforia. This would be first game I've made that never ends, player can continue building the city as long as he wants. He can leave the game at any time and return at any time to continue.
There was two programmers in this project. I did the server using Java and another person in our company made the client using Flash. We are old friends and made many games together, so communication was easy. I eagerly waited to get started with this because server was interesting project. Like often in server-client games, game is actually running on server side and client is "just" interface for player. It takes commands from the player, send them to server, server handle it, server send reply to client and client displays it to player. However, sometimes to make game smoother and faster, client have to assume that what normally happens, will really happen. Because of the internet or busy server, there randomly might be delays before client get response and using any program where each small action is followed by short delay is just frustrating.
During the development, I needed some test client though. So whenever I implemented some new feature that we had planned, I was able to test it well with my own client. Finding the bugs and define exactly what client should send and what server will reply. After it worked well, I handed technical specs of this particular feature to person who made real client, knowing that feature works and commands both way contains all the data needed. Much less hassle.
For comparison, here are screenshots of real client and my test client. Same city at the same time. You can click the pictures to view them in full size.
There is two things that may cause changes in the city. Time and player actions. Often I'd do it so that game clock ("time") and each player would have their own threads and they call whatever city methods needed at their own time. And then do necessary locks and synchronizing to City class so that things doesn't get messed up. But in this game I decided to go with single thread approach. So every command coming from client was added to command queue of that city where player was in. Including commands like "player x enters to the city". And this queue was processed in City.tick() method that was called every time when game clock moved forward one second. This gave full control on what time to execute commands coming from clients. Due the type of this game, players won't notice that up to one second delay in their actions.
That was working nicely in this game. And it works nice even if there are multiple players in the same city. Server just send all changes caused by any actions to all clients connected to particular city. This approach would even allow multiple players to edit and build same city at the same time. Server basically woudn't even notice the difference whatever city editing commands are coming from one or multiple clients. This possibility isn't currently used in this game though.
Most straight forward and naive way to run all the cities on server side is that World.tick() loops all cities and on each city City.tick() loops all the blocks in the city (buildings, fields, trees, roads etc). And do this once in a second, every second. Second being the smallest time unit when something may happen. To get quickly started, first version of the server was like this. But early tests showed that this would limit amount of cities just to thousands, which is of course unacceptable.
Very first change was that not all the blocks will run their own Block.tick() method. Blocks were inactive or turned inactive when for example building on that block was finished. After that block was skipped from any updates unless player changed it. Also cities turned inactive if all the blocks in it were inactive. These cities were skipped also unless player did some actions there.
First beta version for small group was actually published like this. Quickly it came clear that much more is needed, cities started falling behind in time when one World.tick() started to take more than one second. Next thing to do was something so simple that I should have thought it in the beginning. All the tick() methods have parameter telling how many seconds they move forward. This was relatively quick change since each block works as own unit. So now World class was easily able to catch up. If previous World.tick() takes over second, skip next tick and run one after that with parameter 2 to catch up previously skipped one. Falling behind problem was solved and no more 100% CPU usage.
But that wasn't enough. When more players came and more cities were created, some World.tick() runs started to take up to 5 seconds and random peaks nearly 10 seconds. Game was started to get sluggish for the players because in worst case client had to wait up to 10 seconds to get reply to some player command. But thanks to previous change, there was now relatively simple solution to this one too. So far all the cities were kept in memory and running forward all the time since that was the idea of the game that cities live even when player is away. Of course cities were saved to hard storage time by time or immediately if player did some important actions that can't get lost. Memory-wise keeping all the cities active wasn't problem, each city used just few kilos of memory. But also cities where player wasn't online, were taking cpu cycles. This was now changed so that when player quits, city is saved to hard storage, then removed from memory. When city is needed again, it is loaded back to memory and single City.tick() will be run with parameter "current time - city save time". So if player was away 15120 seconds, City.tick(15120) immediately catched up. Player won't even notice difference between this and case when city is running all the time while player is away. I'm ruining the magic for all TinyTown players here :-]
But this solved any CPU or possible upcoming memory problems once and for all. Unless we get 100k players online at the same time. Afterwards this seems so simple and obvious solution that I wonder I didn't do it that way immediately. But this was a bit learning process for me as I've never done any game of this scale before. Best way to learn is to do things wrong first ;-) I'm very happy with the server now.
Whole server isn't actually running in one single thread as mentioned in "Running command". Since each city works as own unit, World.tick() uses thread pool to run multiple cities concurrent. Also for example city loading, parsing and catch-up is done in own thread before ready-to-play city is added to list of currently active cities runned by World.tick()
Back to "Work"-page| www.tiikoni.net/work/tinytown/ | Full page map | Copyright © Pasi Laaksonen |