Converting to a database-driven model

I spent most of memorial day weekend pacing back and fourth in my apartment trying to devise the best possible drupal architecture to share and manage game content. 

Currently the games data consists mostly of URIs and object properties relevant to the various different kinds of game objects (trees, units, buildings, etc). The kind of information that would ideally fit in a database. But because the game runs on the client side, it made sense to put everything into a single JSON file. After all, it isn’t that much data and it isn’t going to change all that much, except when new versions of the game are released.  

Now that I want people to be able to add their own content to the game and share it publicly, I have to rethink that approach. The data isn’t static - it changes, and frequently, sometimes in the same user session.  And it isn’t a small amount of data we’re talking about anymore. The database size has the potential to become extremely large if enough people contribute to the public repo. Game content needs to load dynamically from the server instead of from a single static JSON file. In addition, data is retrieved/output based on parameters specific to users. 

Say Armando uploads a texture and puts it in the public repository. Emilio, browsing the public repository, loves Armando’s texture, so he click the “add to account” button. From that point on, Emilio will always have access to Armando’s texture in the map editor. The next time Emilio browses the public repository and sees Armando’s texture, a button, “remove from account” will appear in place of the “add to account” button. In addition, Emilio can remove the texture from his account from within the map editor interface. 

I won’t go into the nitty-gritty of how I rewrote the data model to satisfy both Armando’s and Emilio’s needs, but the end result is that I’m using the same (or variations of the same) server-side code (a la drupal views) to retrieve and output data from the database, wether that data is specific to Emilio’s account i.e.:

  • Stuff Emilio has uploaded and kept private 
  • Public assets Emilio has added to his account from other accounts

Or, if that data is universal, i.e:

  • Public assets 
  • Official game assets 

And, on the client-side, the JS functions I’m writing to load content into the game world will work from the same output structure by the above scenarios, regardless of wether that content is:

  • Added dynamically, from the database, after the game has loaded when a user clicks “add to account” when browsing the public repo
  • Added Emilio-specific content from the database at the beginning of the browser session 

In summary, i’m killing a whole flock of birds with just one stone. Granted it’s not just any old stone - it’s a special stone, a stone I spent all weekend chiseling, smoothing and perfecting. Because I’m thinking long-term here. I’m thinking about how this thing is going to scale and how efficiently the server will operate running 2 or 3 queries to load game content instead of just one. And it’s a heck of a lot easier to make a change in 1 place as opposed to 2 or 3. 

If it’s anything writing code for this game has taught me it’s how to abstract code in such a way that in can be re-usable in different scenarios. Of course, as a web dev, I’ve always been aware of this concept but it takes on a whole new meaning when developing a game.


Map editor launch date

My goal is to have a publicly accessible beta version of the map editor up by June 20th. By that time, I’ll have finished the sharing functionally which (i’m hoping) will greatly expedite things in the art department. By “sharing” I don’t mean sticking a Facebook Like and Twitter button in every nook and cranny of the game (what ‘sharing’, unfortunately, has come to mean these days), but sharing of custom artwork and maps. Here’s how it will work. 

You register an account at and load up the map editor. Every time you upload a custom art asset, you will be presented with an option to make the asset public. This will put your artwork in a public repository which other players have access to. 

When you have finished creating your map, you can choose to export it which is a way of saving it permanently (outside the browser session). In the export window, you’ll be presented with an option to share your map publicly. If you select that option, the map will go to a public repository on export. I’m putting the option in the export feature to save bandwidth. I need to send all map data and process it on the server-side anyway so it only makes sense that data should make the trip to the server and back only once. 

Here’s what I have planned for the public repository:

  • Sort by asset type: artwork or full map. 
    • Artwork can be sorted by various types (e.g. textures, buildings, trees, bushes, etc) and an author-created tag. Each asset will have an auto-generated thumbnail of the asset, an author (linking to the author’s account), and an option to add the asset to your account. Once the asset is added to your account, you can use it inside the map editor. It will show up in the respective category, marked in some way to indicate it’s a custom asset and will likely have a tiny link to meta info. Clicking the meta info link will bring up various meta data: creation date, author, size, etc.  
    • Maps can be sorted by most recent and most popular (and probably various other sort filters, like size). Each map list item will have an auto-generated thumbnail of the map, an author (linking to the creator’s account), meta info, a download link and a link to open up the map in a separate browser tab. 

I realize that all of this sounds like a lot to accomplish in one month but Drupal, my CMS of choice, will do a lot of the SS heavy-lifting, including: session handling, user login + authentication, and, in general, retreiving, sorting and displaying any content stored in the database. 

Forseeable problems

1. Custom game assets won’t work with pathfinding. I’ve thought about that and have been working on developing a system for users to manually mark walkable/unwalkable areas of their custom game assets. It’s really not rocket science. In 9 time out of 10 scenarios, the walkable area of the image is simply the area with transparency. Then you consider proportion of the object’s width/height and its 3D projection on the plane. It’s something that can be done by eye. An interface with a grid overtop the game asset would work. Users could mark which nodes on the graph are walkable/unwalkable. As it is now, when you upload a custom asset, the system “guesses” as to what areas of the image are walkable and non-walkable. This guess will be the default if the user forgets to mark the image.  

2. Bandwidth. I’m currently trying to figure out a way to cut down on map file sizes. Because my editor performs complex and resource-intensive operations on bitmap objects, there really isn’t an easy way to procedurally regenerate the map by recording user inputs and feeding them back into the engine at high speeds, which is the only way I could possibly get small file sizes of such a data-rich environment. After debating this subject on the html5 game dev forum,I’ve decided to go the server-side route. The basic idea is that I’m cutting out areas of the map which are editing with the brush tool, converting them to string format (base64s), compressing them in some way, sending them (along with other map info) to the server for processing and, finally, receiving a zip file from the server as a response… yeah it’s just as complicated and difficult as it sounds. And it means the map editor isn’t 100% client-side i.e. you need to be online to save maps. I suppose I’ll need to get around this by adding the functionality to save your map to local storage. The data limit for LS is 5MB, so you would only be able to store 2 or 3 maps at a time, unless you manually changed your browser’s limit. I’ll need to explore that later. 


Plans for save/load game functionality in Phaser

The next big thing to work on is the save/load game functionality. I want to get this done ASAP because after that I can release a demo of the map editor for people to mess around with. They can contribute to the game by uploading their own art assets and creating ladder maps. I have a tentative plan for achieving the save/load game functionality in-browser and I'll outline it here.

The meta-data for all my game objects (things like unit type, tree type, etc) are neatly organized in a JSON file. The most important data I need to save for game objects (trees, units, buildings, etc) are x/y coordinates and the key for the JSON file which will tell me everything I need to know about the game object. I can recreate game object attributes by looking up their keyed values in the JSON file. There's also player data like which team the unit belongs to and stuff specific to different game objects types, such as the health of units, and their currently facing direction but I won't worry about that right now.

The second big part of this is saving the terrain (grass, snow, dirt, basically anything you draw onto the canvas using the brush tool) which is in an entirely separate layer from game objects. My plan is to utilize the toDataURL method to save the entire canvas (minus game objects i.e. objects placed on top of terrain) in a string format. Then I can recreate the canvas with my base64 string. 

So, in summary, I can recreate the static aspect of the map with the base64 string and the dynamic aspect (units, buildings, trees, etc) with my JSON. Users will be able to download the current game state in a text file (likely, a JSON) and upload it to load the map. 

Possible problems

  1. I'm not sure if the toDataURL method encodes the entire canvas into a base64 or if it encodes only the visible portion of the canvas. Or, it could be that Phaser is incapable of drawing an image to the entire canvas using image data. My experience in trying to draw the entire canvas has led me to believe that at least one of those is true or needs to be worked around. I know that if you try to draw the entire game canvas into a Phaser.BitmapData object, you only get the visible portion if you try adding it to the game as a  Phaser.Image or sprite. I've worked around this in the past by relying on the BitmapData.drawGroup method, which miraculously draws the entire group into a bmd, regardless of wether or not the entire group is visible. I don't have that option now because, when loading maps, I need to draw a base64 instead of an existing Phaser.Group. 
  2. Because my canvas is in webgl mode, I'll need to use preserveDrawingBuffer for toDataURL to even work. This has performance implications. An alternative is to draw my webgl canvas onto a canvas with 2d context, and save the base64 of the 2d context instead of the original webgl context. Something like this: context2d.drawImage( webglCanvas, 0, 0 ).
  3. If that map size is huge (and my game will have huge maps), the base64 string will be huge. Experimenting with a 2000 x 2000 pixel map, my base64 was 3MB ... that's 3MB of pure text. Probably a lot smaller than if I were to save the canvas as a PNG, but it's still a problematic file size. 3MB is big enough to crash notepad and the file size will only increase exponentially as the map size increases. Games like aoe2 didn't have to worry about this problem. Because texture placement was tile-based, they could procedurally recreate the map with a simple 2D matrix. But I need to save an entire image of the static layer of the map.

So, basically all my problems are related to the fact that I just had to have non-tiled based texture placement. I foresee many long and sleepless nights trying to figure out all the workarounds. Lots of sighing and grunting and hair-pulling... I guess that's the price of innovation! 



Video of map editor prototype


Map editor video coming soon

Within the next week I plan to upload a video demonstration of the Feudal Wars map editor. And based on feedback from that video, I will release an interactive prototype. It only makes sense to release the map editor before the game. After all, I need to build the map editor first almost out of necessity, since other aspects of the game rely on it. 

The main goal for the map editor can be expressed with the following analogy:

Photoshop is to images as my map editor will be to 2d games. 

I say 2d games and not 2d RTS games, because, eventually, the map editor is going to be versatile enough to outright build separate games not limited to the RTS genre. Of course that versatility won’t be there at initial release. But much later on, once I’ve finished the custom game creator components, complete with triggering mechanisms, messaging/cinematics, objectives and all the tools people need to create their own custom games. 

It’s a bit audacious, I know, but I think it can be accomplished with all the tools available in Canvas and webGl. 

One of the big features of the map editor is zero grid restrictions on walkable terrain. In typical 2d map editors, the size of ground tiles is restricted to a grid system. The FW editor does away with that since walkable terrain elements are not represented in the grid and can therefore be any shape/dimension. 

This allows for a lot of freedom in how tiles can overlap and blend with each other:


Feudal Wars Map Editor blending


Note the shape of the texture is that of a circle. Neat huh? I can get away with ground textures not being restricted to the grid because I have independent grids for pathfinding, rather than a single central grid for tiles/game objects. 

Unwalkable terrain elements, such as water and cliffs, will be restricted to a grid. i.e when you place a water texture, it has to be proportionate with the node size of the grid and needs to snap to the grid. In essence the water area placements need to be modular, because the pathfinding system is modular. 

Another big feature of the map editor is the ability to upload custom game assets (including ground textures, terrain objects and buildings). This is something I had planned to add from the very beginning. I want to give people the tool to create an implement their own artwork into custom maps/games. This is a feature anyone with basic photoshop skills can take advantage of. What sort of of content are people allowed to upload and with what spec? 

Here’s a rough idea:

  • Buildings and terrain objects (such as trees, cliffs, etc) need to be in the dimetric perspective (the 3D camera is 30/45). This is optional but your assets will look really stupid if they are not in the correct perspective. I plan on writing detailed instructions for creating artwork in the correct perspective. 
  • Ground textures can be no less than 1000 pixels wide and 800 in height and no greater than 2500 wide and 1500 in height.  

Why so big, you ask? Well, when you load a texture into the map it starts out in a compressed format, such as a JPG or PNG. The second you try to edit that texture in the map editor by, for example, blending another texture on top of it, it converts to an uncompressed format: a bitmap.The HTML5 canvas can easily manipulate bitmap data (cropping, blending, changing hue, etc), but it can’t touch compressed formats. This is why I make the tiles so big. Converting to a BMP is a hefty operation, especially for a browser. A few larger images is much better than having oodles of smaller images because fewer images means fewer conversion operations, which ultimately means smoother performance. 

So yeah, that’s a little of what I have planned for the map editor. Stay tuned for the video demonstration! 



Subscribe to Feudal Wars RSS