Introduction

E.V.E. Paradox is a cross-platform game suite that I developed and maintained over the course of several years. It runs on Windows and Linux systems. It was originally released with three games, and grew to a total of five games with one hundred and seventy levels in its final release. Whilst E.V.E. Paradox was not a commercial success, and did not allow me to launch a self-supporting game development business as I had hoped, it was still a significant and successful technical undertaking, and a solid learning experience.

This page discusses some of the technology behind E.V.E. Paradox. If you are interested in seeing E.V.E. Paradox in action, please visit this page to download the final released version of E.V.E. Paradox. Please feel free to contact me with any questions that you may have.

Overview

E.V.E. Paradox was a reasonably large software development project. The project was written principally in C++, although Ruby scripting featured prominently in the build process. As part of the build process, C++ code was automatically generated from custom definition files, generally through Ruby scripting. Other languages were also used (eg. PHP, LUA), in smaller support roles. The primary development environment was Linux, and the client code was periodically moved to a Windows machine, adapted, tested, and changes returned to the development machine. Server-specific code remained Linux-only, although a port to another Unix-like system (eg. one of the BSDs) would have been relatively easy.

The project was roughly structured into the following components:

In addition, several additional utilities and support programs were developed, some with more general application, and some quite specific. These included:

This particular arrangement has ultimately been beneficial for my follow-on project, which makes use of the common code (such as the 3D engine, utility library, and GUI toolkit), but not the EVE game engine (which is more specific to E.V.E. Paradox).

The Central Server

The E.V.E. Paradox central server ("Central") acted as the main connection point for game clients through the life of E.V.E. Paradox. Use of the Central server the game client was entirely optional- the E.V.E. Paradox game suite could be used offline without difficulty. Online use, through online account creation, allowed highscore submission, awards, and a chat facility. The Central sever managed such use and provided these features. A account-specific profile page was generated for each account holder, showing their accomplishments. It also provided the means to automatically update to the latest version of the software. Whilst many of these things are very common nowadays, they were less so during that time- especially with releases from single independent game developers.

Communication with the server was designed to be low-bandwidth, partly to keep hosting costs down. The games themselves were played on the client machines themselves, with game results and feedback sent to the server.

The Central server was run on a reasonably stable but basic PC, originally in my home via a broadband service. For a brief period of a few months it was run remotely offsite on a far superior connection. It was hoped that in time, as the project grew, it would move to a dedicated host online.

Periodically, a cron job would trigger on the server host would send a signal to the Central server that would cause it to dump a simple text-based (but mostly machine-readable) summary of the current state of the server. This information included the latest on accounts, game highscores, and other relevant information. This information was then processed via scripts into a set of web pages, which were then sent to the web host, and synchronised with the data there. This last step was as simple as setting it all up in a new web directory, along with a skeleton site that was copied in, and then renaming the old and new directories (generally resulting in the site being offline for a tiny fraction of a second).

As the project grew, it was planned to make better use of dynamic-generated content on the webserver itself (as opposed to the Central server), and caching of this dynamic content to reduce the server load.

Automated Testing

When you are developing a game suite with five games, one hundred and seventy levels, seven independent editions, that runs on two major platforms (and with multiple variations on each), you run into a significant problem when it comes to testing releases. Consider, for example, the amount of retesting required after making "just a small change". A significant bulk of the problem was solved through the use of virtual machines and automated testing.

Prior to a release, a number of separate environments were set up in virtual machines, and snapshotted so that the machine state is consistent each time. All of these virtual machines were switched off. A test script would then iterate through each of the major variants of the game release and test environments (a strict intersection of the two was not performed- some environments were not tested as heavily). For each tested combination, the relevant virtual machine would be brought up, the release tested, and the virtual machine shut down. In each case, the relevant test machine was restored to the initial state before use to ensure a consistent test environment.

The release testing stage had a number of steps. Firstly, on Windows systems, an AutoIT script was generated that would interact with the installer. The installer itself would be launched on the target virtual machine, and using AutoIT, the latest release would be installed, from scratch, using simulated mouse movements and clicks. On Linux systems, a simple script accomplished the same outcome. Then, assuming this process worked, the installed executable would be launched.

Testing the executable itself was enabled by special options that would cause the client to open a port for incoming connections, allowing commands to be issued to the client itself. The commands covered a range of areas, including directly simulating mouse, keyboard, and joystick events (movements and button presses), so that the operating system need not be involved at all; as well as a series of commands to query game and GUI state. Using these commands, an external script could query the game state, as well as explore the layout of the GUI, determine the location of certain elements on the screen, read the contents of text boxes or input boxes, and test and interact with them. The test script explored various GUI elements, clicked them each, ensured that the levels loaded properly, warnings appeared with the right actions, the right elements were displayed, license codes were interpreted correctly, and so forth. The test script was written to the level of detail that even included simulated moving the mouse across the screen in a human-like fashion, rather than immediately jumping to the precise location required by the script (as would be entirely possible to do).

If problems were observed, they were logged, and the script would continue as best it could (not always possible, of course). The runtime of the script was generally in the order of 48 hours. If the same level of testing were performed by a human, it would take in the order of a week or more- assuming that a human tester could consistently perform the same tests repeatedly without error. As events were simulated just above the input device level, and data read just before the display level, it meant that the automated testing environment could handle the vast majority of the required testing, excluding testing of real input devices and diagnosing graphical quality issues. These last two areas were simply tested by hand, and the workload was relatively small in comparison to that handled by the scripts.

Additionally, patches for the various editions and releases were tested automatically. The entire set of patches for a new release could generally be generated in a few hours, and tested in under one. Relative to the automated testing performed on the client, this testing was somewhat basic and uninteresting, but it removed the need for a human to manually review the end product.

Cross-Platform Code

I was relatively fortunate to have had significant exposure to cross-platform development in the previously, so I had a reasonable idea what to expect when developing a cross-platform Linux/Windows product. Things such as timers, filesystem access, process launching, and networking were obvious candidates for adaption. The vast majority of cross-platform issues were handled by creating wrapper classes that provided similar or identical external interfaces, but consisted of different implementations, depending on the operating system. Further libraries and utilities simply built upon these abstract entities. Whilst higher-level code sometimes does contain special cases for Linux and Windows (mostly to handle behaviours that were different depending on the host operating system), it was generally uncommon to see specific code switches in such code. This also made it considerably easier to develop and test on a single Linux system and then port code across to the Windows environment. Provided that the code did not contain any significant changes to the abstracted low-level cross-platform entities (or exercise them in unusual ways), a synchronisation port to Windows every month or so could be done in a few hours.

3D Engine

As part of the development of E.V.E. Paradox, I developed my own 3D engine. This significant project was undertaken due to my being unable to find a suitable 3D engine at the time- having investigated several and developed code using two such engines, no suitable one was found. As I had developed a simple 3D engine in the past (Glide-based on early 3dfx cards), I decided it would be worthwhile to develop a more modern one. In hindsight, this was perhaps not the best idea- development of a 3D engine tends to be a considerable undertaking, sometimes for reasons that you would not entirely expect. Again, in hindsight, it might have been more productive to compromise on the features that I was after, and use an existing engine. However, the experience of developing a 3D engine was a valuable one, and the end product, whilst not in the realm of those produced for triple-A game development, has proved to be a very useful tool for my purposes.

The engine was developed from the outset to be adaptable to multiple environments. This allowed me the flexibility to experiment with existing game engines at a low level, and the ability to write code that used the engine itself with minimal regard to the specific implementation in use at the time. For example, E.V.E. Paradox has a single line of code that is implementation-specific, and that is the line of code that creates the initial engine. Using the generic engine object returned from that call, the code can generate all of the other 3D engine entities needed. The application can remain completely unaware of the specific implementation in use if desired.

At present, the main implementation of the 3D engine is an OpenGL engine, with SDL used for initialisation and windowing operations. A second implementation uses the wxWidgets OpenGL interface to allow the engine to be used for 3D rendering inside wxWidgets GUI widgets. These two implementations share much common code. A dummy implementation is also provided that renders nothing, but otherwise behaves (from an application point-of-view) as a working 3D engine. The interface is flexible enough to allow the development of further implementations, with a Direct3D-based engine being an obvious candidate for Windows systems.

The engine itself allows the rendering of an arbitrary number of user-specified views (termed "cameras"), each of which can be set to render to arbitrary scalable regions of windows in the underlying environment. For each camera, a root node of a scene tree (scene graph) is specified. The root node for a camera can be independent of the remaining cameras, in which case the cameras render different scenes; or shared, in which case the cameras simply provide a view into the same scene.

The root node is, or typically contains, a group node, which allows multiple child nodes contained within it. When rendering a given camera, the engine traverses the scene tree from the root node of that camera, altering the rendering context (including the position and orientation) as it goes. This behaviour is typical for a scene graph.

A collection of different node types are available. Some of the key node types are:

Of course, there is no need for a specific node type to fit neatly into these categories- a node type could be all or none of these things. Application-specific nodes are also possible.

GUI Toolkit

As part of the the development of E.V.E. Paradox, I also created a simple GUI toolkit. At the time of development, the options for in-game graphical interfaces with arbitrary back-ends were extremely poor, and whilst I would have preferred using an existing toolkit, I could not find a suitable one.

The toolkit itself was designed to be reasonably generic, so as to be adaptable to other backends, although the effort required to make it completely portable in this way could not really be justified at this point. As such, there are some light dependencies between the GUI system and the 3D engine mentioned above. These dependencies could be removed in the future, with a few days work, if desired.

The toolkit provides a collection of layout widgets, buttons (which can contain arbitrary child widgets), text entry and display, images, and other images.

Events (such as keyboard, mouse, and window changes) are passed down the widget hierarchy, whilst child events (such as button presses, etc) travel up the hierarchy. A number of features are provided to route input as desired, and inject additional events, either directly or in response to other events. Input can be fed into a complex input subsystem for processing, which is useful for providing features such as navigation in 3D worlds.

One important widget provided is one that manages a scene from the 3D engine discussed above. Basically, the widget has its own camera and root node, which the application developer can access, attaching their own scene node to the existing one, or manipulating the built-in camera for the node. This means that an interface can be developed such that any designated area is actually a fully-rendered scene. This functionality is used in E.V.E. Paradox to run the games as part of a customisable rendered area (surrounded by various status widgets that are overlaid); as well as in the menu system before the games, to provide 3D elements to the interface (eg. the level selection screens).

The GUI toolkit is of course part of the common code, and is also useful for projects beyond E.V.E. Paradox. Its functionality and scope has been further developed and refined since its use in E.V.E. Paradox.

Automatic Code Generation

One thing that I very quickly discovered when developing E.V.E. Paradox is how often a small change to a key data structure can result in corresponding similar changes in multiple places. For example, if I was to add an attribute to a game object, then I'd need to adjust the structures for the game, the loading code for the game, the editor file format, the load and save routines in the editor, the export routine, and so forth. In each case, similar corresponding changes were needed in similar places, when the datatype itself was similar. Additionally, maintenance of such things was tedious and error-prone.

I began to develop a tool, specific to E.V.E. Paradox, that would read in a specification file (in a custom format), and emit corresponding code in multiple files, based on the specification. These files would then be included in the build via the C preprocessor. For example, with the attribute change mentioned above, a single-line change in the specification file would cause code to be emitted across two to three files different (depending on the attribute), that could then be used to rebuild the project. Over time, the functionality provided by that tool increased, handling multiple data types and sets of structured data. With the latest release of E.V.E. Paradox, the tool was clever enough to emit sufficient code for use that sometimes absolutely no customisation would be needed at all. Using the tool, you would simply add a single line to the specification, rebuild, and the editor could be used with that new attribute, with correct saving and loading of data, along with everything needed except the code in the game engine itself to make the new attribute actually do something.

This tool was, of course, enormously beneficial. Errors due to mistakes made when repetitively duplicating similar functionality across files basically vanished. If I needed a chance to all attributes of a certain type (say, in response to a bug), it was easy to do- the change needed be made in only one place, rather than several.

In the follow-on project to E.V.E. Paradox, I have used a similar technique for code generation, except this time it is provided through a more general-purpose tool (an application-specific tool is useful, of course, but it not applicable to future projects). The design of the tool was based on the many lessons learned from the tool developed for E.V.E. Paradox.

Level Editor

The E.V.E. Paradox level editor was a complex GUI developed for use with E.V.E. Paradox. It allowed creation and editing of new levels in E.V.E. Paradox. The editor provided a 3D view of the level and objects being edited, along with a collection of editing tools and dialogs that allowed customisation of many aspects of the level and the objects within. The editor was developed initially for internal use, but designed so that it could be adapted for general use in the future.

The E.V.E. Paradox level editor made use of wxWidgets (aka wxWindows) to provide cross-platform GUI functionality. 3D scenes were rendered inside certain GUI widgets as well, using the optional OpenGL interface provided by wxWidgets and the wxWidgets implementation of the 3D engine that I had developed. It provided complex customisation of a large number of game and object parameters.

Maintenance of all of the parameters used in E.V.E. Paradox levels would of course been a massive task- certain game objects, for example, had multiple tabs full of parameters that could be adjusted, and many objects were structured in such a way as to require a mechanism for further editing of complex parameters. Maintaining this manually was of course not feasible, except during initial development. As such, much of the GUI code was generated automatically from the tool mentioned previously. The implementation specifics of certain types of parameters could then be manually developed, with the actual parameters themselves making use of these generic constructs. If the generated code did not produce a satisfactory layout for the dialogs, it could be customised further through these input files to produce a better result.

Further Information

If you have any queries or would like to know more about E.V.E. Paradox, please do not hesitate to contact me. I can be reached at this email address:

garthy

I hope to hear from you soon.

Garth is generally available for part-time, casual, and full-time appointments, ad-hoc, short-term or long-term. Click here for further information, or here for Garth's contact details.

Navigation