2/26/2002 -- The Whys and Hows of Porting
Today's Developer Diary installment is
being guest written by Ryan Gordon, the person responsible for porting
Candy Cruncher to Linux and other platforms. I've interspersed my own
comments, mostly as amplifications of his own statements, but occasionally
with clarifications or rebuttals.
Anyone that is interested in having their games ported to Linux or other
platforms should definitely contact Ryan, he works quickly, finds bugs
quick, and does amazingly high quality work. He can be contacted at
So now I hand the mike to Ryan.
To celebrate the completion of the Linux port of Candy Cruncher, I thought
I'd put down some thoughts about porting games and writing portable
software in general. This is a sort of tip-of-the-iceberg, random HOWTO on
Why port software? Especially in the game industry, is there much to gain
by catering to minorities? MacOS is sitting on 5 percent of the world's
desktops, and while that's a significant number of users, the other 95
percent is almost all Windows boxes, which means you can get most of the
users from writing for one target, and think nothing of the Linux and BeOS
users of the world. So what's the point?
Here are my reasons:
Choice: I like my Linux command lines, others like MacOS X's Aqua,
others like WinXP. Making a portable game lets your users work and play in
the environment they like best. The consumer wins.
Code quality: porting to a new platform exposes bugs you never knew
existed in the first place.
It forces you to remove assumptions from the program. It can make it
easier to maintain. It can make it easier to license your code to others,
or contract out work on that code. Having a game that runs cleanly across
several platforms can actually lower the cost of maintenance if you do it
right. It also helps maintain the programmers' sanity when problems like
buffer overruns become obvious through different memory managers.
Consoles: You want a Playstation 2 port of a game? Your first stop
should be Linux. The migration path is easier than going to the console
directly, it's cheaper to develop for, and yes, Linux and the PSX2 use the
I can't agree with this strongly enough. The porting process involves
many different steps, including handling compiler, library and system
dependencies. Trying to handle all of these simultaneously can be
aggravating and difficult, so it's often easier to do the port in steps.
For example, if you know you're going to port from Windows and Microsoft
Visual C++ to the Mac and CodeWarrior, it might make sense to first port
to CodeWarrior on the PC, THEN port to the Mac. Divide and conquer.
Financial reasons: If a program is done right from the start, the
cost to move it to new platforms is small, but the potential for extra
sales increases. Mac and Linux users do tell their friends about software
they like, and they tend to be very vocal in their praise of the companies
that are willing to support their platforms. Games for Linux tend to have
an infinite shelf life, whereas their Win32 counterparts tend to land in
the bargain bin in a month or so.
I'd like to add that the OS X version of Candy Cruncher is outselling
the Windows version by over 2-to-1. If you look at market size alone,
you'd think that the OS X version would get its butt kicked, but in fact
it's the other way around. Quality of consumer can often make up for a
huge difference in quantity of consumer -- this is why niche markets can
However, I do feel obligated to mention a counterargument, and that is
support costs. We're in a bit of a quandary right now with our BeOS
version. It took almost no time to get Candy Cruncher working on BeOS, but
we don't think we'll sell many (if any!) copies. The cost, for us, to
launch and support the BeOS version is significantly higher than any
income we'd derive from it. So we have three choices: release the BeOS
version even though we lose money on it; don't release the BeOS version,
which is unfair to BeOS users and goes against our techie souls; or
release the BeOS version for free, possibly alienating our paying
customers. It's a tough situation and we're not sure what we can do about
Warm fuzzies: I like supporting the underdog operating systems. It
makes me feel good. Pyrogon's stated intention is to make quality games
that don't cost millions of dollars to make. Having players enjoy their
games is priority one, so it makes sense to get them into the hands of
people that generally appreciate the games more than the twitchy
overclockers that are giving Windows gaming a bad name. Note: not that
we don't love you overclocking Windows users too!
I am speaking of games, but these statements are generally true for any
kind of software.
So that we know why, let's explore how.
To start, I have to express the Tao of porting: no code is portable until
it gets ported. Sure, we all write wonderful code, and choirs of angels
sing while we type, but there will always be unexpected problems that
won't be seen until the source is pushed through a strange compiler on a
strange operating system running on a strange processor. The trick is to
minimize the amount of non-portability right from the start. This takes
diligence and a little bit of know-how on the part of the coder. Knowing
what you're doing can literally reduce the porting time by weeks. This
knowledge is best gathered through experience, but these guidelines can be
a push down the road of that first experience. If any of this seems like
common sense, then it just means you've been down that road before.
Rule #1: Think before you code.
Sounds simple, doesn't it? Unfortunately, even in the video game industry
(ESPECIALLY in the video game industry!), it seems that many developers
jump right in and start coding. This is wrong, wrong, wrong. It doesn't
take long for a project to become an unwieldy, hardcoded mass of
spaghetti, which leads to the usual set of problems; however, if even
maintaining (or, heaven forbid, enhancing) the codebase is difficult, it
will be twice as hard to make it portable. What you need is a blueprint.
Sketch it out, write it down, babble incessantly about your plan to
everyone around. Have an attitude that lets people tell you honestly if a
plan is stupid, and prepare to revise details or whole subsystems. Better
to do this now than find yourself reworking the program during crunch
Rule #2: Make abstractions.
If you're writing a Windows game, sooner or later, you are going to have
to call a Windows-specific function. Things that can be done portably
(like using stdio instead of the win32 API) should be done, but other
things, like blitting to the screen, are system-specific. What you should
do is take a few minutes and wrap things like DirectDraw in a simple
class, and expose their functionality in general ways. Do NOT expose
DirectDraw data types, to prevent the urge to bypass the abstraction. If
something can't be done through the abstraction layer, then the
abstraction layer should expand. Candy Cruncher does this very thing for
audio, video, the "registry", input, etc. For example, Candy Cruncher has
a "H2DDisplayDevice" class. This class is subclassed into a DirectDraw
version, a GDI version, an SDL version (for Unix), and a Carbon version
(for MacOS). There are some immediate benefits here, even in
single-platform development. Note that two of those subclasses are for
Windows; this gives Pyrogon the ability to choose the best balance of
performance and stability at runtime for any given run of the game.
Theoretically, they could add an OpenGL-based subclass of DisplayDevice2D
that renders the sprites as textured quads to increase the framerate more,
at their users' discretion. This would be added to the framework, without
changing the actual game's code, and would benefit the next game they do,
too. Good abstraction makes good design, and the benefits are quick ports
to other architectures, more flexibility for the power users, and better
support for various hardware on the primary platform. It's a huge win.
Rule #3: Be data driven.
Spend as little time in your program as possible. Branch gaming logic out
into scripting languages as quickly as possible. This isn't an argument to
write your whole game in Perl (unless you want to, I guess); instead, get
the stuff that must run fast in C/C++ code (which is almost always the
blitters in a 2D game, reference Rule #2) and get the game logic itself
out to something you don't need to compile every time you tweak it. I say
this not just because it's a good idea, but porting a script interpreter
is frequently easier than looking for subtle problems in game logic in C.
But what scripting language is best? It depends on what you need. If you
need something basic, roll it yourself, but it's better to embed an
existing scripting language; there are many that are portable, debugged,
and supported to choose from. Perl, Python, and Scheme are just some
options. The Pyrogon framework has Lua, which seems to be popular for
scripting game logic.
A clarification here -- while we're big fans of Lua, Lua isn't actually
used in Candy Cruncher. Our upcoming 3D game, ColdStar, however will be
leveraging Lua significantly for its scripting and AI components.
Rule #4: Be sensitive to byte ordering and packing.
If I ever see another game that sends "sizeof (myStructure)" bytes over a
network connection, I'm going to scream. I should scream right now,
because I will no doubt see this again. Candy Cruncher is not a networked
game, but it does run on both Intel (Windows and Linux) and PowerPC (MacOS
X) systems, which means that it has to be careful about reading from and
writing to files. Between processor types, operating systems, and even
compilers, sizes of data types change. I'm talking about something more
subtle than the classic C problem of an-int-is-not-always-the-same-size-everywhere,
although that's important, too. Structures get packed differently (and not
every compiler can understand #pragma pack), data has to be aligned
differently, and data gets stored backwards on different processors. If
you have to read or write structures to disk, a network socket, or
anywhere that a different system may see it, you should send it, one
scalar at a time, in an agreed upon format (bigendian or littleendian),
and rebuild the structure on the other side of the connection. Do not send
floating point numbers ever, if you can help it, since different CPUs have
different precisions, and you can only correct for this so far (I can
think of at least four ports I've worked on that got bitten by the
floating point thing. Be wary.) If you do not do this now, it is nearly
impossible to fix it later in a program of any size.
I'd like to add examples of the above that bit me while working on
Candy Cruncher. The first place -- and this is extremely common -- was in
my image file loading code. It's common practice -- and WRONG -- to try to
format your structures to match the memory layout you expect. Then you see
code like this:
pHeader = ( char * ) someBufferLoadedFromDisk;
imageWidth = pHeader->width; //BAD!!!
The above code would work fine on a PC, since the TGA data is stored on
disk in little endian format, but on a Mac it explodes horribly with
wildly incorrect values. The correct way to handle this would be something
pHeader = ConvertLittleEndianToNativeEndian( pHeader->width );
The second bug I encountered was because the sizeof(bool) changed between
CodeWarrior and ProjectBuilder/GCC on OS X. So when storing values to my
preferences file I'd get mismatched sizes, which caused a lot of subtle
(and difficult to find) problems.
Rule #5: Write what you have to, steal the rest.
I've just told you to write a scripting language and be very careful about
how data gets manipulated. Right now you're probably wondering how any of
this is supposed to make your job easier. Hey, I said this takes
diligence! However, the secret is really in the open source community. Why
should you write image decoders, and audio format decoders, and scripting
languages, when they are freely available for the taking? Candy Cruncher
takes advantage of several cross-platform libraries: Lua, zlib, and Ogg
Vorbis, to name a few. The Linux and Mac Classic ports use SDL, SDL_ttf,
and SDL_mixer, not to mention Loki Setup for the installer. This is
literally years of development time that can just be dropped into place,
and more importantly, all of these libraries are cross-platform to start
with, so you don't have to wonder how you'll get that .OGG file to play on
BeOS; it just will.
Rule #6: Don't use assembly language.
Just don't. If you must, you better write a C version and optimize based
from that, so that there is a working fallback. But don't write assembly
in the first place. 99.5% of the time you think you need it, you don't.
Just say no; Candy Cruncher did.
I'd like to add a corollary here -- minimize system specific
dependencies. For example, Candy Cruncher expects each system to implement
audio mixing capabilities. This makes porting more arduous than necessary.
In fact, I intend to write my own audio mixer just so that I can remove a
lot of the gross system specific stuff I have in place right now.
A further corollary to this is to avoid depending on closed source,
non-portable libraries for core aspects of your game. It might be great to
license XYZ from another vendor, but if they have no plans to port to
Linux, then guess what -- neither should you.
Rule #7: Listen to your beta testers.
Sooner or later, your port will be ready for external testing (you are
going to do a beta test, right?) and you will be unleashing your baby into
an unfamiliar world. If this isn't your primary development platform,
chances are it isn't a platform you know all the ins and outs of. Even
Linux users will find that different distributions do things very
differently, and every Linux user has her own routines and traditions.
Listen to what they ask you for, and give them what you can. One of the
beta testers for Candy Cruncher noted that the game's response was a bit
jerky on his box, and wondered if we could make it use the X11 cursor
instead of drawing a sprite. We added that. Another wanted to have his
Unix login name be the default when entering his high score. It was a good
idea that never crossed my mind: added. People with keyboard layouts I've
never heard of showed up: fixed. People with exotic display targets poked
their heads up: tweaked. Odd sound problems on certain distros: debugged.
These requests are to be expected, and are relatively trivial to
implement, but they lead to happy customers and, again, a more flexible
codebase. You do not want thousands of demo downloads from users that
would have bought the game if only the mouse was a little more responsive.
You could not have predicted it until someone came along and ran their
X-server at an odd color depth. Anticipate possible differences in
platforms, but be ready for anything.
Rule #8: Embrace Murphy.
Not every idea is a good one. If something isn't working, chuck it. If a
subsystem isn't portable, make it so. If you are modular, and abstract,
this makes the code easier to drop into a future product. Like I said,
nothing is portable until it's ported, and all the planning in the world
doesn't beat Murphy's Law. In such cases, don't be afraid to throw
something out and replace it with something that works better. Struggling
only makes it worse.
That's your moment of Zen. Now go forth and write portable software! (and
be sure to buy a copy of Candy Cruncher for Linux, to help a starving
hacker like me!)
Thanks to Ryan for a great Developer Diary! Okay, any developer reading
this doesn't have a reason not to port to at least one other platform. And
Win98 to WinXP doesn't count!