As we move into part 2 of our Quake Source Code project, we are still pretty deep inside Windows programming. All video games, even games that are 15 years old, have substantial initialization to go through when they start up, and Quake is no exception. You may find this pretty dull material, and to be honest, much of it is done differently in Windows 7, but there are also some memory management data structures pulled up that are useful to look at too.

And you can always just browse through the code and move onto the next lesson of course.

Here are the current files for uploading:

requakeStep2 compile
requakeStep2 CLOC

This file has a total of 4881 lines of code in it, up from 4542 last time, making our source code 11.4% complete. This 339 line increase will add 15 functions to the overall project, re-include some of the WinMain() code, and add numerous header declarations and data structures. Some of the functions are important initializers, and some are simple helper functions. Open the solution and the 7 files added to it, and let’s look through what we’ve added.

WinMain() revisited

The last project’s version of WinMain() ended with a call to COM_InitArgv(), which turned the command line arguments into global variables that any piece of code could read. We have extended it a little further now.

The first new thing WinMain() does is to use the new function COM_CheckParm() to see if the -dedicated argument was called. This means that the player wants to use this instance of Quake as a dedicated server for a multiplayer game. After this the line ‘#if 1′ appears. In Fitzquake, a small box that says ‘Starting Quake’ will pop up while the game is loading. By changing this line to ‘#if 0′ you will comment out the following lines of code to prevent that box from appearing. This is all Windows-heavy code, so we won’t dive too deeply into it.

After this, Quake attempts to use malloc() to reserve a chunk of memory for the game to run in. The original game wanted between 8 and 16 MB worth of memory, though Fitzquake was changed to set those limits to 8.5 MB and 32 MB. In modern computers this really isn’t an issue – it takes a truly archaic machine to not have 8 MB of free memory available, so virtually any modern computer will end up with a 32 MB block reserved. Alternatively, you can use the -heapsize argument to ask it to set a specific size. If it cannot obtain the required amount of memory, it uses Sys_Error() to shut the game down.

It then uses another new function, Sys_PageIn(), to access all the requested memory and allocate it into virtual memory. I’m a little fuzzy here, but I believe this would prevent the lag of having to do this while the game is running.

If this instance of Quake is a dedicated server, we have some other initializing to do. We look for a few other parameters and then call InitConProc().

The final thing our current version of WinMain() does is call Sys_Init(), where lots more initializing gets done.

int COM_CheckParm(char *parm)

This function is called to see if a command line argument was sent to the game. If you were to run the game at the prompt by typing in “Quake -windowed”, then calling COM_CheckParm(“-windowed”) would return 2, the position in the arg line of that parameter. Some arguments expect numbers after the argument, so you may have “Quake -maxPlayers 8″, in which case the game searches for “-maxplayers” and then pulls out the following argument from the arg list.

If the argument is not in the list, then the function returns 0.

void Sys_PageIn(void *ptr, int size)

This function is called by WinMain(), and may look a little confusing at first. This function is actually taking the chunk of memory that was just allocated and reading it all into virtual memory. From the notes that John Carmack left, it appears that this was a way to work around Windows 95′s memory management algorithms and to help prevent slowdown during the game due to memory that needs to be read from the disk instead of memory. Chances are this algorithm does not really offer much benefit in modern computers, but it’s an intriguing memory management hack that was likely effective in 1996.

void Sys_Init(void)

This function begins by calling some Windows functions that will set up the timer for the game. This will allow us to maintain a steady framerate. It then calls Sys_InitFloatTime() which will prepare the timer to actually be used. It then looks for the version of Windows you’re using. Since Fitzquake is based on WinQuake, this game requires Windows95 or higher. If you are running DOS or Win 3.1, it will Sys_Error() out of the game. Once it has determined that Windows is up to date, it returns.

void Sys_InitFloatTime(void)

This function prepares the system to use Sys_FloatTime() to keep track of how much time has passed. This will be the seed for the internal clock of our game, which balances everything from frame speed to input to… well, everything. The only thing it does differently from Sys_FloatTime() is that it checks for the ‘-starttime’ line argument to see if the player manually set the time that the game began playing. To be honest, this was probably for debugging purposes, as there’s no compelling reason for a player to want to set this on their own. After this is called, the game uses Sys_FloatTime() to return the current clock time.

void InitConProc(HANDLE hFile, HANDLE heventParent, HANDLE heventChild)

If you add the ‘-dedicated’ argument when running Quake, the game will call this function. WinMain() will search for a few other args, and you can use these args to get Quake to run a separate thread for QHost, but if you do not add -HFILE, -HPARENT, and -HCHILD arguments to Quake (along with numbers after them) this will not be done. It will simply create a console window and return.

DWORD RequestProc (DWORD dwNichts)

This is a function called by the thread that may be created in InitConProc(). It is Windows-heavy, and something that I’m not really digging into right now, so we’re going to pass on it right now.

void DeinitConProc(void)

This function is called when Sys_Error() is called. All is does is close down the thread that may have been created.

int VID_ForceUnlockedAndReturnState(void)

This function, along with the next few functions, is called in Sys_Error(), but it’s actually empty in the Fitzquake code. Perhaps this was something that was meant to be inserted that never was? Nonetheless, it has been added back into Sys_Error().

void VID_SetDefaultMode(void)

Another Sys_Error() function. The only thing this one does is closes down the mouse I/O. You may note that we currently do not initialize the mouse at all. So it effectively does nothing right now, but now that the code is in place, once we actually start using the mouse, we know it will shut down okay.

void Host_Shutdown(void)

Yet another function called by Sys_Error(). This is the function that shuts down the player’s game – its counterpart is Host_Init(), which we will see in the next lesson. As a result, it calls a number of other functions that shut down modules that we have not defined yet. Again, it’s good to have this in place, and we’ll revisit again in the future.

char *Sys_ConsoleInput(void)

The final Sys_Error() function added back in. This function returns immediately if you are not running a dedicated server. It processes the commands typed into the dedicated server console through some fancy char* work.

int Q_strlen (char *str)
int Q_strcmp (char *s1, char *s2)
int Q_atoi (char *str)
float Q_atof (char *str)

These functions are equivalent to the C standard functions strlen(), strcmp(), atoi() and atof(). I’m not sure why John Carmack decided to write his own versions, but he did, and so we’ll be using them for this program.

End Result

Like the last post, we are still heavy in Windows programming. We now get an initialization box when we begin the game, so there is an actual graphic result to running it now. We are now actually reserving the memory that the game will use to run, which is also great. But we have some shutdown functions defined before the initialization functions, which is odd. In the next post we will get into the Host_Init() function, which is a list of initialization functions for the various modules and subsystems in the game. It may take a few posts before we see any graphics, but as it stands we are slowly working our way through the foundation we need to get there. Video games are complex beasts, as this series is going to show.

This is part 1 of a 100 part series that pulls apart the Quake I engine (as represented by the Fitzquake remake) and examines it as we rebuild it. Here are the 5 functions we are adding to the project today:

1. WinMain() – the main function for any Windows function
2. Sys_Error() – a general way for any Quake function to report an error
3. Sys_FloatTime() – a way to track how much time has passed in the game
4. Q_strlen() – a custom string length function
5. COM_InitArgv() – sets up any arguments passed to Quake to be checked by the system

These functions are not going to work 100% when we’re done, because they call further functions that we don’t have a chance to get to yet. But added up they total roughly 400 lines of code, which is what we need to average to completely rebuild Quake in 100 sessions.

The first thing you may want to do is download today’s material. The following file takes the base code from part 0 and adds the 5 functions listed. There are two versions – a compile-ready version that have all the undefined portions of these functions commented out. The CLOC-ready version leaves these lines in, so that you can run CLOC to see how many lines of code have already been defined.

requakeStep1 compile ready
requakeStep1 CLOC ready

As you can see from my running of CLOC, this adds 380 lines to the project, which is right on track to finish this project in 100 steps. We now have 4,542 of our lines of code typed in, which brings the total up to 10.7%. Sadly, however, the program still doesn’t do anything visually – it just runs and shuts down.

Also, as you may have guessed from the names of the functions, there is much more maintenance code here than there is actual game code. These posts are coming from a game engineering background, not a strictly game programming background, so we’ll get to entity states and graphics later. Each lesson will begin with code that is ready for you to compile and look at, so you can jump ahead to lesson 10 or 20 or wherever the game work eventually begins, and you should be able to pick up fairly easily.

If you’re a beginner programmer however, and you want to see how everything comes together, then this is where to start. You may want to use the compile ready version of the code, as this is the version that has only the code we are currently looking at inserted in it. If you can understand everything that is compiling in this version, then you have understood this code.

int WINAPI WinMain(HINSTANCE, HINSTANCE, LPSTR, int)

This is the function where any Windows application begins. In Quake it is about 200 lines long, and goes through 3 general steps.

1. 1. Initialize all the systems of the program.
2. 2. Run the game loop, and continue running it until the player quits the game (or an error occurs).
3. 3. Shut down the system and close safely.

We are currently looking at the portion of the code from the beginning of the function through the call to COM_InitArgv(). Directly above the function are several global variables that will come into play. They are all fairly self-explanatory, though there are some Windows-specific variables, like an HINSTANCE and HWND that are not all that important to dig into.

Inside the function itself, it calls a ‘quakeparams_t’ variable called parms. This will hold a variety of general parameters, including the command lines arguments passed to Quake itself. It is defined in the quakedef.h file.

It also fills a MEMORYSTATUS object. This object is used to pull in system information, such as the total amount of memory in the computer running it, virtual memory available, and so on. This version has been deprecated in favor of the MEMORYSTATUSEX object, and it may not pull in the right information in a 64-bit version of Windows, but modern computers have so much available memory that this won’t really matter that much.

The while loop goes through the command line arguments, moving them all into the global argv variable, and then copying them into the quakeparms_t variable as well. Note that Quake sets a hard limit of 50 command line variables that can be passed into the function – if you send it more than 50 variables it will simply ignore the extras.

It then calls COM_InitArgv(), and this is where we end our current lesson.

Review Questions
1. Do you understand the difference between WinMain() and main() functions? When do you use each one?
2. Look through the while loop and follow it. Imagine you are sending some command line variables to Quake – follow through the loop until you understand how exactly it is processing these variables.

void Sys_Error(char *error, …)

On line 219, you will see that if the program cannot determine the directory that Quake is located in, it will call a Sys_Error(), with the argument “Couldn’t determine current directory”. Let’s look at what this function does.

Our compile-ready version of Sys_Error has several sections highlighted out, as they include further function calls. Nonetheless, the current version still does much of what it needs to do. It begins by preparing an error message that will be sent to the console if you are running a dedicated server. In this situation, you can also see a while loop that will wait to get some feedback from the user in this case.

If it is not a dedicated server, the game switches to windowed mode using the commented out VID_SetDefaultMode() function, and then brings up an error box showing what happened. If it can’t restore to windowed mode it will send an additional error explaining that. It also brings in some extra shutdown functions that are currently ignored, and then calls the exit(1) function to shut down the game. Note that once Sys_Error() is called, the game will always close – this is an error that cannot be recovered from.

Review Questions.
1. Manually put in a call to Sys_Error() function in WinMain(), so that you can see how it operates. Put in some of your own error functions and look at how the messages appear in the error box.

double Sys_FloatTime(void)

This function does some math to send back a 64-bit integer that determines how long the system has been currently running. The math behind the function is interesting, though there are some bit shifts that would take awhile to explain here.

int Q_strlen(char *str)

This is a simple function that manually counts the number of characters in the *str sent along, and returns the count. It does not count the ‘/0′ at the end of the string. Quite simple and straightforward.

Review Questions.
1. How is Q_strlen used in WinMain()? Why is it used the way it’s used?

void COM_InitArgv(int argc, char **argv)

This function will copy the sent argc and argv arguments into the com_cmdline array, which is a duplicating effect added into Fitzquake. It also looks for a few particular arguments (-safe, -rogue, -hipnotic, -quoth) and sets variables for those. We will see those in the future.

Review Question.
1. This function uses pointers to copy over the argv commands. Make sure you can follow what’s happening – there will be more pointers in the future.

End Result

Much of this programming is Windows-specific and pointer work. It may not seem interesting at first glance, and yes, there is nothing to show for our efforts currently – the game just loads and shuts right back down. But in the end, if you want to be a professional game programmer, you will be well off to understand pointers, and at least be familiar with the Windows objects that are used here. If you program Windows desktop programs, you will see them again.

The source code to a video game is one of the smaller elements of the game, on a byte-by-byte comparison. The bulk of a game is its assets – the graphics, models, and sound effects that define a world. I’m not particularly interested in digging into Quake’s wad files. I may look at those in the future, but for now, I’m much more interested in rebuilding the project itself.

To that end, what we’re going to do here is set up a new Visual Studio 2008 project, set up the same options as the Fitzquake project, and then bring in some of the files that were included in the original Fitzquake project.

Download and Run Fitzquake

The first thing you may want to do is make sure you can run Fitzquake itself. The program itself is almost wholly self-contained, you only need to find a copy of a wad file to supply the level, graphics, and sound. An easy, free way to do this is to download the Quake demo from id’s website.

We’ll get there though. Let’s get Fitzquake first. Go to the official Fitzquake site and download both the executable and the source. The executable file is a zip file that contains just two items, the executable itself and a README document. If you try to run it as is, though, you get an error.

This is because it can’t find a wad file. These are the files that define the game you are playing. It’s the game versus the engine that runs it. We’re looking at the engine here, so let’s find a wad to get the engine up and running.

Go to id Software’s website to download the demo of the original Quake. It gives you the old DOS installer, which runs with no problem in Windows XP, hopefully it runs in Windows 7 too. Tell it which directory to install it into, and then go to that directory when it’s done. You will see a few different files, but you’re interested in the folder called ID1.

Copy this entire folder into your Fitzquake folder, so you now have three items together – the Fitzquake executable, the readme, and the ID1 folder.

Now try running Fitzquake, and you should have Quake up and running. The reason I suggest doing this is because the Visual Studio project is going to create this Fitzquake executable, so you can just drop the program you create into this folder and compare it to how it runs with the way Fitzquake runs. It’s not 100% necessary to do this, but it helps later when you’re comparing work.

Fitzquake Source Code

The source code for Fitzquake is quite easy to work with, at least in my experience. Download it from the official website, and you’ll see it’s all kept in a single folder. A quick run of CLOC shows that there are 125 files total, but we don’t need to recreate all of them. In fact, you’ll see shortly that we’re going to be able to save ourselves several thousands lines of code right away.

I have copied over the resource files in this folder. They consist of the following:

1. The dxsdk folder. This folder is full of DirectX headers used when building in Windows. There’s no reason for us to copy over Microsoft’s header files, and if you run CLOC on this folder, you’ll see that it’s over 4000 lines of code long. That’s roughly 10% of the original code that we do not need to worry about!
2. The Visual Studio project files. These are Visual Studio 6 files, so Visual Studio 2008 needs to update them, but no big deal – it should do so without any problems. There are four files (and one invisible one that it creates automatically but don’t worry about that one) that we will copy over.
3. The Fitzquake.ico, progdefs.q1, and winquake.rc files. These we will add to the resources folder in Visual Studio.
4. The winquake.h and resource.h files. These are created by Visual Studio and are easier to just move over than worry about remaking.

That should be it! Now open the project in Visual Studio, it will update it, and then you can recreate the folder structure in your new project (or just download the file I have below).

There are a few other things I’m doing here. I’m going through the Project Properties and matching up the new project with the Fitzquake project, to make sure it compiles using the same rule set. I’m also creating the first of the files we need, sys_win.c, in order to put the WinMain function in. This is so we can get SOME C code into the project, which allows us to set the C/C++ project property settings.

The end result we have is in the following .rar file:

requake step 0

If you run CLOC on this folder, you will see that we already have 4,186 lines of code, which shrinks our 42,659 total down 10%:

Great start! Now we can actually focus on the code in this monster.

In the 16 years since the release of Quake, it has become the most influential 3D game engine in the history of video games.

This is hardly an overstatement. Quake may not be the literal grandfather of all modern 3D engines, but the lessons that John Carmack discovered while created Quake have been one of the single biggest influences in game history. Take a look at this chart (pulled from the Wikipedia page on Quake)…

Family tree of Quake Engines

There are two main points to take from this graph. The first is that there are some massive commercial successes in that tree. Quake of course, which was preceded by Doom, but you also have all of Valve’s games (Half-Life, Counter-strike, Portal) as well as the Call of Duty series, and some older series like Hexen and Heretic. These are some of the most popular first person games of their days, and any engine that spawned them all should not be ignored.

The second thing to note (perhaps actually the first thing you noticed) was that the majority of the titles listed here are mods and ports of the existing Quake games. The mod community around id Software’s games is extremely healthy, and you have here almost 20 years of modding experience and history to draw from. What better way to learn about the programming and creation of 3D games than by looking at how this line of code has evolved over the years?

To that end, I have decided to work my way through the original Quake engine, take it apart and slowly rebuild it, examining each piece of code as I do so. What I hope to gain from this experience is a deeper understanding of how the engine itself operates, and how some of the biggest games in history solved the technical problems of creating enthralling three dimensional spaces.

Rebuilding a Quake Rebuild

The biggest issue that I faced initially was deciding on which version of Quake to rebuild. The purist in me wanted to build the original source code as it was released by id. However, the code they released was quite crusty even when it was released, and it took significant work to build. Today, with our 64-bit operating systems and blinding fast computer speeds, this would turn the project into a series of hacks that would much less informative and much more frustrating to deal with, and I quickly discarded it.

Instead, I have decided to rebuild the Fitzquake mod of glquake. Rebuilding this mod provides several advantages for my current interests:

1. The code base runs well in Windows XP, which is the OS I will be using to rebuild it.
2. Because it’s based on glquake, it uses hardware acceleration instead of the software renderer, meaning the code is much shorter and easier to understand than it would be otherwise (I will go into why I’m more interested in hardware acceleration later).
3. The project has not been updated since February 2009, meaning it is stable and likely will not be updated while I’m working on this project.

Sure, there are advantages to using other versions of Quake, but I’m more interested in digging into the code and not getting into a religious war over the ‘right’ way to learn something.

Tools I’m using

Here are the more important tools I’m using for this project.

1. Windows XP / Service Pack 3
2. Visual Studio 2008
3. Fitzquake version 0.85, updated February 12, 2009
4. CLOC, to track how much code I’ve written
5. DOS prompt, to type in command line arguments
6. My brain, the best debugger I have on hand to do this work!

Why not Windows 7 or Visual Studio 2010? Because they aren’t on the computer I have here. Yes, I have Windows 7. Yes, I also have Linux. I’m not a purist for either operating system. I’m a game developer, and 90% of the United States uses Windows machines, so I use Windows. The industry is also in a state where developing in Windows XP makes sense, as we’re split in the number of people who still use it versus the ones that have updated to Windows 7. I haven’t even tried to compile Fitzquake in Windows 7, but maybe I’ll look into it in the future.

The Method

I’m going to start by creating a new project that has identical settings to the original Fitzquake Visual Studio project. I’ll make the directory structure the same, but I’ll strip out all the files. That way we can start fresh. I will likely leave in any Windows resource files, or any other non-source code files in, since we won’t be building those. The focus here is on code, not the environment outside of that.

Using CLOC on a completely shows the following information:

My heretofore unadulterated desktop

125 files consisting of 42,659 lines of code. The original Quake is much larger thanks to the software renderer. But even at this size, we have quite the project in front of us.

I’m planning on breaking this up into 100 posts. Each post therefore will cover about 425 lines of code. Probably what I’ll do is attempt to break up the functions we look at into roughly equal sets.

Interested yet?

If you’re interested in what I’m working on, feel free to follow me on Twitter, where I’ll try to keep in touch with people. I’m excited about this, and hope it stands as a great learning experience for myself and for other people too. The end.