Since some time has passed since this blog post was written, I think itâs safe to disclose what was really going on with IW4x. Additionally, I completely forgot about one of the two RCEs I mentioned in the previous post. Therefore, Iâd better write about the one I still remember before it slips my mind as well.
In a previous IW4x patch, a jump hook was placed at a function called va.
va is a function used since Quake III to format a string quickly. It avoids initializing a local buffer and using sprintf with its return value. Itâs essentially a utility function similar to the modern std::format
const char* s = va("%s", input);
Internally, va has two thread-local buffers of 1024 characters. It uses the buffers sequentially and overwrites what was previously written to the buffers throughout the life cycle of the calling thread.
enum
{
THREAD_VALUE_PROF_STACK = 0x0,
THREAD_VALUE_VA = 0x1,
THREAD_VALUE_COM_ERROR = 0x2,
THREAD_VALUE_TRACE = 0x3,
THREAD_VALUE_CMD = 0x4,
THREAD_VALUE_COUNT,
};
struct va_info_t
{
char va_string[2][1024];
int index;
};
char* va(const char* format, ...)
{
va_list ap;
va_start(ap, format);
va_info_t* info = (va_info_t*)Sys_GetValue(THREAD_VALUE_VA);
int index = info->index;
info->index = (index + 1) % 2;
char* buf = info->va_string[index];
int len = vsnprintf(buf, 1024, format, ap);
va_end(ap);
buf[1023] = '\0';
if (len < 0 || len >= 1024)
{
buf[1023] = '\0';
Com_Error(ERR_DROP, "\x15Attempted to overrun string in call to va()");
}
return buf;
}
As we can observe, the function will terminate the game if a buffer overflow is detected.
Throughout IW4xâs life cycle, the developers made a series of patches to the game. When they added the IW4x embedded web server to allow for easier mod downloading, there was an issue. As IW4x was building a response to send to the users wanting to download mods, it was making a call to va from a separate thread (the Mongoose thread) which did not initialize the internal va_info_t structure that is returned by Sys_GetValue resulting in a crash.
They fixed the issue by placing a jump hook at the va function, completely bypassing it and replacing it with IW4xâs own implementation, which uses its own set of thread-local buffers that are automatically initialized (fun fact from momo5502: [MS] Implement on-demand TLS initialization for Microsoft CXX ABI ¡ llvm/llvm-project@072e2a7 ¡ GitHub).
IW4xâs own implementation of va allowed for an unlimited buffer size. This seemingly harmless design flaw meant that not only could the client itself use va without restrictions, but because the developers of IW4x completely replaced va with this flawed version, it also meant that the game could use va with unlimited buffer size.
This led to a catastrophic sequence of events:
First of all, the IW developers assumed that va could never exceed 1024 characters and if it did, the game would instantly terminate because of the Com_Error call. By hooking va, this assumption was no longer true.
A malicious attack was detected in September 2022. A user using a simple fuzzer discovered that they could send users connecting to their malicious server an OOB packet:
error\n<error string>
The string exceeded 1024 characters. The game uses localized strings so that some specific errors can be displayed in the userâs native language.
In the function SEH_LocalizeTextMessage, the game will attempt to localize a string from the malicious OOB packet that is larger than 1024 bytes, and a stack buffer overflow will occur, leading to a remote execution vulnerability
The malicious attacker used this to install a tracking program on the victimâs machine that pinged back the attackerâs server to confirm the infection took place and then self-destructed. No users were actually harmed, but this could have turned out much worse.
Various users were saved by Windows Defender or their antivirus software, and the tracking program did not run on their machines.
The malicious attacker later decided to contact the IW4x development team, and the vulnerability was fixed in September 2022.
The vulnerability was fixed by using the gameâs own va implementation in critical functions, and by simply initializing the va_info_t structure from new threads.
#define Com_InitThreadData() \
{ \
static Game::ProfileStack profile_stack{}; \
static Game::va_info_t va_info{}; \
static jmp_buf g_com_error{}; \
static Game::TraceThreadInfo g_trace_thread_info{}; \
static void* g_thread_values[Game::THREAD_VALUE_COUNT]{}; \
*(Game::Sys::GetTls<void*>(Game::Sys::TLS_OFFSET::THREAD_VALUES)) = g_thread_values; \
Game::Sys_SetValue(Game::THREAD_VALUE_PROF_STACK, &profile_stack); \
Game::Sys_SetValue(Game::THREAD_VALUE_VA, &va_info); \
Game::Sys_SetValue(Game::THREAD_VALUE_COM_ERROR, &g_com_error); \
Game::Sys_SetValue(Game::THREAD_VALUE_TRACE, &g_trace_thread_info); \
}
We also enabled DEP by default.
Important notes:
- By default, iw4x.exe does not have DEP enabled.
- Apparently, games released before Black Ops 3 do not have ASLR enabled (the official dedicated server release for BO3 still doesnât have ASLR).
- IW/3arc/Sledgehammer did not enable stack cookies on ship builds of their games until Infinite Warfare/Black Ops 3.
This means that, at the time of the attack, there was nothing to stop a stack-based buffer overflow from occurring.
PC debug builds of CoD games seem to always have stack cookies enabled.
Ship builds of the Mac release of some old CoD games also have stack cookies.
Even if iw4x.dll is a module that uses stack cookies and has ASLR, the base game does not. Therefore, a vulnerability found inside the gameâs own code is much more dangerous.
There is room for speculation as to why, despite IW and 3arc using stack cookies on debug builds of their games, they turned them off on ship builds.
There are anecdotal accounts of people contacting the security team at Activision and reporting various security vulnerabilities (SV_SteamAuthClient RCE) and pleading with them to enable stack cookies. However, they never recompiled old CoD games with them or enabled ASLR.
The theory that I choose to believe is that old versions of Visual Studio 2005, 2008, and 2010, which were used for old CoD games, had issues when compiling with ASLR. I donât think these issues persisted in later versions of Visual Studio; however, I can imagine if IW/3arc had a bad experience with ASLR in any of these old versions of Visual Studio, they were reluctant to enable it even on newer versions, leaving many CoD titles without it.
Additionally, itâs possible they somehow believed that on old hardware, stack cookies would cause performance issues, which is why they disabled it on ship builds.
However, itâs also possible they viewed stack cookies as something that aids with debugging faulty programs, and therefore a feature that only belongs to debug builds and not ship buildsâa very uninformed thought process, but I think this is the most likely scenario.
Here is why I believe that. Letâs look at the T5 Dedicated server debug build:
void SV_SteamAuthClient(netadr_t from, msg_t *msg)
{
unsigned __int64 clientID;
unsigned int authBlobLen;
char authBlob[2048];
clientID = MSG_ReadInt64(msg);
authBlobLen = MSG_ReadShort(msg);
// IW/3arc use a custom assert function
assert(authBlobLen < sizeof( authBlob ));
MSG_ReadData(msg, authBlob, authBlobLen);
LiveSteam_Server_ClientSteamAuthentication(clientID, from, authBlob, authBlobLen);
}
This shows the RCE could have been prevented if the assertion, which is not compiled in ship builds, had an additional in-code check that was later added after the RCE was patched.
assert(authBlobLen < sizeof( authBlob )); // for debugging
if (size >= sizeof( authBlob )) // for ship builds
{
return;
}
This is a recurring theme for many other vulnerabilities found in various CoD titles. If their developers did not rely so much on assertions, they would have prevented many RCEs.
In Black Ops 3, they finally started to do something about this issue, but the solution was still quite undesirable and honestly extremely baffling:
memcpy(dest, src, len);
if (len >= dest_size)
{
_report_rangecheckfailure();
__debugbreak();
}
For some reason, in ship builds of Black Ops 3, they perform a range check after a memcpy is performed and not before. I find that to be unacceptable.
Anyhow, if the length exceeds the size of the destination buffer, the internal MSVC function _report_rangecheckfailure is called. This function does different things depending on whether a debugger is attached or not, and then the intrinsic __debugbreak is called (int 3 in x86 assembly).
Because of Arxanâs DRM, you cannot attach a debugger; therefore, this causes the game to crash. I donât know if 3arc would have been able to debug crashes caused by their own code on ship builds like this, especially with Arxan in the way, but it seems debugging wasnât their strongest suit as their developers added 4 RCEs to the game.