win64-port-playbook

Use when porting or debugging a 32-bit-era C/C++ game engine (DOOM 3 / idTech 4 here) on Windows x64 with a modern MSVC and no original SDKs - build-system bring-up, 64-bit pointer truncation hunts, r

SKILL.md

GitHub

Win64 port playbook (DOOM 3, VS2022, Windows 11)

Distilled from specs/001-win11-amd64-native-build (PR #4). Every claim below has a repo path or a command. Companion detail: doom3_orig/doc/06_win11_amd64_port_findings.md. Boundaries are in the last section.

1. Do it in this order (each step has a gate)

Step Gate (a command, not a feeling)
1 Probe the toolchain before planning cl.exe x64 compiles a sample of real TUs; count __asm, (int)ptr, sizeof(void*) hits with a script
2 Generate the build from the original project files scripts/gen-sources-win11-amd64.ps1 reads *.vcxproj; CMake configure exits 0
3 Get every TU to compile keep-going build (-- -m, never -- /m in Git Bash; Ninja -k 0 to list all errors at once)
4 Link exit 0, dumpbin /headers machine = 8664, dumpbin /dependents has no MFC/ATL DLL
5 Boot with NO data engine stops with a clean "missing data" error, not a crash
6 Boot with REAL data under a debugger cdb stack at first access violation (section 3)
7 Prove with evidence scripts/verify-win11-amd64.ps1 (PE, size, patch reproducibility, menu + map smoke, audio peak)

Step 6 is where most of the real work was: 14 separate x64 bugs were invisible until real data loaded. A green link and a menu screenshot prove almost nothing.

2. Build-system decisions that held up

  • CMake generated from vcxproj lists, not hand-written: targets/win11-amd64/sources.cmake is regenerated by scripts/gen-sources-win11-amd64.ps1. Hand lists drift.
  • Link the game statically (no __DOOM_DLL__, no GAME_DLL): the DLL name is hard-coded gamex86.dll, and Sys_DLL_* handles were int. Static link removed a whole class of bugs. Evidence: dumpbin /symbols on Common.obj.
  • Simd: generic only on x64 (5 asm TUs stay in the source list but compile empty behind _M_IX86).
  • Drop MFC: idlib/precompiled.h pulled tools/comafx/StdAfx.h into every TU. Keep tools/compilers/** (plain C++; the engine references dmap/aas/renderbump), exclude the MFC editors.
  • Missing 2010 SDK bits: replace DxErr with a local macro; provide src/eaxguid_x64.cpp (INITGUID + eax4.h); OpenAL is runtime-LoadLibrary, curl off (ID_ENABLE_CURL=0).
  • Defines that cost hours to find: ID_ENFORCE_KEY=0 (without a CD key, +devmap/+map on the command line are silently dropped), _ALLOW_KEYWORD_MACROS (TypeInfo.cpp does #define private public).
  • x64 must not use CPU_EASYARGS (sys/sys_public.h): the x86 shortcut treats every event argument as a 32-bit int on the stack; on x64 floats go in XMM registers and pointers are 8 bytes. Use the generated per-format callbacks.

3. The debugging loop that found the bugs

  1. Run the engine under cdb with a command file, never inline -c "..." (PowerShell quoting mangles it):

    .lines -e
    sxe -c ".ecxr;r;kb 30;q" av
    g
    q
    

    Launch with Start-Process cdb -ArgumentList '-logo',<log>,'-cf',<cmds>,<exe>,... and a hard WaitForExit timeout.

  2. Read the first frame that is your code, plus rax/rcx of the faulting instruction. A register such as 0x00000826570252bc or 0x2962e260 is a 64-bit address cut to its low 32 bits - that is the whole diagnosis.

  3. Search for the same pattern everywhere before fixing one site (rg '\(int\)\s*&|\(int\)\s*\w*[Pp]tr|_findfirst|\bint\s+\w*handle').

  4. Add a temporary diagnostic that writes to a fixed file (fopen append) when stdout is not visible. Always keep a .bak and restore it, then grep for the marker (DIAG) to prove none leaked. One restore overwrote a real fix once: re-check earlier fixes after restoring.

  5. Rebuild incrementally and re-run the same scripted scenario (scripts/run-win11-amd64.ps1).

4. Bug catalogue (symptom -> cause -> fix), all found at runtime with real data

Symptom Cause Fix (file)
Engine says "Couldn't load default.cfg" with data present _findfirst handle stored in int intptr_t (sys/win32/win_main.cpp)
Crash building GUI expressions wexpOp_t.a held an idWinVar* in int intptr_t field + EmitOp (ui/Window.h/.cpp)
Crash in sound mix start 16-byte alignment done with (int)ptr uintptr_t (sound/snd_system.cpp, snd_world.cpp)
Menu fonts missing len != sizeof(fontInfo_t); struct holds an 8-byte pointer, the file is 32-bit era constant FONT_FILE_BYTES (renderer/RenderSystem.h, tr_font.cpp)
%s crash inside script system calls event args truncated (CPU_EASYARGS) CPU_EASYARGS 0, intptr_t arg slots (gamesys/Class.*, Event.cpp)
Crash in idInterpreter::Execute on object field store OP_ADDRESS writes an 8-byte pointer into a 4-byte temp allocate pointer-sized temp in the compiler (script/Script_Compiler.cpp), type_pointer size
Cinematic crash motion-comp offsets unsigned int, added to a 64-bit pointer int mcomp[256] (renderer/Cinematic.cpp)
Level load error "Missing WEAPON_NETFIRING" 2004 demo script predates 2011 code local test shim only; never commit demo data (scripts/prepare-demo-data.ps1)
inline __asm C4235 / C4980 MSVC x64 has no inline asm #if defined(_M_IX86) + intrinsics (_mm_cvtss_si32 keeps round-to-nearest)
VRAM reported negative WMI AdapterRAM read as intVal ulVal (sys/win32/win_shared.cpp)

Later round (savegame/demo, net, d3xp): the savegame and demo pointer writes are now fixed ((int)ptr became a 0/1 presence flag, SAVEGAME_VERSION 18); idPlayer::ExitCinematic crashed on map restart when scriptThread was NULL (guard in game/Player.cpp). Still open: see targets/win11-amd64/README.md.

5. Evidence tooling (scripted; no manual eyeballing as the only proof)

  • scripts/capture-window.ps1 - client-area capture (exact 1280x720), takes real foreground focus.
  • scripts/smoke-win11-amd64.ps1 - two phases (main menu, then +devmap) with log assertions (using ARB2 renderSystem, entities spawned, SpawnPlayer, forbidden ERROR:), colour-count floor, audio floor.
  • scripts/audio-peak.ps1 - Windows audio-session peak meter for the process: separates "engine silent" from "window muted".
  • scripts/capture-showcase.ps1 - engine-rendered shots via a generated cfg (devmap, wait, god, notarget, setviewpos x y z yaw, screenshot name.tga, quit) - no synthetic keystrokes.
  • scripts/export-target-patches.ps1 / apply-target-patches.ps1 - the modified files as one patch + SHA-256 manifest. Added/deleted files are manifest rows with no hunks, so apply and verify fail closed on them; deliver those through the engine fork/submodule path. The verifier exports to a temp directory and compares hashes on a fresh copy.

6. Traps (each cost real time)

  • DirectSound mutes an unfocused window. Peak 0.0000 at first looked like a broken audio path (the mix buffer was fine). Always measure with the window focused; SetForegroundWindow needs an Alt keypress + AttachThreadInput.

  • A file name with - in screenshot writes an empty shot00001.tga. The engine tokenizer splits on -. Use _.

  • git apply inside a copy that lives inside another repo skips every hunk silently. Set GIT_CEILING_DIRECTORIES and fail on Skipped patch. Patches against CRLF sources need .gitattributes targets/*/patches/*.patch binary, or the committed blob has no CRs and git apply fails.

  • PowerShell: R is an alias for Invoke-History; Try is reserved; [IO.File]::ReadAllText resolves relative paths against the process directory, not the shell cwd; text pipelines strip CR from native output (capture raw bytes with System.Diagnostics.Process when generating patches); $ErrorActionPreference='Stop' turns a git CRLF warning on stderr into a terminating error.

  • cd .. drift between tool calls: use absolute paths, or Push-Location/Pop-Location in one script.

  • A restored .bak can revert an earlier fix. Verify earlier fixes after any restore.

  • Hard links (mklink /H) make the demo pk4 visible as base/demo00.pk4 without a 480 MB copy.

  • Git Bash mangles MSBuild /m. cmake --build ... -- /m becomes a path; write -- -m (also -v:q).

  • Long +wait command lines crash the engine (0xC0000005). Put the commands in a cfg and use +exec <cfg>.

  • A second instance exits at once unless +set win_allowMultipleInstances 1. Give each instance its own fs_savepath and, for loopback tests, +set si_pure 0. Kill by PID, never by process name, while other runs are active.

  • Restart crash after a cinematic camera: MapClear deletes the camera, its dtor calls SetCamera(NULL) -> ExitCinematic() after ShutdownThreads() NULLed scriptThread (game/Player.cpp).

  • Save/demo pointer bug pattern: WriteInt((int)ptr) / ReadInt((int&)ptr) truncates on write and leaves the upper half of an 8-byte field uninitialised on read. Write a 32-bit presence flag (0/1) and rebuild the pointer from the real data that follows; bump SAVEGAME_VERSION when the meaning of a field changes.

  • Sound channel placeholders: leadinSample / soundShader / decoder are saved as presence flags and are re-created on load; testing chan->decoder != NULL on a half-garbage pointer was the original crash source.

  • DOOM3XP.exe is the same build plus the d3xp game code (+set fs_game d3xp, retail data required).

7. When NOT to use this skill

  • Targets other than Windows x64 (use cross-platform-guide, wasm-emscripten).
  • Pure web-game debugging (use game-debug).
  • A bug with no real-data repro: use systematic-debugging first, come back here for 64-bit specifics.
  • Anything that needs retail assets: this repo only has the free demo; do not source game data from anywhere else.