View source for Fixing VPK mounting for older Source SDK Bases

==Background==
When '''[[Valve]]''' rolled out the [[SteamPipe]] update and the existing titles were updated to use the [[VPK]] file format, '''[[Source SDK Base 2006|Source SDK Base]]'''{{src06}} and '''[[Source SDK Base 2007|Source SDK Base]]'''{{src07}} were supplied with "depot" VPKs, but were left unable to properly mount them. Old mods using the Bases were often broken, and 2007 Base mods were unable to reference '''[[Half-Life 2: Episode Two]]''' content, unless it was directly copied into the mod's folder; this is despite the fact that the depot VPKs included those files (but were switched around; see [[Fixing_VPK_mounting_for_older_Source_SDK_Bases#Misplaced_VPKs|below]]).

Apparently, this wasn't meant to happen, and both SDK Base 2006 and SDK Base 2007 ''can'' mount their VPKs with a specific fix.

==Overview==
The [https://gmod9.com/~bt/stuff/vpk.html gmod9.com] blog details an extensive research into this issue and provides a working solution.

It explains that the '''Steam2''' wrapper, meant to handle VPK mounting for the old games, requires a missing file, {{code|steam_vpks.vdf}}, which is not shipped in any known product and whose structure needed to be reverse engineered.
{{expandBox|title=Elaboration on the Steam2 wrapper
|''The following text is copied from the original blog.''<br>
{{quote|The Steam2 wrapper is a steam.dll replacement that acts kind-of as a proxy between the new post-Steampipe Steam and the game. It's the thing that handles the VPK mounting and makes the VGUI friends dialog still work. Note that the version of Steam2 wrapper inside the main Steam directory isn't the one that the old Source games use, instead they come with an old build of it dated Wed Feb 26 2014. The Steam2 wrapper uses an internal Valve library called vpklib to open VPK files and provide it to the game's virtual file system, the library is also used in newer Source Engine builds for the filesystem module and vpk.exe.}}
}}

There are three known methods to fix it, starting from the easiest one:

==Creating custom steam_vpks.vdf== 
The file in question needs to list relative paths to the depot VPKs. 

Create it in the Base's root (e. g. {{path|<nowiki><steam_dir>/steamapps/common/Source SDK Base/</nowiki>}} for ''''06''', {{path|<nowiki><steam_dir>/steamapps/common/Source SDK Base 2007/</nowiki>}} for ''''07'''). You can use [[WP:Notepad++|Notepad++]] (or basic Windows' Notepad) to edit it.

The following example lists depot VPKs for Base 2006:
<syntaxhighlight>
"steam_vpks"
{
	"spewfileio" "true"
	"mount"
	{
		"vpk" "vpks/depot_206" // Base Source Shared Materials
		"vpk" "vpks/depot_207" // Base Source Shared Models
		"vpk" "vpks/depot_208" // Base Source Shared Sounds
		"vpk" "vpks/depot_212" // Base Source Engine 2 (Source 2006 assets)
		"vpk" "vpks/depot_213" // HL2: Episode One shared
		"vpk" "vpks/depot_215" // Source Engine Test shared
		"vpk" "vpks/depot_381" // Episode One content + HL2 VCDs (mislabeled as Episode 1 French)
		// other vpks can be mounted here as well
	}
}
</syntaxhighlight> 

The following example lists depot VPKs for Base 2007:
<syntaxhighlight>
"steam_vpks"
{
	"spewfileio" "true"
	"mount"
	{
		"vpk" "vpks/depot_206" // Base Source Shared Materials
		"vpk" "vpks/depot_207" // Base Source Shared Models
		"vpk" "vpks/depot_208" // Base Source Shared Sounds
		"vpk" "vpks/depot_212" // Base Source Engine 2 (Source 2006 assets)
		"vpk" "vpks/depot_213" // HL2: Episode One shared
		"vpk" "vpks/depot_215" // Source Engine Test shared
		"vpk" "vpks/depot_381" // Episode One content + HL2 VCDs (mislabeled as Episode 1 French)
		"vpk" "vpks/depot_216" // Source 2007 Binaries
		"vpk" "vpks/depot_305" // Source 2007 Shared Materials
		"vpk" "vpks/depot_306" // Source 2007 Shared Models
		"vpk" "vpks/depot_307" // Source 2007 Shared Sound
		"vpk" "vpks/depot_308" // Episode One - Source 2007 Shared
		"vpk" "vpks/depot_309" // Episode One - Source 2007 Content
		"vpk" "vpks/depot_310" // Source 2007 Dedicated Server Windows
		"vpk" "vpks/depot_311" // Source 2007 Dedicated Server Linux
		"vpk" "vpks/depot_421" // Episode Two Content (mislabeled as Episode 2 French)
		"vpk" "vpks/depot_422" // Episode Two Materials (mislabeled as Episode 2 German)
		"vpk" "vpks/depot_423" // Episode Two Maps (mislabeled as Episode 2 Russian)
		// other vpks can be mounted here as well
	}
}
</syntaxhighlight> 
*The "vpk" keys point at files relative to the Base's root folder (as you can see the {{path|<nowiki>/vpks/</nowiki>}} folder right inside the root).
*The <code>spewfileio</code> parameter defines if additional debug info is being spewed; usually you can leave it at {{code|false}}.
*{{Note|Depot 421, 422, 423 will not mount without copying it from SDK Base 2006 ({{code|Source SDK Base\vpks}}), and paste it on {{code|Source SDK Base 2007\vpks}}, due to mistake from Valve. See [[#Misplaced VPKs]].}}

This file allows the Base to load its depot VPKs and, in turn, allows a mod for that Base to properly initiate and reference content. <u>This works without needing to change the mod's {{code|gameinfo.txt}} or moving it to another Base.</u>

{{note|Keep in mind that {{code|steam_vpks.vdf}} disables the automatic dependency enumeration and mounting by taking a different code path, so you have to include the base VPKs in the steam_vpks.vdf file.{{todo|Elaborate. Is just listing all the depots enough?}}
}}
{{note|You can use {{code|steam_vpks.vdf}} to manually specify any number of additional VPKs inside it (within {{code|vpklib}}'s maximum), so long as you use a path respective of the Base's install path. Without this file, '''Steam.dll''' automatically determines what IDs to mount, then finds the respective depot VPK. At this point, it won't load any additional VPKs. With the file, the wrapper instead mounts whatever is listed in the file.}}
===Testing===
A simple way to check it is to try and load an old mod for this Base without the file present (or renamed to anything other than {{code|steam_vpks.vdf}}). The typical behavior is that either the mod would crash and report being ''Unable to load manifest file'' (typically {{code|scripts/surfaceproperties_manifest.txt}} or something similar); another frequent error is ''Can't find background image materials/console/startup_loading.vtf''. Some mods can load, but then will be missing assets; this is frequently the case with Base 2007 mods that need '''Ep2''' content. 

Now place the file you created, {{code|steam_vpks.vdf}}, next to the Base's {{code|hl2.exe}} . The mod should now properly initiate.

==Using mount_vpk function==
A relatively simple alternative in your mod's code. Calling {{code|mount_vpk}} with the path to a depot, ''relative'' to the Base's main directory, should get the target VPK at that path mounted.
{{todo|Elaborate. Is it the same relative paths as in steam_vpks.vdf? And where is the function supposed to be called?}}

==Using SteamMountFilesystem==
According to the blog, this isn't the best way, as it requires using specific paths to the depots -
<source><game_path>/vpks/depot_<depot_id>_dir.vpk</source>

Nevertheless, calling this function in the same manner as '''filesystem_steam.dll''' does it, should work.

==Additional caveats==
===Difference in paths===
The Bases' depot VPKs follow a slightly different filing structure compared to all others. Their tree looks like {{path|<nowiki><root>/<game name>/<content folders></nowiki>}} (materials, models, sound, etc), whereas normal VPKs look like {{path|<nowiki><root>/<content folders></nowiki>}}. 

Thus, even with the fix, they won't properly mount "traditional" VPKs shipped with the games - the resulting path to files would be treated as incorrect due to a 'missing' folder in it. 

It also means that while you can pack the mod's content into a VPK and reference that VPK in {{code|steam_vpks.vdf}}, you have to add a 'fake' game directory into the path:

<source>
root\fake_game_dir\maps
root\fake_game_dir\models
root_fake_game_dir\materials
etc...
</source>

and reference that folder in the mod's {{code|gameinfo.txt}}'s '''SearchPaths''':

<source>Game  fake_game_dir</source>

===Misplaced VPKs===
To make matters worse, SDK Base 2007 does <u>not</u> include files from '''[[Episode Two]]'''. Those VPKs - {{code|depot_421, depot_422, depot 423}} - are all shipped with SDK Base </u>2006</u> by mistake, even though 2006 is supposed to "reflect" '''[[Episode One]]'''.

A crude, but working solution requires having both SDK Base 2006 and 2007 installed at the same directory (same {{path|<nowiki>/<steam_dir>/steamapps/common/</nowiki>}}) and referencing 2006's depots by their relative paths, i. e.:
<syntaxhighlight>
"vpk" "./../Source SDK Base/vpks/depot_421"
"vpk" "./../Source SDK Base/vpks/depot_422"
"vpk" "./../Source SDK Base/vpks/depot_423"
</syntaxhighlight>

Alternatively, if you don't want to have SDK Base 2006 installed, but still want Episode Two content on SDK Base 2007, first install SDK Base 2006 and 2007, then copy (or cut) the following files from {{path|Source SDK Base\vpks}} folder:
<syntaxhighlight>
depot_421_000.vpk to depot_421_005.vpk
depot_421_dir.vpk
depot_422_000.vpk to depot_422_003.vpk
depot_422_dir.vpk
depot_423_000.vpk to depot_423_003.vpk
depot_423_dir.vpk
</syntaxhighlight>

Next, paste these files on {{path|Source SDK Base 2007\vpks}} folder and then uninstall SDK Base 2006. And finally, edit {{code|steam_vpks.vdf}} and add:
<syntaxhighlight>
"vpk" "vpks/depot_421"
"vpk" "vpks/depot_422"
"vpk" "vpks/depot_423"
</syntaxhighlight>

{{note|The alternative method does also works even if you chose to keep Source SDK Base 2006. Just don't verify SDK Base on Steam, as this will redownload depot 421 - 423 to 2006 Base, wasting storage space.}}

===Suspected corruption of the VPKs===
{{expandBox|title=A sidenote about the VPK files coming with the game and the vpk(lib) version used
|''The following text is copied from the original blog.''<br>
{{quote|I deem the VPK files and the vpklib version built into the steam.dll a very specific one. The VPK files coming with all Valve games and licensed games that utilize this mounting system are special, specifically it's pretty much a valid VPK2 file except for the checksums at the end. The tree and chunk checksums have been replaced by something that looks like corruption, which might've been caused by the checksums being uninitialized memory aligned next to each other. Here's how it looks like in the depot_215_dir.vpk file from Source SDK Base 2006. This pattern can also be seen in the depot_307_dir.vpk file from Source SDK Base 2007.<br>
<source>
    {
        "PersonaName"       "martin
</source><br>
Here is another pattern, this one is found in depot_309_dir.vpk and depot_308_dir.vpk from Source SDK Base 2007.<br>
<source>
    {
        "0"     "0100000065a3e27eb
</source><br>
As to what the actual cause of the corruption is, I can only speculate. The VPK files might've been made by some internal vpk.exe build, which licensees would also have, because this can be observed in both Valve and licensed games as I have stated before.}}
}}

=== Base 2007 doesn't draw menus ===
There is a bug that causes Base 2007 and its mods to not display the main menu, [[console]] panel, [[HUD]] or other interfaces. While they are functional (main menu buttons work and produce button sounds, the console can accept input), they are invisible and have to be navigated 'blindly'.

It appears to be caused by mounting {{path|depot_'''317'''|vpk}}. This was later revealed to be files intended to be used for {{src09|1}} games (Valve labeled this as Source 2007 Binaries 2 instead of Source 2009 Binaries). However when unpacked, its contents don't appear to be causing the same issues (if any).
{{fix|Excluding {{path|depot_'''317'''|vpk}} from the listing should be enough.}}

==Credits==
All info on researching and documenting this is taken from the [https://gmod9.com/~bt/stuff/vpk.html gmod9.com] blog.

The original credits include:
{{quote|
[https://steamcommunity.com/id/emv33/ mv] - for providing BGT and The Ship VPK files.<br>
[https://steamcommunity.com/id/nafrayu/ Nafrayu] - for trying to figure this out with me long ago<br>
[https://darkok.xyz/ DarkOK] - for reviewing this<br>
… and everyone else that I've forgotten.
}}
Thanks to '''[https://steamcommunity.com/profiles/76561198058929932 Jai "Choccy" Fox]''' for helping me with the article and the subject.

[[Category:Bug fixes]]
[[Category:Source]]
[[Category:Steam]]