Shader Cache and Generation
This section discusses the shader cache and how to generate shader cache .pak
files.
Shader Cache
The shader cache stores a collection of parsed and precompiled shaders. Since the shader code is written with multiple definitions, Lumberyard can generate an enormous number of different shaders. Compiling shaders on demand at run time is only possible on the PC. On-demand shader compiling can cause freezes during the game play and uses extra memory. To reduce this overhead, all required shader combinations for a game are parsed, compiled, and stored in the shader cache.
The shader cache generally refers to the following files:
-
Shaders.pak
– Contains the shader source files, which is everything inside the
directory, excludinglumberyard_version
\dev\Engine\Shaders\EngineAssets
.Note
The actual shader source code (
*.cfi
and*.cfx
) can be removed from this file for the final released version, and is not required anymore when the binary shaders are valid and available. -
ShadersBin.pak
– Contains the binary-parsed shader information of the shader source code. -
ShaderCache.pak
– Contains compiled shaders for all possible combinations that have been submitted to the remote shader compiler. -
ShaderCacheStartup.pak
– Contains a small subset of the shader cache that is used during game start. This file is loaded into memory for quicker start times, but is not required. This cache is often used to hold the minimum required set of shaders to show a loading screen so that the rest of the loading can occur.
ShaderCache.pak File Generation
Creating a ShaderCache.pak
file consists of running the BuildShaderPak_DX11.bat
batch script, which in turn runs
ShaderCacheGen.exe
to ensure the local cache directory contains all the shaders that are listed in the
ShaderList.txt
file. The
BuildShaderPak_DX11.bat
script packs the contents of the cache directory, creates a ShaderCache.zip
file, and then renames the file to
ShaderCache.pak
.
You can find the ShaderList_
file either from the remote shader compiler server or from the Lumberyard Editor
directory. This file
contains the list of all shaders which platform
.txtShaderCacheGen.exe
uses to produce the shader combinations for your game.
When running Lumberyard Editor, individual shaders are created as you view them. As
such,
you do not strictly need a remote shader compiler server to test game release mode
or
test shader pack generation. You just need access to the
ShaderList_
file
that is created in the
platform
.txt
directory when running Lumberyard Editor.
lumberyard_version
\dev\cache\game_name
\platform
\user\cache\shaders
However, only the shaders you have viewed on your local computer while running Lumberyard
Editor will be listed in the ShaderList_
file. For this reason, it is recommended that you use a remote shader compiler server
if possible.
platform
.txt
Note
During development time, when you run the game or Lumberyard Editor, and before the
shaders are packed into shader cache .pak
files, loose shader
files are created in the following directory:
lumberyard_version
\Dev\Cache\your_game
\platform
\user\cache
The following sections detail the steps used to generate ShaderCache.pak
files:
ShaderCacheGen.exe
Lumberyard ships with ShaderCacheGen.exe
, which is located in the
directory. lumberyard_version
\dev\Bin64ShaderCacheGen.exe
is a stripped-down version of the
Lumberyard game launcher without the render viewport, and is used to populate the
local
shader cache directory with all the shaders contained in the
ShaderList.txt
file.
When running ShaderCacheGen.exe
, it first loads the
ShaderCacheGen.cfg
file, which you can customize to suit
your needs.
If you customized Lumberyard in any way, you must build Lumberyard and your game using
the
all
profile. This argument builds both
ShaderCacheGen.exe
(and ensures that it is up-to-date) and
the game .dll
files that are required.
Enter the following command for your version of Visual Studio:
lmbr_waf build_win_x64_vs
2015
_profile -p all --targets=CrySCompileServer
If you don’t want to (or cannot) build using the all
profile, you can
alternatively just build the game_and_engine
spec and the
shadercachegen
spec.
Enter the following command for your version of Visual Studio:
lmbr_waf build_win_x64_vs
2015
_profile -p game_and_engine lmbr_waf build_win_x64_vs2015
_profile -p shadercachegen
Packing the Shader Cache Using a Batch File
The BuildShaderPak_DX11.bat
file generates the ShaderCache.pak
files, which are saved to the
lumberyard_version\dev\build\
directory. The batch file first calls the
platform
\your_game
ShaderCacheGen.exe
and then calls Tools\pakShaders.bat
.
From a command line, navigate to the
directory, and run the lumberyard_version
\devBuildShaderPak_DX11.bat
, and
specify the location to the ShaderList_
file.
platform
.txt
Example
F:\
lumberyard_version
\dev\BuildShaderPak_DX11.bat C:\shader_compiler_server
\ShaderList_DX11.txt
Once the shader .pak
files are created, you can move them as needed. For example, if you already built
a release version of your game, you can place them with
the rest of the .pak
files.
When compiling shaders for your project, you can customize the BuildShaderPak_DX11.bat
file as needed. The following is an excerpt from a sample
.bat
file:
set SOURCESHADERLIST=%1 set GAMENAME=
your_game_project
set DESTSHADERFOLDER=Cache\%GAMENAME%\PC\user\Cache\Shaders set SHADERPLATFORM=PC rem other available platforms are GL4 GLES3 METAL rem if changing the above platform, also change the below directory name (D3D11, METAL, GL4, GLES3) set SHADERFLAVOR=D3D11
Packing the Shader Cache Manually
If you want to use more complex build pipelines, you can pack the shader cache manually.
To pack the shader cache manually
-
Run
ShaderCacheGen.exe
to generate the shader cache so you can pack it later. -
Zip all the shaders into a
ShaderCache.zip
file, then rename the file toShaderCache.pak
.
Each platform has different .pak
files. See the following
directory mapping for the PC platforms:
The PC platform should copy data from the following directories:
lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\D3D9\lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\D3D10\lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\D3D11\lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\GL4\
into the following destination directories:
shaders\cache\D3D9\ shaders\cache\D3D10\ shaders\cache\D3D11\ shaders\cache\GL4\
ShaderCache.pak
should contain everything from the previously listed subfolders.
ShadersBin.pak
should contain only the *.cfxb
and *.cfib
files.
ShaderCacheStartup.pak
should contain the following files:
lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\lookupdata.bin -> Shadercache\<platform>\lookupdata.binlumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\CGPShader\FixedPipelineEmu* -> Shadercache\<platform>\CGPShader\FixedPipelineEmu*lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\CGPShader\Scaleform* -> Shadercache\<platform>\CGPShader\Scaleform*lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\CGPShader\Stereo* -> Shadercache\<platform>\CGPShader\Stereo*lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\CGVShader\FixedPipelineEmu* -> Shadercache\<platform>\CGVShader\FixedPipelineEmu*lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\CGVShader\Scaleform* -> Shadercache\<platform>\CGVShader\Scaleform*lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\CGVShader\Stereo* -> Shadercache\<platform>\CGVShader\Stereo*lumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\lookupdata.bin -> Shaders\Cache\<platform>\lookupdata.binlumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\Common.cfib -> Shaders\Cache\<platform>\Common.cfiblumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\fallback.cfxb -> Shaders\Cache\<platform>\fallback.cfxblumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\fixedpipelineemu.cfxb -> Shaders\Cache\<platform>\fixedpipelineemu.cfxblumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\FXConstantDefs.cfib -> Shaders\Cache\<platform>\FXConstantDefs.cfiblumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\FXSamplerDefs.cfib -> Shaders\Cache\<platform>\FXSamplerDefs.cfiblumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\FXSetupEnvVars.cfib -> Shaders\Cache\<platform>\FXSetupEnvVars.cfiblumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\FXStreamDefs.cfib -> Shaders\Cache\<platform>\FXStreamDefs.cfiblumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\scaleform.cfxb -> Shaders\Cache\<platform>\scaleform.cfxblumberyard_version
\dev\cache\your_game
\platform
\user\shaders\cache\<platform>\stereo.cfxb -> Shaders\Cache\<platform>\stereo.cfxb
Build Platforms
You can find the build platform subfolders listed in the following table in the
directory.
lumberyard_version
\dev\Cache\your_game
\platform
\user\cache\shaders\
Build Platform | Build Platform Subfolder |
---|---|
PC, DirectX 11 | \D3D11 |
PC, OpenGL 4 | \GL4 |