Menu
Lumberyard
Developer Guide (Version 1.11)

CryPak File Archives

The CryPak module enables you to store game content files in a compressed or uncompressed archive.

Features

  • Compatible with the standard zip format.

  • Supports storing files in an archive or in the standard file system.

  • Data can be read in a synchronous and asynchronous way through IStreamCallback (max 4GB offset, 4GB files).

  • Files can be stored in compressed or uncompressed form.

  • Uncompressed files can be read partially if required.

  • File name comparison is not case sensitive.

  • Supports loading of .zip or .pak files up to 4GB in size.

Unicode and Absolute Path Handling

Internally, all path-handling code is ASCII-based; as such, no Unicode (16-bit characters for different languages) functions can be used—this is to save memory and for simplicity. Because games can and should be developed with ASCII path names, no real need for Unicode exists. Game productions that don't follow these requirements have issues integrating other languages. For example, because a user might install a game to a directory with Unicode characters, absolute path names are explicitly avoided throughout the whole engine.

Layering

Usually the game content data is organized in several .pak files, which are located in the game directory. When a file is requested for an opening operation, the CryPak system loops through all registered .pak files. .pak files are searched in order of creation. This allows patch .pak files, which have been added to the build later, to be in a preferred position. It is also possible to mix .pak files with loose files, which are stored directly in the file system (not in a .pak file). If a file exists as a loose file as well as in a .pak archive, the loose file is preferred when the game is in devmode. However, to discourage cheating in the shipped game, the file stored in the .pak is preferred over the loose file when the game is not run in devmode.

Slashes

Usually forward slashes ( / ) are used for internal processing, but users may enter paths that contain backslashes.

Special Folder Handling

You can use the path alias @USER@ to specify a path relative to the user folder. This might be needed to store user-specific data. Windows can have restrictions on where the user can store files. For example, the program folder might not be writable at all. For that reason, screenshots, game data, and other files should be stored in the user folder. The following are examples of valid file names and paths:

Copy
@USER@/ProfilesSingle/Lisa.dat game/Fred.dat

Internals

  • A known implementation flaw exists where using more than approximately 1000 files per directory causes problems.

  • Format properties:

    • The .zip file format stores each file with a small header that includes its path and filename in uncompressed text form. For faster file access, a directory is listed at the end of the file. The directory also stores the path and filename in uncompressed text form (redundant).

Creating a pak file using 7-Zip

To create a .pak file with 7-Zip's 7za.exe command line tool, use the following syntax:

Copy
7za a -tzip -r -mx0 PakFileName [file1 file2 file3 ...] [dir1 dir2 ...]

Dealing with Large Pak Files

The zip RFC specifies two types of .zip files, indicated by .zip format version 45. Old .zip files can have a 4GB offset, but if legacy I/O functions are used, it is only possible to seek +- 2GB, which becomes the practical limit. The 4GB offsets have nothing to do with native machine types and do not change size across platforms and compilers, or configurations. The offsets for older versions of .zip files are in a machine independent uint32; the offsets for the new version .zip files are in uint64, appended to the old version structs. The version a .zip file uses is located in the header of the .zip file. Applications are free to not support the newer version. For more information, see the .ZIP File Format Specification.

Manual splits are not necessary, as RC supports auto-splitting:

  • zip_sizesplit – Split .zip files automatically when the maximum configured or supported compressed size has been reached. The default limit is 2GB.

  • zip_maxsize – Maximum compressed size of the .zip file in kilobytes (this gives an explicit limit).

Splitting works in all cases and supports multi-threading and incremental updates. It expands and shrinks the chain of necessary zip-parts automatically. Sorting is honored as much as possible, even in face of incremental modifications, but individual files can be appended to the end of the parts to fill in the leftover space even if this violates the sort order.

For more information about zip files, see Zip File Format Reference by Phil Katz.