Menu
Lumberyard
Developer Guide (Version 1.12)

Serialization Library

The CryCommon serialization library has the following features:

  • Separation of user serialization code from the actual storage format. This makes it possible to switch between XML, JSON, and binary formats without changing user code.

  • Re-usage of the same serialization code for editing in the PropertyTree. You can write the serialization code once and use it to expose your structure in the editor as a parameters tree.

  • Enables you to write serialization code in non-intrusive way (as global overloaded functions) without modifying serialized types.

  • Makes it easy to change formats. For example, you can add, remove, or rename fields and still be able to load existing data.

Tutorial

The example starts with a data layout that uses standard types, enumerations, and containers. The example adds the Serialize method to structures with fixed signatures. 

Defining data

Copy
#include "Serialization/IArchive.h" #include "Serialization/STL.h" enum AttachmentType { ATTACHMENT_SKIN, ATTACHMENT_BONE }; struct Attachment { string name; AttachmentType type; string model; void Serialize(Serialization::IArchive& ar) { ar(name, "name", "Name"); ar(type, "type", "Type"); ar(model, "model", "Model"); } }; struct Actor { string character; float speed; bool alive; std::vector<Attachment> attachments; Actor() : speed(1.0f) , alive(true) { } void Serialize(Serialization::IArchive& ar) { ar(character, "character", "Character"); ar(speed, "speed", "Speed"); ar(alive, "alive", "Alive"); ar(attachments, "attachments", "Attachment"); } };   // Implementation file: #include "Serialization/Enum.h" SERIALIZATION_ENUM_BEGIN(AttachmentType, "Attachment Type") SERIALIZATION_ENUM(ATTACHMENT_BONE, "bone", "Bone") SERIALIZATION_ENUM(ATTACHMENT_SKIN, "skin", "Skin") SERIALIZATION_ENUM_END()

Why are two names needed?

The ar() call takes two string arguments: one is called name, and the second label. The name argument is used to store parameters persistently; for example, for JSON and XML. The label parameter is used for the PropertyTree. The label parameter is typically longer, more descriptive, contains white space, and may be easily changed without breaking compatibility with existing data. In contrast, name is a C-style identifier. It is also convenient to have name match the variable name so that developers can easily find the variable by looking at the data file.

Omitting the label parameter (the equivalent of passing nullptr) will hide the parameter in the PropertyTree, but it will be still serialized and can be copied together with its parent by using copy-paste.

Note

The SERIALIZATION_ENUM macros should reside in the .cpp implementation file because they contain symbol definitions.

Serializing into or from a file

Now that the data has been defined, it is ready for serialization. To implement the serialization, you can use Serialization::SaveJsonFile, as in the following example.

Copy
#include <Serialization/IArchiveHost.h>    Actor actor; Serialization::SaveJsonFile("filename.json", actor);

This will output content in the following format:

Copy
{ "character": "nanosuit.cdf", "speed": 2.5, "alive": true, "attachments": [ { "name": "attachment 1", "type": "bone", "model": "model1.cgf" }, { "name": "attachment 2", "type": "skin", "model": "model2.cgf" } ] }

The code for reading data is similar to that for serialization, except that it uses Serialization::LoadJsonFile.

Copy
#include <Serialization/IArchiveHost.h>   Actor actor; Serialization::LoadJsonFile(actor, "filename.json");

The save and load functions used are wrappers around the IArchiveHost interface, an instance of which is located in gEnv->pSystem->GetArchiveHost(). However, if you have direct access to the archive code (for example, in CrySystem or EditorCommon), you can use the archive classes directly, as in the following example.

Copy
#include <Serialization/JSONOArchive.h> #include <Serialization/JSONIArchive.h> Serialization::JSONOArchive oa; Actor actor; oa(actor); oa.save("filename.json");   // to get access to the data without saving: const char* jsonString = oa.c_str();   // and to load Serialization::JSONIArchive ia; if (ia.load("filename.json")) { Actor loadedActor; ia(loadedActor); }

Editing in the PropertyTree

If you have the Serialize method implemented for your types, it is easy to get it exposed to the QPropertyTree, as the following example shows.

Copy
#include <QPropertyTree/QPropertyTree.h>   QPropertyTree* tree = new QPropertyTree(parent);   static Actor actor; tree->attach(Serialization::SStruct(actor));

You can select enumeration values from the list and add or remove vector elements by using the [ 2 ] button or the context menu.

In the moment of attachment, the Serialize method will be called to extract properties from your object. As soon as the user changes a property in the UI, the Serialize method is called to write properties back to the object. 

Note

It is important to remember that QPropertyTree holds a reference to an attached object. If the object's lifetime is shorter than the tree, an explicit call to QPropertyTree::detach() should be performed.

Use Cases

Non-intrusive serialization

Normally when struct or a class instance is passed to the archive, the Serialize method of the instance is called. However, it is possible to override this behavior by declaring the following global function:

Copy
bool Serialize(Serialization::IArchive&, Type& value, const char* name, const char* label);

The return value here has the same behavior as IArchive::operator(). For input archives, the function returns false when a field is missing or wasn't read. For output archives, it always returns true.  

Note

The return value does not propagate up. If one of the nested fields is missing, the top level block will still return true.

The global function approach is useful when you want to:

  • Add serialization in non-intrusive way

  • Transform data during serialization

  • Add support for unsupported types like plain pointers

The following example adds support for std::pair<> type to the Serialize function:

Copy
template<class T1, class T2> struct pair_serializable : std::pair<T1, T2> { void Serialize(Serialization::IArchive& ar) { ar(first, "first", "First"); ar(second, "second", "Second"); } } template<class T1, class T2> bool Serialize(Serialization::IArchive& ar, std::pair<T1, T2>& value, const char* name, const char* label) { return ar(static_cast<pair_serializable<T1, T2>&>(value), name, label); }

The benefit of using inheritance is that you can get access to protected fields. In cases when access policy is not important and inheritance is undesirable, you can replace the previous code with following pattern.

Copy
template<class T1, class T2> struct pair_serializable { std::pair<T1, T2>& instance; pair_serializable(std::pair<T1, T2>& instance) : instance(instance) {} void Serialize(Serialization::IArchive& ar) { ar(instance.first, "first", "First"); ar(instance.second, "second", "Second"); } } template<class T1, class T2> bool Serialize(Serialization::IArchive& ar, std::pair<T1, T2>& value, const char* name, const char* label) { pair_serializable<T1, T2> serializer(value); return ar(serializer, name, label); }

Registering Enum inside a Class

Normally, SERIALIZATION_ENUM_BEGIN() will not compile if you specify enumeration within a class (a "nested enum"). To overcome this shortcoming, use SERIALIZATION_ENUM_BEGIN_NESTED, as in the following example.

Copy
SERIALIZATION_ENUM_BEGIN_NESTED(Class, Enum, "Label") SERIALIZATION_ENUM(Class::ENUM_VALUE1, "value1", "Value 1") SERIALIZATION_ENUM(Class::ENUM_VALUE2, "value2", "Value 2") SERIALIZATION_ENUM_END()

Polymorphic Types

The Serialization library supports the loading and saving of polymorphic types. This is implemented through serialization of a smart pointer to the base type.

For example, if you have following hierarchy:

IBase

  • ImplementationA

  • ImplementationB 

You would need to register derived types with a macro, as in the following example.

Copy
SERIALIZATION_CLASS_NAME(IBase, ImplementationA, "impl_a", "Implementation A"); SERIALIZATION_CLASS_NAME(IBase, ImplementationA, "impl_b", "Implementation B");

Now you can serialize a pointer to the base type:

Copy
#include <Serialization/SmartPtr.h>   _smart_ptr<IInterfface> pointer; ar(pointer, "pointer", "Pointer");

The first string is used to name the type for persistent storage, and the second string is a human-readable name for display in the PropertyTree.

Customizing presentation in the PropertyTree

There are two aspects that can be customized within the PropertyTree:

  1. The layout of the property fields. These are controlled by control sequences in the label (the third argument in IArchive::operator()).

  2. Decorators. These are defined in the same way that specific properties are edited or represented.

Control characters

Control sequences are added as a prefix to the third argument for IArchive::operator(). These characters control the layout of the property field in the PropertyTree.

Layout Control Characters

Prefix Role

Description

! Read-only field Prevents the user from changing the value of the property. The effect is non-recursive.
^ Inline

Places the property on the same line as the name of the structure root. Can be used to put fields in one line in a horizontal layout, rather than in the default vertical list.

^^ Inline in front of a name Places the property name before the name of the parent structure. Useful to add check boxes before a name.
< Expand value field Expand the value part of the property to occupy all available space.
> Contract value field Reduces the width of the value field to the minimum. Useful to restrict the width of inline fields.
>N> Limit field width to N pixels Useful for finer control over the UI. Not recommended for use outside of the editor.
+ Expand row by default. Can be used to control which structures or containers are expanded by default. Use this only when you need per-item control. Otherwise, QPropertyTree::setExpandLevels is a better option.
[S] Apply S control characters to children. Applies control characters to child properties. Especially useful with containers.

Combining control characters

Multiple control characters can be put together to combine their effects, as in the following example.

Copy
ar(name, "name", "^!<Name"); // inline, read-only, expanded value field

Decorators

There are two kinds of decorators:

  1. Wrappers that implement a custom serialization function that performs a transformation on the original value. For example, Serialization/Math.h contains Serialization::RadiansAsDeg(float&) that allows to store and edit angles in radians.

  2. Wrappers that do no transformation but whose type is used to select a custom property implementation in the PropertyTree. Resource Selectors are examples of this kind of wrapper.

Decorator Purpose Defined for types Context needed
Serialization/Resources.h
AnimationPath Selection UI for full animation path.

Any string-like type, like:

std::string,

string (CryStringT),

SCRCRef

CCryName

 
CharacterPath UI: browse for character path (cdf)  
CharacterPhysicsPath UI: browse for character .phys-file.  
CharacterRigPath UI: browse for .rig files.  
SkeletonPath UI: browse for .chr or .skel files.  
JointName UI: list of character joints ICharacterInstance*
AttachmentName UI: list of character attachments ICharacterInstance*
SoundName UI: list of sounds  
ParticleName UI: particle effect selection  
Serialization/Decorators/Math.h
RadiansAsDeg Edit or store radians as degrees float, Vec3  
Serialization/Decorators/Range.h
Range Sets soft or hard limits for numeric values and provides a slider UI. Numeric types  
Serialization/Callback.h
Callback Provides per-property callback function. See Adding callbacks to the PropertyTree. All types apart from compound ones (structs and containers)

Decorator example

The following example uses the Range and CharacterPath decorators.

Copy
float scalar; ar(Serialization::Range(scalar), 0.0f, 1.0f); // provides slider-UI string filename; ar(Serialization::CharacterPath(filename), "character", "Character"); // provides UI for file selection with character filter

Serialization context

The signature of the Serialize method is fixed. This can prevent the passing of additional arguments into nested Serialize methods. To resolve this issue, you can use a serialization context to pass a pointer of a specific type to nested Serialize calls, as in the following example.

Copy
void Scene::Serialize(Serialization::IArchive& ar) { Serialization::SContext sceneContext(ar, this); ar(rootNode, "rootNode") } void Node::Serialize(Serialization::IArchive& ar) { if (Scene* scene = ar.FindContext<Scene>()) { // use scene } }

Contexts are organized into linked lists. Nodes are stored on the stack within the SContext instance.

You can have multiple contexts. If you provide multiple instances of the same type, the innermost context will be retrieved.

You may also use contexts with the PropertyTree without modifying existing serialization code. The easiest way to do this is to use CContextList (QPropertyTree/ContextList.h), as in the following example.

Copy
// CContextList m_contextList; tree = new QPropertyTree(); m_contextList.Update<Scene>(m_scenePointer); tree->setArchiveContext(m_contextList.Tail()); tree->attach(Serialization::SStruct(node));

Serializing opaque data blocks

It is possible to treat a block of data in the archive in an opaque way. This capability enables the Editor to work with data formats it has no knowledge of.

These data blocks can be stored within Serialization::SBlackBox. SBlackBox can be serialized or deserialized as any other value. However, when you deserialize SBlackBox from a particular kind of archive, you must serialize by using a corresponding archive. For example, if you obtained your SBlackBox from JSONIArchive, you must save it by using JSONOArchive.

Adding callbacks to the PropertyTree

When you change a single property within the property tree, the whole attached object gets de-serialized. This means that all properties are updated even if only one was changed. This approach may seem wasteful, but has the following advantages:

  • It removes the need to track the lifetime of nested properties, and the requirement that nested types be referenced from outside in safe manner.

  • The content of the property tree is not static data, but rather the result of the function invocation. This allows the content to be completely dynamic. Because you do not have to track property lifetimes, you can serialize and de-serialize variables constructed on the stack.

  • The removal of the tracking requirement results in a smaller amount of code.

Nevertheless, there are situations when it is desirable to know exactly which property changes. You can achieve this in two ways: 1) by using the Serialize method, or 2) by using Serialization::Callback.

  1. Using the Serialize method, compare the new value with the previous value, as in the following example.

    Copy
    void Type::Serialize(IArchive& ar) { float oldValue = value; ar(value, "value", "Value"); if (ar.IsInput() && oldValue != value) { // handle change }  } 
  2. The second option is to use the Serialization::Callback decorator to add a callback function for one or more properties, as the following example illustrates.

    Copy
    #include <Serialization/Callback.h> using Serialization::Callback;    ar(Callback(value, [](float newValue) { /* handle change */ }),  "value", "Value");

    Note

    Callback works only with the PropertyTree, and should be used only in Editor code.

    Callback can also be used together with other decorators, but in rather clumsy way, as the following example shows.

    Copy
    ar(Callback(value, [](float newValue) { /* handle change*/ }, [](float& v) { return Range(v, 0.0f, 1.0f); }), "value", "Value");

Of the two approaches, the callback approach is more flexible, but it requires you to carefully track the lifetime of the objects that are used by the callback lambda or function.

PropertyTree in MFC window

If your code base still uses MFC, you can use the PropertyTree with it by using a wrapper that makes this possible, as the following example shows.

Copy
#include <IPropertyTree.h> // located in Editor/Include   int CMyWindow::OnCreate(LPCREATESTRUCT pCreateStruct) { ... CRect clientRect; GetClientRect(clientRect); IPropertyTree* pPropertyTree = CreatePropertyTree(this, clientRect); ... } 

The IPropertyTree interface exposes the methods of QPropertyTree like Attach, Detach and SetExpandLevels.

Documentation and validation

QPropertyTree provides a way to add short documentation in the form of tool tips and basic validation.

The Doc method allows you to add tool tips to QPropertyTree, as in the following examples.

Copy
void IArchive::Doc(const char*)
Copy
void SProjectileParameter::Serialize(IArchive& ar) { ar.Doc("Defines projectile physics.");   ar(m_velocity, "velocity", "Velocity"); ar.Doc("Defines initial velocity of the projectile."); }

The Doc method adds a tool tip to last serialized element. When used at the beginning of the function, it adds the tool tip to the whole block.

The Warning and Error calls allow you to display warnings and error messages associated with specific property within the property tree, as in the following examples.

Copy
template<class T> void IArchive::Warning(T& instance, const char* format, ...) template<class T> void IArchive::Error(T& instance, const char* format, ...)
Copy
void BlendSpace::Serialize(IArchive& ar) {  ar(m_dimensions, "dimensions, "Dimensions"); if (m_dimensions.empty()) ar.Error(m_dimensions, "At least one dimension is required for BlendSpace"); }

The error message appears as follows.

Warning messages look like this:

Drop-down menu with a dynamic list

If you want to specify an enumeration value, you can use the enum registration macro as described in the Defining data section.

There are two ways to define a drop-down menu: 1) transform your data into Serialization::StringListValue, or 2) implement a custom PropertyRow in the UI.

A short example of the first approach follows. The example uses a custom reference.

Copy
// a little decorator that would annotate string as a special reference struct MyReference { string& str; MyReference(string& str) : str(str) {} }; inline bool Serialize(Serialization::IArchive& ar, MyReference& wrapper, const char* name, const char* label) { if (ar.IsEdit()) { Serialization::StringList items; items.push_back(""); items.push_back("Item 1"); items.push_back("Item 2"); items.push_back("Item 3"); Serialization::StringListValue dropDown(items, wrapper.str.c_str()); if (!ar(dropDown, name, label)) return false; if (ar.IsInput()) wrapper.str = dropDown.c_str(); return true; } else { // when loading from disk we are interested only in the string return ar(wrapper.str, name, label); } } 

Now you can construct MyReference on the stack within the Serialize method to serialize a string as a dropdown item, as in the following example.

Copy
struct SType {  string m_reference; void SType::Serialize(Serialization::IArchive& ar) { ar(MyReference(m_reference), "reference", "Reference"); } };

The second way to define a drop-down menu requires that you implement a custom PropertyRow in the UI. This takes more effort, but makes it possible to create the list of possible items entirely within editor code.

On this page: