The NxuStream FAQ
What is NxuStream? NxuStream is a serialization library for the Ageia PhysX SDK. The Ageia PhysX SDK uses a data driven approach to create objects for
the simulation. Rather than creating a rigid
body and then calling numerous methods to configure it, instead all of the data
needed to define the properties of a rigid body are placed into a data
structure called a ‘descriptor’ and submitted in a single creation call.
The PhysX SDK likewise offers an API method that will save the
current state of an object back out to this data structure so that a developer
can preserve the state.
Since there are
somewhere on the order of over 1,200 unique data items that can be represented
in the Ageia PhysX SDK
(this include every single bit flag and enumerated type) it takes a substantial
development effort to read and write all of this state to and from single data
resource.
NxuStream will take all or any sub-portion of the
contents of the SDK and copy them to a collection of descriptors (referred to
as an ‘NxuPhysicsCollection’) and then can save this
collection to a number of file formats.
It can also read that collection back into memory and, finally,
instantiate it onto the SDK.
The uses for this
include capturing a single object, an entire scene, or the whole SDK to a file
for debugging purposes. These files can
also be sent back to technical
Probably the most
exciting use for NxuStream is to represent discrete
data assets for your game. Combining the
NxuStream data with authoring tools such as CreateDynamics, 3D Studio Max or Maya can be very
powerful.
NxuPhysicsCollections can be instantiated at any location in the
world. In this fashion you could have a
single data asset that represents an object like a ‘tree’ or a ‘ragdoll’ and then place then instantiate them all over a
game environment.
How do I use the library? NxuStream is delivered as a collection of source code
that can compile against all versions of the PhysX
SDK 2.4.0 and higher. Simply add the
source code to your project (make sure your include paths are set up to point
to the PhysX SDK) and build.
Only a single
header file is needed to take advantage of the NxuStream
API. Simply include ‘NXU_Helper.h’
and make a few calls.
The NxuStream library needs a small amount of persistent memory
to keep track of name fields and the instantiation of mesh assets. To release this memory your application
should, on exit or any time you do a full reset of the SDK, call ‘NXU::releasePersistentMemory’.
Why isn’t it just part of the SDK? NxuStream is a tool that, even though it is tightly bound
to the SDK, is also independent of any specific version. The goal is to target past and future
releases of the SDK with the same code base.
This kind of backwards and forwards compatibility is critically
important to help maintain data assets.
Another reason not
to put NxuStream directly into the SDK is because it
is a rather large code base and not everyone is going to want to use it in
their product.
It is also quite
possible that NxuStream would be revised independent
of the SDK release schedule since new features such as COLLADA improvements
would be unrelated to the SDK directly.
Why is it delivered as source code instead
of a DLL or library? NxuStream
is delivered as source code to provide the maximum flexibility for developers
to incorporate it into their project. Of
course it could be easily placed into a library or DLL by any individual
developer if they wanted to.
Since NxuStream is entirely tools related code it is valuable for
a developer to easily step into the source code when debugging or binding it to
their own system. It is also quite
likely that some developers will choose to place the NxuStream
code into part of their tools chain as part of a game editor or used in plug-in
components.
The NxuStream source code is entirely inside a namespace ‘NXU’
so it should have no conflicts with other source in an application.
Does NxuStream use
the STL? NxuStream does not
use the STL (Standard Template Library).
For containers it simply uses the simple vector template ‘NxArray’ that is provided by the PhysX
SDK.
However, even
though NxuStream does not use the STL is still does
quite a few memory allocations using the standard ‘new’ and ‘delete’
operator.
How much source code is in NxuStream? NxuStream comprises 42 source files, 22 CPP files and
20 header files.
This contains
33,308 lines total which breaks down as:
25,761
lines of code.
4,510 empty lines.
2,066 comment lines.
971 empty comment lines.
Why is there so much source code?
The quantity of
source code simply reflects the amount of data that NxuStream
has to represent. It must first be able
to extract all data from the SDK and copy it into an ‘NxuPhysicsCollection’. Next it needs to be able to both read and
write this data in a fault tolerant and human readable way. All memory ownership needs to be taken into
account during this process. There is
also code to instantiate the data back into the SDK and this too needs to be
fault tolerant and keep memory ownership into account.
Finally, the
biggest reason for the quantity of source code is having
to
What file formats does NxuStream
The COLLADA
What is COLLADA? COLLADA
is an industry standard 3d asset interchange format. It is the first such format to provide
Why are there so many formats? Each
of these three formats serves a particular need.
COLLADA is useful when working with 3rd party authoring
tools that recognize and
NxuStream
XML is useful when
authoring data assets that require complete control over every feature in the PhysX SDK. It is
especially useful for debugging and troubleshooting problems since the XML can
easily be visually inspected.
NxuStream
Binary is useful for final
production assets for high speed data loading.
The binary format
Why aren’t binary files backwards
compatible? Since the binary version of the data is
designed to loaded and saved at high speed directly to and from various
descriptor data structures it is extremely challenging to maintain forwards and
backwards compatibility when those data structures change across SDK versions. Additionally any time new features are added
to NxuStream itself it would invalidate the previous
data assets.
The XML version is
much more flexible and robust and does not suffer from these same
problems. It is best to keep source data
in XML and only burn it into a binary cooked file as a production process.
What is ‘cooked’ data? ‘Cooked’
data represents triangles meshes and convex hulls in a format that the PhysX SDK can directly operate on. Before a triangle mesh or convex hull can be
submitted to the SDK it must first be ‘cooked’ which involves a lot of
operations to manipulate the source data into an optimal representation for
high speed collision detection.
Internally various data structures are built as well as general
cleanup. Once data has been ‘cooked’ it
can be loaded into the SDK almost instantaneously.
Is NxuStream
multiplatform? Yes, NxuStream has been tested on Windows, Unix,
and an XBOX-360.
What XML parser does NxuStream
use? NxuStream
uses the ‘TinyXML’ parser to load XML data
assets. TinyXML,
as the name implies, is very tiny. It is
a stable, robust, and fast DOM XML parser in only a few thousand lines of
code. This implementation of TinyXML has been wrapped into a namespace and redirects
file IO to
What is the purpose of the individual source
files provided?
NXU_Helper.cpp
and NXU_Helper.h
This is the main
public interface to NxuStream. An application should only ever need to
include this single header file to have access to all of the functionality
needed.
NXU_Asc2Bin.cpp
and NXU_Asc2Bin.h
This is a utility
that helps convert ASCII data into a binary representation.
NXU_BinaryStream.cpp and NXU_BinaryStream.h
This is the implementation
of the stream class to read and write the binary version of the data. NxuStream uses a
pure virtual interface so that reading and writing both binary and the XML
versions is largely the same code.
NXU_ColladaExport.cpp and NXU_ColladaExport.h
This code saves out
the contents of an NxuPhysicsCollection to a COLLADA
1.4.1 physics file.
NXU_ColladaImport.cpp NXU_ColladaImport.h
This source parses
a COLLADA file and extracts all of the relevant physics content. This physics data is then copied to an NxuPhysicsCollection.
NXU_Cooking.cpp and NXU_Cooking.h
This is a utility
that invokes the version controlled cooking library.
NXU_File.cpp and NXU_File.h
This is a utility
that can vector standard file IO routines to a buffer in memory if needed.
NXU_Geometry.cpp and NXU_Geometry.h
This is a utility
that can build a renderable mesh from primitives such
as box, sphere, or capsule.
NXU_Hull.cpp and NXU_Hull.h
This is a utility
that generates a convex hull from a point cloud.
NXU_PhysicsExport.cpp and NXU_PhysicsExport.h
This source
extracts the contents of individual SDK objects into NxuStream
descriptors that can be added to an NxuPhysicsCollection.
NXU_PhysicsImport.cpp and NXU_PhysicsImport.h
This source invokes
the stream read routines to load a physics collection into memory.
NXU_PhysicsInstantiator.cpp and NXU_PhysicsInstantiator.h
This
source instantiates the contents of an NxuPhysicsCollection
into the PhysX SDK.
NXU_schemaparse.cpp and NXU_schemaparse.h
This is an internal
utility to read the NxuStream XML schema and produce
debugging data.
NXU_Stream.cpp and NXU_Stream.h
This is the main
streaming code that handles both reading and writing all of the contents of an NxuPhysicsCollection
NXU_StreamFactory.cpp and NXU_StreamFactory.h
This is a small
utility to retrieve the appropriate input or output stream interface based on a
particular file type.
NXU_Streaming.cpp and NXU_Streaming.h
This is a small
utility to stream cooked data to and from blocks in memory.
NXU_String.cpp and NXU_String.h
This is a small
utility that keeps track of persistent strings and asset bindings.
NXU_tinystr.cpp and NXU_tinystr.h
This source
provides string
NXU_tinyxml.cpp and NXU_tinyxml.h
This source contains
the core of the TinyXML data structures and API.
NXU_tinyxmlerror.cpp
This source is used
by TinyXML to keep track of the current error state.
NXU_tinyxmlparser.cpp
This source
contains the core XML parsing routines used by TinyXML
NXU_XMLStream.cpp and NXU_XMLStream.h
This source
provides