Coordinate System Conversion Between Unity And A Java Game Server

Requirement from game

For the purpose of FPS attack predication I have to synchronized Hitboxs/BoxColliders between Unity Client And Java Game Server

To be more specific:

• Each game character is exported to a DAE file for Server and to a FBX file for Client. These files contain geometry, joint and animation information.
• HitBox information is manually defined and stored as a XML file and shared between Client&Server. Each Hitbox is attached to a Joint/Bone of the corresponding character.
• When a player shoot a gun, both client and server will need to calculate the hit information. The server do so because it’s the arbitrator. The client do so to predicate the result and response immediately to the player.

The problem becomes complicated when:

• For some reason we can’t ask unity to do the ray tracing. However there is a shared library between client and server that can calculate whether a bullet ( ray ) has hit a character and the bone being hit if so.
• Client and server have different Coordinate Systems.
• When imported to unity, character’s coordinate system is also changed.

Defining Problem

However, if we think about it, we can find the the problem is actually as simple as following:

For each hit box, given the transform (denoted as $M^u$) of the joint it attached to in Unity’s coordinate system, we need to find a matrix (denonated as $M^s$) that can transform it to the same state in server’s coordinate system.

In the game a character is rotated and translated after animated. Suppose a point $P$ has the hierarchy of the following:

- Translate Node
    - Rotate Node 
        - Character Pivot Node
            - Joint1 Node
                - Joint2 Node
                    - Joint3 Node
                        - P

Then in Unity $M_u$ can be defined as:
$$M^u = M^u_{t} * M^u_{r} * M^u_{pivot} * M^u_{joint1} * M^u_{joint2} * M^u_{joint3}$$
Where:

• $M^u_{t}$ corresponds to translation
• $M^u_{r}$ corresponds to rotation
• $M^u_{pivot}$ corresponds to extra rotation introduced while imported by Unity
• $M^u_{joint1-3}$ corresponds joint transform

Given $M^u$ we can transform the point $P$ from its local space coordinate $P^u_{local}$ to its world space coordinate $P^u_{world}$:
$$M^u * P^u_{local} = P^u_{world}$$
Equally, in server we have $M^s$:
$$M^s = M^s_{t} * M^s_{r} * M^s_{joint1} * M^s_{joint2} * M^s_{joint3}$$
And can use it to transform $P$:
$$M^s * P^s_{local} = P^s_{world}$$
Note that there is a $M^u_{pivot}$ on Unity side, but no $M^s_{pivot}$ on server side. This will be explained latter.
We can easily convert between $M^u_{t}$ and $M^s_{t}$ and between $M^u_{s}$ and $M^s_{s}$, because we know how two coordinate systems are defined. The unknown part is:
$M^u_{pivot} * M^u_{joint1} * M^u_{joint2} * M^u_{joint3}$
That is what Unity do when importing fbx files.

Coordinate Systems

The following image shows two coordinate systems:

Server’s on the left, Unity’s on the right. We can see that Unity’s left handed while server’s right handed. A point $P^s(P_x, P_y, P_z)$ in server will be $P^u(-P_y, P_z, P_x)$ in Unity. The transform matrix would be:
$$M_{s \rightarrow u} = \begin{bmatrix} 0 & -1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ \end{bmatrix}$$
$$M_{s \rightarrow u} * P^s = P^u$$

What unity do when importing

After importing a FBX file, we can find that characters has the following changes:

• If it is originally facing +X axis, it is now facing -X axis. This effectively changes it’s handness
• It’s pivot is rotated around +x axis for +90 degrees. This step makes it head pointing to up direction again. Here is where the above mentioned $M^u_{pivot}$ comes from.

Given a point $P$, if we change the character from binding pose to another pose, say run, we first transform $P$ from its model space to joint local space:
$$P_{local} = (M_{joint1Binding})^{-1} * (M_{joint2Binding})^{-1} * (M_{joint3Binding})^{-1} * P_{modelBinding}$$
Where:
- $P_{modelBinding}$ is $P$’s coordinate in character’s binding pose.
- $M_{joint1Binding}$ transform a point in joint1’s local space to joint1’s parent’s local space. So $M^{-1}_{joint1Binding}$ will have the reverse effect.
Then we transform $P_{local}$ to a new model space coordinate.
$$P_{modelRun} = M_{joint3Run} * M_{joint2Run} * M_{joint3Run} * P_{local}$$
But if unity negates all model’s x axis. $P^s(P_x, P_y, P_z)$ becomes $P^u(-P_x, P_y, P_z)$. Suppose there is a matrix $M_{negX}$ does the transform, that is: $P^u = M_{negX}* P^s$.
So we have:
$$\begin{aligned} P^u_{modelBinding} &= M_{negX}* P^s_{modelBinding} \\ P^u_{modelRun} &= M_{negX}* P^s_{modelRun} \\ P^s_{modelRun} & = M^s_{joint3Run} * M^s_{joint2Run} * M^s_{joint3Run} * P^s_{local} \\ & = M^s_{joint3Run} * M^s_{joint2Run} * M^s_{joint3Run} * (M^s_{joint1Binding})^{-1} * (M^s_{joint2Binding})^{-1} * (M^s_{joint3Binding})^{-1} * P^s_{modelBinding} \\ P^u_{modelRun} &= M^u_{joint3Run} * M^u_{joint2Run} * M^u_{joint3Run} * (M^u_{joint1Binding})^{-1} * (M^u_{joint2Binding})^{-1} * (M^u_{joint3Binding})^{-1} * P^u_{modelBinding} \end{aligned}$$
Replacing $P^u_{modelRun}$ with $P^s_{modelRun}$ and $P^u_{modelBing}$ with $P^s_{modelBinding}$, we have:
$$
$$\begin{aligned} P^s_{modelRun} & = M^s_{joint3Run} * M^s_{joint2Run} * M^s_{joint3Run} * (M^s_{joint1Binding})^{-1} * (M^s_{joint2Binding})^{-1} * (M^s_{joint3Binding})^{-1} * P^s_{modelBinding} \\ M_{negX}*P^s_{modelRun} &= M^u_{joint3Run} * M^u_{joint2Run} * M^u_{joint3Run} * (M^u_{joint1Binding})^{-1} * (M^u_{joint2Binding})^{-1} * (M^u_{joint3Binding})^{-1} * M_{negX}* P^s_{modelBinding} \end{aligned}$$
$$
For both of the above equations to be true. We can:
$$M^u_{jointXPosX} = M_{negX} * M^s_{jointXPosX} * (M_{negX})^{-1}$$
That way, the following equation also is true:
$$(M^u_{jointXPosX})^{-1} = M_{negX} * (M^u_{jointXPosX})^{-1} * (M_{negX})^{-1}$$
Here, $M^u_{jointXPosX}$ stands for a matrix that can transform a point from jointX’s local coordinate to it’s parent’s local coordinate at posture posX.
Having the above information, we can now finish our task. Below is a code snippet of what I have written in our game:

public static Matrix3D Unity2SSJJ(Matrix4x4 result)
    {
        double[] data = new double[16];
        data[4 * 0 + 0] = result.m00;
        data[4 * 1 + 0] = result.m10;
        data[4 * 2 + 0] = result.m20;
        data[4 * 3 + 0] = result.m30;

        data[4 * 0 + 1] = result.m01;
        data[4 * 1 + 1] = result.m11;
        data[4 * 2 + 1] = result.m21;
        data[4 * 3 + 1] = result.m31;

        data[4 * 0 + 2] = result.m02;
        data[4 * 1 + 2] = result.m12;
        data[4 * 2 + 2] = result.m22;
        data[4 * 3 + 2] = result.m32;

        data[4 * 0 + 3] = result.m03;
        data[4 * 1 + 3] = result.m13;
        data[4 * 2 + 3] = result.m23;
        data[4 * 3 + 3] = result.m33;
        return new Matrix3D(data);
    }
    public static Matrix3D TransformToMatrix3D(Transform boneTransform, Matrix4x4 pivotWorldToLoal, PlayerEntity player)
    {
        Matrix4x4 mImportNegX = Matrix4x4.TRS(Vector3.zero, Quaternion.identity, new Vector3(-1, 1, 1));
        Matrix3D m  = Unity2SSJJ(mImportNegX * pivotWorldToLoal * boneTransform.localToWorldMatrix * mImportNegX);
        m.AppendRotation(player.orientation.ViewYaw, Vector3D.Z_AXIS);
        m.AppendTranslation(player.position.Origin.x, player.position.Origin.y, player.position.Origin.z);
        return m;

    }