Xj Conversion Notes

[Soly]

Doesn't seem to be default, but is on the store so no problem I guess if you want to mess with it.
https://www.microsoft.com/en-us/store/a ... zdncrfj3t6

 

[Andy]

Ooops...that's embarrassing...
Thanks guys! I'm excited to see where this goes!

 

[kion]

Well.... fuck. At least I figured out rotation, which I was stuck on how to convert Sega's int's into an angle. Then I realized that most of the rotations are probably going to be 90 degrees. So I took one of the given values, 16384, set that as 90, and it worked. Now I need to figure out this parent, sibling tree structure and how to apply translations and rotations for each element over the structure.



Edit:

Need to fix rotations across 2 axis, and translations. Starting to actually look like something.

 

[Soly]

Sweet... you almost got that Pwand xD

 

[vermillion]

I just cloned your project to play around with things. I'm glad to see you're making progress!

 

[Aleron Ives]

Well.... fuck. At least I figured out rotation, which I was stuck on how to convert Sega's int's into an angle. Then I realized that most of the rotations are probably going to be 90 degrees. So I took one of the given values, 16384, set that as 90, and it worked.

I wish you had given examples of the numbers involved, as I could have told you that to save you some trouble. Quest object rotation works the same way; rotation is stored as a signed int, and 32768 is 180 degrees.

 

[kion]

I just cloned your project to play around with things. I'm glad to see you're making progress!

Sorry for the lack of any kind of coding style. My main focus is mostly to playing around with different files, so it's a massive mess of bodged code. I published ninja-lib on npm for functions that are more stable, and I'm trying to make steps to migrate more over to npm once they work. For now the archives: prs, gsl, bml and xvm seem to be working pretty well.

I wish you had given examples of the numbers involved, as I could have told you that to save you some trouble. Quest object rotation works the same way; rotation is stored as a signed int, and 32768 is 180 degrees.

I read in Sega's ninja guide that positive 0xFFFF corresponds to 360 degrees, which is what I had in my code before. Maybe something didn't click or I messed up the model's tree structure when writing out the obj file. It's one of those things where you take your break and comeback to realize you were stuck on some non-issue.

And I dropped the psychowand .xj file into schthack's model viewer to find that it worked perfectly. Aside from his map editor, did that guy ever release any source or write any documentation? It's good experience, but it'd be nice not having to go back and re-reverse engineer everything.

 

[Aleron Ives]

I wish he had released source for more tools, because we are having to reinvent the wheel and remake tools from scratch to bypass the limitations of the programs he never finished. Supposedly he even figured out how the map animations work so as to make the maps animate in the final version of Qedit, but he never even released that version publicly, let alone the information on how to manipulate the map animations.

 

[kion]

You're pretty sour about those map animations. Now that I think about it though, animations might not be too difficult. One of my issues is that I'm not very familiar with the .nj file type and .xj actually makes more sense to me at the moment. But for the Dreamcast and Version 2 on PC, models are in .nj format, maps are just a collection of .nj models. The njm file format is actually pretty trivial and one of the easier formats to deal with.

PsoBB uses mostly .xj for models are maps are pretty much a collection of .xj models. Sega uses a slightly different animation format for .nj models in PsoBB which mostly just has a few bits moved around. I looked at the items (warps etc) that use the .xj format and have animation and it looks like Sega has a different variation of .njm for those as well.

I refer to them in my head as: ".njm", ".njm2", ".xjm"

I haven't gotten the chance to play around with them, since I'm still in the process of parsing out the .xj model, but at first glance the .xj animation format didn't seem to be too drastically different from their previous versions . Not sure how viable it would be to parse out the animations from .nj style maps and port them to the .xj animation format, but that might be a possibility if someone wanted to get their hands dirty.

My priority is to try and parse out .xj, possibly work on textures and then get back to trying to export maps with the correct rotations and translations. So my hands are kind of full with the amount of time I have to spend on this stuff. Though I really don't mind explaining anything or drawing things out, since it helps me understand things better and gets more documentation out there.

In terms of programming ability, I didn't even know what an unsigned int was when i started working on these things, so I don't mind starting from the very basics or possibly demonstrating my lack there of. Also if anyone is familiar with the .nj file format, it gives me no end of trouble. Being able to parse or convert to and from .nj would be a pretty decent advantage.

 

[kion]

I wish he had released source for more tools, because we are having to reinvent the wheel and remake tools from scratch to bypass the limitations of the programs he never finished.

I was super sour about the model viewer not being open source. It displays njm2 animations and .xj models, Both of which were pretty high on my priorities. I managed to convert njm2 to njm, so being able to export xj would be a plus.

Writing this here so I have something to refer to.

Node id: 0
Parent: null
Sibling: null
Position: {x : 0.000000, y : 0.000000, z : -0.061831}
Angle: { x: 0, y: 0, z: 0 }
Center: {x : 0.000001, y : -0.073414, z : -1.684910}
Radius: 3.977135
Eval Flags: 0 0 0 0 0 1 1 0

Node id: 1
Parent: 0
Sibling: null
Position: {x : 0.000004, y : 0.000000, z : 6.000000}
Angle: { x: 90, y: 0, z: 0 }
Center: {x : -0.000018, y : 4.695141, z : 0.096502}
Radius: 2.860878
Eval Flags: 0 0 0 0 0 1 0 0

Node id: 2
Parent: 1
Sibling: null
Position: {x : -0.009434, y : 5.046088, z : 0.038424}
Angle: { x: 0, y: 0, z: -59 }
Center: {x : 0.910859, y : 2.340958, z : 0.000000}
Radius: 2.511920
Eval Flags: 0 0 0 0 0 1 0 0

Node id: 3
Parent: 2
Sibling: null
Position: {x : 0.000000, y : 0.000000, z : 0.000000}
Angle: { x: 0, y: 0, z: 0 }
Center: {x : 0.304303, y : 2.868946, z : 0.000000}
Radius: 1.863142
Eval Flags: 0 0 0 1 0 1 1 1

Node id: 4
Parent: null
Sibling: 2
Position: {x : 0.000000, y : 0.000000, z : 0.000000}
Angle: { x: 0, y: 0, z: 0 }
Center: {x : -0.000020, y : 2.251833, z : 0.055300}
Radius: 7.483905
Eval Flags: 0 0 0 1 0 1 1 1

Node id: 5
Parent: null
Sibling: 4
Position: {x : 0.009434, y : 5.046087, z : 0.038424}
Angle: { x: 0, y: 0, z: 59 }
Center: {x : -0.910860, y : 2.340960, z : 0.000000}
Radius: 2.511921
Eval Flags: 0 0 0 0 0 1 0 0

Node id: 6
Parent: null
Sibling: 4
Position: {x : 0.009434, y : 5.046087, z : 0.038424}
Angle: { x: 0, y: 0, z: 59 }
Center: {x : -0.910860, y : 2.340960, z : 0.000000}
Radius: 2.511921
Eval Flags: 0 0 0 0 0 1 0 0

 

[MagicLuthee]

Nice project there kion, and good progress ! Could it open up the way to higher-poly models in PSOBB later ?

 

[kion]

I'm amazed at my ability to disappoint modders. Ideally I think there are better options than bending over backwards to writing custom tools for a closed source binary with no possibility of modifying the code.

My main focus is exporting assets, but I'm making everything open source and explaining anything and trying to document everything I can in terms of file formats to try and make the information easier to access for anyone in the community.

In terms of progress:
For Version 2 on PC and Dreamcast pretty much anything can be extracted. All of the models are .nj, animations are .njm, and archives are pretty easy to extract. I also managed to export the player motions into individual .njm files, so the only thing left there is maps.

Right now I'm focusing on PSOBB. Most of the models are in .nj. Enemy animations are in .njm2, which can be pretty easy to convert back to .njm. Right now I'm working on exporting .xj. And I can kind of export map files, but they don't have the right rotations, which is why I'm focusing on .xj to try and debug those problems.

My trial & error github repository is here: https://github.com/seiche/Pso_File_Formats
Once I get functions working, I'm moving them over to https://www.npmjs.com/package/ninja-lib

 

[MagicLuthee]

It's true that there are better ways to do things than shove stuff into an old engine, a dream would be some kind of FOSS implementation of a PSOBB client, which would work with official assets as well. Kind of what the RObrowser project is to Ragnarok Online (only development seems to a halt on this one ?)

 

[Aleron Ives]

You're pretty sour about those map animations.

I want a moving aurora without having to boot my DC. :lol: All of the map animations are stored in the *n.rel file for each map, though, so I have no idea how they might differ from the animations that are attached to specific weapons. In theory, if the format were documented, it would be possible to add and remove animations to the maps at will by removing or appending animations to the rel file.

 

[kion]

I didn't notice anything that stood out in the n.rel file, but then again I didn't look into the animation part very much. The file has two sets of vertex groups, A which contains the model and a minimal B, which Schthack in his program comments said might be the animation. Though in the map viewer a lot is still incomplete. I haven't looked into the B group because my main focus is getting a clean extraction of the models and not the animations. I published the source for my Nodejs edit, and it has a similar structure to his map editor if someone wants to get their hands dirty.

Right now .xj really isn't making any sense. The header for njcm is given by:

typedef struct cnkobj {
Uint32 evalflags; /* Evaluation method optimization*/
NJS_CNK_MODEL *model; /* Model structure pointer*/
Float pos[3]; /* Parallel motion*/
Angle ang[3]; /* Rotation*/
Float scl[3]; /* Scale*/
struct obj *child; /* Child object pointer*/
struct obj *sibling; /* Sibling object pointer*/
} NJS_CNK_OBJECT;

The first variable is an unsigned int for flags. Flags are defined as:

#define NJD_EVAL_UNIT_POSBIT_0/* Motion can be ignored */
#define NJD_EVAL_UNIT_ANG BIT_1/* Rotation can be ignored */
#define NJD_EVAL_UNIT_SCL BIT_2/* Scale can be ignored */
#define NJD_EVAL_HIDE BIT_3/* Do not draw model */
#define NJD_EVAL_BREAK BIT_4/* Break child trace */

The first node is the shaft of the rod given by:
Position: {x : 0.000000, y : 0.000000, z : -0.061831}
Angle: { x: 0, y: 0, z: 0 }
Center: {x : 0.000001, y : -0.073414, z : -1.684910}
Radius: 3.977135
Eval Flags: 0 0 0 0 0 1 1 0

Basically it lies flat across the x-axis and doesn't have any kind of rotation, so every other node in the model should line up with this one. It has the flag to ignore rotation, of which it has none, so that doesn't really matter.

The second node is the decoration on top of the staff.
Position: {x : 0.000004, y : 0.000000, z : 6.000000}
Angle: { x: 90, y: 0, z: 0 }
Center: {x : -0.000018, y : 4.695141, z : 0.096502}
Radius: 2.860878
Eval Flags: 0 0 0 0 0 1 0 0

The problem is that it is straight up and down in the y direction. That's not so much of a problem since it has a 90 degree rotation in the x direction, that aligns it with the axis of the shaft of the rod. The issue is that the node has a 6 unit translation in the z direction, which puts it a significant distance away from the shaft of the rod.

So it would make sense if it has the flag to ignore translation, but it seems to have 0's on the bits for ignoring angle and rotation. Unless there's something I need to do with applying flags from the parent node or something. It seems really weird to have both on or both off in this case, since it doesn't come anywhere close to being right.

With no translation or rotation:

With both translation and rotation:

Also the values match up with what Kryslin had on his xj parser from exmldnet. So I think those are right, I'm just not sure how to get them to cooperate.

 

[Aleron Ives]

At the risk of asking a stupid question, are you taking into account the fact that the Psycho Wand extends and becomes about twice as long when you swing it or get close to an enemy (which is also what causes the tip to expand)? Maybe that's what the 6 unit translation is for?

 

[kion]

The relation is hard to see in a 2d image. Here is the output for the models: link. In the zip is a write out of the first two nodes of the pwand. "applied.obj" has translation and rotation applied. You can see how 6 units in the z axis messed everything up. "default.obj" has nothing applied. And "rotation.obj" has rotation but not translation applied. It lines up with the shaft of the wand, which is nice, but the problem is I can't find any flags in the file to write that out automatically.

I played around with different variations of translations, and stuff more or less started lining up once I applied translations to everything. So I might need to go the whole distance and apply everything before things start lining up. I'm going try different models to see if there are any patterns that stick out with different models.

 

[kion]

Started writing up some documentation to track progress. Link. If anyone has any information or changes they want to mention let me know.

 

[Soly]

Scthack shouldn't be Scht... ? xD (In the readme of git)
Besides that I have nothing to offer, haven't even looked deep into your work, but is a damn good one so far.

 

[Aleron Ives]

He usually managed to spell our names wrong (among many other things), so it's only fair that we should misspell his.