Xj Conversion Notes

Been putting off fixing my PsoBB njm2 to Pso njm converter. Would like to be able to say it's finished, but at least for the most part it seems to be working, so I published it on NPM.



To use it, download and install Nodejs.

From the command line (directory doesn't matter), run
Code: 
> npm install convert_njm -g

Once that is installed, you can run my utility from anywhere. In a directory with a PSOBB formated .njm run
Code: 
> njm_conv <filename.njm>

And the animation should be exported to a Noesis readable filename_n.njm
 
Soly said:
Pretty good!! Congrats

I ported part of your lib (pretty much the archive handling) to c# ... and went to try the ExMDL or however is named and saw that it needs .Net 2 lmao, I don't use it and don't plan to activate it so, I might try and remake it in C#/net 4.5
Click to expand...

.Net framework is backwards compatible. The runtime has been written in such a way that it can and will run all previous assemblies built against previous versions. If you're referring to the source then it's an even simpler process for porting to later versions. At most things may have become obsolete or deprecated. You shouldn't need to explictly install .Net 2.0 framework.

Here is an MSDN article about .Net version compatibility: http://msdn.microsoft.com/en-us/library ... 00%29.aspx
 
I know, yet windows asked me to install 3.5 (which has 3 and 2)
Either way... I'd like to port the code to C#, mostly to use C# rather than a newer version of the runtime.
 
Another tool and another video on how to use it. This time for extracting player animation data. I do some light explanation in the video for the general concept of how to extract the animations. Source code is here: link.



1. Install nodejs
2. Open a command line
3. > npm install plymotion -g
4. > PRCTool.exe /D PlyMotionData.pr2 motion_data.bin
5. > plymotion motion_data.bin

And then my plymotion data script will make an "output" folder in the current directory with all of the player animation .njm files exported into it.
 
Made a small tool for simple bml exporting.



You can install the bml export util by openning the command line and typing in "> npm install export_bml -g". This will add "bml_expt" to your path. From any directory, you can hold shift while right clicking and select, "open command line here" from the context menu. From there you can type in "> bml_expt <filename.bml>" and the script will export all of the contents from the bml, decompress them and put them in a folder with the basename of the bml file.
 
Lool... I like gui way more than cmd, and at the end, we could argue about
-opening a custom program vs cmd
-drag&drop/open file vs typing
It's not really that much different ...

I had assembled this a few days ago, but have been working on other stuff and not much into what you have done.
 

Attachments

  • Untitled.jpg
    Untitled.jpg
    9.2 KB · Views: 241
Soly said:
Lool... I like gui way more than cmd, and at the end, we could argue about
Click to expand...

No need to argue. For me it's mostly personal preference, I'd rather be able to be able to do everything from the command line, but drag and drop is nice, and GUI's are simple and intuitive to use. That's part of why I'm trying to document everything and make my source as open and clean as possible, the more options the better.


Also, that looks great. It'll be nice to have one tool with everything as opposed to always having to jump back and forth between different tools.
 

Attachments

  • Untitled.jpg
    Untitled.jpg
    9.2 KB · Views: 241
Oh no, don't take me wrong... this line "It's not really that much different ... "
Same as you, is preference, I have been interested in interface since my very beginning on java (that whole semester was pure command line).

I haven't uploaded that program anywhere (afaik), but it uses some of your stuff, AFS and BML (ported to C# obviously)... and not sure if it has drag&drop :P ... I did on something but don't recall what.
 
Okay, updated my Github page so it almost looks like a project, almost but not really.

Also Soly, I'm not sure if you're using any of my poorly written source code, but you might want to look at my updated source for BML archives. PSO version 2 has 16 bytes of zeros between each file in the archive, while PSOBB seems to have each file one right after the other. My old source didn't accommodate for this and failed miserably at PSO version 2 archives.

Also not sure if anyone else is using File Pointer on NPM besides me, but I added PRS decompression into that library for convenience if anyone's interested.

Next thing I'd like to take a look at is PVM. Problem is that for all of the utilities out there, there doesn't seem to be that mush source available and most of the documentation only includes the file header and not the actual image structure. So if anyone knows of any resources let me know. Not sure how long it's going to take if I have to start from scratch.

Edit: Oh nice, Puyotool has a decent wiki entry on PVM. A C implementation, stashing it here so I don't lose track of it. link.
 
Okay did some npm cleaning. I removed all of the existing packages and re-published them with names that made more sense.

1. BML_Export
2. NJM_Convert
3. Ply_Motion
4. PVM_Export

Also I started playing around with PVM and managed to export an archive all to png format from the command line. I think I still have to add support for different data format types, but here's a quick video of it in action, and you can give it a try from the NPM links above.

 
I've spent the last couple of days playing around with NJ. The file format is pretty simple, node, model definition, vertex list and polygon list. The node and mode definition are both static types, so reading those is super easy.

The vertex list has a 8 byte header on the front [type][length][offset][num vertexes]. The length and offset can pretty much be ignored. There are a few different kind of vertex types, such as x,y,x with vertex color or with normals, but the basic definition is normally just x,y,z,nx,ny,nz. You can also cheat by dividing the length of the section by the number of items to get the size of each vertex.

For the polygon list things get a little more interesting, but is also pretty simple once you get an idea of what's going on. In the polygon list, there are three possible chunks that can be declared, material, texture and triangle strip. Material contains ambient and diffuse information, right now I'm ignoring this, but I can at least recognize it and know how long it is to do so. The texture, which is just a dword which contains the texture id as a byte inside of it, and a triangle strip which contains a list of integers describing the faces of an object.

Weird thing about the triangle strip is that the UV data and vertex color and normal lines can also be declared in there. Though again, the structure isn't too hard to figure out and for nor I'm ignoring everything except the triangle list itself and the uv's.

Surprising problem with NJ's and models in general is while parsing out the information seems like a challenge, it's not as ugly as actually writing out the model. Applying the rotations and getting the translations right, getting the flags and trying to figure out what Sega was thinking when they were making these models is the hard part.

I can get the faces and rotations, but when I add in translations things tend to not go so well. Though looking into NJ was mostly to allow me to read model information from the PsoV2 maps. And those are in a list and not a child-sibling tree-node structure, so i'll kind of tended to look at the maps again to see if I can avoid dealing with any flags.
 
I was hoping to cut corners by focusing directly on stages, but it looks like pso version 2 stages use nested nj objects and not just listed ones like in PSOBB, which means I need to go back and finish exporting nj before I can get back into stages.

This is mostly to help me write out information, but if anyone else is interested, the NJCM structure used in PSO version 2 is as depicted:


As a reference, the file structure will start with the interchange file format header of "NJCM" followed by the length as a dword. The evalflags for the first NJS_OBJECT will follow immediately after that.

Basically a NJCM model is made up of "nodes". Each node has eval flags, rotation, translation and scale, and a model. The model describes the center pointer and radius of the node (for bones). It has a vertex list describing a list of xyz coordinates the model uses, and an indices list which describes the material, texture and triangle strip describing the faces of model.

Each "node" has a child and a sibling pointer which describes how the model should be written out. Rotation, translation and scale attribute pass down from parent to child, but not sibling to sibling.

※１. The eval flags is a 32 bit unsigned integer which contains bit wise flags for how the Rotation, translation and scale attributes are to be evaluated when writing the model to create the shape of the overall object.

※２. The structure of the NJS_VERTEX depends on the vertex_type declared in the header of the VERTEX_LIST. The vertices may be in <x,y,z,nx,ny,nz> or <x,y,z,color> or <x,y,x,nx,ny,nz,color> depending on the type.

※３．The NJS_INDICE_LIST doesn't have an exact defined structure. It contains three smaller sub-chunks:
1) Materials which declares the diffuse, specular, and ambient settings.
2) Texture which indicates the texture id to use from the PVM archive
3) Strip List which contains clockwise or counter clockwise indices for faces, and generally includes UV information as well
 
I finally managed to parse and render .nj thanks to Kryslin's ExMLD.Net source. I'm going to start working on textures now, but I finally got the model structure working.

Y5xgbMqm.png
 
Still missing textures and a other details when it comes to processing ninja-chunks, but the source code is up on github here if anyone wants to take a look.
 
Good news, stage models can now be exported.


Source code is here if anyone likes that stuff or anything. Basically the stage files are a list of sections with an offset. Inside each section is a list of nj files. The nj files are rendered for each scene, and then rotated and moved as a scene to their respective part of the stage.

What this means is that it doesn't take a lot of code to actually read them. Stage.js parses out the parts of the stage, njcm.js parses the inner-nj files and MatrixUtil.js are matrix functions ported from Kryslin's exmldnet for rendering models.

I need to go back and get stuff like textures and make the models double sided, but for the most part I think this means that pretty much everything can be exported from Pso version 2. As for stages n.rel files seem to be non-interactive models in the stage, while d.rel files make up the walls and floor.
 
kion said:
Good news, stage models can now be exported.


Source code is here if anyone likes that stuff or anything. Basically the stage files are a list of sections with an offset. Inside each section is a list of nj files. The nj files are rendered for each scene, and then rotated and moved as a scene to their respective part of the stage.

What this means is that it doesn't take a lot of code to actually read them. Stage.js parses out the parts of the stage, njcm.js parses the inner-nj files and MatrixUtil.js are matrix functions ported from Kryslin's exmldnet for rendering models.

I need to go back and get stuff like textures and make the models double sided, but for the most part I think this means that pretty much everything can be exported from Pso version 2. As for stages n.rel files seem to be non-interactive models in the stage, while d.rel files make up the walls and floor.
Click to expand...

Wow thats not good news, thats very good news.

What about import stage models? Are you planning to do it aswell?

Congrats!
 
Oskitar1981 said:
Congrats!
Click to expand...
Thanks!

Oskitar1981 said:
What about import stage models? Are you planning to do it as well?
Click to expand...

Planning to? Short answer: no.

With exporting you can skip over a lot of tiny details to export the model. Going the other way you'd need to perfectly understand everything. On top of that, there's collision and lighting, qedit stuff. So it's a pretty massive time investment.

So I'm trying to make my source code and documentation available for anyone who really wants to do that stuff, but personally I don't think I could be paid to do it. You can just export the models to unity and get results like this in the fraction of the time with well established tools available to anyone.
 
This is absolutely brilliant! I had just posted about finding a way to get these models exported to C4D, thanks to Soly I was redirected here. You did it all! I'm not a programmer, the most experience I have is a CSE 114 intro to Java class but I've been trying to keep up with what you've been documenting. This is some really impressive stuff. Thanks again.
 
You must log in or register to reply here.
Back
Top