Ninja ASCII (.NJA) format conversion from .OBJ [Theory]

To actually work on implementation of bones, njaPatcher will require a large rewrite, this means separating the different functions such as loading OBJ and generating triangle strips into isolated JavaScript promises, as working with multiple OBJ will need random access to those functions and njaPatcher should also retain support for single OBJ export of NJA and NJ.

Fortunately the code for njaPatcher can be reused for the majority of this reimplementation and the new NJA with bones gives a good idea of how to break up the existing code (which I wasn't sure of earlier). It can be expected that the next major version of njaPatcher will aim to have some different functionality in support of loading multiple OBJ objects as a single NJA, but before it is in development, V0.21a will be pushed, to fix the OBJ smoothing group error.

The next major version will take a while to write, since there are some new elements to program for, and the focus will shift back to making NJA files until one can accurately be made with a bone structure.

To explain why bones are important, there are several PSOBB weapons that utilize bone positioning as projectile spawn points, or for spawning techs like Gifoie. For example, Baranz Launcher has a bone for each projectile that is shot, but also bones for the actual weapon model, on top of that, each bone has an ID. So 4 bones in Baranz Launcher have ID for a projectile start point function, the tricky part is figuring out which bone ID is associated with which action per weapon model, but there is a solution to this, which is stepping through a weapon with 1 bone, then 2 bones, then 3 etc. - It should also be noted that character models generally have around 64 bones, so it is not really that much to step through, most weapons seem to be around 4 - 10 bones.

It's simple with a weapon like Magical Piece, since it only has 4 bones. I believe 1 of them is where it's held, 2 of them are representing the shadeless/emitting parts of the model, and then a final one which displays where the particle system for Magical Piece spawns, not entirely sure how they are ordered though. Can see Magical Piece bone structure below:

1631784479552.png​

Here is how Magical Piece's special attack functions with only a single bone; the Zonde still hits whatever is targeted, but the particles spawn at the map origin or (0, 0, 0). Certain weapons like Psycho Wand and Mercurius Rod with Foie techs, will spawn that Foie tech also at the map origin, generally Barta, Zonde and Anti all spawn on the character model, but this is likely to do with the bone structure per weapon described and how those tech call functions are associated with it. Below is an example of Magical Piece, and then Psycho Wand:

1631785342347.png

1631786809154.png
​
 
So I have implemented some new features into njaPatcher, which is mostly the re-write of code that I mentioned in the last post, but also some completely new ideas. Unfortunately my research with bone structures has turned into limbo. Before explaining why, I must remind anyone keeping up with this thread that this research is a lot of guess work and theory, even if I do now have systems that can accomplish the original aim of this thread. With this in mind, I will also say that the next version of njaPatcher is still a while away, and that the smoothing group error probably won't be fixed until then.

About bones (I will refer to them as nodes from now on), my thoughts about whether they could be used to control projectile spawn positions seems to be partially wrong. I still believe they are used as projectile spawn points, but I don't believe the function for that can be exported from njaconv.exe for example, it seems to be a proprietary export option for PSO development. To test if it worked from njaconv.exe, I reproduced the Handgun model from PSOPC using ExMLDNet's scan feature, which gives the exact positioning of nodes for an NJ loaded into it.

With those node positions I wrote an NJA file that chains the nodes in the exact same order as the original Handgun model, but the model still spawns projectiles at the wrong place. My guess is still that the property of projectile spawn position is validated in some way by the model file, since it changes per model, but just not sure how it's communicated from the model file to the game client. The Handgun model details can be seen below, compared to the original model.

1632075311076.png

I didn't check if node structure controls Foie tech spawn points yet, since I feel like that will also be inconclusive. Instead I have shifted my efforts into writing a parser for ExMLDNet's scan output, the purpose of this is to reverse engineer already existing PSO/NJ model files back into NJA, this way, existing node structures can be reproduced with custom models, which will allow better testing for stuff like swapping enemy models or character models (not saying it will work).
 
Today I managed to copy some node structures, like HUcast and Ill Gill. The process works by taking every node position from an ExMldNet scan of an NJ object from PSOBB, njaPatcher can parse these scans when it creates an NJA file, and can bind it with the loaded model, e.g, cube. All models stay within the domain of the old model node structure. This kind of appearance is how models are loaded for NJ with nodes, a skeleton structure is created, and it draws model data (3D objects) at important points of the skeleton, e.g, head, legs, arms.

Not every node structure builds correctly so far, Dragon for example seems to have too high of a node count, or is formatted in a way that the scan parser reads it wrong.

1632169184731.png​
 
Today I started writing a rigging system for player models, unfortunately certain chars will require unique rigging systems since each char has a different node structure. For instance, FOmarl's arm, uses a different node ID to HUcasts arm. The same thing goes for any enemies in the game, they will require unique rigs to mod.

The rigging system currently implemented supports HUcast for now, still unsure if other models like RAcast use roughly the same rig. The rig works by assigning a model to each part of the player model. It looks something like this in code:

Code: 
rig.leftHand = "left_hand.obj"
rig.leftElbow = "left_elbow.obj"

While this works there are some issues with 'up' and 'forward' facing direction for models, since each model in a node structure has a different 'up' and 'forward' direction. Can be seen in the image below, where blue represents the 'up' facing direction, green represents the 'forward' facing direction, and red represents the 'right' facing direction of each model (cube). The labelled numbers are also in the order of models that are loaded for HUcast:

1632335020799.png​

It may be possible to hardcode facing direction by reading how each model is rotated through the node structure, and then winding back the rotation so it's as if the object was never rotated. The real problem with facing direction is that it's even awkward to configure in Blender, when each part of an object has to be rotated to a specific angle before exporting, it can be quite confusing and hard to keep track of what you're doing.

It is possible to get a neat outcome though if you follow the structure properly - this is a model that I have started working on to build the armature system today. I'm not saying it's a good model, but it can be seen how the armature system is modified:

1632335927581.png​
 
Last edited:
HUcast rig is mostly complete and will be available for the next release, to finish it up I spent the day figuring out how to make all nodes have the same orientation. In the previous post, the orientation/face direction problem was described, and I have now solved it for HUcast at least, as solving this is not an autonomous implementation. It's unique for each character and enemy as it's part of the rigging object. The difference between the image below and the one in the previous post, is the facing direction of each node:

1632587373095.png
​
Having solved this, it should now be possible to box model a complete character in Blender with reference to a character rig, and when the model is finished in Blender, each part that is a node just needs to be separated into it's own object with a defined origin. Being able to box model the whole character is what this new implementation allows, since previously you would have to mess with facing direction per node on export, which is a lot of extra work! To explain better how this works, each node object still needs to be exported from Blender as a separate OBJ, but it doesn't need to be modified in terms of it's orientation or location. I will make a video on how to use njaPatcher's rigging tools at another time.

I will still need to test this with some different 3D models before I confirm everything works, but it seems good so far, one thing I have noticed is that certain parts of the player model allow less memory than others, e.g., complex models on certain nodes (HUcast shoulder shield thing) will cause PSOBB to crash as the model loads and sometimes it will just load extremely warped (reducing vert count fixes this). Something cool that I've found is that section ID badge also seems to be it's own model, which is a pretty neat custom object since it's texture is based on section ID.
 
There is a new version of njaPatcher up with the previously mentioned node structures implemented. There are some bug regressions with this version of njaPatcher, mostly relating to making NJA files without textures, but it is nothing major.

https://github.com/je-mappelle-egg-yolk/njaPatcher

This version only converts node structures to NJA at the moment, although with this version released I have started to study how node structures are implemented into NJ files. So far it has been a quite difficult to figure out, this is due to the amount of different outcomes that require testing for node structures, since there can be more than one model in the node structure version of NJ and also nodes that don't use model data. This means that the new NJ conversion should be able to detect abstract model sizes and adjust variables accordingly, fortunately what I had learned making the single OBJ to NJ converter has helped me out a lot so far, and I actually think I have figured out the hardest part of node based NJ today. I believe what I figured out today was sibling pointers, although I can't say for sure, at least it is the same as njaconv.exe output anyway...

Node structure to NJ will take a long time to finish, since it seems vastly more confusing than anything else I've programmed so far, but I do believe I can do it if I take my time and write stuff down.

I am planning to make a video pretty soon, to go over the current version of njaPatcher, but if I don't make a video the next post will probably be when I have finished node based NJ or have hit a roadblock with it.
 
Seems like there's a geometry issue with a specific vertex, does it open correctly in Noesis?

EDIT: Just read description, it were a direction test :)
 
Last edited:
For the past couple of weeks I have been designing a 3D character model for PSOBB, what's cool is that I am also reusing any mods that I make for PSOBB in other projects that I run, so it is not wasted time.

I did learn some new things when doing this, since I had tested the entire process. I realized that even with njaPatcher, the process for exporting/testing models is still very tedious, firstly it will always require old programs like texture manager and AFS manager, and secondly there is some heuristic practice to exporting models.

The heuristics are that on export, each body part origin must match where it is on the HUcast armature and also be centered within the 3D scene on export. The entire set of models also need to be scaled massively (not sure of %, but to the scale of original HUcast model) and rotated 180 degrees on the up axis. This becomes quite time consuming to redo when you have to make edits to the model. Generally when you have all the programs set up in a particular way though, the process is fast to test, but you are still swapping between 5-6 windows with some manual operations for each patch to the game files. Fortunately it's usually only 2 or 3 models that need re-editing at that point though.

The reason I had to patch often is due to polygon count limitations, which I've realized are tied to the entire character model for character model mods. For instance, you could put 600 verts into HUcast's chest model, but trying to then load arms and legs would cause crashes.

Below is an example of the reductions I did after finishing the model in Blender, the left model is 702 verts, while the right is 1,178 verts. So somewhere around 700 verts seems like the limit for HUcast at least. Some of the reductions were made on parts where there are body part joints, so there are new gaps in between leg and knee for example, this will probably be a trend with my own character model mods, since it's pretty awkward to make the joint look right. njaPatcher does not make blend node models like in PSOBB, instead models just clip through each other.

polygonCount.png
​
Lastly this is an image of the model in game, I decided to use a different head model for this screenshot, but the cool thing about PSOBB is that a character mod could have several different head models. I also didn't encounter any crashes while playing with this model so far. I am planning to release this mod at a later date, I will continue to crash test it for now, but I would also like to finish the texture and design some other head models before releasing.

1636893734407.png
​
 
Today I tried out the new character model from the previous post on Ephinea server and there are some new memory related discoveries.

To explain, the original HUcast body model is 25kb in memory, whereas the custom model is 40kb. HUcast head models range from 2kb - 5kb and the custom model is 8kb. So it is 30kb~ vs. 48kb (original model vs. custom model).

What I've found is the custom model loads fine in single player or an empty lobby, but if there are too many players in the lobby it will crash. So I decided to reduce the custom model memory size again by unloading some parts.

With the custom model reduced to 40kb (including HUcast head 1), bursting into a packed lobby is fine, even with multiple custom HUcast standing around.

So the conclusion to this post is that custom HUcast model memory should total at no more than 40kb. Many areas are still unvisited, such as larger ep4 areas where it's expected more memory is used for map/enemy data, or in general areas with lots of mobs.

My suspicion is that, rather than there being a limit for models total, there is perhaps a separate limit for player models total, but I will continue to test it out.
 
Lately I have been making a folder with decompiled BML contents from data.gsl. Within data.gsl there are lots of game objects such as trap models, item drop models, map cursor models etc.

I have been attempting to mod a few of these BML files, and figured I would make a post about it here so that I can have a reference to my findings. I will perhaps make a new thread about BML files later on, so there is a reference for anyone that mods in the future.

The first finding was when trying to mod item.bml, which has item drop models. In PSOBB these models use the XJ model format, although it is different to the XJ format used for weapons. This different format can be recognised by viewing the end of the XJ file in hex, where it reads "P0F08". Most other models will just have "P0F0" followed by some kind of total data size.

There are a few other BML files that contain the same type of XJ with "P0F08". To name some, trap models and the tower that appears before you fight Dark Falz are formatted like this also. Unfortunately this file format doesn't seem to be modifiable with the currently known model swapping methods, and the "P0F08" tag cannot be reproduced using SEGAs exporter.

Another BML I tried to edit was obj_f_enemy.bml, which contains the cursor objects for the map, they are 3D objects for whatever reason. This mod didn't work out either, but it was an interesting BML to look at, since it actually contains cursor objects as GJ files, which are the GameCube counterpart to NJ files, and I also think they are the smallest 3D model files in the game. I found something else when modding this file too, it's possible to load the V2 version of the BML in place of the PSOBB one, I had thought it might be a good idea to instead mod the V2 version since it contains NJ files, but the bml tool I use (BlueCrab) doesn't seem to work that well with V2 BML files. This concept may prove to be useful for other mods, so it's something I will try more often, if I can find a more compatible bml tool.

The last BML I tried to mod was onlineending_dat.bml, which is the posing scene that plays after you beat a major boss. This actually worked for model swapping, but I was unable to modify the texture. I did realize when doing this that VMT can be used to directly repack a BML file though, the options for saving change when you open a BML with VMT, I am unsure how well it works though, since it didn't work for this particular BML file. I will maybe mod onlineending_dat.bml to use new models, but with the same textures, so that the screen doesn't look so weird in widescreen mode and also so that the player models don't clip.

1637922799955.png​
 
I have been testing character model swaps lately, trying out different quests with the new model loaded. Today I tried TTF and unfortunately the attack where Dark Falz spawns a smaller version of the player model causes PSOBB to crash. So far this has only happened when the custom model is textured, there could be a couple reasons for this.

Either the game is expecting a smaller file, since the NJCM version works here (smaller in file size) or, the game is expecting a file without UV mapping, since the original character models are not textured.

To explain how this works, there are at least 3 versions of each player class model in the game files. The first set is in the "data" folder, and these are models without textures. The next set is in "plZsmp.afs", which are morph targets for proportion, and the final set is in each class's own BML file, "plAnj.bml", "plBnj.bml", etc., it's not currently possible to mod these BML files.

The mod I'm using adds textures coordinates to the "data" versions of the models, which don't usually have texture coordinates. The mod also changes any corresponding model in "plZsmp.afs".

EDIT: The "data" versions do have texture coordinates, or "texture map assignment", according to ExMLDNet. They don't have a texture list though (NJTL header), which is really confusing. It is possible for the custom model to load by removing it's NJTL header, but the same crash occurs. I can also confirm that problem is not file size, as I tested it with a low file size body model.

The character model below works in any situation, but it does lose some visual appeal without having a texture.

1638295520435.png​
 
Last edited:
I figured out a way to resolve the issue described in the last post, so it is now possible to play all EP1 areas with a custom textured player model. I'm not certain that I have checked every possible enemy attack in EP2 and EP4 yet, but it is looking fine from what I have played.

To explain how the fix works, each model node in NJA includes an Evalflag setting, and while I am not certain what each Evalflag value does, I decided to experiment with them. When using the SEGA exporter, textured models will use this Evalflag setting, "EvalFlags( FEV_UT|FEV_UR|FEV_US|FEV_BR )," and non textured models will use "EvalFlags( 0x00000016 ),". So I decided to try the non textured Evalflag setting on a textured model, which is not something SEGA's exporter can do. It bypassed the animation where Falz replicates the player model, stopping the game from crashing. I believe the game is expecting an untextured model, and perhaps 0x00000016 tells the game that the model isn't textured, even though it is, very speculative idea though.

The new model doesn't seem to appear when Falz does this attack, as seen in the last post, Falz only clones the players head model, however, the fight continues as it normally would.

I will write a feature to add arbitrary values into the Evalflags setting with njaPatcher, which should be getting an update soon.

This discovery was pretty simple in the end, but I'm happy I found it since it may lead to developments for other types of model swapping that I have not been successful with in the past.

Can see the attack working below. The enemy radar cursor here is a new loaded model, that I was using to see if enemies have a facing direction on the radar, unfortunately they don't.

1638386900072.png​
 
Since njaPatcher is at a point where it is usable for creating valid NJA files, I have started to program a new application named njaGenFE, which is a frontend user centered version of njaPatcher using Angular. I am essentially using njaPatcher as an endpoint for generating NJA file output, so njaPatcher updates will still be separate.

I am mostly creating this app to practice front end web development, so I am unsure how far I will go with it. I am thinking to add a 3D view of the model, where anyone using the app can drag and drop models to their respective parts (arm, leg, chest, etc.). There should be a drop down for cloned NJ armatures, which will start as just PSO objects, e.g., HUcast or RAcast., but also a creative mode for creating abstract node structures.

I will open source the project when I have certain components in place.

1638726348231.png​
 
Since my last post, various development changes have taken place for my new program:

-Renamed from njaGenFE to njaGen
-Environment change from Angular to React and Electron in order to release it as a desktop application
-Removed 3D view concept to simplify things

Some other changes have occurred as well, which make me hesitant to update the njaPatcher repository further, since a modified version of it is now utilized in njaGen and the updates wouldn't make sense in the original context of njaPatcher.

njaGen is currently in a bug testing phase and could see a release next week if nothing major is found. It will allow anyone to create a single mesh OBJ into an NJ file (Mag mods), and also supports multi OBJ compiling into a HUcast body armature, this option is limited to NJA file output though. I have also implemented a series of articles into the program to help guide the user into getting a working output.

I will make a new thread when I release this tool.

1641677914013.png​
 
You must log in or register to reply here.
Back
Top