Looking for a bulk texture extraction tool

[ZabaZu]

Hi, I was wondering if there is a preexisting bulk method of extracting textures from the game files. I'm trying to use an AI algorithm (ESRGAN) to upres all the textures in the game (the results are absolutely incredible in other games and images in general). Problem is I can't locate a tool that allows me to bulk extract textures and needing to go through 1 by 1 for both export and import would be a real pain in the ass. Thanks in advance for any help!

EDIT: Alternatively, if there isn't one, if anybody has source code for an existing texture exporting/importing tool that they'd like to share I'd love to take a look at it to see if I could add that functionality myself.

 

[Wil]

This.

I have a really bulky editing PC that can do massive workloads like this. If someone can find a way to do bulk texture extraction and repackaging, I'll help upscale all of the textures.

 

[ZabaZu]

This.

I have a really bulky editing PC that can do massive workloads like this. If someone can find a way to do bulk texture extraction and repackaging, I'll help upscale all of the textures.

Having played around with it more now and tried it out on some textures, this is gonna end up being a far more manual process than I anticipated, most likely. The problem is mostly that all textures in the game are DDS textures, and upscaling that really blows up compression artifacts in super ugly ways. So each texture will need to be cleaned up and it's going to require getting eyes on each texture for cleanup and then upscaling afterwards, as I'm not convinced there will be a good generic setting for all textures to get cleaned up while retaining detail. It also doesn't help that ESRGAN doesn't work with alpha elements, so you gotta strip the alpha channel out, upscale it more traditionally, then merge them back. I'm probably going to give it a shot for one area, just to determine how much individual effort each texture will require (all these complications are technically possible to handle with a script, provided I can find some decent CLI software for my needs).

Basically, I'm finding out that manually exporting textures is the least of my concerns with this project, which would explain why someone hasn't done this yet lol.

 

[Wil]

Having played around with it more now and tried it out on some textures, this is gonna end up being a far more manual process than I anticipated, most likely. The problem is mostly that all textures in the game are DDS textures, and upscaling that really blows up compression artifacts in super ugly ways. So each texture will need to be cleaned up and it's going to require getting eyes on each texture for cleanup and then upscaling afterwards, as I'm not convinced there will be a good generic setting for all textures to get cleaned up while retaining detail. It also doesn't help that ESRGAN doesn't work with alpha elements, so you gotta strip the alpha channel out, upscale it more traditionally, then merge them back. I'm probably going to give it a shot for one area, just to determine how much individual effort each texture will require (all these complications are technically possible to handle with a script, provided I can find some decent CLI software for my needs).

Basically, I'm finding out that manually exporting textures is the least of my concerns with this project, which would explain why someone hasn't done this yet lol.

Oh, yikes >.<

 

[Echelon]

Having played around with it more now and tried it out on some textures, this is gonna end up being a far more manual process than I anticipated, most likely. The problem is mostly that all textures in the game are DDS textures, and upscaling that really blows up compression artifacts in super ugly ways. So each texture will need to be cleaned up and it's going to require getting eyes on each texture for cleanup and then upscaling afterwards, as I'm not convinced there will be a good generic setting for all textures to get cleaned up while retaining detail. It also doesn't help that ESRGAN doesn't work with alpha elements, so you gotta strip the alpha channel out, upscale it more traditionally, then merge them back. I'm probably going to give it a shot for one area, just to determine how much individual effort each texture will require (all these complications are technically possible to handle with a script, provided I can find some decent CLI software for my needs).

Basically, I'm finding out that manually exporting textures is the least of my concerns with this project, which would explain why someone hasn't done this yet lol.

I've been there before

Spoiler

It requires your life.

Months of work and it's not finished, ran out of Gigapixel AI free trial.

 

[ZabaZu]

I've been there before

Spoiler

It requires your life.

Months of work and it's not finished, ran out of Gigapixel AI free trial.

Damn. It looks pretty good though! I'll probably still give it a real shot on the off chance I can find a way to automate it with some of the newer tools available today. I just gotta stop playing games with my free time and start figuring out how I can do this

 

[Echelon]

You probably can automate it with a script (with Gigapixel I could do it in batch) but i would'nt do that, because you want to try different settings for different textures. The results vary widly from one to another, so you have to play with noise reduction/remove blur, x2/x4 depending on the texture and most of the time you'll need to edit/clean it in photoshop afterwards (i wouldn't clean them before upscaling because you would loose some details the AI could have detected). Also i had to make several iterations for some of them because what you see in photoshop does not always look the same ingame.

 

[ZabaZu]

You probably can automate it with a script (with Gigapixel I could do it in batch) but i would'nt do that, because you want to try different settings for different textures. The results vary widly from one to another, so you have to play with noise reduction/remove blur, x2/x4 depending on the texture and most of the time you'll need to edit/clean it in photoshop afterwards (i wouldn't clean them before upscaling because you would loose some details the AI could have detected). Also i had to make several iterations for some of them because what you see in photoshop does not always look the same ingame.

I spent a couple hours last night experimenting more. I'm getting some okay results with a more generic upscaling model as long as I first run the textures through an artifact removal model first. I'm starting to think I could quickly make a "better than nothing" upscaled texture pack if I could export/import the textures in bulk. Currently, I can automate image conversion (dds to png and vice versa), image cleanup, and image upscaling. It's not perfect but honestly, upon scrutiny, the textures in the game contain some strange details (or lack thereof).

Some quick examples

Spoiler

There's weird soft patches in the grass texture, but when you zoom in on the source image, that weird blur exists there as well (in much lower fidelity lol). I still need to test it on more varied textures to know for sure, but I'm hopeful!

EDIT: This image hosting seems to be resizing the upscaled images...
You'll have to use these links to get the full res image apparently. Struggling to find an image hosting site that can just flat out give me an image link, support png, and allow me to upload images over 10 MB lol.
https://ibb.co/2dxKmF0
https://ibb.co/2kZK0xw
https://ibb.co/QP3vmMN