Question for John P (kind of)

potterr on 20/8/2005 at 21:27

John, As far as I am aware your texture upgrades (and nice that you have a new one now) work by editing the ibt files with either pointers to diferent files or replacing the textures themselves in the IBT files (to be honest its not that important for what I'm thinking).
Now I'm not sure if this is done specifically by editing knwon locations in the IBT file or by searching for byte chunks that match a pattern.
I was thinking that would it be possible to create an installer for each texture pack that would work on T3 Fan missions?

The reasoning being that authors are having a bit of trouble using custom images at the moment, so I'm thinking it may be possible to use custom images after the FM IBT is created?

Didn't want to PM this as it may be possible for someone else to take up the challenge if you are unable to.

ProjectX on 21/8/2005 at 07:20

I get John P's textures in my custom FMs, I think the installer actually edits the matlibs, so that anyone who plays a map that uses a texture from those matlibs sees the new texture.

potterr on 21/8/2005 at 08:51

Cool for that, I was also thinking along the lines of the 'installer' being able to change certain images (portraits, etc) after the ibt file is created so that it can be distributed out (as authors are getting problems with custom stuff at the moment).
Also a though occurs to me in that if an FM author could actually do this, would it work on peoples PC's that don't have the textures? Not sure about the full process that the installer uses.

Or am I barking up the wrong tree here?

John P. on 21/8/2005 at 10:38

My installers are pretty 'crude' in the way they swap textures, and not intelligent. Actually, it only works thanks to how the game was programmed:

Essentially, my installers(for Thief: DS) opens .ibt files using a hex editor, which changes the names of the textures I want to swap out. The game then can't find the texture inside the .ibt file, and so it goes to the PCTextures\DynamicallyLoaded folder to see if there's such a texture there. And if it finds it, the game will use it, regardless of if it's high res. or not.

Making the installers is a lot of work, 'cause it's all been manual labour. For each given texture, I have opened each and every .ibt file, searched it, and if the texture name was there, I would write the name of that .ibt down. After having searched through all the .ibt files in this manner, I would then have a list of .ibt files to alter, for that texture alone. And so on to the next texture (unless of course there were several in a "family" of textures; for instance, if the upper body of a guard was in an .ibt, I would conclude that the rest of his textures would be there too).

The reason for doing it that way, was to shorten the time it takes to override the original textures. It already takes a lot of time for my biggest packs, but if I had just let them search through every single .ibt for each texture, it would have taken forever to install.

So - to the point: my installers work for the .ibt files that are there by default, with the default texture names. If an FM author has created custom .ibt file(s) and named them something other than one of the default .ibts, my installers won't open that .ibt file at all, and won't try to change its content.
It has to find specific .ibt names, and specific texture names inside those .ibts. So I guess my installers will work for some parts of FMs, where the FM uses the original resources, but not for the parts where it uses resources specially made for the FM.

potterr on 21/8/2005 at 13:15

Quote:

Essentially, my installers(for Thief: DS) opens .ibt files using a hex editor, which changes the names of the textures I want to swap out. The game then can't find the texture inside the .ibt file, and so it goes to the PCTextures\DynamicallyLoaded folder to see if there's such a texture there. And if it finds it, the game will use it, regardless of if it's high res. or not.

Ok so then its perfectly possible to fool the game into using custom textures.

Quote:

For each given texture, I have opened each and every .ibt file, searched it, and if the texture name was there, I would write the name of that .ibt down.

This would not be required the way I see it the installer program thingy would be pointed at a custom ibt by the author before distribution

Quote:

The reason for doing it that way, was to shorten the time it takes to override the original textures. It already takes a lot of time for my biggest packs, but if I had just let them search through every single .ibt for each texture, it would have taken forever to install.

Again not really an issue as it would only be the author doing this.

Quote:

It has to find specific .ibt names, and specific texture names inside those .ibts. So I guess my installers will work for some parts of FMs, where the FM uses the original resources, but not for the parts where it uses resources specially made for the FM.

Ah but it would be possible to adjust the process to look at list of input texture names against a list of change texture names, against an input value of a custom ibt file? Although I am slightly confused by this, as you mention that it would work on some parts, presumably if the FM ibt has not been edited then it should use the resources inside it?

[SIDEPOINT]
From what I can gather the custom textures are the dds files (noticing that the Labyrinth FM has Labyrinth.dds and Schism has zSchismEntry.dds and zSchism.dds which I am presuming are the loading screen images), what exactly is a dds file (just a plain image file or a combination of layers and textures and objects) and how is it made?
[/SIDEPOINT]

This is kind of the process I am thinking of:

* FM author creates an ibt file without custom textures.
* A souped up installer program thingey (not talking about installing anything as such just using the scripting process in the installer) can then be run over the ibt file. This will allow the FM author to specify the ibt file to run through and which images to replace (providing they are the same sizes, etc).
* The program will then do the required changes giving the IBT all the custom images required.
* This edited IBT file can then be zipped up with the extra textures in the PCTextures\DynamicallyLoaded folder and distributed

NOTE: this would not be the end user doing any of this, just the author.

I can see a few problems with it though, it means that the zips will be slightly bigger leaving redundant files in there and could possibly cause file location problems if not done right.

Effectively the program would be less of an installer and more of a texture changer, without needing to know what ibt's contain what textures are needed to change, etc.

I'm not saying you need to do any work here as I could probably do something (or another programmer could take up the challenge), I'm just wanting to know the ins and outs of the process to see if its entirely possible to create such an "installer/ibt changer" for FM authors to use.

Do you see where I am going with this?

John P. on 21/8/2005 at 14:07

Quote Posted by potterr

Again not really an issue as it would only be the author doing this.

No, I was just saying how my installers are doing this; I guess it was a digression.

Quote Posted by potterr

Ah but it would be possible to adjust the process to look at list of input texture names against a list of change texture names, against an input value of a custom ibt file?

The process could undoubtedly be both sped up and made more efficient; I just don't have any kind of programming knowledge to do so. However - for the game DX:IW, a programmer in the community made a texture overrider that does something like that. Not sure how it works, but it seems to be a lot more efficient than my hex editing approach(although it does edit the .ibd files in pretty much the same way that I edit the .ibt files), and I also don't have to go through the .ibd files in Deus Ex: IW to see which one contains this and that texture when I use his program. It's called ibd_patcher.exe, and the guy who made it is called Boye.

Quote Posted by potterr

Although I am slightly confused by this, as you mention that it would work on some parts, presumably if the FM ibt has not been edited then it should use the resources inside it?

I said that because ProjectX over here said he got my textures in his FMs, so I guess some FMs use the original .ibt files for most of the resources(textures etc.), and then maybe one custom made .ibt file for the FM map and possibly some new textures and objects, if the FM author has made any. If all the textures in the FM are used by linking to the original game .ibt files, then my textures would turn up in the whole FM, where applicable.

I don't know anything about making FMs or using the editor, so that's as far as my understanding goes.

Quote Posted by potterr

[SIDEPOINT]
From what I can gather the custom textures are the dds files, what exactly is a dds file (just a plain image file or a combination of layers and textures and objects) and how is it made?
[/SIDEPOINT]

A .dds file is a DirectDraw Surface. I don't know of any other use than for image files. They can be textures(colour maps) and normal maps, and they can have an explicit alpha channel or not, and they can be compressed using DXT compression, or uncompressed. They cannot contain objects or layers.

If the author of an FM has made custom textures for his mission, he would surely just include them in his .ibt file, and the game would then use them automatically?

Or is there a problem I'm not aware of? I haven't been following the editor thing at all.

If that IS a problem, the author could do the same thing I have done, but he could do it in advance; he could edit the texture names inside his .ibt, distribute the altered .ibt, and let his installer leave the textures in DynamicallyLoaded for the game to use.

Or, more likely, just make the FM .ibt file link to the DynamicallyLoaded folder directly when it comes to textures. But again, I haven't used the editor, so I don't know these things work.

GlasWolf on 21/8/2005 at 23:26

Quote Posted by potterr

From what I can gather the custom textures are the dds files (noticing that the Labyrinth FM has Labyrinth.dds and Schism has zSchismEntry.dds and zSchism.dds which I am presuming are the loading screen images), what exactly is a dds file (just a plain image file or a combination of layers and textures and objects) and how is it made?

There's some info in (http://www.ttlg.com/forums/showthread.php?t=99138) this thread too.

Bardic on 22/8/2005 at 09:26

I have a couple of questions about this. Hopefully most of it will make sense.

From what I've read here and another thread (http://www.ttlg.com/forums/showthread.php?t=97632&highlight=ibt+file)

The ibt contains the name of a texture, and the texture files (colormap and normalmap). Your installer goes through and hex edits the name of a texture to the name of your hi-res texture.

Does the installer only have to rename wall1.dds to wall1hires.dds once in each ibt that uses it? or say a mission has 10 rooms with all four walls set as wall1, the installer has to search through and change 40 instances of wall1 to wall1hires?

if it does have to change 40 or 50 instances for each texture, is that why you can't just have it search every texture in every ibt file?

Quote:

Can the Hex editor be made to delete the files instead of the names in an ibt?
if your hires textures were named the same as the original lowres ones and put into the dynamically loaded foder like they are now, and you deleted the textures out of the ibt, the game would have to go use the hires textures.

Also, if 90% of the regular missions (and future fan missions) used wall1, rather than having it written into every ibt the file could be in the dynamically loaded folder.

Lastly, the other thread I posted a link to said that ibts are cooked and perform 5x better. Have you found a huge decrese in performance, and do you think it is because the textures are higher quality, or because they are being dynamically loaded?

I'm just curious about all of this. Even if all the textures were in 1 big block in the ibt and the hex editor just removed all of them rather than individually selecting them it would save space per ibt. Oh, one thing about it, you wouldn't be able to undo this as easily as having your installer change the wall1hires back to wall1.

Shadowspawn on 22/8/2005 at 13:41

This relates to the work I've been doing on an IBT "compiler" so we don't have to ship massive IBT files with the FMs.

An IBT file contains a version of the .DDS file, but it looks like it's already loaded as a surface, or at least an in memory file. So far, the IBT textures seem to compose 60% to 80% of the IBT bloat, and probably why they don't compress well. (The .DDS is already somewhat compressed).

I could use this technique to dynamically load the texture, instead of banging my head repeatedly on the wall trying to figure out the format of the internal IBT DDS component.

I can easily parse the IBT file now, and could massively reduce the size by just fixing the texture component. Does this sound like a good approach?

Oh, in answer to a previous question, the textures are scattered throughout the IBT file. The IBT file contains headers in the first section, which basically define the resource, and an offset in the file of the location of the actual texture. Not only could I change the name easily in the header, I could drastically reduce the size of the file by eliminating or reducing the texture block size.

I need to try this out.

Dark Arrow on 22/8/2005 at 14:22

I have no idea what you wrote Shadowspawn, but from what I do understand, it sounds good. :angel: