Question for John P (kind of)

John P. on 22/8/2005 at 14:52

Quote Posted by Bardic

I have a couple of questions about this. Hopefully most of it will make sense.

From what I've read here and another thread (http://www.ttlg.com/forums/showthread.php?t=97632&highlight=ibt+file)

The ibt contains the name of a texture, and the texture files (colormap and normalmap). Your installer goes through and hex edits the name of a texture to the name of your hi-res texture.

It doesn't really matter what you rename the texture to(inside the .ibt) - you could rename wall1.dds to donald_duck.dds, and put wall1.dds in the DynamicallyLoaded folder, and the game would then load wall1.dds from DynamicallyLoaded. I usually just change one single letter in the name, usually the first letter.

Quote Posted by Bardic

Does the installer only have to rename wall1.dds to wall1hires.dds once in each ibt that uses it? or say a mission has 10 rooms with all four walls set as wall1, the installer has to search through and change 40 instances of wall1 to wall1hires? If it does have to change 40 or 50 instances for each texture, is that why you can't just have it search every texture in every ibt file?

There's only one instance of a given texture name inside an .ibt. -But a couple of times I have had to be more careful, because there have been objects/models or possibly something else using pretty much the same name, so when I have made my installer change the first letter automatically, I have inadvertently changed the name of something I shouldn't have. Never in an installer I have actually released though - I have figured it out while working on them.

The reason I did all the manual work with checking what .ibt had what texture, was also because some of my installers let you choose which changes to apply. So if you only chose ONE texture change, my installer will only open the .ibt where that texture is, and only change that .ibt (or more, if the texture is in more than one .ibt). If the installer would check all the .ibts every time, it would as mentioned take ages to install(it already does with my largest packs in spite of this process).

BUT - had I made the large installers in such a way that you didn't have any choice of what to install, then the install process would have been much faster, because then the installer would of course change the names of all the textures in each .ibt file it had opened. As it is now, the installer opens an .ibt, changes a name, and if you have chosen to install several texture options, the same .ibt will be opened, changed and closed several times, one for each texture option.

Quote Posted by Bardic

Can the Hex editor be made to delete the files instead of the names in an ibt?
if your hires textures were named the same as the original lowres ones and put into the dynamically loaded foder like they are now, and you deleted the textures out of the ibt, the game would have to go use the hires textures.

I have never tried this, mostly because my installers need to be able to put the game back to its original state, but I guess you could do that. Since the name can be changed to whatever, I guess the whole name could just as well be deleted. But I don't know if that would accomplish anything; the game still will look for the texture inside the .ibt first. -Unless you altered the model/mesh so that it pointed directly to the DynamicallyLoaded folder for its texture. Then maybe it would speed up the loading process a bit. I'm just theorizing here though, haven't tried anything like that.

Quote Posted by Bardic

Also, if 90% of the regular missions (and future fan missions) used wall1, rather than having it written into every ibt the file could be in the dynamically loaded folder.

Yes, I would think so (I like to make 'clean' installers though; when you've uninstalled my textures(properly), there shouldn't be any textures or other files or registry entries left on your machine from that install).

Quote Posted by Bardic

Have you found a huge decrese in performance, and do you think it is because the textures are higher quality, or because they are being dynamically loaded?

I think it's both. More data to load = longer time to load. More data to process = slower performance.

Quote Posted by Bardic

I'm just curious about all of this. Even if all the textures were in 1 big block in the ibt and the hex editor just removed all of them rather than individually selecting them it would save space per ibt. Oh, one thing about it, you wouldn't be able to undo this as easily as having your installer change the wall1hires back to wall1.

Exactly - I'm doing it this way mostly to be able to properly uninstall the textures afterwords.

Shadowspawn on 22/8/2005 at 15:28

Quote:

Originally Posted by John P.
It doesn't really matter what you rename the texture to(inside the .ibt) - you could rename wall1.dds to donald_duck.dds, and put wall1.dds in the DynamicallyLoaded folder, and the game would then load wall1.dds from DynamicallyLoaded. I usually just change one single letter in the name, usually the first letter.

I take this to mean that you basically "broke" the IBT in a controlled way, and in doing so the game looks for textures it can't find in the dynamically loaded folders? If this is so, I can just strip texture headers and texture data blocks out of a custom IBT and it should dynamically load them?

This would make the IBT compiler alot simpler, if people can live with the performance hit...

Bardic on 22/8/2005 at 16:24

So it does sound like we might be able to strip 15 or 20 megs of textures out of a fan mission ibt and release one with textures and one without for those people that have the editor and have already copied all the normal textures into the dynamically loaded folder.

Any FM that worked with a stripped ibt and the normal textures, we could copy your textures replacing the regular ones and have it work in the FM also.

That way we could also do a performance comparisson of:
ibt
stripped ibt dynamically loading normal textures
stripped ibt loading hires textures.

Thank you so much for the quick reply, we really appreciate all the textures you've done, they enhance the levels quite a bit.

I suppose I should probably go to work now. Later guys.

John P. on 22/8/2005 at 16:53

-Forgot to mention that I did come across a problem once; there was a space I didn't notice in the script file I used to batch hex edit the .ibts with, so that it would replace the first letter in the texture name like I "told" it to, but it would also append a space after the texture name. This resulted in a crash when that .ibt was being loaded. As long as I don't introduce any new spaces, it doesn't seem to matter if the name is longer than the original though.

For the game DX:IW, we did much the same thing, as they work in a very similar way. The most used texture override program we used there, would append _overridden to the texture names we wanted to replace inside the .ibd files, and that doesn't cause any problems.

So - I think I'm right when I say that you could rename the texture to whatever you like, but there are still some limitations. Exactly why introducing a space 'ruins' the .ibt, I have no idea, but that is what I've experienced.

Shadowspawn on 22/8/2005 at 17:13

Some (not all) of the texture names have a size in the word before the string. If you lengthen the string without changing that size, it could cause problems.

Or, possibly, later on the game parses the name and looks for whitespace. The space could cause problems there also.

potterr on 22/8/2005 at 17:49

Quote Posted by Shadowspawn

I can easily parse the IBT file now, and could massively reduce the size by just fixing the texture component. Does this sound like a good approach?

Oh, in answer to a previous question, the textures are scattered throughout the IBT file. The IBT file contains headers in the first section, which basically define the resource, and an offset in the file of the location of the actual texture. Not only could I change the name easily in the header, I could drastically reduce the size of the file by eliminating or reducing the texture block size.

WoooWhooo Thats where I was heading with this...

Quote Posted by Shadowspawn

Again where I was going with this, getting the ibt file sizes down by stripping out extra stuff that is not required but added still, having a STANDARD custom texture pack that is located as dds files in teh dynamicallyloader folder (I actually found some command line exes that could be used within GarrettLoader to convert jpg files to dds files for authors, or simply by GarrettLoader itself as part of an install). The FM package though would require a flag of some kind to indicate what texture pack must be installed (which leads onto another problem of who will be able to create standard texture packs as say 50mb installs...we are not talking replacement ones here, more additional textures that can be used).

Quote Posted by Bardic

See point above (taking this thought to the next stage for custom texture packs).

Dark Arrow on 22/8/2005 at 18:05

I have always understood that we would be aiming for a texture release of the editor that players could download. Then the textures would never need to be downloaded again with the FMs, unless the author added some custom textures.

potterr on 22/8/2005 at 18:28

Quote Posted by Dark Arrow

That's true but it was not really understood how to do it. I thought that as John P's installers actually changed the ibt files themselves it would be a good base to work from for this process. The main reason for this is to partly work out how to do it in a standard way and partly if I need to define some funtions in GarrettLoader to do this.

Bardic on 23/8/2005 at 04:38

I love good problem solving.

Correct me if I am wrong on any of this.

Currently you can download John P's textures and install them and it will place the hi-res textures into the dynamically loaded folder. You can then copy those into your Thief3edit PCtextures folder and when you create your ibt they will be included automatically. They don't even have to be in there when you start building your mission, just added before you compile the final product (correct?). So John P doesn't need to make new installers that search through FM ibts because they will most likely already have them cooked in.

Do we have John P's textures in a zip without the installer, then for people who only want to create FMs with them you don't have to go through the process of patching all the regular mission ibts (which you may not normally play anymore) Just copy them into the Thief3edit folder over the low-res textures.

On to the ibt files and I guess Shadowspawn knows most of this. Depending on which methods are possible...

The simple solution is to have the utility parse the ibt, change the names, and delete as much of the texture block as it can safely. We know this will work, but the performance hit might be a bit high. For one thing, instead of making the game dynamically load the 225 or so custom textures that are in John's pack, it will have to load all 1,899 textures dynamically.

Another way would be to take those texture blocks out and save them either into 1 big file, or into 1,899 .hex files or something. The parser could be run by the designer to remove any texture blocks before the mission was sent out for release, and garretloader could add them back in when a fan mission was loaded. That way we wouldn't have to figure out how to convert a .dds file into the "in memory" block in the ibt.

I suppose most of the second option depends on the block of data for a particular texture being identical in 2 different ibt files. Would an FM with say 200 identical cubes or 1200 flat planes and each wall painted with the 1200 largest textures help in easily examining a stripped down ibt? Just throwing out ideas. Thinking outside the box is fun :D

Crispy on 23/8/2005 at 08:13

Quote Posted by Bardic

Another way would be to take those texture blocks out and save them either into 1 big file, or into 1,899 .hex files or something. The parser could be run by the designer to remove any texture blocks before the mission was sent out for release, and garretloader could add them back in when a fan mission was loaded. That way we wouldn't have to figure out how to convert a .dds file into the "in memory" block in the ibt.

I like that idea. No, scratch that - I like that idea a lot. :) Treating it as "black box" data makes a lot of sense, seeing as it's apparently so bothersome to figure out.

'Course, those .hex files probably won't compress well, but it's still an improvement on the current great-big-whopping-IBT situation. And if the .dds files are significantly smaller, the dial-up users could just take the speed hit of DynamicallyLoaded (after all, they're used to waiting :ebil: ) - it's only the loading speed that's affected, right? Once the DynamicallyLoaded textures are in memory, they're just as fast as any other texture.

Unless TDS actually swaps textures in and out of memory during the level, but I don't see that happening - there doesn't seem to be much point.

Quote:

I suppose most of the second option depends on the block of data for a particular texture being identical in 2 different ibt files.

I'd be surprised if that wasn't the case - but Shadowspawn is probably in a better position to answer this than I am.