Thief 1/2 & SShock 2: DDFix and Enhanced Resolution Patch - discussion - by bikerdude
Hiatus on 3/4/2008 at 23:53
ok, as promised I just tested latest beta (1.3.5) for a while (~15 minutes), trying various things and I didn't notice anything wrong with it during my time-limited (but intense ;)) testing:
- no pixelated/sharpened object textures (like doors), fog works
- loading test 24-bit 1024x1024 .tga worked fine, no texture aliasing visible now (wow!) (made some new screens, maybe will post tomorrow at least some of them)
- custom resolutions work fine (tried 1280x960 on my CRT - will try 1152x864 etc tomorrow) - taking screenshots works fine with them; no performance hit noticed (other than usual)
- didn't notice any perf. issues with Thief2Extensions=1 (both when used for custom resolutions and for 32-bit tex loader)
- didn't test new gamma comp. option for ATI nor alt-tabbing but I never alt-tab in Thief anyway (maybe will test tomorrow)
Timeslip, do you want me to test for sth specific/combination not mentioned above?
X1950 + Cat 8.3 as usual, XPSP2
Impressive job overall as always :thumb:!
Timeslip on 4/4/2008 at 07:41
1.3.6 is up
Code:
Added support to the GUI for modifing gif and bmp textures, instead of just pcx
The GUI checks for the existence of bmps and dds's before warning about missing replacement textures
Removed gamma option again (It didn't work...)
Performance improvements when overriding multiple textures
I managed to convince .NET's image saving code to do most of the work for me, so I'm still living in blissful ignorance of how gif files work. :)
Quote Posted by 242
PS: Just tested it, the gamma fix actually doesn't work, with 1.1.1 in-game gamma never changes after alt-tab...
This may help: when I alt-tab with 1.1, the desktop is affected by Thief ingame gamma setting (same gamma), but with beta - it's not.
Woah. It keeps the gamma setting when you alt-tabbed to the desktop? That's an even worse bug than thief loosing its gamma. :wot:
In any case, I was explicitly setting the gamma ramp after each alt-tab, so if that doesn't work then ATi must be deliberately blocking it. (Presumably that was their way of fixing the desktop gamma issue... :rolleyes:) I've no idea why they don't do the same thing for DX9.
Quote Posted by Nameless Voice
EDIT: Now I remember what my other point was. Any chance of getting this to work in DromEd? It won't do new FM authors much good if they can't see the new textures they're using in the editor...
Depends. Does dromed use the full dark engine? If so, getting it to work with dromed wouldn't be much harder than getting it to work with SS2. I thought it had some software based renderer thingy though...
Quote Posted by ZylonBane
Could someone please recap exactly what the current mechanism is for informing ddfix that a terrain texture has been replaced with a high-res equivalent?
* Create a new texture, and save it as a .tga, .bmp or .dds. A tga or bmp must be in either R8G8B8 or A8R8G8B8 format. With a dds you get a little more freedom, but it still needs to be 24 or 32 bit. If using a dds, you need to pregenerate the mipmaps. For tga or bmp ddfix will handle it for you.
* Put the new texture in '\res\ddfix\'. You can use subfolders, but the total length of the path between ddfix\ and the file extension must be less than 128 characters.
* Get the .pcx, .gif or .bmp that you want to replace out of whatever archive it's in. Open the GUI, click 'replace texture', enter the path of the new file relative to the ddfix\ folder, (don't include the extension,) click replace and browse to the texture you want to replace. (Overwritten textures must be a minimum of 16x16 in size.)
* The GUI will have created a new file in the same place as the overwritten one with a .override extension. If you want to pack the file back into an archive, you need to rename it back to its original name and overwrite the original texture with it. If the file is loose inside the res folder, you can leave both files as they are, and ddfix will force thief to preferentially load the .override file.
Hopefully that's everything. :)
Quote Posted by Hiatus
- didn't notice any perf. issues with Thief2Extensions=1 (both when used for custom resolutions and for 32-bit tex loader)
I don't notice any either, but fixing the problem with loosing mipmaps on alt tab required blocking thiefs texture cache and forcing it to recreate textures from scratch. That'll definitely reduce performance: it's probably not noticeable on anything modern, but you might see see some slowdown in loading speed on something much older. If no-one complains about performance issues with it in the next few days, I'll remove that warning.
Quote Posted by Hiatus
Timeslip, do you want me to test for sth specific/combination not mentioned above?
The last two things I want to test are replacing object textures, and replacing a large number of the textures in a mission. I was in the middle of batch replacing all textures, but I'll have to do some learning before I can test objects. (e.g.
Where are the actual gifs? Packed into the .bin files? Edit: Bah. Why on earth are they in a folder called
txt. :weird:)
Edit: I batch replaced all textures used in mission 1, and it looks like there's some world textures that aren't being replaced correctly. Judging from the shape of sections that aren't working, it looks like it might be a problem with non-square textures. :confused:
Edit2: There's definitely a problem with textures which are higher than they are wide. I've probably got the width and height variables backwards somewhere.
242 on 4/4/2008 at 08:06
Quote Posted by Timeslip
Woah. It keeps the gamma setting when you alt-tabbed to the desktop? That's an even worse bug than thief loosing its gamma. :wot:
In any case, I was explicitly setting the gamma ramp after each alt-tab, so if that doesn't work then ATi must be deliberately blocking it.
Without the fix the game also loses gamma setting after alt-tab, so it's not your fix's fault after all.
In any case, I think you should keep to host 1.1 for download on your site, as Thief with current drivers is barely playable at best with 1.2+ on newer Radeon cards, and it's not known will this be fixed in feature drivers or not.
Hiatus, PM me your email, I'll send you the installer version of 1.1.1
Hiatus on 4/4/2008 at 08:09
Quote Posted by Timeslip
replacing a large number of the textures in a mission
by large you mean how many? 20? 30? more? BTW: *where* FM authors should put their HR *custom* textures (i.e. their own ones, not replacements for stock textures from shipped *.crf's)? Normally in [game_root_dir]\fam, overriding everything else in load path order and ddfix will implicitly load these custom HR textures (even though they are NOT any replacement for anything, just new resources)? Just making sure.
I wonder how long it takes before we start seeing HD editions of existing/already released FM's appearing :). (Not to mention new FM's using HD textures, of course).
Quote Posted by Timeslip
Bah. Why on earth are they in a folder called txt
yup, gifs are in \txt
and also in \txt16 sometimes (don't know why); bin file contains an ascii "reference" to related .gif (file name in ascii near start of file)- I'm sure others will give you more info on how it works exactly. Mesh.crf and obj.crf are used (I think mesh.crf is mainly for character/creature models and obj.crf for other models, but not sure here).
Quote Posted by Timeslip
Depends. Does dromed use the full dark engine?
AFAIK yes - hardware accelerated mode should be available in DromEd/ShockEd (at least in some form). And I've seen pre-patched ddfixed DromEd.exe (version for T2 and version for TG) posted here (ddfixed ShockEd.exe as well for that matter) I think so ddfix definitely has similar use in editor as in regular game.
Quote:
Edit: I batch replaced all textures used in mission 1, and it looks like there's some world textures that aren't being replaced correctly.
any screen(s) with large number of textures replaced, please :)?
re gamma on ATI (1xxx at least): in latest beta I can alt tab from and to game and both the game and desktop retain their gamma setting. So it looks like another thing broken for HD 2xxx/3xxx owners in ATI drivers :/.
I actually "alt-tab" from game using Win key (the one between left ctrl and alt), because I bound alt to 'move backward' in game, so alt-tabbing to desktop doesn't work for me - on purpose (alt-tab from desktop to game works of course). Here I got a small feature request: could you block Win key while in game so it's not possible to accidentally go to desktop by hitting it by mistake (it invokes start menu)? It's immersion breaker when you play in dark room and hit it by mistake and bah, you're sent to desktop. Best would be to make it an option/toggle in .ini. Is it possible to do with ddfix?
@242: PM sent. TIA for 1.1.1 installer.
Nameless Voice on 4/4/2008 at 08:51
Quote Posted by Timeslip
Depends. Does dromed use the full dark engine? If so, getting it to work with dromed wouldn't be much harder than getting it to work with SS2. I thought it had some software based renderer thingy though...
DromEd uses the full Dark Engine in game mode, but only software rendering inside the editor window itself. Even if you could only get it to load the textures in game mode, that would already be great.
(If you could make the 3D view render in proper hardware, that would be insanely awesome, but I doubt you'd manage that...)
Quote Posted by Timeslip
Edit: Bah. Why on earth are they in a folder called
txt. :weird:)
Txt. For
te
xtures. Seems pretty obvious to me. You wouldn't want to keep your textures in the same folder as the .bins, that would be extremely messy.
But, txt is a bit obsolete. Thief 1 originally used only txt because every texture needed to use the same 256-colour palette (for software rendering). Since textures in Thief 2 and SS2 can each use their own palette, they go into txt16 (for 'textures to be used in 16-bit colour mode'). Note that DromEd's in-editor software renderer displays txt16 textures correctly.
Any textures in txt which do not use the standard palette will be made to use the standard palette, in by index conversion - their palettes are completely ignored.
Note that there are still a lot of textures in txt, mainly ones left over from Thief 1, but also it makes sense to use it if your object looks good with that palette, because of Thief's ~250 palette limit.
Timeslip on 4/4/2008 at 09:13
Quote Posted by Hiatus
by large you mean how many? 20? 30? more?
any screen(s) with large number of textures replaced, please :)?
I meant
all of them. :p
I've got the non-square textures problem fixed now, and apart from the skyline, (which uses alpha, so I knew it wouldn't work,) (
http://img257.imageshack.us/img257/1722/screenshot2qv5.jpg) everything seems (
http://img257.imageshack.us/img257/6159/screenshot3uw4.jpg) to be (
http://img257.imageshack.us/img257/919/screenshot4ua7.jpg) working, so I can consider the stress test safely passed. (For the record, replacing every world texture in the first mission with a 1024x1024 tga adds about a second to the loading time and causes no fps drop in game on my E6600/8800 GTS.)
Edit:
Replacing object textures doesn't seem to work yet. I suspect they get loaded in a different place to the world textures, so I'm missing them atm.
Quote Posted by Hiatus
BTW: *where* FM authors should put their HR *custom* textures (i.e. their own ones, not replacements for stock textures from shipped *.crf's)? Normally in [game_root_dir]\fam, overriding everything else in load path order and ddfix will implicitly load these custom HR textures (even though they are NOT any replacement for anything, just new resources)? Just making sure.
It doesn't matter. As long as they're somewhere where thief will see and load them, ddfix should spot that they need to be replaced. The only cavet is that the .override trick only works for loose files, and wont work for files packed into an archive.
Quote Posted by Hiatus
AFAIK yes - hardware accelerated mode should be available in DromEd/ShockEd (at least in some form). And I've seen pre-patched ddfixed DromEd.exe (version for T2 and version for TG) posted here (ddfixed ShockEd.exe as well for that matter) I think so ddfix definitely has similar use in editor as in regular game.
Sorry, I wasn't very clear on that one. I thought that there were seperate editing and game modes, and that the editing mode used a different renderer.Edit: Answered already. :)
Quote Posted by Hiatus
I actually "alt-tab" from game using Win key (the one between left ctrl and alt), because I bound alt to 'move backward' in game, so alt-tabbing to desktop doesn't work for me - on purpose (alt-tab from desktop to game works of course). Here I got a small feature request: could you block Win key while in game so it's not possible to accidentally go to desktop by hitting it by mistake (it invokes start menu)? It's immersion breaker when you play in dark room and hit it by mistake and bah, you're sent to desktop. Best would be to make it an option/toggle in .ini. Is it possible to do with ddfix?
Yes, filtering out the windows key is easy enough. Will add for .8. :)
Edit:
Quote Posted by Nameless Voice
Txt. For textures. Seems pretty obvious to me.
I suppose so. I just automatically associate 'txt' with 'text'. :(
Hiatus on 4/4/2008 at 10:03
Quote Posted by Timeslip
It doesn't matter. As long as they're somewhere where thief will see and load them, ddfix should spot that they need to be replaced. The only cavet is that the .override trick only works for loose files, and wont work for files packed into an archive.
so FM authors still *have* to include standard low res .pcx/.gif etc (for vanilla T2) and associated *.override file? Putting *only* HR .tga/.dds etc version (for whatever reason) w/o low-res .pcx/*.override pair won't work? (ddfix won't load hi-res texture by itself, w/o being pointed to it via *.override (with loose files) method? HR textures should be always placed in ..\ddfix (incl. subdirs), no matter where put in search path?
Quote Posted by Timeslip
I've got the non-square textures problem fixed now, and apart from the skyline, (which uses alpha, so I knew it wouldn't work,) everything seems to be working, so I can consider the stress test safely passed. (For the record, replacing every world texture in the first mission with a 1024x1024 tga adds about a second to the loading time and causes no fps drop in game on my E6600/8800 GTS.)
looks good :D! Have you checked how video/system memory (by thief2.exe) usage rises by replacing all textues on that pretty small level (just curious)?
Quote Posted by Timeslip
Yes, filtering out the windows key is easy enough. Will add for .8.
would be fine - TIA!
Timeslip on 4/4/2008 at 10:26
Quote Posted by Hiatus
so FM authors still *have* to include standard low res .pcx/.gif etc (for vanilla T2) and associated *.override file? Putting *only* HR .tga/.dds etc version (for whatever reason) w/o low-res .pcx/*.override pair won't work? (ddfix won't load hi-res texture by itself, w/o being pointed to it via *.override (with loose files) method? HR textures should be always placed in ..\ddfix (incl. subdirs), no matter where put in search path?
Yes. Thief needs to see a valid file, so you still need an 8 bit .pcx or .gif even on completely new textures. :(
You don't need a pcx/override pair though; just the override file with the .override extension removed will work fine. The .override thingy was just so that you could replace existing textures without having to overwrite the old ones. That's not going to be an issue for new textures.
Edit:
Quote Posted by Hiatus
looks good ! Have you checked how video/system memory (by thief2.exe) usage rises by replacing all textues on that pretty small level (just curious)?
No, I hadn't actually. I just gave it a check, and the results are every bit as bad as you might expect from replacing every single world texture in the game with something with 4x the bit depth and a 1024x1024 resolution. System memory usage jumped from 60 to 657mb, and video memory usage jumped from 88 to 172mb. That's actually better than I was expecting; replacing a 64x64x8 world texture with a 1024x1024x32 one requires 64 times the amount of memory in system ram and 32 times in video ram, so the 10x/2x multipliers aren't all that bad. :)
There's no way thief should have been using 88mb without any extra textures though. The extra framebuffers used by ddfix are responsable for about 16mb of it at 1280x1024 resolution, but even so usage shouldn't have been that high. Windows was probably using the rest. :confused:
I could cut system memory usage by about half by storing textures in 24 bit and not generating mipmaps until they got loaded into video ram. That would probably cause stuttering when the textures were loaded into video memory though. If anyone tries writing a whole game texture replacer at that sort of resolution, I'll add it in as an option.
Hiatus on 4/4/2008 at 11:13
thanks for these numbers... that 88MB vid mem is quite strange indeed, though. How do you check vid/sys mem usage in any given moment? With Rivatuner/Everest/sth else (custom written maybe)?
Quote Posted by Timeslip
I could cut system memory usage by about half by storing textures in 24 bit and not generating mipmaps until they got loaded into video ram. That would probably cause stuttering when the textures were loaded into video memory though. If anyone tries writing a whole game texture replacer at that sort of resolution, I'll add it in as an option.
luckily, system RAM is so cheap nowadays it (bigger sys RAM usage for improved performance) shouldn't be a big issue I think :). I'd be more concerned with Thief hitting 2GB address space allocation limit/barrier for 32-bit processes (or sth like that - described here (
http://www.anandtech.com/gadgets/showdoc.aspx?i=3034) ) on some huge missions with lots HD textures. 64-bit .exe version would be helpful here (Thief source code anyone ;)?)
Quote Posted by Timeslip
Yes. Thief needs to see a valid file, so you still need an 8 bit .pcx or .gif even on completely new textures
but on the other hand, this implicit loading mechanism is useful for keeping compatibility for those with vanilla T2 (so 2 versions of FM's - 1 HD and 1 normal - are not needed).
Timeslip on 4/4/2008 at 11:47
I'm not having any luck with objects. Their creation is handled by a whole different section of code, so to override them I effectively need to start everything again from scratch. Probably going to take another few weeks. :(
Quote Posted by Hiatus
thanks for these numbers... that 88MB vid mem is quite strange indeed, though. How do you check vid/sys mem usage in any given moment? With Rivatuner/Everest/sth else (custom written maybe)?
ddfix sits between thief and dx, so all I need to do is call IDirectDraw4->GetAvailableGraphicsMemory. ;)