Huge size of T3FMs, something needs to be done.

242 on 1/3/2006 at 11:11

I believe the most important problem of T3FMs right now - unecessarily huge size (I mean download size ;) ).
More and more missions are released and even though they are small in map size, they are like 50Mb or more each, people with dialup or limited traffic can't afford to download the same stuff they already have on their hard drives again and again. I remember there was a promise to do something with that problem. So, how it's going? It's important to do something soon while there isn't many FMs released.

New Horizon on 1/3/2006 at 14:52

Quote Posted by 242

It would require finding a way to strip the ibt's of their content and reassembling them on the users computer. That way, only 'custom' content needs to be shipped. I don't know how difficult this would be to program, but I don't suspect it would be a simple process. Coders can be very hard to find. I've been trying to find a coder to help us rewrite the GTKRadiant Level Editor for user with Dark Mod. We want it to be as familiar to the Thief community as possible. If I can't find someone soon, it's highly unlikely we will have a custom editor for Dark Mod and users will be forced to use Doom Edit. We lose, the community loses.

The other option for T3 FM's is to 'not' use blockloading, and require the 'user' to have then entire contents of the 'PcTextures' folder in their T3 installation. This could already be done and would likely be a one time download of 50 to 80 megs. Non-Blockloading increases load times of course...so preparte to brew a cup of tea while the level loads.

There doesn't seem to be any simple answer to T3 FM's. It's a poorly optimized engine, there really isn't much we can do. As many have said before, if Eidos would release the source code this could all be fixed, but we're never going to see the source code...for ANY of the Thief games. I've personally given up believing that Eidos would care enough to give this to the community. Most recently, Eidos has been selling Thief Deadly Shadows off it's website as a 'full game' 1 hour limited demo. You pay to get the restriction lifted.

1. They're selling the unpatched version. Since it's a new installer package, they obviously could have take the time to include the patched .exe.

2. They forgot to put the Videos folder in the installer package. The game is pretty pointless without a good number of those videos.

Sorry, I kind of got off topic....but as you can probably tell, I'm pretty pissed off at the way Eidos has been handling this wonderful series. They've show it and the community very little respect.

Shadowspawn on 1/3/2006 at 15:28

This is on my plate to do. I've done some research on the IBT format, and I think I can build an IBT "compressor", which will remove entries for textures (which seem to be the worst offenders) and other large packed objects.

Once "compressed", the IBT can re re-inflated once the FM is downloaded. GarrettLoader has provisions for running something like this.

All I need now is time! I haven't forgotten about this, work has just been killing me lately. The IBT uses something like 30-some odd types for textures, and I'm still parsing through them, trying to figure out which are the most commonly used and largest offenders to large IBT sizes.

Hey, at least I got a new laptop so I can work on this stuff when I'm on the road.

str8g8 on 1/3/2006 at 16:26

Yay for Shadowspawn! :D The biggest problem is not so much downloads (most people have, or have access to, broadband these days) so much as the hosting.

Krypt on 1/3/2006 at 19:02

If you just included the .unr (plus any custom content) and have the end user compile it and build the blockfiles themselves each map would probably only be a couple megs. That kind of sucks for the person downloading it, but that's really the only way you can get the size down.

I think I may have mentioned this before, but I suppose I will again since it might be relevant to this topic. You can cut down the size of a .unr file drastically for distribution if you use an autosave. Autosaves don't save out Flesh BSP or Navmesh data, which cuts out a huge amount of data and reduces the filesize. If anyone wanted to run it they would need to rebuild Flesh BSP and Navmesh, but at least the filesize is way smaller. Just turn on autosave, wait for it to save, then grab the Auto##.unr from the maps directory and rename it to whatever you want.

bukary on 1/3/2006 at 19:17

We, T3Ed users, can easily rebuild the missions etc., but it might be, I guess, a real problem for an average player.

Do you think that it would be possible to integrate some program that does all that rebuilding / reassambling into GarrettLoader? I wonder if such program is even possible... Let's hope Shadowspawn will, once again, move his magic wand... He's the one who did all those incredible things for dromedites. :thumb:

OrbWeaver on 1/3/2006 at 22:04

Quote Posted by New Horizon

I've been trying to find a coder to help us rewrite the GTKRadiant Level Editor for user with Dark Mod. We want it to be as familiar to the Thief community as possible. If I can't find someone soon, it's highly unlikely we will have a custom editor for Dark Mod and users will be forced to use Doom Edit. We lose, the community loses.

OT, but I might end up having a go at that myself if you don't find anyone more suitable. The design of the code doesn't look too bad and the stuff we want to do is not all that complex (no major rewrites of the graphics engine, for example). Even if we were just to fork the code and start stripping out the irrelevant Q2/Q3 stuff that would be a good start.

Quote Posted by Shadowspawn

Once "compressed", the IBT can re re-inflated once the FM is downloaded. GarrettLoader has provisions for running something like this.

My impression was that IBTs contain data that is already present in either PCTextures or the game IBTs, so maybe it is not even necessary to re-inflate them on the destination PC.

New Horizon on 1/3/2006 at 22:10

Quote Posted by OrbWeaver

That would be very cool. I didn't want to take anyone from within the team away from what they were doing, but if you would like to start working on GTK Radiant, that would be absolutely fantastic.

I'll talk to Sparhawk and see if he would be willing to setup a GTKRadiant repository on CVS.

Spog, of the Radiant team, has said that he will likely get a few of the Q4 features I mentioned into the current version, just not sure when. The main thing I'm after him to do is to make the light radius boxes resizable by dragging, just like Quake 4 Ed.

Sorry for the hijack. We'll take this back to Dark mod.

Komag on 1/3/2006 at 22:40

My firm impression from hosting a major Thief site and answering emails over the years and stuff is that the most important priority of all is to make it absolutely simple and idiot-proof on the user end. Having the end user upen up an .unr file and recompile the mission, generate the .ibt, etc, is just not realisitic, unless you want to severely limit those who will ever play the mission.

I feel that the current situation of having large files is worth it (from a hosting and downloading perspective) in order to make it as easy as it can be (and fast loading) for the end user. Plus, hosting is getting cheaper for more and more storage/bandwidth, and more and more people are getting broadband in one form or another, so I think the problem will continue to diminish.

But if some ibt stripper thing could work very smoothly and not require the end user to do anything extra, of course I'm all for it :thumb:

I would say though, that Krypt's suggestion of using an autosave file is perfect for collaborators, partners, teams, and possibly even beta-testers, working on a mission set, etc, to transfer stuff around to each other.

Shadowspawn on 1/3/2006 at 22:58

As I recall, there is a major benefit to loading via an IBT file, just because it's much faster.

I think I can generate a "CBT" (compressed IBT) file, which just provides filenames for the textures. GarrettLoader can run the inflator if it doesn't see an IBT with the distributed zip, and not run it again as long as the IBT exists. (potterr, some cleanup code might be needed to remove stale IBT).

Things are starting to ease off at work, so I can probably get to this pretty soon. It might not be perfect at the start, but if we can unload 70% of the bulk, it will be a good enough start. And it seems like the textures take up most of the space.