Crispy on 14/1/2006 at 02:02
Quote Posted by New Horizon
I'll likely throw a custom schemameta file up for download, so people who only want to play the missions properly don't have to download the full editor.
How big will this be? Are we talking hundreds of megabytes, dozens, or just a few? If everyone who wants to play an T3EnhancEd FM has to download it, we don't want it to be too big...
Quote:
Barring any major issues, I should be able to have the installer ready for a Sunday release. I won't make an absolute promise on that, but cross your fingers.
Cool! Don't rush needlessly, I'm sure we'd all rather have it done properly than done quickly. :)
New Horizon on 14/1/2006 at 05:12
Quote Posted by Crispy
How big will this be? Are we talking hundreds of megabytes, dozens, or just a few? If everyone who wants to play an T3EnhancEd FM has to download it, we don't want it to be too big...
Yeah, I'm not exactly sure how I should handle it. Damn T3 for being so unfriendly to modding.
Quote:
Cool! Don't rush needlessly, I'm sure we'd all rather have it done properly than done quickly. :)
Heh,heh. Yeah, I hear you. It's just that at this point, I want it done quickly. My energy is as thin as a thread.
Crispy on 14/1/2006 at 10:12
Quote Posted by New Horizon
Yeah, I'm not exactly sure how I should handle it. Damn T3 for being so unfriendly to modding.
Aye.
It occurs to me that this would be a perfect problem for binary patching to solve - assuming that most of the schema metafiles stay exactly the same, it should be possible to generate a patch that just updates the data that's changed.
I did a bit of Googling and found this app: (
http://sites.inka.de/tesla/f_others.html#bsdiff)
(Download: (
http://sites.inka.de/tesla/download/bsdiff4.2-win32.zip))
You will also need bzip2 in your PATH in order to run it, which you can get here:
(
http://gnuwin32.sourceforge.net/packages/bzip2.htm)
(Download: (
http://prdownloads.sourceforge.net/gnuwin32/bzip2-1.0.3-1.exe?download))
Then you could redistribute bspatch with the installer (both the editor and a separate installer for players) and automatically update their existing metafiles. Provided they haven't modified their metafiles already, it should work fine...
I just tried it myself to generate a patch between the original gamesys and the one for my C6 mission - installed bzip2 and then just chucked bsdiff and bspatch into its "bin" folder:
Code:
D:\Apps\bzip\bin>bsdiff
usage: bsdiff oldfile newfile patchfile
D:\Apps\bzip\bin>bspatch
usage: bspatch oldfile newfile patchfile
D:\Apps\bzip\bin>bsdiff D:\Games\Thief3Edit\System\T3Gamesys_original.t3u D:\Games\Thief3Edit\System\T3Gamesys.t3u t3gamesyspatch
D:\Apps\bzip\bin>bspatch D:\Games\Thief3Edit\System\T3Gamesys_original.t3u D:\Games\Thief3Edit\System\T3Gamesys_patched.t3u t3gamesyspatch
D:\Apps\bzip\bin>
I ended up with an identical duplicate of T3Gamesys.t3u called T3Gamesys_patched.t3u; and the patch file, D:\Apps\bzip\bin\t3gamesyspatch, is only 86 KB. That's less than 1/20th of the size of the gamesys file itself. I imagine the savings for metafiles would be even better; both in absolute and percentage terms.
Hey potterr, any chance of getting this kind of functionality into GL? Imagine zipping up an 86 KB T3Gamesys_patch.t3u file instead of a 2024 KB T3Gamesys.t3u... could be easily applied to other files too. Metafiles being a case in point; heck, even OM IBTs.
Quote Posted by New Horizon
Heh,heh. Yeah, I hear you. It's just that at this point, I want it done quickly. My energy is as thin as a thread.
Fair enough. You've put a lot of effort into this; don't fade on us now! :)
New Horizon on 14/1/2006 at 16:18
Hmmm, so how does this work? Does it simply compare both files and extract the differences somehow? Just not 100% sure I understand.
Crispy on 15/1/2006 at 00:03
Quote Posted by New Horizon
Hmmm, so how does this work? Does it simply compare both files and extract the differences somehow?
Exactly. It finds the difference between two files and produces a "patch" that can be used to turn the first file into the second file.
So say you added a single sound to the DVD1 metafile (SchemaMetafile_DVD1.csc). The sound itself only takes up 1 MB, but the updated metafile is 201 MB. Normally you'd have to distribute the ENTIRE 201 MB file just because of that single change. But the bsdiff program figures out that only that 1 MB has changed, so that's all the information you need to turn the original SchemaMetafile_DVD1.csc into the shiny new T3EnhancEd SchemaMetafile_DVD1.csc.
It produces a ~1 MB patch file which summarises these changes. You distribute the patch file and bspatch.exe with an installer. The installer finds SchemaMetafile_DVD1.csc (the original metafile) on the player's hard drive, and runs bspatch.exe against it and the 1 MB patch. Tada! Your updated 201 MB metafile is present on the user's system, but you only had to send 1 MB of it.
New Horizon on 15/1/2006 at 00:51
Sweet.
I'm just trying it out on my system right now. The program is pretty memory intensive, so the DVD_n files are taking an uber long time to work through.
I have a gig of memory and it has gone well beyond that into virtual memory. I can hear my poor hard drive cranking. :) It will be truly sweet if I can do this. Great find. :)
Crispy on 15/1/2006 at 01:14
Heh, yeah, they'll do that. The patch generating process is pretty inefficient; the readme admits as such. Luckily applying the patch isn't quite as bad; it should only use a few hundred megabytes of memory for patching each metafile. (Yes, "only". :rolleyes:)
New Horizon on 15/1/2006 at 05:09
Quote Posted by Crispy
Heh, yeah, they'll do that. The patch generating process is pretty inefficient; the readme admits as such. Luckily applying the patch isn't quite as bad; it should only use a few hundred megabytes of memory for patching each metafile. (Yes, "only". :rolleyes:)
Sadly, I had to stop it. I had left it running for hours on the DVD1 file and it never finished. I might try to slap in a bit more ram and try again.
New Horizon on 16/1/2006 at 15:37
You might have noticed that the enhanced editor hasn't been released yet. Heh, some bumps. Mainly that I'm sick as a dog. I'll pick at it as much as I can and get it out there. Sorry guys.
OrbWeaver on 16/1/2006 at 16:16
Quote Posted by Crispy
So say you added a single sound to the DVD1 metafile (SchemaMetafile_DVD1.csc).
You can't add a single sound to any of the DVD metafiles without updating the HardDrive metafile which contains the metadata. In this case you would be better off adding the sound directly to the HD metafile (which IIRC is about 60 megabytes in size) and distributing this.
I have noticed also that there is a 1-byte field in the metafile header called something like "Use metafile", although I can't remember the specifics off the top of my head. It would be interesting to flip this to 0 (it is set to 1 automatically by the compile script) and see if the game could load sounds directly from the folder structure.
