vfig on 10/12/2022 at 17:49
ah fuck it
ever since i went to collect this stuff to dump it here, it has got back in my head and wont leave
guess i should try to finish it before i die, huh
R Soul on 10/12/2022 at 19:49
I suggest focussing on what parts help to tell the FM's story and ruthlessly cutting out the rest. Beware the sunk cost fallacy!
gamophyte on 10/12/2022 at 20:05
Yeah I'm watching your lets plays. It's hard not to get some inspiration from that too.
vfig on 10/12/2022 at 21:20
Quote Posted by R Soul
I suggest focussing on what parts help to tell the FM's story and ruthlessly cutting out the rest. Beware the sunk cost fallacy!
oh sure, but scoping has never been the issue. never had a problem cutting things that are better off cut.
Quote Posted by gamophyte
Yeah I'm watching your lets plays. It's hard not to get some inspiration from that too.
yeah, thats why when i put the fm on the shelf, i had to stop playing other fms too.
vfig on 31/12/2022 at 16:17
dec 21-30, 2022grief in stages: after the detail pass on the mansion, the mission hit 78238 raw cells, and would not even portalize! this was a big surprise to me, because its not all that big, even taking into account the three copies for the different time periods. and i had reached this particular layout after cutting down in size
twice from the original blockout. but what can you do?
i started out by bargaining: if i amputated whole sections of the mansion, could i save enough cells to get it to portalize again?
deleting the dining hall saved ~3000 cells (as measured when portalizing just the nonruined mansion section):
Inline Image:
https://i.imgur.com/3daLTvu.pngInline Image:
https://i.imgur.com/dyCX7YU.pngdeleting the colonnade saved ~1500 cells:
Inline Image:
https://i.imgur.com/M7RDeHS.pngInline Image:
https://i.imgur.com/lyv0Evi.pngboth were significant losses to the visual appearance of the mansion, but neither loss would be catastrophic for gameplay. and even then, the savings were not looking significant enough to save the mission: bearing in mind that the ruined version of the mansion still needed much of the interior detailing brushwork to be carried across and adapted into ruined form, which will eat more cells.
i almost despaired entirely again. but then decided to move onto the denial stage. i
would have my cake, er, mansion and eat, er ruin it too! back in 2021 i had written a blender addon that would import the worldrep of a mission (including lightmaps, so that unlike the newdark dromed's .obj export, the exact in-game appearance of the mission could be replicated); and that meant i had some understanding of the .mis file format and how the worldrep was stored.
(the worldrep, if youre unfamiliar with the term, is the output from portalize/optimize. its a bunch of cells—convex polyhedral volumes—that make up the entire space of the level, together with the portals that connect them and the polygons that are rendered to the screen. in addition, the worldrep also contains a bsp tree of the level that is used for spatial queries and physics and stuff. and the worldrep also contains some extra information thats used only by dromed to identify which polygons were generated from which brushes, which is needed so you can select brushes in its 3d view.)
so i came up with the idea of writing a tool to merge the worldreps from multiple .mis files together into one. after all, each of the three parts of my mission, when isolated by an area and optimized, was around 6000-8000 raw cells, and optimized down to 2000-3000 cells. the three parts of the level are completely isolated, so i wouldnt need to do any shenanigans around changing cell faces into portals or anything like that. and the bsp tree would just need to get a couple extra nodes at the very top, with each of the input .mis's bsp trees as separate branches below.
in theory, pretty doable.
well, progress on that was slow. the .mis format—the worldrep specifically—is only partially understood as far as public knowledge goes. many others have done work with it before (e.g. for the OpenDarkEngine project, and PinkDot's editor). and with the t2 source code to read, the olddark worldrep structure was at least
knowable in all its details (though understanding it all is a different story!). newdark though made a number of changes to the worldrep data, only some of which are known. outputting a non-crashing .mis is going to require figuring out those unknowns, too.
as of dec 30, after some false starts, i had my code successfully reading both olddark and newdark worldreps entirely. but hadnt yet made a start on writing it back out unchanged (straightforward) or the actual merging (the hairy part with lots of exciting crashes to come).
vfig on 31/12/2022 at 16:31
dec 31, 2022understanding binary data and writing code to manipulate it is a much more mentally intensive task than faffing about in dromed, and i struggle a lot to find impetus to work, and to keep focus when i have found some. so the going has been slow. i have been playing more fms in the past ten days, which, though it eats a lot of time, is better than anything else i have found in getting my brain into gear to actually work. and hey, that gives me more lets play episodes to post on my youtube channel.
today while finishing up the last couple objectives of an fm, and with convex cells and bsp planes still floating around my mind, i wondered what would happen if instead of stacking the three parts of the mission vertically, one directly above the other, i offset them horizontally? this would prevent the vertically-oriented planes of the brushes in one area being shared with the correlated brushes in the other areas, which—... well, it would definitely have
some impact on the csg process, though exactly what wasnt obvious.
but it is a trivially easy experiment to try, so. this is the vertically stacked layout that wont even portalize:
Inline Image:
https://i.imgur.com/mH69qc7.pngthen shoving the top and bottom layers over in X and Y to remove the overlap... resulted in a raw cell count of 69091, csgmerge input of 32521, and optimized output of 8308 cells.
it actually portalizes again!Inline Image:
https://i.imgur.com/ZdxRqRS.pngthis is good news for the mission—i might not need to do any fancy merging shenanigans now, together with the awkward workflow that would need (working on the merged result would be kinda like working with a stripped .mis, where only objects and maybe lights can be changed). but bad news for the merging tool: maybe i wont need to write it after all??
well, im gonna keep trying a few more variations in laying out the three parts now, and see how they go.
PinkDot on 31/12/2022 at 19:59
Good to hear you managed to pass through the block in such a simple (yet not obvious) way!
I do think however that the idea of merging multiple worldreps is interesting enough and maybe worth getting back into at some point.
Quote:
the .mis format—the worldrep specifically—is only partially understood as far as public knowledge goes. many others have done work with it before (e.g. for the OpenDarkEngine project, and PinkDot's editor). and with the t2 source code to read, the olddark worldrep structure was at least knowable in all its details (though understanding it all is a different story!). newdark though made a number of changes to the worldrep data, only some of which are known. outputting a non-crashing .mis is going to require figuring out those unknowns, too.
Over last couple of days I managed to decipher the rest of WREXT chunk. That includes CSG Internal database and additional New Dark arrays, which you may not encounter unless you mess around with Zones. Let me know, in case you're interested.
vfig on 31/12/2022 at 20:29
Quote Posted by PinkDot
I do think however that the idea of merging multiple worldreps is interesting enough and maybe worth getting back into at some point.
Over last couple of days I managed to decipher the rest of WREXT chunk. That includes CSG Internal database and additional New Dark arrays, which you may not encounter unless you mess around with Zones. Let me know, in case you're interested.
i havent given up on the merging idea yet—for one thing, even though the mission portalizes again now, there is very little headroom, and the ruins still need a lot of work on them. so the merge might still be necessary.
and yes, i am very interested in everything else youve figured out about the .mis!
vfig on 18/1/2023 at 19:03
jan 18, 2023almost three weeks and a lot of distractions later... i finally got back to the merging tool today, and wrote a first draft of the worldrep merge program: just enough to see if the whole idea can actually work.
to test it, i made two single-brush maps, not far off the simplest possible. the first has a six-sided cylinder above the zero plane:
Inline Image:
https://i.imgur.com/CU3BqRY.png(why i am using olddark dromed will become clear soon.) and the second, a boring cube below the zero plane:
Inline Image:
https://i.imgur.com/1qkZEcK.pngonce i had fixed a few silly typos in my program—oh, sorry, i forgot to introduce it properly! it is a program that
does things to
.mis files, so quite obviously it is called
misdeed.exe—aaaanyway, got it compiling and fed it the top and bottom .mis files, and it duly spat out a result.
next step is to load up the result in dromed: and
of course dromed immediately crashed! i fully expected this: the worldrep is complicated, and there are little bits of it we dont quite 100% understand yet, and also i took a few possibly unjustifiable shortcuts for the sake of the first draft. thankfully debugging what went wrong was made about ten million times easier because by luck a certain source code repository happened to have a build of dromed in it
with its .pdb! and so:
Inline Image:
https://i.imgur.com/5GVAykM.png(for the non-windows-programmers, a .pdb is a file full of debug information about a program: its so that when the compiled program is running, visual studio can know what the function and variable names are and which source code files and line numbers correspond to any part of the .exe). turns out the source code files in the repo arent a 100% match for those that this dromed was built for, so the line numbers are a bit offset (visual studio thinks the program is at the line shown in green; but its actually at the line i highlighted in blue a little further below). but close enough, and the local variables here all match up, and it meant i could instantly understand what my program had done wrong to cause the crash. so i fixed that problem, and ran my program again. and loaded the output into dromed, fully expecting another crash. but:
Inline Image:
https://i.imgur.com/5AzD4ft.pngit worked?!? im showing the game mode view here, because my first draft isnt copying the brushes themselves over, so the editor view is less than helpful. but in game mode, you can see both cells are there! i can fly into either one, and everything about them works just fine! i am stoked.
