Jump to content
COMBATSIM Forum
Krycztij

TAW terrain format

Recommended Posts

Well, there's no specific instruction I'm stuck on at the moment … and I'm currently all busy with the transformation, which is finally moved from fixed-point to floating-point math. Here is, however, the current instruction list, just pick something:

struct Opcode { enum Value {

	// fully implemented

	_0000Return						= 0x0000,

	_0001DrawFlatShadedLine			= 0x0001,

	_0002DrawFlatShadedTriangle		= 0x0002,

	_0003DrawFlatShadedQuad			= 0x0003,

	_0004DrawFlatShadedPolygon		= 0x0004,

	_0008Call						= 0x0008,

	_0015JumpIfTriangleIsInvisible	= 0x0015,

	_001ADrawSemiTransparentTriangle= 0x001A,

	_0020JumpIfParameterIsBelow		= 0x0020,

	_0021JumpIfParameterEquals		= 0x0021,

	_0022JumpIfParameterIsAbove		= 0x0022,

	_0027JumpIfDistanceAbove		= 0x0027,

	_002DDrawDisplacedSubmesh		= 0x002D,

	_002EFetchPolygonUV				= 0x002E,

	_002FDrawSolidTexturedPolygon	= 0x002F,

	_0030DefineUnitLength			= 0x0030,

	_0031PushPolygonColors			= 0x0031,

	_0033DrawPoint					= 0x0033,

	_0034DrawPoint					= 0x0034,

	_0037PushFogDistance			= 0x0037,

	_0039DrawPoints					= 0x0039,

	_0047FetchPolygonUV				= 0x0047,

	_0048ChooseActiveParameter		= 0x0048,

	_0049DrawTransformedSubmesh		= 0x0049,

	_004BDrawStaringSubmesh			= 0x004B,

	_004CFetchTriangleColors		= 0x004C,

	_004EFetchQuadColors			= 0x004E,

	_004FDrawGouraudShadedQuad		= 0x004F,

	_0053WriteXYZs					= 0x0053,

	_005CDrawSphere					= 0x005C,

	_005FDrawSpheres				= 0x005F,

	_0062WriteRandomXYZ				= 0x0062,

	_0063WriteNextXYZWithXYZDelta	= 0x0063,

	_0064WriteNextXYZWithXDelta		= 0x0064,

	_0065WriteNextXYZWithYDelta		= 0x0065,

	_0066WriteNextXYZWithZDelta		= 0x0066,

	_0067WriteNextXYZWithXYDelta	= 0x0067,

	_0068WriteNextXYZWithXZDelta	= 0x0068,

	_0069WriteNextXYZWithYZDelta	= 0x0069,

	_006AWriteRandomXYZWithXYZDelta	= 0x006A,

	_006BWriteRandomXYZWithXDelta	= 0x006B,

	_006CWriteRandomXYZWithYDelta	= 0x006C,

	_006DWriteRandomXYZWithZDelta	= 0x006D,

	_006EWriteRandomXYZWithXYDelta	= 0x006E,

	_006FWriteRandomXYZWithXZDelta	= 0x006F,

	_0070WriteRandomXYZWithYZDelta	= 0x0070,

	_0071DrawNextFlatShadedTriangle	= 0x0071,

	_0072DrawNextFlatShadedQuad		= 0x0072,

	_0073DrawNextFlatShadedPolygon	= 0x0073,

	_007DDrawFlatShadedQuad			= 0x007D,

	_0080DrawGouraudShadedTriangle	= 0x0080,

	_0081DrawGouraudShadedQuad		= 0x0081,

	_0083UseTexturePortion			= 0x0083,

	_0084UseWholeTexture			= 0x0084,

	_0088DrawBlendedTexturedPolygon	= 0x0088,

	_008DDrawAfterburnerTriangle	= 0x008D,

	_008EDrawMaskedTexturedPolygon	= 0x008E,

	_009DDrawMaterialShadedTriangle	= 0x009D,

	_009EDrawMaterialShadedQuad		= 0x009E,

	_009FFetchPolygonMaterial		= 0x009F,

	_00A3JumpIfAGPAvailable			= 0x00A3,

	_00A7EnableTextureColor			= 0x00A7,

	_00A8DisableTextureColor		= 0x00A8,

	_FFFFEndOfLOD					= 0xFFFF,

	// somehow understood; not yet implemented

	_0007DrawSubmeshIf___			= 0x0007,

	_001BDrawQuad					= 0x001B,

	_0024							= 0x0024,

	_0025							= 0x0025,

	_0026							= 0x0026,

	_0038DrawFlatShadedQuad			= 0x0038,

	_003EDrawQuad					= 0x003E,

	_003FDrawLightSubmesh			= 0x003F,

	_0040							= 0x0040,

	_0041							= 0x0041,

	_0042							= 0x0042,

	_0043JumpIfTimeOfDayEquals		= 0x0043,

	_0045							= 0x0045,

	_0051MarkPylonPosition			= 0x0051,

	_005DDrawSemiTransparentSphere	= 0x005D,

	_0060DrawSemiTransparentSpheres	= 0x0060,

	_0061CompleteXYZFetching		= 0x0061,

	_0075MarkSpawnPosition			= 0x0075,

	_0076							= 0x0076,

	_007B							= 0x007B,

	_007C							= 0x007C,

	_007EDrawQuad					= 0x007E,

	_0086MarkJointPosition			= 0x0086,

	_0087							= 0x0087,

	_0093							= 0x0093,

	_0095DrawSmallPoint				= 0x0095,

	_0098SetTransparencyLevel		= 0x0098,

	_00A2DrawFlatShadedQuad			= 0x00A2,

	// unknown

	_004A							= 0x004A, // postponed

	_004D							= 0x004D,

	_0052							= 0x0052,

	_0054							= 0x0054,

	_0077							= 0x0077, // postponed

	_007F							= 0x007F,

	_0094							= 0x0094,

	_0096							= 0x0096,

}; };

Consider that "somehow understood / not yet implemented" includes instructions where single words of the operand are yet unknown, e.g. 005D & 0060 are implemented and look fine, but I haven't yet worked out what the first word means … and then there are others of which we know nothing but that their operands are addresses — but not if they are jumps or calls and what conditions must be met to be executed.

I'll try to implement indentation into the debug output, so when you produce a debug dump, you see what is executed as a function and what isn't. This could help us seperate jumps from calls … I hope.

Oh and the new vertex color / texture coordinate handling is implemented and works fine, too.

Share this post


Link to post
Share on other sites

OK, let's start with an easy one. 007f does not exist. Don't know where that came from since we still have 99.

Share this post


Link to post
Share on other sites

Nothing exciting to report, but I'll post this as fuel for speculation.

Here is a list of files which have 0054 as the first opcode:

newhoriz.3 0000; 0054    ;

nvg_hori.3 0000; 0054    ;

rwyend02.3 0000; 0054    ;

rwyend06.3 0000; 0054    ;

rwyend07.3 0000; 0054    ;

rwyend09.3 0000; 0054    ;

rwyend11.3 0000; 0054    ;

rwyend13.3 0000; 0054    ;

rwyend14.3 0000; 0054    ;

rwyend16.3 0000; 0054    ;

rwyend17.3 0000; 0054    ;

rwyend20.3 0000; 0054    ;

rwyend24.3 0000; 0054    ;

rwyend25.3 0000; 0054    ;

rwyend26.3 0000; 0054    ;

rwyend27.3 0000; 0054    ;

rwyend31.3 0000; 0054    ;

rwyend32.3 0000; 0054    ;

rwyend34.3 0000; 0054    ;

rwyend35.3 0000; 0054    ;

rwymid00.3 0000; 0054    ;

rwymid01.3 0000; 0054    ;

rwynd15l.3 0000; 0054    ;

rwynd15r.3 0000; 0054    ;

rwynd16c.3 0000; 0054    ;

rwynd16l.3 0000; 0054    ;

rwynd16r.3 0000; 0054    ;

rwynd18.3 0000; 0054    ;

rwynd33l.3 0000; 0054    ;

rwynd33r.3 0000; 0054    ;

rwynd34c.3 0000; 0054    ;

rwynd34l.3 0000; 0054    ;

rwynd34r.3 0000; 0054    ;

rwynd36.3 0000; 0054    ;

vasi.3 0000; 0054    ;

Apart from the two horizon files, they are all runway related.

....but there are two runway files, rwymid02.3 and rwymid03.3 which don't have 0054 at the start, but the rest of the files look very similar.

The only thing I can guess from this is that whatever 0054 is, it doesn't require a terminator.

Share this post


Link to post
Share on other sites

While it remains in my conscious memory, I thought it good to mention a point clarification which is based on previous work with 00a2.

Current:

// 00A2 <color> <position0> <position1> <position2> <position3>

// Draw a flat-shaded quad with the color at palette index <color>. The indices of its vertices'

// positions are given in clockwise order.

// The exact purpose of these instructions is unknown — they seem to be useful for shadow rendering

// only; the polygons they render are not visible in the actual game.

just-in-time note: 00A3 is a jump on AGP, so 00A2 could mean "draw an AGP-textured quad". This would

explain why 00A2 draws black quads at the tires: <color> would be the index of the AGP tire texture then

(if such a texture exists, it´s really just a spontaneous idea).

I would suggest that it <color> is not the index for the AGP texture, AGP textures begin at index number 201 in the known texture index. It seems as if 00A2 texture selection is hardcoded and uses the same texture for shadow rendering as the associated shadow-caster. I'll keep working on it.

Share this post


Link to post
Share on other sites

00a2 is temporily drawn with the same syntax as 0003 to make the polygons visible. Hence the bright green empennage on the F22 in earlier screenshots.

Shadow casters provide the best explanation for 00a2, but I don't understand when the resulting shadows are applied....

Share this post


Link to post
Share on other sites

Two short questions:

1. How are texture indices mapped to red****.ini? More specifically: I must add 1 to each index, but how can I seperate the .3 file's hard-coded texture (at index 0) from the first red****.ini texture (which is referenced through 0, too)? Or do hard-coded names override the first entry of red****.ini? This is relevant since we still cannot render f22usa1.3 properly, and some explosion files crash because the hard-coded texture is referenced by accident.

2. How exactly is transparency stored in the 0098 opcode?

Share this post


Link to post
Share on other sites

I've always interpreted it like this:

1. 002e uses the hard coded texture, and 0047 uses the list from redXXXX.ini

2. 0098 acts a modifier to the transparency value of the current mask, so 0098ffff would change the transparency level by one unit.

Haven't you already implemented it this way? The building shadows seem to behave as I'd expect, with the shadow lightening with distance.

Share this post


Link to post
Share on other sites

I've always interpreted it like this:

1. 002e uses the hard coded texture, and 0047 uses the list from redXXXX.ini

Works great, thank you! f22usa1.3 does look even better than f22usa2.3 now. Body damage and position lights are fully functional while f22usa2.3 still has mistaken textures (brown) there.

2. 0098 acts a modifier to the transparency value of the current mask, so 0098ffff would change the transparency level by one unit.

Haven't you already implemented it this way? The building shadows seem to behave as I'd expect, with the shadow lightening with distance.

Well, that must be a coincidence then — I did indeed experiment with the opcode over the last days, but I never got it working to my satisfaction. Well possible that my implementation was close when I sent you the current build, but it doesn't work well at the moment. I'm going to test your interpretation immediately. Implemented. Does work well; you need, however, to multiply the operand by eight — just like alpha maps, the value is in the range of [0, 31], inclusively. Also, it does not define transparency but coverage (nitpicking here ;) ).

I need to re-compile the shaders because of all the mess with texture portions and transparency in the last few days; then I'll post the current build.

I'd also be happy with information on program invariants that help me optimize buffer sizes — for example:

  • the highest vertex index ever written in any model
  • The highest number of vertices in a single polygon instruction (002E, 002F, 0047, 0088, 008E and the like)

There are also numbers like the highest number of subroutines being called simultaneously, but we need to decode all jumps and calls carefully until we can work that out. Speaking of jumps and calls: ivan.3 draws all its damage models instead of only one, and so does para.3 draw multiple animation steps. Still a long way to go :(

Share this post


Link to post
Share on other sites

Well, it didn't turn out as easy as I thought. First of all, here's the link:

http://www.mediafire.com/?7htdsk7d6iba948

Spheres don't work any more, but that's because I'm currently at implementing them via sprites instead of meshes, just like TAW does. I've also corrected texture coordinate transformation, which results in less color bleeding; implemented texture portions and updated the F12 button so it only dumps one frame to the console.

Shadows attenuate fine, and so do single explosions (like ex_nblt1.3). Multiple smoke puffs (like sm_nblt1.3), however, … well see four yourself. It might be possible the value is popped from the stack when the subroutine returns … I think I got it now. On 0098, you need to bloat the 5-bit value up to eight bits (or whatever bit depth your transparency works with) and then add it to the full coverage value (e.g. 255). That's the final transparency then; the value is not manipulated later any more. Not updated in the build I posted, though.

On the topic with calls vs jumps, we should look into chm_tnk2.3 — it draws tens of thousands of points (literally!) …

… and in blakhawk.3, 0024 controls the secondary rotor while parameter #2 controls the primary.

… aaand I see the build I posted has a slight bug reducing texture quality when seen from certain distance. I've already fixed it in my internal builds.

Share this post


Link to post
Share on other sites

Nice indentation on the console output!

I've only tried a few models, but have to report that the viewer crashes with F22usa1.3 when extending the landing gear.

...will now look at a few more.

Share this post


Link to post
Share on other sites

Oh no, that's a pity. It works fine here. Is anything written to the console? Did you try disabling AGP textures via F4 before extending the gear?

Share this post


Link to post
Share on other sites

Any idea what's happening to a couple of models with gun turrets?

Here's a ZSU-23-4 which shows more polys than it should. While I'm at it, the top picture shows the ZSU from EF2000...

zsu.jpg

Share this post


Link to post
Share on other sites

No idea. I guess it's the same problem with chaparal.3, chinook.3 and comanche.3 …

I'll do a debug dump and compare the vertex positions to your script's, maybe I find something.

Share this post


Link to post
Share on other sites

Your vertex output:

http://pastebin.com/wiiDPUqT

Mine, with inverted Y axis:

http://pastebin.com/SedGp37K

No difference. So it's either a transformation problem or our interpretations of vertex instructions are wrong.

This is the model without any transformation applied to any vertex:

pretran.jpg

So there are vertices pre-transformed and the 0049 transformation must not be applied to them again.

Oh, dear. I hope it's not the 0061 opcode marking vertices as "final, don't transform".

Share this post


Link to post
Share on other sites

I guess the problem is, I store the XYZ I get from the file and transform it when the triangles are drawn. I should transform the XYZ with the current transformation matrix immediately upon reading: If some of the barrels' vertices are defined with the lower part of the tank, and others with the turret, they will be transformed wrong.

This was planned anyway, and I've already spent some days moving the transformation from the renderer to the .3 processor. I'll try to implement the rest now and see if it works.

Share this post


Link to post
Share on other sites

Oh no, that was just a big fat waste of time:

zsu234trns.jpg

It looks even worse in movement; the barrels turn twice as fast as the turret. The vertices which were misplaced before are now correct, but now the others are wrong instead. I need to revert it to the previous version … so we're stuck in square one.

Share this post


Link to post
Share on other sites

Sorry for the triple post, but I have some real progress here:

Another puzzle solved, and as proof, the correctly-rendered ZSU-23-4:

yeahn.png

Well, this one was hard. TAW obtains two pools with XYZ data. The first pool is the one we already know, and which works fine with almost all models. Then, there is a second pool with transformed XYZ.

Whenever a 0061 instruction is encountered, a block of vertices is transformed (the current transformation first, then the current scale factor, then the view and projection matrix). It is crucial to understand that this may happen long before triangles using these positions as vertices are actually drawn — in case of the ZSU, some positions were written and transformed in a subroutine and others weren't. Therefore, the turret's vertices actually used two seperate coordinate systems simultaneously. I transformed them at the time they were drawn, which happened always in the innermost subroutine, i.e. in one coordinate system — and that's where those glitches came from.

So, the transformed XYZ data is used for rendering exclusively, while the raw XYZ data is still used for delta compression, as reference points to 0049 transformations, to mark spawn/pylon positions etc.

Anyway: I just wrote a quick hack and now many models are screwed up. Also, the shortest night of the year is over and I really need to get some sleep. It will take me a while to rewrite the vertex handling. We are, however, on the right track! :thumbsup:

Share this post


Link to post
Share on other sites

Sorry for the triple post, but I have some real progress here:

Well, this one was hard. TAW obtains two pools with XYZ data. The first pool is the one we already know, and which works fine with almost all models. Then, there is a second pool with transformed XYZ.

Whenever a 0061 instruction is encountered, a block of vertices is transformed (the current transformation first, then the current scale factor, then the view and projection matrix). It is crucial to understand that this may happen long before triangles using these positions as vertices are actually drawn — in case of the ZSU, some positions were written and transformed in a subroutine and others weren't. Therefore, the turret's vertices actually used two seperate coordinate systems simultaneously. I transformed them at the time they were drawn, which happened always in the innermost subroutine, i.e. in one coordinate system — and that's where those glitches came from.

So, the transformed XYZ data is used for rendering exclusively, while the raw XYZ data is still used for delta compression, as reference points to 0049 transformations, to mark spawn/pylon positions etc.

Anyway: I just wrote a quick hack and now many models are screwed up. Also, the shortest night of the year is over and I really need to get some sleep. It will take me a while to rewrite the vertex handling. We are, however, on the right track! :thumbsup:

Thank you, especially for the explanation regarding the 2 pools of data.

You certainly deserve a good sleep, although I must say, I didn't think you ever took sleep ;)

Share this post


Link to post
Share on other sites

Brilliant! I wake up and find that the problem has been solved. :thumbsup:

In the last list of codes, you have 0095 in the 'sort of understood' category but 0096 is unknown, but it would seem that any difference between these is the same as the difference between 0033 and 0034.

From 707.3:

0040; 003f0080000a    ;  If 003f (Time??) test(128), Jump to line 43

0041; 0033000e0015    ; Light (Long range): Palette:14  Vertex: 21

0042; 0000    ;

0043; 00950005000e0015    ;

0044; 0000    ;


0364; 003f0080000a    ;  If 003f (Time??) test(128), Jump to line 367

0365; 003400de0008    ; Light (Short range): Palette:222  Vertex: 8

0366; 0000    ;

0367; 0096000500de0008    ;

0368; 0000    ;

I guess the 003f opcode is making the decision whether or not a light should be on or off.

If on, the 0033/0034 opcodes are used and 0095/0096 if off.

Since 0095/0096 use the same palette index as the associated 0033/0034, I assume that (in this case) 0005 is a modifier which darkens the colour somehow.

Share this post


Link to post
Share on other sites

Thank you :) I could also fix the glitches: 0075 mark spawn position causes the vertex of the given index being transformed, too. Although not many 0075 are used for rendering, this does make sense because you want spawn positions to be in world space — but it also means that the processor performs world transformation and scale only; view and projection transformation must be applied in the renderer (because screen coordinates should be unusable as spawn positions). The same should be with 0051 mark pylon position, but I couldn't verify it yet because these vertices are not used for rendering.

The 0086 mark joint position (which copies F-22's rudders to their according positions) is a little trickier, because I can image it does not only need a point, but a whole coordinate system for rotation, translation and scaling.

I can gladly announce that the glitches regarding incorrect transformations are all gone: F-22 weapon bay, T-80 barrel, Comanche, C-17 loading bay, ZSU-23-4, Challenger tracks, Chinook loading bay — these are rendering fine now.

There are, however, new glitches with lights in the runway files. Also, spheres are still under construction. After I've finished that, I'll have a look at your lights, Mike :) You may also want to look at air_hq.3 — its lights are drawn as point sprites, actually. I never saw this in-game. Also, 003F is a call, not a jump.

Share this post


Link to post
Share on other sites

I'll look at it later. Restricted to theoretical research only for the next few hours.

Is 004d really unknown? It would seem to be the triangle version of 004f, ie draw gouroud shaded triangle.

Share this post


Link to post
Share on other sites

You are right; it draws the tips of Chaparel's missiles. Solved & implemented. :thumbsup:

More on lights: The strangest thing right from the start was that there are two point sizes in TAW with glide wrapper: One is always 1×1 pixels; the ordinary point we know. The other one is scaled with resolution, and it's about 2×2 pixels in full HD.

When analyzing the bytecode, I also noticed that many lights are drawn in their own subroutine — more specifically, a 004B draw staring submesh subroutine. (This is the one used for pilots to always look at you). This implies that the "large" lights are not actually points, but flat-shaded sprites; quads (with a size given in pixels?) which always face the viewer. Probably the same sprites as used for spheres (spheres are used as lights in the KC-135).

Therefore, we should identify which instructions are used inside of 004B subroutines (which do nothing but write vertices, colors, and render one light) exclusively. These must be handled as sprites; the others must be handled as points. The ref_*.3 files should be a good start as they have hundreds of lights inside.

You could even do this with the viewer; it should be easy to spot in the indentation. I was already on it, but then the whole seperate-XYZ-buffers thing kicked in and I've lost the thread :)

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...