Krycztij

TAW terrain format

979 posts in this topic

If you just want to see what the model looks like, then its fine to just render everything and let the depth buffer handle it.

No no, you are completely correct! Depth buffering is completely wrong because TAW uses overlapping for multitexturing. With depth buffering, there are artifacts then:

artefacts.jpg

Without depth buffering, this is fine:

87487678.jpg

But now I will have to perform the 0015 tests to handle overdraw correctly. I really hope it is just a if(front_visible) test or something other easy!

002F vs 008E seems consistent. Didn't you have palette problems in the glide wrapper thread with a tile like the one in my screenshot? The ground is drawed with 002F and the river with 008E. Maybe TAW applies other palette/color keying algorithms when drawing 008E?

Share this post


Link to post
Share on other sites

Welcome Krycztij !! :thumbsup:

And what an entrance you've made!

I wish you much success - You seem to be making progress rapidly!!

I watch with much appreciation and anticipation, and will pitch in if I can help in any way... :popcornsmilie::icon_salute3:

(Also don't forget, some of us join up and fly weekly - We're friendly, and we enjoy flying TAW)

Share this post


Link to post
Share on other sites

The 0015 test should be as easy as you describe.

TAW uses palette position zero to denote transparency when reading the textures, but colour keying is used when the primitives are sent to Glide (and I presume D3D). Colour keying is only enabled when it's needed and this confused me for a while

Share this post


Link to post
Share on other sites
TAW uses palette position zero to denote transparency when reading the textures, but colour keying is used when the primitives are sent to Glide (and I presume D3D). Colour keying is only enabled when it's needed and this confused me for a while

I can't use paletted textures and I must convert them to RGBA when I load them. If color keying is used in specific situations, I must seperate the color keyed polygons and load a color keyed copy of the texture for them. This is a change in program structure with certain cost. I'll make it now and when all cases with color keying are identified, I'll activate it.

The visibility testing is a problem, too. My virtual machine is much too slow to execute the .3 file again in each frame, but this is required for real-time visibility testing. I will have to build program tree and pre-allocate all buffers :( But I'm sure I'll have it done next weekend.

Share this post


Link to post
Share on other sites

You also need to consider the 0088 opcodes that you will eventually come across. These make the texture behave as a transparency mask, where there are about 20 or so (not sure how many) levels of transparency corresponding to the first 20 or so palette positions.

Share this post


Link to post
Share on other sites

Update time!

I'm very very glad now:

I rewrote the virtual machine and the renderer to draw triangles the same way TAW does. No depth buffering, instead 0015 tests. And it works!

The river (nlarbn_2.3) draws perfectly:

riverk.jpg

The cliff (ctclfc_1.3) does not draw perfectly, but with the same quick little glitches we see in TAW sometimes:

15059422.jpg

The 0015 is indeed a "jump if triangle of indices in clockwise order is not facing forward" test.

I clean up now, because it was much rewriting and the source code is very messy now. Then tomorrow I hope I can start to treat new opcodes. Also thank you for the 0088 consideration, mikew!

Share this post


Link to post
Share on other sites

Cool!

Being able to handle the 0015 opcodes now will pay off if you want to continue with this since most .3 files involve them.

So, what next?

Share this post


Link to post
Share on other sites

So, what next?

I don't know yet. I'm still tidying up the code base. And also adding gadgets like switch the wireframe and the level of detail on and off, and choosing random palettes :)

But I see there are still some mistaken textures. Also, many tiles fail because I had no time yet to analyze the 003F opcode which is very common I think. If you already have explanations for it, I'm sure it would speed up my progess ;)

Share this post


Link to post
Share on other sites

I don't know yet. I'm still tidying up the code base. And also adding gadgets like switch the wireframe and the level of detail on and off, and choosing random palettes :)

But I see there are still some mistaken textures. Also, many tiles fail because I had no time yet to analyze the 003F opcode which is very common I think. If you already have explanations for it, I'm sure it would speed up my progess ;)

Great, things do seem to be moving along for you.

The 003f opcode is a decision on night or day, and has control over the lights as to whether they should be on or off. 003f is frequently used with the 0033 opcode which are lights, such as small runway light as dots on the ground lining both sides and middle of runway.

What do you mean by 'mistaken textures'?

Share this post


Link to post
Share on other sites
The 003f opcode is a decision on night or day, and has control over the lights as to whether they should be on or off. 003f is frequently used with the 0033 opcode which are lights, such as small runway light as dots on the ground lining both sides and middle of runway.
Okay, this makes sense, I think. Thank you!

What do you mean by 'mistaken textures'?

My bad: I forgot to add 1 to all indices in redxxxx.ini. It's okay now.

Share this post


Link to post
Share on other sites

Looking at the 003f opcode again, I now think it is a simple decision of whether a light should be on or off.

From 707.3, we have this:

0040; 003f0080000a    ;  If 003f (Time??) test(128), Jump to line 43

0041; 0033000e0015    ; Light (Long range): Palette:14  Vertex: 21

0042; 0000    ;

0043; 00950005000e0015    ;

0044; 0000    ;

I now interpret this as some test which reads a flag and if the light is on:

0033000e0015 draws a point at vertex 21 with palette colour 14

...and if the light is off

00950005000e0015 uses the same colour and the same vertex, but the colour is darkened by some amount.

Share this post


Link to post
Share on other sites
tiletest.jpg

Looking a bit closer at this screenshot, I think all of the terrain tiles were displayed with the same rotation - However in game, 'corner pieces' of mountain areas wound be rotated to round off the mountain.

Maybe the 'random' number is actually telling which 1 out of 4 rotation positions the texture has?

..Just pure speculation though..

Share this post


Link to post
Share on other sites

No, while EF2000 used rotation as you describe, TAW uses separate .ssd and .3 files for each orientation.

Speculation is good though. :)

Share this post


Link to post
Share on other sites

Evidence and testing and results are usually better ^^

So the random number just gives us the possibility to include up to 4 variations of a tile, and the game will randomly select from 1 of those 4 tiles? Or is it predefined which of the 4 variations will get placed at a certain position on the map?

Share this post


Link to post
Share on other sites

I assume that it isn't totally random, but I'm not sure exactly how it's done.

Note, that the stock TAW uses 1 or 4, but this could be other numbers as well. I know that 5 works.

Unfortunately, I think (but not 100% sure) we hit some limit on total ssd numbers (1024) so use of this tactic is limited.

Share this post


Link to post
Share on other sites

mikew is right.

The screenshot is old. I did not use texture coordinates then. I.e.: All texture coordinates on the screenshot are 0|0 to 96|96. I've got a new screenshot where the minimum and maximum texture coordinate is used to select portion of the texture (0-95 or 96-191 on every axis) and there 80 % of all textures look correct (the remainer is wrong because I was not considering mirroring, rotation of textures). So TAW indeed uses one tile per rotation.

The image blocks my PC for 25 minutes (it's almost 40,000 x 40,000 large) and I don't have time right now, but I'll post it in a few hours so you have empirical evidence.

I'm currently only working on the .3 file viewer. The whole terrain image is too complicated when I don't know the exact outcoming of the .3 files. I need the model viewer to compute rotation and mirror of all tiles correctly.

Share this post


Link to post
Share on other sites

Looking a bit closer at this screenshot, I think all of the terrain tiles were displayed with the same rotation - However in game, 'corner pieces' of mountain areas wound be rotated to round off the mountain.

Maybe the 'random' number is actually telling which 1 out of 4 rotation positions the texture has?

..Just pure speculation though..

I must admit I was wrong (version with correct texture offsets but without texture rotation or mirroring shown here):

tles.jpg

The correct-incorrect ratio is more 50-50 than 80-20 and all it seems to me that there is no other action to perform an all incorrect tiles than mirror them vertically (rotation by 180° would not match). So 1/4 COULD indeed be a "mirror vertically" flag but ONLY IF we can verify that correct tiles and incorrect counterparts refer to the same .3 file and differ only in the flag.

But we must consider: Mirroring is very very complex. It changes the order of triangle vertices from clockwise to counter-clockwise. Backface culling and visibility tests must consider that, and collision detection in physics and AI, too. I was surprised if DiD indeed did that, because it seems like horror from a programming point of view.

Share this post


Link to post
Share on other sites

I think I came across this before when trying to display tiles in relation to each other and seem to remember having to invert the z axis of each tile for it to work properly. This was some time ago and I've forgotten exactly why.

Share this post


Link to post
Share on other sites

Looking at the 003f opcode again, I now think it is a simple decision of whether a light should be on or off.

From 707.3, we have this:

0040; 003f0080000a    ;  If 003f (Time??) test(128), Jump to line 43

0041; 0033000e0015    ; Light (Long range): Palette:14  Vertex: 21

0042; 0000    ;

0043; 00950005000e0015    ;

0044; 0000    ;

I now interpret this as some test which reads a flag and if the light is on:

0033000e0015 draws a point at vertex 21 with palette colour 14

...and if the light is off

00950005000e0015 uses the same colour and the same vertex, but the colour is darkened by some amount.

Thank you for the information! I have not found lights yet, so the 003F opcode is used for normal rendering, too. Maybe it's the "jump if distance below" counterpart to 0027?

Out of 15 tiles of jeddah, 12 load perfectly. The others (jeddah_b, jeddah_j, jeddah_k) fail at the following opcodes:

1. 0075

Earlier you said:

The 0061 (and 0075) codes are not really needed to understand what a model will look like, but must be an important part of how the engine works.

So I can just skip it for now?

2. 006B, 006D, 006F

They seem similar to 0067, 0068 and 0069 (new vertex with XY/YZ/XZ delta). Is there any difference?

Share this post


Link to post
Share on other sites

The opcodes 006a to 0070 are the same as 0063 to 0069 regarding the x,y,z calculations, but the extra bytes at the end give the vertex index that the calculated coordinates apply to. For the 0063 to 0069 opcodes, the vertex index just increments each line.

You can ignore the 0075 for now. It would be nice to eventually know what it does though.

Which .3 file are you having the 003f trouble with?

Share this post


Link to post
Share on other sites

0075 adds vertices, too. I think it means "copy vertex with the following index", but I can't say that until I loaded a 0075 mesh successfully. It's near 0071 ("repeat previous line"), too.

Which .3 file are you having the 003f trouble with?

Sorry, I forgot what I wanted to write on that when I posted :) I implemented 003F and it runs fine on many files, but some always jump to a 00DF. jeddah_j.3 is an example I'm currently working on (reached the first 0001 opcode).

Share this post


Link to post
Share on other sites

0001 just draws a line:

0001002c001e0002 ; Line, Palette:44 Vertices: 30,2

Here's how I've interpreted the 003f block at the end of jeddah_j:

0498; 003f00800004    ;  If 003f (Time??) test(128), Jump to line 500

0499; 0000    ;

0500; 003400df0015    ; Light (Short range): Palette:223  Vertex: 21

0501; 003400df0010    ; Light (Short range): Palette:223  Vertex: 16

0502; 003400df0011    ; Light (Short range): Palette:223  Vertex: 17

0503; 003400df001b    ; Light (Short range): Palette:223  Vertex: 27

0504; 003400df0014    ; Light (Short range): Palette:223  Vertex: 20

0505; 003400df000e    ; Light (Short range): Palette:223  Vertex: 14

0506; 0039002400df00060015    ; Line palette: 223 verts 6->21 ?????

0507; 0039002400df00150010    ;

0508; 0039002400df00110010    ;

0509; 0039002400df00110005    ;

0510; 0039002400df00140007    ;

0511; 0039002400df001b0014    ;

0512; 0039002400df0014000e    ;

0513; 0039002400df00150014    ;

0514; 0039002400df001b0010    ;

0515; 0000    ;

0516; 0000    ;

0517; ffff    ;

which I think means, that if the 003f test is true, some points are drawn using palette colour 0xdf, then maybe some lines.....although I have no idea what function the 00390024 performs.

Share this post


Link to post
Share on other sites

Oh no, I had accidentially defined a 4th word in the 0034 instruction and that is why he jumped into 00DF. This always happens with copy & paste <_<

Thank you SOO much :icon_bow: I have all tiles of jiddah up now.

jeddahj.png

The positions of the lights and the line are not yet correct, but I'm working on it. A lot of new instructions were implemented today and I have some debugging and testing to do now.

btw I'm impressed: TAW uses multitexturing by overdraw really often. Sometimes tiles are almost drawn thrice. But it does indeed look good when you consider the hardware back in the days.

mulex.png

Share this post


Link to post
Share on other sites

Here's a picture approaching Jeddah at night from the north. The bottom right quadrant involves jeddah_j.

I've changed the palette so that index 0xdf shows magenta. While most the magenta areas are defined by the texture, the strings of the streetlghts are probably defined by the 003f section of the .3 file.

jed_mag.jpg

Share this post


Link to post
Share on other sites

First: I found a bug in my viewer where the call stack is popped after a line is drawn. Now that I fixed it, jeddah_j is ruined again by street polygons which are spanned all over. I'll have to check my vertex delta code.

BUT: Your screenshot is very very helpful. Look close to street lights on the right. You can see, they are rendered in pairs. And if you count them, you get 36 pairs per street segment, so 24 hex. Look at line you posted earlier:

0506; 0039002400df00060015    ; Line palette: 223 verts 6->21 ?????

I am sure it means: "draw 36 pairs of street lights with color 223 from position #6 to position #21". But I have not implemented it yet. I post screenshots when I have.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now