Krycztij

TAW terrain format

979 posts in this topic

Look what just loaded:

28761006.jpg

Not very beautiful, but it's officially the first non-terrain .3 file I ever rendered :icon_rock:

I also loaded logo.3, but that looks correct if I ignore the rotations. Don't know if I follow the 0049 issue.

Share this post


Link to post
Share on other sites

That looks like the low LOD B2 model, which brings us onto the subject of LODs which are handled in a couple of different ways.

For the B2, there are two LOD levels. The distances where each LOD is visible is defined by the second and third words of the header:

fff501f407d0000000007fff......

So, the most detailed LOD is drawn at a distance of less than 01f4 (500) and the low detailed LOD up to 07d0 (2000)

After the header, the low detail LOD is defined first. The total number of vertices in the model is given in the word before the start of the bytecode. In this case 000e, but not sure if this is a general rule for all .3 files.

The LOD model then terminates with the ffff opcode.


0000; 0030000a    ; Scale Factor 10

0001; 00480007    ;

0002; 00220007012e    ; If Parameter 0007 > 7 then jump to line 45

0003; 00620000000000000032    ; Vertex :0  X=0  Y=0  Z=50

..

..

0043; 00710002000c0007    ; Flat Shaded Triangle, Palette:180  Vertices: 2,12,7

0044; 0071000c000a0007    ; Flat Shaded Triangle, Palette:180  Vertices: 12,10,7

0045; 0000    ;

0046; ffff0078    ; End of LOD 1, and there are 0078 (120) vertices in LOD2

Now we continue on to the next LOD. I forget whether the starting point of this can be found in the header, but I scan through the file looking for a ffff opcode not near the end of the file.

0047; 0030000a    ; Scale Factor 10

0048; 00480007    ;

0049; 0022000711d2    ; If Parameter 0007 > 7 then jump to line 690

0050; 00620000000000000032    ; Vertex :0  X=0  Y=0  Z=50

0051; 0066ffaa    ; Vertex :1  X=0  Y=0  Z=-36

0052; 0068003c0016    ; Vertex :2  X=60  Y=0  Z=-14

..

..

0687; 008800040074007600770075    ; Transparency = TASKER2

0688; 0000    ;

0689; 0000    ;

0690; 0000    ;

0691; ffff    ; end of file

Anyway, just showing that things aren't necessarily straightforward. :)

Share this post


Link to post
Share on other sites

COINCIDENCE! I am just now writing that I never find the correct LoD! Great, you answer my questions before I send them! :thumbsup:

Just rendered the F-22's shape (f22simp2.3):

f22shape.png

Now I'll implement what you have just written!

Share this post


Link to post
Share on other sites

OK, but I'm having trouble keeping up with you. :)

Actually, the F22 is an exception in that it has separate .3 files for each LOD.

Share this post


Link to post
Share on other sites

No. Look at f22simp2.3:

header ;fffd012c3e80000000007fff00000000fc180000000000020003000300000000008300100007000400050006000000000020000000200000002000000001

0000; 00480007    ;

0001; 002200000016    ; If Parameter 0007 > 0 then jump to line 5

0002; 00620000000000000000    ; Vertex :0  X=0  Y=0  Z=0

0003; 00750000    ;

0004; 003400c00000    ; Light (Short range): Palette:192  Vertex: 0

0005; 0000    ;

0006; ffff00e4    ;

0007; 00300050    ; Scale Factor 80

0008; 002600641b82    ;  Comparison with (100), Jump to line 1131

First six lines for LoD 0 (just one point). I started at instruction 7 for the screenshot.

Share this post


Link to post
Share on other sites

Ah, Ok

Of maybe interest in that sequence is that 'Parameter 0007' is usually (and maybe always) the damage level. So this is telling us that if there is any damage we jump to the end of the model without drawing the light.

Also, on line 8 it is possible that the 0026 opcode means 'If Distance is less than 0x0064 then jump to line 1131', the end of the model.

0027 is 'If Distance is greater than', so maybe not so far fetched.

Share this post


Link to post
Share on other sites

Also: If Parameter is >1 while rendering the simple F-22, the landing gear is drawn. But at a wrong position (in the weapons bay).

The 0038 instruction renders something at the same place. Maybe landing lights?

Share this post


Link to post
Share on other sites

The 0038 instruction renders something at the same place. Maybe landing lights?

I don't think so, since the 0038 seems very similar in form to the 0003 lines just above:

0529; 000300b400a000a100ac0099    ; Flat Shaded Quad, Palette:180  Vertices: 160,161,172,153

..

..

0533; 003800b40096009f00980097    ;

They both seem to have a palette index followed by 4 vertices, and since the palette entries are the same, I assume it's not a light.

I guess you could substitute 0003 for 0038 to see what gets drawn.

Share this post


Link to post
Share on other sites

Yes, you are right! Disregard my comment.

I have a pre-alpha prototype here. You can load models and control the following values:


lights (on/off)

parameter

damageLevel

0024

0025

002D (true/false)

004B

0076 (t/f)

007B (t/f)

0087

0093 (t/f)

I'll send it if that's okay with you. Many simple models load and it should be easier to test the flags this way than reading hex codes :)

Share this post


Link to post
Share on other sites

Okay, it's on its way. If you have problems to get it running, you can tell me here. If there are just some files crashing or not loading LoDs, that's why it is not an official release ;)

The ground tiles should all work. Planes? Don't know. I tested only three or four of them. Missiles, buildings etc not tested at all.

I already found out something interesting in jeddah_j.3: "parameter" controls there if alpha blending is on or off.

Also, I just found out what you meant with color keying:

ftruck1.jpg

(The shadow of the truck)

ftruck2.jpg

The damage is controlled by "parameter" being 1 (O.K.) or 2 (destroyed). 002D flag controls whether a black cross is drawn inside of the truck. I'm sure it is the tires and one of the opcodes we have not yet decoded is responsible for XYZ movement to their final positions when rendered.

Share this post


Link to post
Share on other sites

:beer:

Well, that's just awesome!

It works great within the known constraints.

Controlling parameters with function keys is perfect.

Thank you for doing this. :thumbsup:

002d still mysterious, but enlightenment is sure to come.....

Share this post


Link to post
Share on other sites

Oh GREAT! I was SO afraid of unexpected problems. Now I'm sure I can get a public alpha up until end of the week.

I'm having problems with the missiles. This doesn't look like AIM120C:

aimi.jpg

But I'm working on it :icon_salute3:

Share this post


Link to post
Share on other sites

:thumbsup:

'fp' is prefix for Pylons.

Yes yes, I am so silly. I looked at the hex codes and double-checked, but I never realized this was just a pylon :banghead: Thank you.

I implemented zoom (using mouse wheel) and now finally I worked out what it is with the lines being drawn at roads. 1000 distance ---- the road (in the middle at the bottom) is textured polygons:

jeddah0.png

2000 ---- the textured polygons disappear. Now the road is drawn through the brown zigzaggy lines:

jeddah2000.png

6000 ---- flat shading.

jeddah6000.png

Little difference, but now I know what it's good for. Step by step forward.

Share this post


Link to post
Share on other sites

I noticed the program hangs on the 009f opcode. This is used quite a lot and needs a bit of explanation.

Here's an example from tent_3.3

0039; 009f000e0070040100b700b700bc00bc009e0000000100070006 ; Quad

0040; 009f000c0072030100b800ba00b7009d000500000006 ; Triangle

Taking the quad, and splitting it up:

009f 000e 0070 0401 00b7 00b7 00bc 00bc 009e 0000 0001 0007 0006

This can either render a gouroud shaded polygon or a textured polygon depending on an 'Options' menu setting.

In both cases, the 4 words at the end give the vertex indices.

For the shaded polygon, 00b7 00b7 00bc 00bc give the palette colours for each vertex.

The textured polygon is more complicated. The 0070 is an index to the [surface] section of a 'redXXXX.ini' file.

Here's part of the one from red1000.ini:


[surface]

0=txtr 4,(0,0),(31,0),(31,31),(0,31)			;N	CONCRETE

1=txtr 4,(32,0),(63,0),(63,31),(32,31)			;NE	CONCRETE

2=txtr 4,(64,0),(95,0),(95,31),(64,31)			;E	CONCRETE

3=txtr 4,(96,0),(127,0),(127,31),(96,31)		;SE	CONCRETE

..

107=txtr 5,(224,128),(255,128),(255,159),(224,159)    	;NW	ROCK DEAD

108=txtr 55,(192,128),(223,128),(223,159),(192,159)    	;BOT	ROCK DEAD

109=txtr 55,(224,128),(255,128),(255,159),(224,159)    	;TOP	ROCK DEAD

110=txtr 5,(0,160),(31,160),(31,191),(0,191)		;N	CANVAS DEAD

111=txtr 5,(32,160),(63,160),(63,191),(32,191)		;NE	CANVAS DEAD

112=txtr 5,(64,160),(95,160),(95,191),(64,191)		;E	CANVAS DEAD

113=txtr 5,(96,160),(127,160),(127,191),(96,191)	;SE	CANVAS DEAD

..

119=txtr 55,(224,160),(255,160),(255,191),(224,191)    	;TOP	CANVAS DEAD

So 0070 is hex for 112, so we need this line: 112=txtr 5,(64,160),(95,160),(95,191),(64,191) ;E CANVAS DEAD where 'txtr_5' is itself a pointer to the actual texture given in the first part of red1000.ini: 5=mattdead ....so, 'mattdead.tm' is the texture to use with UV coordinates given by the figures in the brackets, in this case: (64,160),(95,160),(95,191),(64,191) There are a couple of variations of this opcode with flat shading, where only one colour is defined. This part of my script which determines the lengths of the various variants:

            elif opcode=='009f':

                gen1=data[ind+3]

                gen3=data[ind+5]

                gen4=data[ind+7]

                gen2=""

                mm=0

                if gen1=='0401':

                    while mm<13:

                        gen2+=data[ind+mm]

                        mm+=1

                    self.parsed.append(gen2)

                    pind+=1

                    ind+=13

                elif gen1=='0301' and gen4<>'009e':

                    while mm<11:

                        gen2+=data[ind+mm]

                        mm+=1

                    self.parsed.append(gen2)

                    pind+=1

                    ind+=11

                elif gen1=='0301' and gen4=='009e':

                    while mm<13:

                        gen2+=data[ind+mm]

                        mm+=1

                    self.parsed.append(gen2)

                    pind+=1

                    ind+=13

                elif gen1=='0900':

                    while mm<18:

                        gen2+=data[ind+mm]

                        mm+=1

                    self.parsed.append(gen2)

                    pind+=1

                    ind+=18                    

                elif gen1=='0103' and gen3=='009d':

                    while mm<9:

                        gen2+=data[ind+mm]

                        mm+=1

                    self.parsed.append(gen2)

                    pind+=1

                    ind+=9                    

                elif gen1=='0103' and gen3=='009e':

                    while mm<10:

                        gen2+=data[ind+mm]

                        mm+=1

                    self.parsed.append(gen2)

                    pind+=1

                    ind+=10

Share this post


Link to post
Share on other sites

Thank you for the explanation, it will save a lot of time :icon_bow: But I can not say that I get this into the release.

I am having major trouble with LoDs: You may have noticed, the difference between your programs and mine is that you parse the files while I run them. For you, it is no problem to find the FFFF markers; you go straight through. For me, it is. Because I jump through the code and they are unreachable for the processor. Even worse: I cannot just look for them when my program fjinishes (executes the last "0000 return" instruction) because there is no guarantee that the program finishes with the last 0000 in the LoD; depending on flags the program can return at the beginning, in the middle or a few words before the end of current LoD.

So I have to write a parser now, which finds the LoDs in the bytecode and lets the user select a LoD for execution. This not big trouble, because I already have most opcodes implemented and I find all missing pieces in your scripts. But it means a restructuring of the program layout a litte, it is much copy and paste and costs much time.

Also, I have no working INI parser. I have one here, but it is written long ago and I have to test it before I use it. Takes time again. That is why the red1000.ini is hard-coded at the moment. And that is why 009F will be much afford to implement.

I hope you agree with me that LoD selection has higher priority for the first release. But I'll definitely put it in later! :icon_salute3:

Share this post


Link to post
Share on other sites

No problem, and I agree that the LODs and program flow is more important.

Even if you don't implement some opcodes now, I guess you need to know how much space they take up so you can move on to the next.

I believe we did find a way to determine the bytecode starting position for each LOD from the header, but I don't remember the details.

I'll look at it again....

EDIT:

Well, I've looked at it and if the 17th word of the header is 0083 and there are 2 LODS, then the 18th word gives the number of words from the start of the bytecode to the first ffff. The bytecode of the 2nd LOD should then start a couple of words later.

If there are 3 LODs then the 18th and 19th words need to be read...etc.

This may need some adjustment, but I think that's how it works. I didn't bother implementing this in my parser since I'd already used the 'scan for ffff' technique.

Share this post


Link to post
Share on other sites
Well, I've looked at it and if the 17th word of the header is 0083 and there are 2 LODS, then the 18th word gives the number of words from the start of the bytecode to the first ffff. The bytecode of the 2nd LOD should then start a couple of words later.

If there are 3 LODs then the 18th and 19th words need to be read...etc.

This may need some adjustment, but I think that's how it works.

I can not confirm it. First: 0083 is the type word, correct? The one with 0000, 0020, 0087, 00a3 or 0083 which also says where to find the bytecode?

I looked in b2.3, and there, it says 57h (87) for the next LOD. But actually, it is 9Eh (158) words away. I find no number in the header which corresponds to that value. Not even slightly :( Also not in f13.3 or any other I tested.

Share this post


Link to post
Share on other sites

I did a quick scan through the files last night, and while it is not as straightforward as I proposed, there is some hope. There is a definite pattern for the 0000 and 0083 types, but 0087 seems more problematic.

I will investigate further and try to come up with an algorithm....

Share this post


Link to post
Share on other sites

Okay. I have prepared processor and file class so it can technically handle different LOD's. As soon as you find out something, just post and I'm ready to implement it.

Other, unrelated news:

You can choose "open with" on .3 files, search for other programs and choose the model editor. When you double-click on .3 files then, an instance of the model viewer will open and render the file. This should work with the version I sent you; I just forgot to say it.

I had to re-design the stack to remove some dependencies from the processor which would have interfered with LOD's. An interesting thing is the stack size: arleigh.3 uses more than 16 subroutines! I got it to render now, but the superstructures disappear from certain angles. Also, the Gouraud-shaded vertex colors are still not in correct order. And I suspect that their color palette indices are wrong, tool. Don't know if it is a bug in my code or if the palette is addressed in a special way for Gouraud shading. But it is progess.

arleigh.png

I am also trying to open all terrain tiles once so I can be sure the whole terrain renders properlj. I found that aden_b.3 uses lines to simulate street lights. (They are rendererd orthogonal to the street from left to write, then the street is rendered to cover the middle of the line.)

And I implemented your code to find the hard-coded texture at the beginning of the file instead of searching for "TEX_" etc. So much models render now with texture.

Share this post


Link to post
Share on other sites

I did a quick scan through the files last night, and while it is not as straightforward as I proposed, there is some hope. There is a definite pattern for the 0000 and 0083 types, but 0087 seems more problematic.

I will investigate further and try to come up with an algorithm....

I also did a quick scan through the files last night, and although the pattern for 0083 is not exclusive, I am trying to identify the file type pattern.

By the way, no joy on getting the viewer up. I suspect it's related to the very old and limited Intel 845G chip. I did install the d3dx9_43.dll and yet I still get this error:

suspecting bytecode offset at 45 words

Direct3D 9 (SDK version 32)

the Direct3D device does not support the requested multi-sample level; using def

ault

auto-selected software vertex processing

trying to create a Direct3D 9 device (954+ù690, 0+ù AA, windowed)

successfully created a Direct3D device on Intel® 82845G/GL/GE/PE/GV Graphics C

ontroller

provides 36 MiB of VRAM

supports vs_3_0 and

Any help would be appreciated :)

Share this post


Link to post
Share on other sites
By the way, no joy on getting the viewer up. I suspect it's related to the very old and limited Intel 845G chip. I did install the d3dx9_43.dll and yet I still get this error:

...

Any help would be appreciated :)

You, Sir, are very lucky in your bad luck! I have a Eee PC lying around here, and it should have an Intel Express, too. I just had not tested it there yet. I'll debug it immediately :) Does the message end after supports vs_3_0 and or comes more?

Oh, and I found a misplaced light in aswan_h.3:

aswanh.jpg

Not a bad problem, but I hope it does not mean some vertex positions are incorrect again :(

Share this post


Link to post
Share on other sites

Sorry for the bad news, DrKevDog, but: Yes, your hardware is too old. My Notebook has 945 Chipset, not 845. From what I see, the 845 came out in 2001/2002 and does not support DirectX 9 pixel shaders.

I tried to downgrade my shaders to DirectX 8, but: error X3539: ps_1_x is no longer supported; use /Gec in fxc to automatically upgrade to ps_2_0

Microsoft has quit DirectX 8 support one and a half years ago, and I cannot use it through the libraries they give me any more.

The very last chance would be software rasterizing. Considering the age of your hardware and the performance of the viewer on current GPUs, a single frame would probably take at least 30 seconds and up to two minutes to eventually show up. Is there any chance to get a new graphics card? Or do you already have a second graphics card isntalled (I could make the viewer use it then)?

Unrelated: A few new screenshots. I didn't know the dams were hard-coded in the terrain; it requires "damage" and "parameter" both to be set to make damage visible:

damz.jpg

A train, still without chroma key alpha blending

trainc.jpg

A glowing mountain: The flat-shaded area is actually switched on and off with the light switch.

lightfz.jpg

And this is Dave:

daver.png

Share this post


Link to post
Share on other sites

Unrelated: A few new screenshots. I didn't know the dams were hard-coded in the terrain; it requires "damage" and "parameter" both to be set to make damage visible:

damz.jpg

Again great to see the work going on here. If we can maintain our current improvement in the number of MP fliers, TAWBC mission planners and your work here; TAW2.0 is shown to be live and healthy. By the by, the Dam shot is so familiar to many of us who fly MP. It forms part of one of out favorite missions. :thumbsup:

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now