Krycztij

TAW 3View — A Renderer for TAW's .3 Files

56 posts in this topic

TAW 3View aims to help modders by providing quick feedback on palette/texture/mesh modifications (quicker than starting the game, at least :) ). In the long term, we hope to get a better understanding of how TAW manages and renders its content.

I will use this topic to release new versions, to assist on technical problems, and to discuss bug reports and feature requests. TAW 3View's internals, the encoding of .3 bytecode and other details should be discussed in the Development Thread.

Let's get started with a first Beta:

TAW 3View Beta 1

3view.png

System Requirements:

I tested it on Windows 7 only, but it should work with Windows XP and Windows Vista, too.

You need a Direct3D 9-compatible graphics card (ATI Radeon 9500 / Nvidia GeForce MX or higher -- basically any decent GPU you bought after 2003).

Download:

The software is still in an early stage of development; not well tested; I will not be held responsible if this software causes any damage to your computer.

http://www.mediafire.com/?yrnyd6r96d6i650

Setup:

If you have not already installed the DirectX Runtime June 2010 Update (x86), get it here. Otherwise, TAW 3View will not start due to D3DX9_43.DLL and other files missing.

Extract the ZIP to your TAW\PROGRAM folder.

Start 3View.exe. A console and a window should open and a tile of jeddah should show up. If not, look for errors on the console and post here. You can close the program again by hitting ESC, or closing the window or the console.

Navigate to "TAW\PROGRAM\3" and find "abu_sim.3" (a file of that I know it works).

If you have already associated .3 files with a program, right-click onto the file and choose "Open with...", "Select...".

Else double-click onto the file, check "Select a program from a list of installed programs" and click OK.

Click "Browse..." in the lower left of the new window; navigate to TAW\PROGRAM and select 3View.exe. Click OK.

Check "Always use the selected program to open this kind of file" (if you are not a modder and want to open .3 with a hex editor instead); then click OK. A tile of Abu Simbel should pop up.

Usage:

This is difficult, but -- sorry, I didn't have time to build a fine user interface. And as an old saying goes: It was hard to program, so it should be hard to use! :)

The first thing to notice is that it is very slow (performance has not priority at the moment). It becomes a little faster if you minimize the console.

You can rotate the model with Numpad 2, 8, 4, 6. F1 toggles wireframe mode, F2 toggles lights. The mouse wheel can be used to adjust the distance. This may seem a little strange, but the engine auto-adjusts the zoom so the model has always the same size on screen.

F3–F9 toggles various flags. Parameters can be increased and decreased with A/Y, S/X, etc. We don't know the meanings yet and most of them work in combination, but if you see a damaged model, it is very likely to get the undamaged version by decreasing "parameter" to 0 (or to get the fubar version by increasing it).

The top-left says which LOD is loaded ("LOD x of n"). If n > 1, you can switch to the next LOD with the up arrow key (and back with down key, respectively).

Known Bugs & Constraints:

  • Many models are not rendered correctly: Wrong colors, wrong textures, missing polygons, flickering, etc. We're on it. More than 95 % of terrain tiles, however, should show up perfectly.
  • It's slow. I'll consider performance later; currently, the primary goal is to get all models rendering.
  • It's difficult to use. I'll build a user interface when most models show up properly and when we know how exactly the flags and parameters are controlled. It's wasted time otherwise.
  • You cannot maximize, minimize or resize the program (I had no time to implement this yet).
  • Double-clicking a .3 file while the viewer is running results in a second viewer popping up.
  • Currently, the viewer uses the "1000" version of the textures and the "1200" palette only. I'll try to implement time of day, palette switching etc. for the next version.
  • Although all vanilla models should load, I cannot guarantee that it doesn't crash if you load modified files.

If you encounter an error popup, hit CTRL+C and paste it in your answer along with the name of the crashing model. If something crashes / hangs / does not start / etc, let me know here.

Finally: Happy modding!

I'd like to thank mikew and DrKevDog for their help. Basically, I just translated their advice to C++ and plugged a renderer on it ;)

-- Pjotr

Share this post


Link to post
Share on other sites

Excellent.

My vote for first improvement is to the 'parameter' system.

There are a total of 0x19 parameters, which are selected by the 0048 opcode.

So, there should be a way to increment/decrement the parameter index...then for the current index, there should be a way to increment/decrement the value of the parameter.

So, for instance, the damage level can be changed by incrementing the 'parameter' index until it reads 7. The actual damage level would then be changed by increasing it's 'value' from zero (undamaged) to maximum damage (which depends on the model).

No hurry though. :)

Share this post


Link to post
Share on other sites

Excellent.

My vote for first improvement is to the 'parameter' system.

There are a total of 0x19 parameters, which are selected by the 0048 opcode.

So, there should be a way to increment/decrement the parameter index...then for the current index, there should be a way to increment/decrement the value of the parameter.

So, for instance, the damage level can be changed by incrementing the 'parameter' index until it reads 7. The actual damage level would then be changed by increasing it's 'value' from zero (undamaged) to maximum damage (which depends on the model).

No hurry though. :)

Technically no problem; keyboards have not enough keys for 25 parameters anyway. The real challenge is to identify all 25 parameters :)

Share this post


Link to post
Share on other sites

Announcing

TAW 3View Beta 2

Almost all models rendering now!

3vbeta2.png

System Requirements and Setup are unchanged.

Download:

The software is still in an early stage of development; not well tested; I will not be held responsible if this software causes any damage to your computer.

http://www.mediafire.com/?zam9on129e281nz

Usage:

The usability has significandly improved since Beta 1. Just like in TAW, you can use the mouse while holding the left mouse button to rotate the model, and hold the right mouse button to adjust the distance. The mouse wheel is used for zoom.

The top-left says which LOD is loaded (LOD x of n). If n > 1, you can switch to the next LOD with the up arrow key (and back with down key, respectively).

Parameters control the model's state. The left and right arrow keys are used to select parameters. You see the currently selected parameter in the top left behind parameter #:. Use A and Y to increase or decrease the parameter, respectively.

We have yet to identify all parameters, but for example, parameter 7 often means "damage" and the parameters 1, 2 and 3 control landing gear and the orientation of rotors and turrets.

You can see aswan_j.3 breaking by pressing the right arrow key until parameter #7 is selected and increasing the parameter by holding A.

If a parameter is not used by a certain model, it is marked (idle) so you don't need to waste time on it.

F1 toggles wireframe mode, F2 toggles lights. F3 and F4 are used for flags whose purpose we have not identified yet; S/X and D/C increases and decreases other unknown values. F12 enables debugging mode, which will write a log to stdout (which is usually the console, but can be a file if you started the application from the command line).

Known Bugs & Constraints:

  • It's still difficult to use. But it's getting better and better.
  • You cannot maximize, minimize or resize the program (I had no time to implement this yet).
  • Double-clicking a .3 file while the viewer is running results in a second viewer popping up.
  • There are glitches in the 180 and 90 versions of many models. The cause is known; I'm working on it.
  • There are glitches in the Comanche, Challenger, Chinook, F-14 and ZSU-23-4 models. This is still under investigation.
  • Many canopies (e.g. MiG 21's) are drawn dark blue.
  • Currently, the viewer uses the 1000 version of the textures and the 1200 palette only. I'll try to implement time of day, palette switching etc. for the next version.
  • Although all vanilla models (except ian*****.3, neil_10.3, radar.3, rdr_90.3 and rdr90.3) should load, I cannot guarantee that it doesn't crash if you load modified files.
  • ufo2.3 crashes if parameter #7 is above zero.

If you encounter an error popup, hit CTRL+C and paste it in your answer along with the name of the crashing model. If something crashes / hangs / does not start / etc, let me know here.

-- Pjotr

Share this post


Link to post
Share on other sites

Announcing

TAW 3View Beta 3

It's been a long time since Beta 2 and I'm still not satisfied, but I guess you can use an unfinished version better than waiting for another ten months …

3viewbeta3.png

System requirements and Setup are unchanged.

Download:

The software is still in an early stage of development; not well tested; I will not be held responsible if this software causes any damage to your computer.

http://www7.zippysha...87128/file.html

Usage is unchanged except for Red Sea Explorer (see Features below).

Features:

  • Red Sea Explorer (by just starting the program from the TAW\PROGRAM directory)
    • left mouse button glides forward
    • repeatedly pressing S increases viewing range (it's four tiles default, because that's what my Eee PC can still render smoothly); Y decreases it
    • NUMPAD 8 / 2 moves North / South
    • NUMPAD 4 / 6 moves West / East
    • F5 disables target names

    [*]time of day can be switched via F2 (V toggles night vision; F4 toggles Moon)

    [*]implemented TAW's virtual file system using john_doe's RA decompression and mikew's hash algorithms (preferring DID.DAT greatly reduces startup time)

    [*]x86-64 compatibility

    [*]TAW 2.0 compatibility

    [*]new input system (the mouse will be tracked above the client window exclusively now)

    [*]you can now minimize, maximize and resize the window

    [*]using a self-made TAW font and a dedicated font renderer for on-screen information

    [*]introduced fixed-point arithmetics and rebased the Red Sea Explorer to use it

    [*]declared data types and coordinate systems as necessary to simulate TAW scenarios

    [*]listed all code names, plane types, and city and tower names, and associated them with voice fragments for the upcoming radio system

Fixes:

  • fixed overdraw issues with the terrain in exploration mode
  • fixed cloud coverage issues
  • fixed many bugs regarding minimization / resizing the window too small
  • fixed crashes on missing supershapes
  • dotted lines look more familiar now

Miscellaneous:

  • performance improvements:
    • all geometry is batched now
    • x64-optimized math library with 30 % gain over XNA Math
    • reduced geometry data overhead

    [*]restructured the source code so the TFX / EF2000 / ADF / TAW data structures (libTFX now) can be used independently of 3View (I'm planning to outsource the Red Sea Explorer into a seperate program)

    [*]introduced TFX nomenclature according to strings found in TAW's binaries:

    • SSD's are now supershapes
    • 3 => shape
    • 3 type => shape class
    • LST => supershape list
    • ENV => world

Known Bugs:

  • LOD is wrong when viewing SSD's
  • very few SSD's can be viewed (in fact, viewing SSD's is still limited to landscape)
  • red****.ini is still hard-coded, so any changes will not work in the viewer (e.g. TAW 2.0's Aswan dam texture modification)
  • fog does not adapt viewing range
  • there are glitches in the Comanche, Challenger, Chinook, F-14 and ZSU-23-4 models — this is still under investigation

There's still so much on my to-do list that I don't even know where to start … therefore, I could not test this version intensively. Please report any problems and bugs you encounter; I'll try to help and fix.

Share this post


Link to post
Share on other sites

Epic!

Had a quick play scrolling across the terrain and all seems to be working well.

On my keyboard it's F5 that toggles target names.

I wish I could produce stuff like this, but wouldn't know where to start. :)

Share this post


Link to post
Share on other sites
On my keyboard it's F5 that toggles target names.
That's correct, I must have mistyped it … fixed.

I wish I could produce stuff like this, but wouldn't know where to start. :)
Considering that much of what you see is based on your work, you already did that ;)

Share this post


Link to post
Share on other sites

I'm sure TAW 3View Beta 3 is top shelf :thumbsup: (ilivid ???) Can't wait to see where this will go. I will continue working on fleshing out the ENV2 format, I suspect it will be of some benefit here.

Share this post


Link to post
Share on other sites

Just been sightseeing around the red sea region and notice a couple of white tiles, one at djibouti and the other at a nearby tile labelled 'obock ew'.

It would be nice to be able toggle the view slaved to the mouse position. Just when I've lined up for the perfect screen shot, I can't easily move the mouse curser out of the way.

Pressing alt seems to do this but when I press alt again, the view changes.

These are minor points though. I could spend hours just looking around. :)

Share this post


Link to post
Share on other sites

From what I see, vive block_4_pointer_1 errors are shown when loading is complete; i.e. five SSD's could not be loaded. I guiess these are the blank tiles.

I hope these go away automatically when my SSD re-implementation is complete. The current SSD loader is merely a copy of your Python code, and I must re-write the whole thing to include our new findings like collision meshes.

If you are keen on investigating: The error occurs in TAW_Supershape.cpp, line 137. The five files are:

1. ss\desewrs2.ssd

2. ss\desewrs4.ssd

3. ss\dunewrs2.ssd

4. ss\dunewrs4.ssd

5. ss\djibou_b.ssd

You probably have spotted 1., 2., and 5. in the viewer.

I'll have a look at the mouse handling; it annoys me, too.

Share this post


Link to post
Share on other sites

Ah, yes. It's my fault. :)

Not sure why right now, but there is definitely something strange with djibou_b.

I can't remember exactly what I was doing, but something is going wrong with block identification.

In this summary, there are two Block4_pointer1 entries and an empty Block 7.


;File ;djibou_b.SSD

;Size ;2720

;Block1   0   10

;Block2   10   100

;Block3   110   0

;Block4   110   8

;Block5   118   6

;Block6   124   6

;Block7   130   0

;Block4_Pointer_1   130   0

;Block4_Pointer_1   130   8

;Block4_Pointer_2   138   2

;COLLISION_TRIANGLES   140   2370

;TERRAIN   2510   2

;WATER   2512   2

;NB_ROUTES   2514   2

;ROUTES   2516   180

;Block4_Pointer_3   2696   2

;Lab_Block   2698   22

;Block1 Starts 0 Length.10

0001e803011effff0400

;Block2 Starts 10 Length.100

00000000000000000000

00000000000000000000

00000000000000000000

00000000000000000000

00000000000000000000

00000000000000000000

00000000000000000000

00000000000000000000

00000000000000000000

00000000000000000000

;Block3 Starts 110 Length.0


;Block4 Starts 110 Length.8

040009000c000a05

;Block5 Starts 118 Length.6

7a00

bb05;    DJIBOU_B

ffff

;Block6 Starts 124 Length.6

03000000bb05; Object 0  DJIBOU_B

;Block7 Starts 130 Length.0


;Block4_Pointer_1 Starts 130 Length.0


;Block4_Pointer_1 Starts 130 Length.8

07000000

01000005

;Block4_Pointer_2 Starts 138 Length.2

0000

;COLLISION_TRIANGLES Starts 140 Length.2370

etc

etc

I'll try and find the time to investigate further.

Share this post


Link to post
Share on other sites

This is more interesting than what I'm supposed to be doing.....

Anyway, the root cause of this is that our 'standard model' of the ssd file structure is incomplete.

For TAW, we know that 'zsam' and 'c17' are anomolous, but it appears that my sanity checks were not very rigorous because the five files mentioned above don't conform either....and I guess there must be others.

I know that there are further problems if I try and apply the same rules to the EF2000 ssd files.

So, what to do?

I suppose the answer is to start again, and try and find rules which all ssd files conform to.

Share this post


Link to post
Share on other sites
So, what to do?

I suppose the answer is to start again, and try and find rules which all ssd files conform to.

I have not yet begun with re-writing the SSD code; so, from my point of view it's no problem or code waste if you want to start from scratch. I will, however, not have much time to help you since the to-do for 3View etc is quite long and my spare time is rather short …

Share this post


Link to post
Share on other sites

Well, not exactly starting from scratch, but it would be nice to make all .ssd files parseable in a consistent way, even if we don't know what each part does.

After a brief look at djibou_b.ssd, it would seem that Block7 comes after the data pointed at by Block4_Pointer_1.

My code has assumed that Block7 immediately follows Block6 but this is obviously incorrect.

My problem is that I can not find any direct pointer to Block7, which is why I made that assumption. The number one priority (at least when I get the time to look at it) is to find this pointer, otherwise we're in trouble.

How much of the ssd file are you using for 3View? I'm guessing you don't need the Block4_Pointer_XX stuff for now, so you can probably ignore this problem, at least for the terrain explorer function.

Share this post


Link to post
Share on other sites
How much of the ssd file are you using for 3View? I'm guessing you don't need the Block4_Pointer_XX stuff for now, so you can probably ignore this problem, at least for the terrain explorer function.
As far as I can see, I do the following:
  • skipping block 1
  • skipping block 2
  • reading positions from block 3
  • skipping block 4
  • reading shape indices from block 5
  • reading object list from block 6
  • skipping block 7
  • reading object list behind block 7, accessing data from blocks 3, 5 & 6

I guess the problem is, just as you stated, we don't use the pointer from block 4 to access the data after block 7 but instead just parse after block 7.

Share this post


Link to post
Share on other sites

I'm now using block 4 offset + (block 4 pointers[1] + 1) × sizeof(UInt2B) to get to the object data. Works pretty good — instead of five, there is only one supershape error (djibou_b.ssd) … the EWR sites are now loading o.k..

Share this post


Link to post
Share on other sites

Good, but I don't really understand th problem with 'desewrs2.ssd'. This should work just like an airbase or town.

'djibou_b' is a bit different in that the position of Block7 is unusual, but the Block4_Pointer_1 and Block4_Pointer_2 offsets are OK.

The problem may be that there is no "terminating 0000" between the (07000000) and Block7 (01000005).

Share this post


Link to post
Share on other sites

3View Beta 3 is working smoothly.

I can't get night vision with "V", but I'll troubleshoot that later, everything else looks good.

Thanks again :)

Share this post


Link to post
Share on other sites
I can't get night vision with "V", but I'll troubleshoot that later, everything else looks good.
You must toggle the time of day to night time using F2 before you can enable night vision.

Any idea why these blue lines occur in the game but not in the Viewer?

Short explanaition: Because I've made it better ;)

Long explanation:

TAW realizes color keying as alpha blending with linear texture filtering. The lines occur because, halfway between a transparent and an opaque pixel, the transparency is interpolated to 50 %, but the pixel color is also interpolated 50 % between the opaque color and the color key (which, in this case, is blue).

I, on the other hand, have implemented color keying using premultiplied alpha to avoid this exact issue. The result of the blending is the same if you consider only pixel centers, but it avoids colored borders between transparent and opaque pixel centers. (Moreover: it avoids artifacts with non-linear fog like the one I'm using.)

Share this post


Link to post
Share on other sites

I've seen this problem reported many times over the years, and experienced it myself with a TNT2 card back when it was the thing to have.

It's really a driver issue for the DX5.2 that TAW uses, but I guess not serious enough for AMD or NV to fix.

I also assumed it was because the key colour is blue....which it may be. The key colour for Glide is something different though.

...and back on topic:

It would be really good to have some basic indication about which direction we are looking at in the terrain view.

Share this post


Link to post
Share on other sites

The DX that TAW uses went out of fashion long before Win XP.

And today even OpenGL 1.0 and 2.0 are End Of Life meaning any issues that are still there, regardless of severity won't be fixed.

This is what is going to hit a certain company in San Francisco REAL hard in the coming years (Linden Labs and Second Life which still is single threaded and uses legacy Open GL today)

Share this post


Link to post
Share on other sites

Wow!, is 'Sadville' still going? You don't see much reported about it these days.

Where did we get to with rendering order?

There are quite a lot of rendering order problems in TAW, at least as seen on the ground. So. maybe DID didn't bother to calculate the order except a basic Terrain->Buildings->Mobile sequence, as from the air there is unlikely to be any problem.

A "problem" with 3view is that we can observe the scene from any angle, so some rendering order needs to be calculated based on distance for the scene to be rendered properly.

Share this post


Link to post
Share on other sites
Where did we get to with rendering order?

There are quite a lot of rendering order problems in TAW, at least as seen on the ground. So. maybe DID didn't bother to calculate the order except a basic Terrain->Buildings->Mobile sequence, as from the air there is unlikely to be any problem.

A "problem" with 3view is that we can observe the scene from any angle, so some rendering order needs to be calculated based on distance for the scene to be rendered properly.

The Terrain->Buildings->Mobile sequence is very unlikely, because it does no solve the problem of concrete foundations accidentially covering runways. Distance sorting does not solve this either (I have already tried). There must be something precalculated in the supershape files … If I remember correctly, we had found something like a binary tree in one of the data blocks — if there was one object in each leaf, this would be the data structure we'd have to traverse for rendering.

Share this post


Link to post
Share on other sites

It's been a while since I've looked at this, but I don't remember any such binary tree. There are a few blocks without explanation though...

Maybe Block2 is involved. Terrain objects have entry 00, runway 01, buildings 08 etc. On the other hand, EF2000 didn't have this block, and if I start the Luxor Takeoff training mission, I can see the runway lights through the hanger behind me. Hmmmn. :(

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now