Jump to content
COMBATSIM Forum
Krycztij

TAW 3View — A Renderer for TAW's .3 Files

Recommended Posts

I'd forgotten the limitations of the TFX shape system, so we can't have single objects like this without BSP techniques.

I'm not going to bother fixing the texture orientation right now. :(

ac_taw1.png.482b4f9ae62429da844c7d91dfb18a5f.png

 

Share this post


Link to post
Share on other sites

It looks a bit better at night, and at a better angle...but then again, so do most things. :)

ac_taw2.png.e7190cd410cda0426a5fb5ce08c62a70.png

My conclusion is that TFX and hence 3View uses the 'Painter's algorithm' technique while Lean Viewer parses the VRML and renders based on geometry.

I suppose I always knew that, but nice to see it confirmed...

Share this post


Link to post
Share on other sites

Yes, absolutely. This dependency on Painter’s Algorithm is the main performance bottleneck because it can’t be done in hardware and doesn’t scale with cores.

 

Lean viewer’s thing is called depth buffering and TFXplorer uses it as well, but in a very limited scope to avoid problems with TAW’s geometry.

 

The future VRML-based terrain will use depth buffering exclusively.

Share this post


Link to post
Share on other sites

Is there any advantage in expanding the use of depth buffering and solving problems as they come up by modifying the shape files?

That way the basic file handling would be the same and would give us something to do on the data side.

 

The model in my last picture is just a VRML derived vertex list, texture coordinate list and polygon list converted to TAW's format...and I'm sure you recognize where I got the raw data from.

 

 

 

 

Share this post


Link to post
Share on other sites
39 minutes ago, mikew said:

Is there any advantage in expanding the use of depth buffering and solving problems as they come up by modifying the shape files?

That way the basic file handling would be the same and would give us something to do on the data side.

 

I always try to be as compatible as it comes with the vanilla data. There’s always the uncertainty of whether our project will be banned somewhere *cough SimHQ cough* if we deliver our own TAW-based content instead.

 

Yes I recognized it ❤️ Regarding scale, because this will certainly be a problem: When I last checked, the tiles were ca. 1024 m wide (should be 420 TFX units). This must be a giant building.

Share this post


Link to post
Share on other sites

The base is 2048x2048 units and the tallest building 717 units. Does that map to meters in that version of AC?

Nothing wrong with giant buildings in a futuristic world.

 

The building size can be easily scaled if the underlying tile is a separate shape as in TFX. The tile scaling may be another matter.

 

 

 

Share this post


Link to post
Share on other sites
5 hours ago, mikew said:

The base is 2048x2048 units and the tallest building 717 units. Does that map to meters in that version of AC?

 

From my memory, it should map to meters. But also from my memory, it should have been 1024×1024 units in the VRML file. I’m confused and I don’t find my notes on the experimental checks, so I’d like to withdraw my statement … :(

Share this post


Link to post
Share on other sites

An excerpt of that bit of the original file that relates to the base tile:

	geometry IndexedFaceSet {
		coord Coordinate {
			point [
1024 102 1024,
-1024 102 1024,
1024 102 -1024,
-1024 102 -1024,

so it's +/-1024

 

There's a 102 value for the vertical axis. I've made this zero for my test and taken 102 off each of the building vertical coordinates, then inverted the sign as TFX's vertical coordinates are all negative This probably explains why the texture coordinates are all wrong as I shouldn't do things like that in a coordinate system. But this is also an experiment. :)

 

As an aside, I thought I'd look at the 3View code in VS2015 (community edition) on a PC that hasn't been online a while. Apparently, the license has gone 'stale' so they want me to go online to renew it and won't let me use it until I do. That's annoyed me somewhat as I don't remember the same behaviour with 2010/2012 Express (or whatever it was called)

Share this post


Link to post
Share on other sites
38 minutes ago, mikew said:

As an aside, I thought I'd look at the 3View code in VS2015 (community edition) on a PC that hasn't been online a while. Apparently, the license has gone 'stale' so they want me to go online to renew it and won't let me use it until I do. That's annoyed me somewhat as I don't remember the same behaviour with 2010/2012 Express (or whatever it was called)

 

Yes, it’s very user-offensive. My „trial“ license runs out tomorrow so I’ll have to enter an email address 😞

Share this post


Link to post
Share on other sites

The same PC (Win8.1) is also complaining about out of date Windows Defender and also claims it has updates ready to install. :angry:

While I could quite easily connect it up right now, there are a few times where I don't have (or want) any internet access and I'd be extremely annoyed if it happened then.

The problem is VS201x is just so nice even for my 'Hello world!' level projects, and I haven't seen an alternative that's easy to set up since maybe 'Dev-C++'

 

Anyway, enough complaining...

 

My recent .3->VRML and VRML->.3 experiments have helped me remember how the .3 files go together. I had assumed that the best way to dump out the basic model data would be based on the the 3View code, but it seems I could do this myself by modifying my various Python analysis scripts. This is what I'll probably look into next if time allows, but only for the terrain/buildings to begin with.

We've already agreed that I can forget shadows, but what about the less detailed LODs, particle effects etc?

 

 

Share this post


Link to post
Share on other sites

I don’t have an alternative to Visual Studio either. Eclipse is very clunky and buggy IMHO. I’m forced to use Qt Creator at work, and it has some advantages over Visual Studio (source code browsing/refactoring is easier) but you have to configure a lot regarding makefiles and stuff.


(Thumbs up for Dev-C++, if I hadn’t found it in 1999 or so, I would probably never have started programming.)

 

Please keep the LODs; there is an LOD node in VRML. Samples here, but remember that I don’t support “inline” so you’d place (or USE) your IndexedFaceSet there. Lean Viewer does not automatically support LoD, but you can select individual levels to display by double-clicking them in the scene graph. (If I’ve done it right, it should only ever display one of them at a time.)

Share this post


Link to post
Share on other sites

I don't seem to have any way of controlling anything with Lean Viewer except to turn shapes on and off. Probably not surprising, as VRML is designed to work in conjunction with other scripts in any interactive way.

This is a bit of a problem in navigating through the .3 file, where we need to know the LOD,TOD, damage level etc in order to determine what to render.

 

For a simple model like tent_3.3, I can manually identify the 2 LODs and the 2 damage levels so could just produce 4 separate shapes, ignore the shadows  and worry about which one to display later. Not really realistic for the thousand or so building models though.

Share this post


Link to post
Share on other sites

Your strategy for handling the issue in my last post is already described here:

http://community.combatsim.com/topic/41326-back-to-tfx1/?do=findComment&comment=5194353

For my own personal enjoyment, I've been reading up a bit on VRML and it may be possible to do almost a direct conversion from .3 to .wrl by using ROUTEs to handle the different combinations.

What might work is something like this:

1. Parse the .3 file and identify the decision structure to a allow a ROUTE table to be constructed.

2. For each route, sort out the vertices and polygon descriptors to produce one SHAPE per route

3. Save in VRML format.

Some form of test framework is needed though, and it should roughly match any upcoming API.

In the Python world, there's a couple of utilities that may help, an OpenGL sandbox:

http://pyopengl.sourceforge.net/context/tutorials/

and a VRML library for it

http://pyopengl.sourceforge.net/context/vrml97.html

Although quite old, all the examples I've tried work OK.

 

This is kind of fun, although it won't take much for me to give up. :)

Share this post


Link to post
Share on other sites

No, don’t use ROUTE. Apart from not being supported by Lean Viewer, that’s exactly what made TAW so slow and it would make your new terrain just as slow!

 

The Python VRML library is interesting. I will take a look in the future …

Share this post


Link to post
Share on other sites

Ah, I misunderstood. I thought the slowdown was due to not having the vertices, polygons, texture coordinates etc in IndexedFaceSet-like lists.

For example, how would we deal with things like the windsock? One way might be along the lines of this spinning cylinder example here

Share this post


Link to post
Share on other sites

Windsocks and parachutes are a very difficult corner case, and I don’t know yet how to handle them properly. Please just skip them for now.

 

Some of TAW’s other models have moving/rotating parts: The turret of the Shilka, or some antennas on ships, and so on. If you find any way to do it, please …

  1. isolate the moving parts into a separate object
  2. do not store the entire rotation animation; just store the first frame
  3. if you have any hint at the axis or origin of the rotation, move the coordinate origin of the sub-object to this point

To be honest, I’m not quite sure how to accomplish all that, but that’s what we need …

 

1 hour ago, mikew said:

Ah, I misunderstood. I thought the slowdown was due to not having the vertices, polygons, texture coordinates etc in IndexedFaceSet-like lists.

 

The slowdown is due to the CPU making decisions during drawing: Are we on side XY of the model? Then draw this polygon first then that and skip the other one. After five or six polygons, a similar decision, and so on.

 

The GPU expects a contiguous stream of polygons, but if we skip a polygon, then the stream is not contiguous any more. So all later polygons must be moved to an earlier position and close the gap. This can only be solved with at least one copy per polygon, and this takes ~200 cycles (not counting the overhead for decisions like “what time of day is it”, “how far away is the shape”, and – worst – “on which side of a plane defined by three points are we” (for Painter’s algorithm).

 

At 200 cycles per polygon, you can quickly estimate an upper limit for smooth rendering: 3.2 GHz CPU @ 60 fps makes 53.3 million cycles per frame, so we can’t ever surpass 266 thousand polygons/frame. Assuming 100 % CPU for rendering and 0 % for gameplay here.

 

On modern systems, it’s best to not invoke the CPU at all. Rule of thumb is, if the model has less than thousand polygons, don’t even try to check visibility on the CPU because this visibility computation will likely eat more time than the GPU needs for rendering. Just render it. (Exceptional is excessive material/shader use, e.g. each of the 1000 polygons using a unique material, but that’s rare.)

 

That’s why we need one complete set of polygons per object and per LOD. Just let the GPU handle it.

 

(Lean viewer renders everything blindly and without CPU interference, and it gets ~1000× TFXplorer’s polygon count.)

 

Same for moving/rotating parts: Let the CPU only change the rotation matrix (16 values) and let the GPU rotate thousands of vertices to the new position (because GPUs are pretty good at that). That’s why I only want one indexed triangle set per moving part, and not TAW’s stuttering animation: We’ll just change the matrix and the part will move smoothly and with high efficiency.

 

And as soon as that works, we’ll add Skeletal animation.

Share this post


Link to post
Share on other sites

I was always going to try to resolve the 0015 BSP-style blocks and sub-meshes into one set of polygons, but the routing was to handle the other decisions,  ie distance/LOD/damage/parameter/TOD. I can ignore that completely for now by creating separate files for each route.

 

There are possibly better ways of representing TOD these days than changing the palette every 2 hours, but I'd like to try to preserve as much of the data as I can. The whole process has to be done programmatically as I don't want to be making manual decisions at run-time. 

This may turn out to be a step too far, but it would be nice to have the data from a  known world in a generic format.

 

 

Share this post


Link to post
Share on other sites

I misunderstood, sorry!

 

If you want to preserve as much data as possible, how do you want to add dynamic lighting? Imagine:

  1. A building has a bright texture at day
  2. … and a dark texture at night
  3. At night, something explodes right next to it and casts orange light on the building
  4. … but the building’s texture is dark, so there is no effect. It’s just dark under all circumstances.

 

I would just always use the bright day texture and let the engine darken the lighting at night, not the actual texture.

 

(Lights are a different beast. If it’s a lit area, put that into the material’s emissive component. If it’s building lights etc, we make a separate texture, e.g. with _EMISSIVE prefix and the engine will treat it automatically. I did that here, for example.)

Share this post


Link to post
Share on other sites

I should have used 'extract' rather than 'preserve'. It doesn't need to be used, but it's there if required.

 

Without assuming any level of commitment, how do you see the transition to a more flexible engine will take place?

From my point of view, TFXplorer looks wonderful as it is, but I understand things aren't ideal under the hood.

Is it worth using the current assets to create the same look and feel with a new engine, or to make a big leap straight away?

 

Lighting is a good example, as things like streetlight effects would need to change from a simple palette/texture system to a bunch of point sources that need to be placed. A seemingly exponential increase in level of complexity.

 

Share this post


Link to post
Share on other sites
24 minutes ago, mikew said:

Without assuming any level of commitment, how do you see the transition to a more flexible engine will take place?

From my point of view, TFXplorer looks wonderful as it is, but I understand things aren't ideal under the hood.

Is it worth using the current assets to create the same look and feel with a new engine, or to make a big leap straight away?

 

The current look of EF2000/ADF/TAW in TFXplorer will be preserved 99 % after the transition. I try to not break anything. I had to remove the canopy reflections, but that’s it.

 

I would love to make a big leap straight away. Consider that terrain is logically separate from vehicles (planes, ships and tanks) – we can proceed with TAW’s F-22 cockpit chasing TAW enemies in a VRML-powered terrain of our choice.

 

24 minutes ago, mikew said:

Lighting is a good example, as things like streetlight effects would need to change from a simple palette/texture system to a bunch of point sources that need to be placed. A seemingly exponential increase in level of complexity.

 

Make it a point list with emissive material. Lean Viewer doesn’t support points yet, but it will.

 

Consider streets preliminary. I’d love to have OpenStreet import and fill the streets with a million cars (that’s four million lights, isn’t it?) and this requires special case handling for streets, probably procedural generation of streets, street lights and bridges along predetermined paths. Until we get there, just place some bright points for street lights. Still better than TAW’s :) 

Share this post


Link to post
Share on other sites

I've been having another look at Openstreetmap, and there's some nice tools for it. In a homage to TAW, I downloaded the dataset for Eritrea and trimmed it down (using a tool called Osmosis) to an area of Eastern Asmara, roughly corresponding to this:

asmara.png.17df746c9a12cedea5b0c06b1c0b0bd8.png

There's a tool called QGIS which is useful to visualize the bounding box for the osmosis command line parameters.

Once we have the area of interest, another tool OSM2World was used to create a 3D representation based on the osm data, which can be exported as an .OBJ file.

osmtools.png.bd9d8588989c1bed5fe79179789b6106.png

From there, we can convert to VRML, although we've lost whatever material we had along the way.

asmara_vrml.png.4e97b95e057859d7a75683ba839bde8c.png

 

I just used the default settings, but there is some flexibility in what osm data is used, and what models are used to produce the .OBJ file.

 

Some alternatives here:

https://wiki.openstreetmap.org/wiki/3D_development

 

I don't know if this is a practical way to create a flight sim world as even that vrml file is 6MB, but it's fun to play with this stuff. :)

Share this post


Link to post
Share on other sites
22 hours ago, mikew said:

I don't know if this is a practical way to create a flight sim world as even that vrml file is 6MB, but it's fun to play with this stuff. :)

 

 

Let’s not care about the practical part – the VRML will be preprocessed and cached as a binary file. Let’s just make the most out of it, and your work is just stunning!

 

Share this post


Link to post
Share on other sites

Not my work, but the result of tools produced by German students who appear to have embraced OSM in a big way.

I'll look into this a bit more since my enthusiasm for producing a 3->VRML conversion tool has diminished somewhat.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×