Jump to content
COMBATSIM Forum
Krycztij

TAW terrain format

Recommended Posts

:anim_bounce:

 

1 hour ago, mikew said:

I can't get the same orientation as 3View by inverting either X or Z. In fact if I do that, the Z-buffering seems to fail.

 

Oh! Screenshot of the Z buffer fail, please. Could it be that the triangles are inverted? Mirroring an axis swaps the winding order of indices from clockwise to counter-clockwise and vice versa …

Share this post


Link to post
Share on other sites

I'm sure there is no problem with the Z Buffer handling, but not much can be done if the data in it is wrong.

The orientation is the least of my worries now.

 

This model specifies 4 vertices, draws a concrete pad under the hanger then reassigns the vertices for the hanger itself. I wasn't expecting that sequence and it will be awkward to fix.

That pales into insignificance though when we see the result of 2 textures fighting for the same polygon. As I understand it, a texture can override a shaded polygon in VRML, but you can't have coplanar textures. :(

Hopefully, I've missed the chapter on decals. :)

hdha_180wrl.png.c866039820dcbe869922256067745b2f.png

 

Share this post


Link to post
Share on other sites
7 hours ago, mikew said:

This model specifies 4 vertices, draws a concrete pad under the hanger then reassigns the vertices for the hanger itself. I wasn't expecting that sequence and it will be awkward to fix.

 

Yes, this broke my ZSU-23-4 rendering back in 2012. And I guess to some degree it’s the reason for the odd canopy in 3rd person view in TFXplorer :(

Wait, two textures on the same polygon? I need to check this file out.

Share this post


Link to post
Share on other sites

The 'two textures on the same polygon' in that picture was due to my misunderstanding. If I rotate the model, we get z-fighting around the hanger door area.

However, we will have that for terrain tiles where the textures are piled on each other. Luxor has a base texture, scruff, taxiway and runway...all coplanar.

Share this post


Link to post
Share on other sites

For that model (hdha_180), I'm sure it does.

 

The problem comes when we come to a file like tluxor.3, which is the base tile for Luxor airbase. It contains the base texture, then a 'scruff' layer is placed on top.

With 3View's painters algorithm, it's fine, but I can't find VRML's method of handling this.tluxor_wrl.png.36a19c7bbc9f00d4043e4ef087aa0464.png

3View on the left. :)

What you see in Lean Viewer depends on the viewing angle. The transparency is another problem, but there's maybe a setting for that.

Share this post


Link to post
Share on other sites

Yes, the black areas have palette position 0 which means transparent in the TFX world, so we should just see the underlying texture.

The problem is that VRML doesn't support putting one texture on top of another. I've done that, and the outcome is Lean Viewer sometimes shows the bottom texture or sometimes the top one. Not very visually appealing...

There is a switch in the tluxor bytecode to not draw the 'scruff' at all, corresponding to some TAW detail setting, but I'm trying to handle the highest level of detail and we'd have to deal with this scenario anyway when it comes to tiles with roads and rivers on them.

The workaround for this is to somehow combine the textures in my code and create a new texture, tluxor.bmp that should look like the picture to the left in my previous post.

That technique should be good enough for road/river tiles, but for Luxor airbase (and similar) I'll have to modify the texture again to bake in the runway, building bases etc based on analysis of the shapes used.

Share this post


Link to post
Share on other sites

VRML has no method of handling this; no modern game engine has. There is a concept of decals, but it doesn’t apply here.

 

TFXplorer renders fine due to two facts:

  1. a special depth buffer setting („draw pixel if its distance is smaller than or equal to previous pixel’s distance“); this avoids Z fighting (the flickering) in most cases
  2. GPUs must guarantee ordering of triangles, i.e. pixels of later triangles must cover those of earlier triangles if the depth buffer value is equal

In essence, the later pixel always wins. I programmed it this way specifically for TFX’s painter’s algorithm. I do not want this in the new VRML renderer because it prevents efficient grouping of triangles by texture. Assume three triangles:

  1. with a grass texture;
  2. with a dirt texture;
  3. again with a grass texture.

These would generate the following command sequence with classic TFX-era rendering:

  1. select grass texture (costly)
  2. draw triangle #1 (cheap)
  3. select dirt texture (costly)
  4. draw triangle #2 (cheap)
  5. select grass texture (costly)
  6. draw triangle #3 cheap

More efficient would be:

  1. select grass texture (costly)
  2. draw triangles #1 and #3 (cheap)
  3. select dirt texture (costly)
  4. draw triangle #2 (cheap)

(This grouping is typically done offline, during loading of the scene.) Keeping the ratio of triangles to texture changes high is paramount for performance – imagine a million triangles collapsing into four commands if they just use two textures.

 

But obviously it breaks when all three triangles are in the same place, because now we’ll see dirt instead of grass!

 

For now, I see two solutions:

  1. The Right Thing to Do: create a new texture from merging the overlapping polygons into one. This is pure math horror, it’s academic and we won’t do it.
  2. Cheap Trick: When you define the materials, define them in the order of drawing. I may not be able to guarantee the order of triangles, but I may be able to guarantee the order of materials. I.e. define the grass first and the dirt later. I’ll take measures to assure it’s rendered in this order.

… and I’ll have a look at Lean Viewer this weekend so you can inspect your output properly.

 

Share this post


Link to post
Share on other sites
1 hour ago, Krycztij said:

The Right Thing to Do: create a new texture from merging the overlapping polygons into one. This is pure math horror, it’s academic and we won’t do it.

That was exactly what I was thinking of doing, but 'pure math horror' isn't exactly filling me with confidence. :)

 

Lean Viewer is fine for now although being able to 'inline' shapes will probably be useful later.

Share this post


Link to post
Share on other sites

For these terrain tiles with two or more textures, my plan was something like this:

1. Convert all .tm textures to .bmp files, with the alpha value of palette position=0, and 255 for all other positions.

2. Use OpenGL to render the base texture

3. Add the next layer, but using blending.

4. Save the resulting frame buffer as a texture.

5. Use that texture in the VRML model of the tile.

 

Step 3 is proving a bit of a problem. I can't get the top layer to be completely opaque despite trying nearly all combinations of the 'glBlendFunc' parameters.

The transparent part seems to work at least.

Here's the some test textures with and without blending.

noblend.png.b36767d6ba8c35c7fc7b0ce5f4e1cb61.png

Hopefully, it's some incompetence on my part, but have not had much time to investigate this weekend. :(

Share this post


Link to post
Share on other sites

Did you write the BMP files yourself? Did you load them yourself?

 

If you use 3rd-party libraries, please consider that very few software supports transparency in BMP at all. Lean Viewer does (because its BMP loader was written by me), but when I switch to Windows’ built-in BMP loader, all transparent parts come out as black.

 

Could this be a problem? Did you try to use PNG instead (which supports transparency from the ground up)?

 

Apart from that, I’m really impressed by that workflow!

Share this post


Link to post
Share on other sites

Good point. I assumed the A values were being used since there are transparent parts in the blended image, but we'd get the same effect if the RGB values in the images were simply added.

I made the BMPs, which consist of an appropriate header, then the palette (the RGB from .col converted to BGRA) followed by the .tm contents, but with the raster starting from the bottom. The loader is what is built into PyGame/PyOpenGL and from the sparse documentation I see that there may be a problem with indexed colour bitmaps regarding transparency.

 

I'll see what happens with a 32bit RGBA BMP as that should be fairly easy to make, at least compared to PNG format.

Share this post


Link to post
Share on other sites

32-bit BMP support is very spare as well; 90 % of facilities built into windows don’t support it either. (Yes, the situation *is* this bad, and it was one of the original reasons for me to make Lean Viewer.)

 

I really recommend PNG, it will work well even with transparency in indexed colors.

Share this post


Link to post
Share on other sites

I would like to take a shot at this. In doing so i would assume WYSIWYG for your images and describe what I believe I see.

On the first image I see what you later described as Additive Blend Mode layers (E = min(M + I),255) and I would agree that there is no real transparency of an alpha type.

This is the additive-image I see in my nightmares, ever since I was introduced to 24 bit, AGP  color in TAW using 0088.

On the second image I disagree with the descriptor 'without blending'. What I believe I see is 2 layers in Normal Blend Mode (E = M). This is probably the default blend mode and, with 100% opacity, only the upper layer is visible. You mentioned there may be a problem with indexed colour bitmaps regarding transparency and that might constitute the be all and end all unless a way to effect transparency can be found.

Share this post


Link to post
Share on other sites

Thanks DKD, but it's fixed now. 32-bit BMP didn't work, so converted to PNG using mtPaint since MS Paint doesn't recognize the Alpha at all so doesn't it save to the PNG.

I did look at producing the PNG file directly, but gave up after looking at the spec for a minute or so. It reminded me of those .lbm files I was looking at for the moving map.

 

Anyway, the transparency didn't work then either which was a bit worrying, but that was the fault of OpenGL which has far too many parameters for its own good. :)

Changing a 3 to a 4 did the trick. It turns out that the 32-Bit BMP would have worked after all, but not the indexed colour BMPs I started with.

gl_hell.png.959c156270f9b27bb0feaf5090283802.png

 

...and why does this site's spell-checker complain about the correct spelling of 'colour'?

I thought Combatsim was based in Canada...

 

Share this post


Link to post
Share on other sites

Pretty awesome, as always! And very elegant, too … I’m so keen on using all these shiny new things …

 

Mike, here’s a new Lean Viewer build. No things of great interest, but just to synchronize our state. If you hit CTRL+C while displaying a mesh, you can paste a screenshot into Gimp or other tools. (It doesn’t work with all programs because Windows clipboard is fubar, but Gimp is compatible and even supports my transparent background.)

Share this post


Link to post
Share on other sites

Thanks! The build of Lean Viewer I've been using is quite old, but my VRML files are not very sophisticated. :)

One thing that might be useful is to view PNG files directly. While they work when called from the model, the only format that seems to be supported for viewing is BMP.

Share this post


Link to post
Share on other sites
2 hours ago, mikew said:

One thing that might be useful is to view PNG files directly.

I fall back to Windows’ GDI+ for loading them as textures, and it’s okay for that purpose. Treating them directly in the viewer is a different beast though. The spec is too complicated for me to implement right now, mostly because it requires my own zlib/DEFLATE implementation (and honestly, JPEG would have higher priority because with my own JPEG decoder I could finally apply the arithmetic coding extension to all my pictures and save 15 % of hard drive space :) )

 

2 hours ago, mikew said:

the only format that seems to be supported for viewing is BMP.

Yeah, the only one of practical usage :) Just for the sake of it: You can also view IFF/ILBM/HAM/SHAM, ACBM, PBM, TIM, DIB/RLE, PCX/PCC, WAD, and SPR (see the filter on the right side in the Open File dialog). That’s what it was originally for, viewing TFX’s Amiga image formats.

 

I’d rather go for inline files. Textures are currently just hacked in, and together with inline files, they should be treated as “external dependencies” with asnychronous loading and failure-proof placeholders. This would solve lots of problems the viewer will embrace with future formats (e. g. OBJ meshes have MAT files with material properties as external dependencies).

 

Anyway, just trying to get you, me, and the few other users (mostly Ace Combat paleontologists) to a synchronized state to simplify my maintenance.

Share this post


Link to post
Share on other sites
1 hour ago, Krycztij said:

Anyway, just trying to get you, me, and the few other users (mostly Ace Combat paleontologists) to a synchronized state to simplify my maintenance.

 

Another post for the bigger picture – sorry but I think it’s important for understanding.

  • Lean viewer started in April 2016 as help for us to read the LBM files of Total Air War and EF2000.
  • In August 2016, I extended it for support of 3D meshes to simplify workflow in my job. I could have started a separate project for it, but I didn’t. Looking back, this was indeed a good choice because now we have all the WRL goodness.
  • People from the 3D printing world got noticed of my tool and urged me to release it publicly.
  • IMHO, Lean Viewer was too unstable and too messy to release. Instead, I started the STL viewer project, which is basically the stable portion of Lean Viewer – a stable, minimal UI and the well-tested STL loader. No support for 30 years old image formats. No buggy VRML.
  • The STL viewer is not exactly popular, but many people from the 3D printing communities use it. Removing the messy parts from Lean Viewer really paid off – it’s rock solid; I only had to provide hard technical support once.
  • I get lots of mail about it. I get donations once in a while. The feedback is very positive, but there’s also feature requests, and a few bug reports.
  • Now, whenever I fix a bug, I have to fix it twice – in the STL Viewer and in Lean Viewer. Whenever I add new features, I have to add them twice – in the STL Viewer and in Lean Viewer. That’s bad. It’s eating up my time. And even though I constantly merge STL Viewer progress into Lean Viewer, it doesn’t seem to get any closer to a state that could be released to the broad public.
  • So I’m currently trying hard to unite the code bases. This is not exactly work for TFX, so I’m a little uncomfortable bothering you with it while you’re working even harder to get a new terrain, but I’m sure that eventually, we’ll all profit from it. 
  • Some things that I’m currently trying to unite:
    • There’s this awesome guy from South America who uses Lean Viewer to paste screenshots of Ace Combat models over his own photographs. I added support for screenshot-to-clipboard via CTRL+C- for him. Currently porting this to STL viewer.
    • Lean Viewer’s UI has many tabs and menus, and my wife has greatly improved the code to support variable sized tabs. Trying to roll this out in both viewers.
    • Adjustable material colors (STL viewer). Currently trying to port this to Lean Viewer.
    • Wireframe/outline rendering (STL viewer). Porting this to Lean Viewer.
    • Drag’n’Drop support (STL viewer). Porting to Lean Viewer.
    • Better lighting (STL viewer). Porting as well.
    • Top/left/side views (STL viewer). Porting to Lean Viewer.
    • STL viewer has a setup with proper support for file extensions (instead of Lean Viewer’s ugly, hacky menu command). Lean Viewer gets one as well.
    • Lean Viewer’s scene tree supports fancy things like instancing and material editing (and is generally incredibly fast, even with thousands of objects and millions of polygons). Porting to STL viewer.
    • STL viewer had a tab with GPU settings added per user request, and Lean Viewer just got it as well. (It’s the one on the far right.)
  • I hope to have a single code base for all of this in the future. I wish I had done this in 2017 already.
  • I hope you understand that PNG display is currently the least of my problems …

Share this post


Link to post
Share on other sites

OK! :D

I hadn't realized that viewing the bitmap and using it as a texture invoked two different processes, so please don't waste time on a PNG viewer. Inline files would be useful, but on my current trajectory they won't be needed for some considerable time.

For example...while the 009F textured polygons seem to be rendering better, my shaded polygon code needs some tweaking... :)

rammess.png.b7b74c6976fa791fb953f53db511ee90.png

 

No need to feel guilty about not spending 100% of your time on TFX and nice to hear that you're getting good feedback on your other projects.

 

Share this post


Link to post
Share on other sites

I've been out of this for some weeks, but was having some fundamental problems so need to think over the strategy a bit.

My current code does something like this:

1. Break down the .3 file into blocks of  'ffff' terminated bytecode.

2. For each block, map the decision opcodes to work out the possible paths through the bytecode and try and work out the ones that are relevant.

3. For each relevant path, step through it and produce a text file with the opcodes still as they originally were.

4. Evaluate the result of step 3. and de-duplicate polygons, convert vertices from relative to absolute coordinates etc. Save the result as a text file for debug.

5. Sort the polygons by material and produce a separate VRML file for each path through the bytecode.

Not yet done, but then would come:

6. Combine the VRML files in some way.

 

Only step 1 can be said to be complete. Steps 2 to 5 work quite well for a lot of models and it's gratifying to see the end result in Lean Viewer.

There are some serious problems with my approach to steps 2 and 4 though, so they need a rewrite. I should try and break these steps down further and into some form that is fun to code.

 

 

 

Share this post


Link to post
Share on other sites

Not much TFX time this holiday period, but have been looking at bit at OpenSceneGraph and its file format and it looks very similar to VRML. The conversion tool works fine for the simple VRML models that I've already produced which unfortunately isn't that many.

wrl_osg_tent3.png.42317be98069c08f64c9c8866abc90ed.png

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...