mikew

Back to TFX1

49 posts in this topic

Ha! That's where they went..

The whole thing is a proof of concept. You may notice bits of TFX3 terrain in the Libya map due to a naming clash which I couldn't be bothered to sort out. It shows the versatility of TFXplorer to render the different TFX generations at the same time though. :)

Now to make those edits....

Share this post


Link to post
Share on other sites

While you’re editing: In TFXplorer.cpp (1807), replace

	if(supershapesToRasterize) {

with

	if(supershapesToRasterize) {

		// Draw one quad per tile in ground color (160 in TFX 3 palette). This is a quick-and-dirty solution to transparent ground
		//  in TFX 1 levels.
		auto const y000      = floatx4Zero() - x000From(yInTFXUnits(eyePoint));
		auto       toNextJob = supershapesToRasterize.toBeginning;
		auto       index     = 0;
		do {
			// From XZ and Y, interleave XYZ:
			Geo::XYZ_float_reg eyeSpaceOffset = { interleavedXY(load(toNextJob->offsetFromEye).xy__, y000) };

			auto const objectToEye = preTranslated(eyeSpaceOffset, worldToEye);

			auto const a = Geo::transformed(Geo::xyzFrom(-1024.0f, 32.0f, -1024.0f), objectToEye);
			auto const b = Geo::transformed(Geo::xyzFrom(-1024.0f, 32.0f,  1024.0f), objectToEye);
			auto const c = Geo::transformed(Geo::xyzFrom( 1024.0f, 32.0f,  1024.0f), objectToEye);
			auto const d = Geo::transformed(Geo::xyzFrom( 1024.0f, 32.0f, -1024.0f), objectToEye);
			rasterizeFlatShadedTriangle(myRasterizer.tfx, a, b, c, 160);
			nextVertex(myRasterizer.tfx, d, 160);

			if(0 == index % 512) {
				myRasterizer.tfx.flushCommandList(); // prevent overruns
			}
		} while(++index, ++toNextJob < supershapesToRasterize.toEnd);
		myRasterizer.tfx.flushCommandList();

Which really is a big waste of CPU time with TFX 2 and TFX 3, but it looks so incredibly good with TFX 1 …

lib2.png

lib3.png

Share this post


Link to post
Share on other sites

Wonderful! :thumbsup:

An empty SSD Block5 might be a way of identifying a TFX1 tile if such a test is needed.

That second picture would be even more awesome with some tile randomization. How hard would it be to implement something?

Share this post


Link to post
Share on other sites

Another problem. In docks_1.ssd, Block6 isn't where I expect it, so it appears empty in my parser, and I used the same logic to recompile the shape list.
In game, the effect is a plethora of TFX3 shapes, as can be seen around tile 121,182.

I'm not sure what to spend time on. One of the reasons for choosing the Libya map to experiment with, was to investigate the possibility of replacing the terrain with that from TFX3. TFX1 terrain has its own charm though, so it may be worth 'perfecting' that instead.

Share this post


Link to post
Share on other sites

That 'empty Block6' problem seems to be widespread, with at least 'fact.ssd' displaying the same behaviour. I need to analyze what's happening, but I'm running out of TFX time.
EDIT: Totally my fault. I need to make some new SSDs.

Also, the airbase on Malta looks different to that Youtube clip I linked to earlier. It seems that the game uses '36_rway2.ssd' in that position instead of '36_rway1.ssd'. This implies that randomization is used even for complex tiles in TFX1.
Obviously, it can't be a completely random system, so we need to work out the rules. One way to do this would be to use custom textures in TAW2.0, where I think we have 5 different sea ssd tiles and see how they are arranged in game.
Is DrKevDog still with us? This is his area of expertise. :)

Share this post


Link to post
Share on other sites

As I recall, and as you alluded to, in tfx3 the randomization process never applies to any terrain with shapes on it, must have been a problem in earlier versions (perhaps like the above variable runway).

tfx3 randomized tiles:

Desrtm_1,3,4,2

Dunesm_1,2,3,4

Desrth_3,2,1

Island_4,2,1,3

Nlarbs1a,1d,1b,1c

Nlarbs2b,2a,2c,2d


IIRC the order is defined in the .lst file and loops through.

Share this post


Link to post
Share on other sites

We probably shouldn't be using the word 'randomization' for the randomization process. :)
For a particular map, the same ssd should be chosen for a particular tile every time, and I'd expect the calculation to be simple.
Something like:
rand_index=(y*400+x)%rand_number
..might work, but in TAW, the most often used rand_number is 4. This would cause the terrain to look like stripes with that formula, so it probably isn't that simple.

Share this post


Link to post
Share on other sites

Is there a way to find out all values without flying to these points in the game?

If I had a table with all 400×400 actual indices, I could compare that to the raw indices and pattern matching could be as easy as looking at the diff image.

Share this post


Link to post
Share on other sites

Not that I know of, and in game we don't get to see many tiles a the same time, so maybe the only way to see a pattern is to modify all the textures to put big numbers on them, then fly in straight lines from a known tile.

That's a lot of hassle, so even looking in IDA is an alternative....although looking at this, maybe not... :lol:

int __cdecl sub_4FC25C()
{
  signed __int64 ST114_8_0; // ST114_8@0
  int v1; // edx@1
  int v2; // ecx@1
  int v3; // esi@1
  int v4; // ST11C_4@2
  int v5; // ST120_4@2
  int v6; // ST124_4@5
  int v7; // ebx@6
  int v8; // ebp@11
  int v10; // eax@6
  int v11; // ecx@7
  int v12; // [sp-138h] [bp-138h]@21

  strcpy_();
  strcat_();
  dword_A2F070 = 0;
  dword_A2F078 = 0;
  dword_A2F07C = sub_4FBD8C();
  dword_A2F06C = sub_4FBD8C();
  v3 = 0;
  dword_A2F074 = sub_4FBD8C();
  dword_A2F070 = v1;
  dword_A2F078 = v1;
  if ( dword_690394 > 0 )
  {
    *((_DWORD *)&ST114_8_0 + 1) = v1;
    v5 = v1;
    v4 = v1;
    do
    {
      if ( dword_690398 )
        dword_690398(v2, (ST114_8_0 >> 32) % dword_690394);
      *(_DWORD *)&ST114_8_0 = dword_A2F070;
      v6 = 0;
      if ( j_opendir_() )
      {
        do
        {
          v10 = sub_599D18();
          v7 = v10;
          if ( v10 )
          {
            strlwr_();
            if ( strstr_(v11, ".sss") )
            {
              if ( dword_A2F070 >= 2000 )
                off_683A98("BuildShapeList() - out of ix space at tag %d");
              if ( (unsigned int)(dword_A2F078 + strlen_()) >= 0x186A0 )
                off_683A98("BuildShapeList() - out of names space at tag %d");
              *(_DWORD *)(4 * dword_A2F070 + dword_A2F06C) = dword_A2F078;
              strcpy_();
              v8 = dword_A2F074;
              while ( *(_BYTE *)(dword_A2F078 + dword_A2F074) != 46 )
                ++dword_A2F078;
              dword_A2F074 = dword_A2F074;
              *(_BYTE *)(dword_A2F078++ + v8) = 0;
              ++v6;
              ++dword_A2F070;
            }
          }
        }
        while ( v7 );
        sub_599D28();
        if ( !v6 )
          off_683A98("MkList() - No entries in directory %s, specified in taglist");
        *(_DWORD *)(v4 + dword_A2F07C) = v6;
        *(_DWORD *)(v4 + dword_A2F07C + 4) = ST114_8_0;
      }
      else
      {
        off_683A98("MkList() - Unable to open directory %s, specified in taglist");
      }
      ++v3;
      v2 = v4 + 8;
      *((_DWORD *)&ST114_8_0 + 1) += 100;
      v5 += 63;
      v4 += 8;
    }
    while ( v3 < dword_690394 );
  }
  if ( !sub_516B3D((int)&v12) )
    off_683A98("BuildShapeList() - Unable to open file %s");
  fwrite_();
  fwrite_();
  fwrite_();
  fwrite_();
  fwrite_();
  fwrite_();
  fclose_();
  j_j__nfree__0();
  j_j__nfree__0();
  return j_j__nfree__0();
}


Share this post


Link to post
Share on other sites

I modified mikes tawbig script to output a map with the values for each tile, only the script doesn't address "randomization". Perhaps including " rand_index=(y*400+x)%rand_number" or some other such similar guesses would help in matching it up?

Share this post


Link to post
Share on other sites

The problem is getting something out of the game to match it up with.
It's not straightforward, and maybe not worth the effort...but our world editing capability is improving, so we may be able to do it some day.

Share this post


Link to post
Share on other sites

The problem is I'm having too much fun creating new custom tools in 3P-4tfx :D Last night I created a way to analyze the randomization's in Tfx2. I started with the seven file randomization's for the ground file set (cm0_gx). Let me finish up my Norway map and I'll post some images.

Share this post


Link to post
Share on other sites

Reverting back to tfx2 for this part of the analysis while I figure out a standard to compare.

Here are the tfx2 randomization files as defined in the .dat file:

00 02
____________

FCM1_P.FCM1_P2 /
CLIF1S2S.CLIF1S_S
CLIF2S2S.CLIF2S_S
TEST.TSTPLANE
CLIF1S2S.CLIF1S_S

40 02
____________

CLIF1S2W.CLIF1S_W
CLIF2S2W.CLIF2S_W
CLIF1S2W.CLIF1S_W

80 02
____________

CLIF1S2N.CLIF1S_N
CLIF2S2N.CLIF2S_N
CLIF1S2N.CLIF1S_N

C0 02
____________

CLIF1S2E.CLIF1S_E
CLIF21S2.CLIF2S_E
CLIF1S2E.CLIF1S_E

00 03
____________

CM1_P.CM1_P2.CM1_P3
COAST_S.COAST_S1.COAST_S2
COAST_S.COAST_S1.COAST_S2
COAST_S.COAST_S1.COAST_S2

40 03
____________

COAST_S.COAST_S1.COAST_S2
COAST_S.COAST_S1.COAST_S2
COAST_S.COAST_S1.COAST_S2

80 03
____________

COAST_S.COAST_S1.COAST_S2
COAST_S.COAST_S1.COAST_S2
COAST_S.COAST_S1.COAST_S2

C0 03
____________

COAST_S.COAST_S1.COAST_S2
COAST_S.COAST_S1.COAST_S2
COAST_S.COAST_S1.COAST_S2

00 04
____________

CM3_P.CM3_P2.CM3_P3.CM3_P4
CM2_P.CM2_P2.CM2_P3.CM2_P4
MARSH_M.MARSH_M2.MARSH_M3.MARSH_M4
MARSH_M.MARSH_M2.MARSH_M3.MARSH_M4

40 04
____________

none

80 04
____________

none

C0 04
____________

none

00 05
____________

TREEISL1.TREEISL2.TREEISL3.TREEISL4.TREEISL5
ICEBERG1.ICEBERG2.ICEBERG3.ICEBERG4.ICEBERG5

04/08/0C 05
____________

none


00 06
____________

ISL_1.ISL_2.ISL_3.ISL_4.ISL_5.ISL_6


00 07
____________

CM0_G.CM0_G2.CM0_G3.CM0_G4.CM0_G5.CM0_G6.CM0_G7

0028    n:\3dshapes\tfx2\sh\enviro\ground\
        0 2 2 2 2 2 2 2 2 2


I am working on the CM0_G, randomization.

Here's what it looks like in-game:

 

 

 

capture_001_02012017_163356.jpg

Share this post


Link to post
Share on other sites

I have to admit that I'm not sure what that picture is displaying. :unsure:

Share this post


Link to post
Share on other sites

 

 

F6 01 00 
00 01 00 53 45 41 00 00 
00 04 00 43 4D 33 5F 50 00 43 4D 33 5F 50 32 00 43 4D 33 5F 50 33 00 43 4D 33 5F 50 34 00 00 
00 01 00 43 4D 33 5F 53 00 00 
40 01 00 43 4D 33 5F 53 00 00 
80 01 00 43 4D 33 5F 53 00 00 
C0 01 00 43 4D 33 5F 53 00 00 
00 01 00 43 4D 33 5F 43 00 00 
40 01 00 43 4D 33 5F 43 00 00 
80 01 00 43 4D 33 5F 43 00 00 
C0 01 00 43 4D 33 5F 43 00 00 
00 01 00 43 4D 33 5F 49 00 00 
40 01 00 43 4D 33 5F 49 00 00 
80 01 00 43 4D 33 5F 49 00 00 
C0 01 00 43 4D 33 5F 49 00 00 
00 04 00 43 4D 32 5F 50 00 43 4D 32 5F 50 32 00 43 4D 32 5F 50 33 00 43 4D 32 5F 50 34 00 00 
00 01 00 43 4D 32 5F 53 00 00 
40 01 00 43 4D 32 5F 53 00 00 
80 01 00 43 4D 32 5F 53 00 00 
C0 01 00 43 4D 32 5F 53 00 00 
00 01 00 43 4D 32 5F 49 00 00 
40 01 00 43 4D 32 5F 49 00 00 
80 01 00 43 4D 32 5F 49 00 00 
C0 01 00 43 4D 32 5F 49 00 00 
00 01 00 43 4D 32 5F 43 00 00 
40 01 00 43 4D 32 5F 43 00 00 
80 01 00 43 4D 32 5F 43 00 00 
C0 01 00 43 4D 32 5F 43 00 00 
00 03 00 43 4D 31 5F 50 00 43 4D 31 5F 50 32 00 43 4D 31 5F 50 33 00 00 
00 01 00 43 4D 31 5F 53 00 00

My first assumption (norway.dat) is that the first 2 bytes in each line indicate, respectively: 1. rotation and 2. number of files in that line and which also correlates to the number of files randomized for that index number.

Agree ?

Share this post


Link to post
Share on other sites

For TFX1 and TFX2, the .dat file has two bytes for rotation, then two bytes for the number of files in the randomization list.

EDIT: This site's formatting software has probably porked the following Python code, but hopefully you get the idea...

 

#Handle the .dat file
#First two bytes contain number of shapes, then each entry has rotation, number of shapes used, then the ssd names
input=file(filepath+"angola.dat","rb")
raw_dat=input.read()
dat_size = len(raw_dat)
#print dat_size

dat_num_shapes=ord(raw_dat[0])+ord(raw_dat[1])*256
print dat_num_shapes

e=2
dat_rot=[]
dat_rand=[]
dat_ssd=[]
while e< dat_size:
    rot=ord(raw_dat[e])+ord(raw_dat[e+1])*256
    dat_rot.append(rot)
    rand_num=ord(raw_dat[e+2])+ord(raw_dat[e+3])*256
    dat_rand.append(rand_num)
    e=e+4
    f=0
    ss=''
    while f<rand_num:
        temp=ord(raw_dat[e])
        if temp<>0:
            ss=ss+chr(temp)
        else:
            ss=ss+' '
            f+=1
        e+=1
    ss_list=[]
    vals=ss.split()
    for val in vals:
        ss_list.append(val)   
    dat_ssd.append(ss_list)

 

Share this post


Link to post
Share on other sites

...but you are right, there are indeed 7 CM0_G ssd files in the list:
 

noraxe.png

Share this post


Link to post
Share on other sites

Okay, so here is the .dat section for :

n:\3dshapes\tfx2\sh\enviro\ground\
        0 2 2 2 2 2 2 2 2 2

 

00 00 07 00 43 4D 30 5F 47 00 43 4D 30 5F 47 32 00 43 4D 30 5F 47 33 00 43 4D 30 5F 47 34 00 43 4D 30 5F 47 35 00 43 4D 30 5F 47 36 00 43 4D 30 5F 47 37 00

It has the 3rd largest volume of tiles (index = 2800) on the Norway map.

1. I modified the ssinfo.fn to accommodate a list new files:  cmx_g.3 thru cmx_g6.3

2. I modified new .3 file/tiles to render different flat color polygons

3. I modified the Block5 and 6 of SSD files CM0_g thru CM0_g7 (maintaining ssd file name integrity) to call the new .3 files

What is displayed is the visual rendering of the randomization of index # 2800 = cm0_g as I am flying over SE Norway.

Anyone with experience at working with high level graphics will immediately recognize the pattern and recite the mathematical randomization equation(s) ;)

Share this post


Link to post
Share on other sites

Reverting back to tfx2 for this part of the analysis while I figure out a standard to compare.

Here are the tfx2 randomization files as defined in the .dat file:

00 02
____________

FCM1_P.FCM1_P2 /
CLIF1S2S.CLIF1S_S
CLIF2S2S.CLIF2S_S

*[ TEST.TSTPLANE ]

CLIF1S2S.CLIF1S_S...

 

*I suspect this inclusion has some significant analytical meaning. The 2 named SSD defaults do not jibe with the terrain, despite them being included in the .dat.. Therefore I am needing to experiment with substitutions to find ones useful for analyzing the  randomization process.

Share this post


Link to post
Share on other sites
16 hours ago, DrKevDog said:

What is displayed is the visual rendering of the randomization of index # 2800 = cm0_g as I am flying over SE Norway.

That's what I don't understand. Is that some test mode where you get a bird's eye view of a chunk of terrain? What does each colour signify?

For TFX2, just over 400 of the 502 positions in the dat file are 'used' by the .4ev/ev2 files. 

Share this post


Link to post
Share on other sites

It is in Tfx2 Debug Mode. Each color is 1 terrain tile. I flood filled an area in the 4EV/4E2 norway map with 2800, which maps to cm0_g.ssd, I assigned it the color blue, however, the randomization can assign any one of the 7 ssd's (there are seven files included in that set) to a 2800 tile. I have assigned each one a different color:

cm0_g = blue

cm0_g2 = white

cm0_g3 = green

cm0_g4 = red

cmo_g5 = slate gray

cm0_g6 = yellow

cm0_g7 = black

While 7 colors are attractive, they make recognizing a definitive pattern more difficult.

 

capture_002_03012017_140251.jpg

PS: I found one in the Libya map that randomizes to 10 which is the highest I've found this far.

 

Share this post


Link to post
Share on other sites

It seems the Norway.dat entry "test.tstplane" is a real test entry. According to what I get from the debug mode, or maybe it's more of a setup-level mode, using suite "01", the test generates tracking lines/vectors that are color coded. Yellow vectors for planes to ? and red vectors for SAM's to planes:

capture_001_13012017_133942.jpg

capture_003_13012017_134522.jpg

capture_001_14012017_100622.jpg

 

The 2 SSD files called, test.ssd and tstplane.ssd use SAM's (sa_11, sa11mis) and aircraft (antonov, efa) respectively. Considering that the 2 SSD's are randomized, this may evaluate the changing spatial tracking relationship between SAM's and aircraft and aircraft and aircraft.

This may also be considered an image of the 2-tile randomization (C600, C700) used for that 4ev/4e2 randomized pointer (C600)  :

capture_003_10012017_165045.jpg

C600 = red

C700 = black

Share this post


Link to post
Share on other sites

No time to participate in 2017 so far, but that randomization process is far more random than I was expecting. :(

Share this post


Link to post
Share on other sites

No time either. I was holding my breath in hope for some simple patterns to emerge … now I guess this will require disassembly :( 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now