[FF8] world map and objects (world.fs)

[makipl]

I probably found a way to create 32BPP (ARGB) textures using multi generated textures with CLUT palettes, block UV and texture mixing. I'm finishing slowly this original algorithm on battle stages. I have correctly working palette 8BPP indexed texture drawing, UV bounding boxes locations and auto multi CLUT generator. Only mixer and CLUT resolver is TODO. Let's see how will it work. I'll edit this post if I make it to finish texture generator today...

 

[kaspar01]

That would be awesome! is it supposed to work whith every tim file?

 

[makipl]

Only if I got bounding box of segment's UV. I have some problems with CLUT ID. Looks like it works strictly bit style...

CLUT ID for battle stage is probably like this:

#1:00000000 00111100
#2:01000000 00111100
#3:10000000 00111100
#4:11000000 00111100
#5:00000000 00111101
#6:01000000 00111101
#7:10000000 00111101
#8:11000000 00111101

This comes a little familiar as I was coding palette.
The palette was example:
ABCD> 10101011 11001101
And spec was: A, B, G, R

So AB should be moved behind CD so it's:
11001101 10101011  and should be read from left.

In this way:
#1:00000000 00111100 > 00111100 00000000
#8:11000000 00111101 > 00111101 11000000

So #9 will be probably:
00111111 00000000

Maybe... Need to test it.

Random tests:
Pavement is #5, and #5 clut is gray like so it pass. a0stg027.x
Specifical blue element on a0stg012 is 403c so  00111100 01000000  and is #2. Passed!
Okay. Made it! Max CLUT count is 1111 (with 0, so 16)

Complete CLUT reference:
Code: [Select]

#1:  00000000 00111100 (00 3C)#2:  01000000 00111100 (40 3C)#3:  10000000 00111100 (80 3C)#4:  11000000 00111100 (C0 3C)#5:  00000000 00111101 (00 3D)#6:  01000000 00111101 (40 3D)#7:  10000000 00111101 (80 3D)#8:  11000000 00111101 (C0 3D)#9:  00000000 00111110 (00 3E)#10:  01000000 00111110 (40 3E)#11:  10000000 00111110 (80 3E)#12:  11000000 00111110 (C0 3E)#13:  00000000 00111111 (00 3F)#14:  01000000 00111111 (40 3F)#15:  10000000 00111111 (80 3F)#16:  11000000 00111111 (C0 3F)

Dirty sample code #updated (Forgot about reversing I mentioned above):
Code: [Select]

                       byte[] bt = new byte[2];            Buffer.BlockCopy(buffer, 1, bt, 0, 1); //00             Buffer.BlockCopy(buffer, 0, bt, 1, 1); //3C            BitArray ba = new BitArray(bt);            BitArray CLUTbit = new BitArray(4);            CLUTbit[3] = ba[15]; CLUTbit[2] = ba[14];            CLUTbit[1] = ba[1]; CLUTbit[0] = ba[0];            int[] ClutArray = new int[1];            CLUTbit.CopyTo(ClutArray, 0);            return ClutArray[0];

Only texture mixing left... Like damn, I was coding today for whole day... Just this, and I'm taking a break.

Nah, reading image by image and transferring pixels algorithm would take a lot of time. I'll leave it for tomorrow or something...

 

[kaspar01]

Good job Maki!

Can't wait to see it finished.

I always had to fix textures manually but I was not always sure if I choose the right palette for every clut (some of them were really similar)..

And it's been a pain in the ass every time XD

 

[makipl]

Done!

Also corrected code from yesterday's post, as it had wrong indexes:
Code: [Select]

            byte[] bt = new byte[2];            Buffer.BlockCopy(buffer, 1, bt, 0, 1); //00             Buffer.BlockCopy(buffer, 0, bt, 1, 1); //3C            BitArray ba = new BitArray(bt);            BitArray CLUTbit = new BitArray(4);            CLUTbit[3] = ba[1]; CLUTbit[2] = ba[0];            CLUTbit[1] = ba[15]; CLUTbit[0] = ba[14];            int[] ClutArray = new int[1];            CLUTbit.CopyTo(ClutArray, 0);            return ClutArray[0];

All done. Only wmsetus object remaining to do and maybe world map.

 

[kaspar01]

That's so cool!

I'm really happy to finally see so many progress in ff8 tools development 

 

[Halfer]

Oh damn, that's incredible!!

I guess I'll have to up the pace with the wmx tool since you are making this much progress with the textues :S

 

[Halfer]

Little update now!

Since I have a week off from school I started to make the importer finally. I feel kinda sorry that I haven't contributed as I planned but the fact is, that now the project is back and running.

I've built a simple algorithm that can import a model to ff8 world map format. This was the result:

So it's a cube (little stretched) with material properties that I manually set because the importer currently generates only vertices and faces.

Just a reminder though, although importing is possible, it's probably necessary to make many options for user because for example if user wants to add object over segment, it has to iterate through blocks. And same goes for multiple segments. Importing to one block however is rather easy as done in picture above.

So that concludes to my question, what options people would like to have in the importer? Also if you don't know what can be done through world map file, feel free to ask.

 

[makipl]

Oh Halfer, you're a God! That's awesome! :3

There should have an option to select faces to block player movement. Also CLUT selection and texture selection. I keep my fingers crossed and wish you best. We want to see this finished! :3
Take care!

 

[Halfer]

Oh Halfer, you're a God! That's awesome! :3

There should have an option to select faces to block player movement. Also CLUT selection and texture selection. I keep my fingers crossed and wish you best. We want to see this finished! :3
Take care!

Definitely there eventually will be those options! I have couple of ways to implement those features but i'm open to suggestions!

 

[Halfer]

Update.

Texture coordinates are being worked on, need to replace the algorithm though, but working so far. Also I manually set texture indices to FF8 model since I haven't done that algorithm yet at all.

Seems like some combination of material property values are yet to discover because I just found about this:

 

[Halfer]

Update, Current phase.

Texture coordinates and indices are now working. Here's a segment that I filled with water blocks. Water blocks were corrected so that 1 quad shows the texture instead of 4 quads as in original file, so it's 4x upscale actually.

I tried to subdivide it to upscale it more but seems like the developers were aware that the blocks including only water can have only 32 faces (32 triangles or 16 quads). So subdividing gave 128 faces which goes beyond the limit (0x900h) for one block (16 blocks in segment).

The importer currently creates a block from an imported wavefront object with:

• Block header                                                               (Face count, vertice count, normal count, and unknown count (always 0x00))
• Faces(vertice indices)
• Texture Coordinates                                                  (Minor tweaks for calculation on works)
• Manually set Material Properties for whole block
• Vertices
• Manually set 1 normal pointing upwards

(minimum information to form the block)

Features the importer doesn't do yet, but are planned to be added:

• Write normals to FF8 format. (pretty fast to implement, not implemented yet because testing does not need them "yet")
• Material property editing. (Meaning that user can't currently edit material for each face separately, but can set material properties for whole block. Materials include: Texture pages, CLUT id, info for walkability and landability)
• Write the exported file straight to wmx.obj. (Meaning that the importer exports a separate FF8 format block which has to manually be set to wmx.obj through hex editor. This also means that segment header must be manually fixed if you change offset of blocks.)
• Write wmx.obj straight to world.fs. (Using Deling currently to replace wmx.obj)

 

[Halfer]

I can't believe how I didn't think this sooner but Squaresoft was very lazy or stupid or even both with their world map formats.

The segment includes 16 blocks with their positions in segments header indicating where the block is in file. Well off course you can change the offset, but what I didn't think until now is that why the heck do blocks like the sea/water or any other block that repeats itself need to be separated? Well THEY DON'T!
You can have only 1 instance of that block and indicate all the 16 block headers to that offset. This way the block don't have to have maximum size of 0x900h, in theory it can be 0x9000 long. Only thing that limits it, is block header with the count of faces, vertices and normals which can only have maximum of 256 value.

I noticed that if I put all the block headers to indicate 1 block, texture coordinates changed drastically. However I think I learned how the engine tries to "fix" texture coordinates and I may need to reformat some things in my algorithm for that.

But this sure is really important information since any block that needs to be repeated inside segment can be referenced multiple times while having one instance of it in segment.

*I also did some experiment with the offsets and off course you can set the offset to be over the segment itself. This however doesn't work by itself, because  FF8 engine is changing the location of segments in memory each time you rotate camera. In other words, making it to work needs a hell lot modification on engine algorithms or memory managment.

 

[Yagami Light]

Great progress Halfer, has anyone delved into the world map chara.one file ? I think it contains geometry for ragnarok/garden/lunatic pandora/etc I also reversed these bytes and found possible beach animation data? Not sure, if anyone wants to have a look

 

[makipl]

No, you affected only GPU operation opcode. Maybe I'm calling this wrong, but in many cases by different researches thing like this was just called "GPU related", nothing more. Changing this makes you extremely crash sensitive. Ragnarok indeed is part of chara.one file as well as Squall on chocobo, Squall on feet, Zell on feet and etc. but not lunatic Pandora nor wave animation. These are objects from wmsetus/fr/it... file.

Check the updated wiki about wmset files:
http://wiki.qhimm.com/view/FF8/WorldMap_wmsetxx
Check section 16 (textures are in section 42), and for wave animations see section 38/39. It's all I researched at the moment. 

Cheers! :3

 

[Halfer]

Great progress Halfer, has anyone delved into the world map chara.one file ? I think it contains geometry for ragnarok/garden/lunatic pandora/etc I also reversed these bytes and found possible beach animation data? Not sure, if anyone wants to have a look

Took a quick look and seems like you reversed bytes from one the 3 TIM textures for ragnarok. As Maki said, by doing this, values can get insane and may cause crashes or affect other elements in bad way which in theory shouldn't be affected by the values.

By a quick note about the structure I can say that Chara.one first 4 bytes indicates the size of the whole file. There are 15 textures and it seems like that after every corresponded texture (for example 3 textures for ragnarok) starts a new section with some kind of data.

Edit: Well, just looked again at the file for curiosity and the structure seems to go like this:

Code: [Select]

u32 Chara.one (whole file) size;    // only once in the beginning of the fileTim texture(s)           // Every texture for one object comes before .mch structure.MCH file                   

http://wiki.qhimm.com/view/FF8/FileFormat_MCH

 

[Vehek]

I briefly wrote about that before. That type of chara.one has a a footer (I mean a structure at the end of the file), much like the regular chara.one header, but with the order of the entries reversed, where the final 4 bytes of the file are the model count, the 4 bytes before that are the first model's first texture offset, and so on. Also, no EE EE EE EE separator.

http://wiki.qhimm.com/view/FF8/FileFormat_ONE

 

[Halfer]

I briefly wrote about that before. That type of chara.one has a a footer (I mean a structure at the end of the file), much like the regular chara.one header, but with the order of the entries reversed, where the final 4 bytes of the file are the model count, the 4 bytes before that are the first model's first texture offset, and so on. Also, no EE EE EE EE separator.

http://wiki.qhimm.com/view/FF8/FileFormat_ONE

Seems like you are right, didn't cross my mind to look from the end. Also it looks that EE EE EE EE separators are FF FF FF FF in this file.

 

[Blue]

Just dropping a quick note to this topic, for someone might not read the wmx2obj thread as actively: I refactored the source code of wmx2obj for readability and efficiency purposes. Known bugs and inefficiencies should finally be removed.

wmx2obj

Those interested in using parts of the wmx2obj source code in other software or upgrading wmx2obj, feel free to do so. I released the program under a copyleft license, so you are allowed to do pretty much anything you want with the source code. Sharing is caring.

 

[Quantize]

Awesome work!
As a long time FF8 fan and a tinkerer I love these creative projects.

I have a question regarding the content this tool can export. I am currently working on a project to recreate the Balamb Garden in 3D and figure every inconsistency out to be able to make it a full building with a 3D/VR walkthrough.
So although i have a lot of reference material, getting original 3D models and textures could help out a lot for me to make it as accurate as possible, is this project capable of exporting this kind of data at of yet?

Thanks!