Decent FF7 Model Viewer

  • Thread starter Thread starter Sephiroth2000
  • Start date Start date
Status
Not open for further replies.
Guys... you might want to consider putting such a huge code block in a separate text file and linking to it, instead of including it in your post. This page of the topic alone is a near-300k download.

EDIT: Sigh, of course my post ended up on a new page... Right, the PREVIOUS page of the topic alone is a near-300k download.
 
Believe me that is what I wanted to do.
But I do not have anywhere to put it.
I don’t have access to my own site.

Find me free web space and I would definitely put it there instead.

Plus we should really move this to a topic specifically for figuring out these files.


L. Spiro
 
You could also use 250free.com. They give a fairly decent amount of space (250 MB) and bandwidth (250 Mbit).
 
Alright I'll post links instead of the images as well.  I have them set to expire quickly out of Mirex's trash bin though :)

Spiro you aren't coming across too coherently too me.

Code: [Select]
Code: 
struct s_FF7AAanimHdr {   unsigned long   rec_a;      //0   unsigned long   rec_b;      //4   unsigned long   block_len;   //8   unsigned short   block_a;   //12   unsigned short   real_data_len;   //14   unsigned short   translat[3];   //16   unsigned char   u1;            //22} FF7AAanimHdr; //size = 23 bytes

Code: [Select]
Code: 
struct s_FF7AAanimHdr {   unsigned long   rec_a;      //0   unsigned long   frames_1;      //4   unsigned long   block_len;      //8   unsigned short   frames_2;      //12   unsigned short   real_data_len;   //14   unsigned char   u1;      //16} FF7AAanimHdr; //size = 17 bytes

All right .. why the moving of u1? is this the 'new' U1 value that is a certain number of rotations that are ignored?  Then the frame XYZ offset?  I'm not sure that this works right.  All the integers in the playstation version are little endian.  So this gives me better nasty and weird values.  do these new values work for you?  How do you discern the frame count per animation?

Cyb
 
Actually I have found that the new u1 value does not actually depict the number of rotations to ignore, but it seems very close.  In Red XIII’s animation set, the values are as I showed, but if you do as I mentioned and skip 2 rotations, you get values that are VERY close to what they should be, but down the line they become more and more off.
It could be a key to some type of compression (and most likely is).

I removed the “unsigned short   translat[3];   //16” completely from the header because it is actually part of every frame.
And also, they actually come one byte later in the file than previously thought, decoded as I showed.
In other words, since they come one byte later, “u1” is actually the last byte of “translat[3]”, which is why I moved it to the front of “translat[3]”.
But after I moved it to the front I then removed “translat[3]” entirely.

As for the frame counts, they are in RAM (the same place I got all my data).  Red XIII’s and Barret’s first animations are posted and have 20 frames each.  I have checked others as well, but since “rec_b” and “block_a” are always the same I am unsure which one is actually depicting the frame count.

Most likely it is “block_a”, since that is the logical place to put the frame count.


If your values are already in little endian, you don’t need to translate them to little endian.  Just use them as they are and you should get the values I posted.


L. Spiro
 
Actually I have found that the new u1 value does not actually depict the number of rotations to ignore, but it seems very close.  In Red XIII’s animation set, the values are as I showed, but if you do as I mentioned and skip 2 rotations, you get values that are VERY close to what they should be, but down the line they become more and more off.
It could be a key to some type of compression (and most likely is).

I removed the “unsigned short   translat[3];   //16” completely from the header because it is actually part of every frame.
And also, they actually come one byte later in the file than previously thought, decoded as I showed.
In other words, since they come one byte later, “u1” is actually the last byte of “translat[3]”, which is why I moved it to the front of “translat[3]”.
But after I moved it to the front I then removed “translat[3]” entirely.

As for the frame counts, they are in RAM (the same place I got all my data).  Red XIII’s and Barret’s first animations are posted and have 20 frames each.  I have checked others as well, but since “rec_b” and “block_a” are always the same I am unsure which one is actually depicting the frame count.

Most likely it is “block_a”, since that is the logical place to put the frame count.


If your values are already in little endian, you don’t need to translate them to little endian.  Just use them as they are and you should get the values I posted.


L. Spiro
Click to expand...
Spiro since I don't know which animation is Barettes normal animation, might it be possible for you to indicate which one this is? I have many possibilities and no way of knowing which one you are looking at (1 of 90 some odd animations). Guessing isn't too helpful in this case :D  I tend to use the first animation myself.

I am going to attempt to see if your data is matching mine and what's up.

And here are the results (text list), here is a marked up hexdump (A Big JPEG).
I likely have a misalignment however whats the length of your animation packet? Mine appears to be 879 bytes which can in no way store 20 frames of animation. The first frame matches yours quite nicely however.

Cyb - still puzzled.
 
Maybe the problem is not compression, but a different storage system for rotational information.
 
Yes, mine is the first animation and your first frame matches it fine.
But remember, I got these animations from RAM and not the file.
When it comes to the file, I still don't know how to decode past the first frame, so that is all I can document.
Each frame DOES have a rotational and translational offset, but after the first frame it is stored differently.

So, keep in mind that all I post so far covers the first frame.

Also, about your translations, they are backwards.

You got 4862, which is hex “12 FE”
You need hex “FE 12” to get -494.

Since you say the PlayStation values may already be reversed, I don’t think you need to reverse them, but no matter what the case, the final result should be -494.


L. Spiro
 
I remeber reading somewhere that PSX animation can be compressed. I somehow remember this being an issure with FF9 and MGS. In MGS's case it was an interview with So Toyota and how he was able to cram all those moving models around and have room left for the game.

Let us theroize about animation compression for a little bit.

What we know.
1) The system creates a series of distinct frames when it's loaded into memory.
2) The model has diffrent "sets" of animation that are used at differrent times. (Throwing, slashing, running)
3) As a file, the animation starts with a "keyframe" and then unknown data spews past it.

Ok, here's some rules about FF7 on the PSX.
1) Do not do a file access unless you absoluty have to. The PSX is not a fortune teller, It bearly has enough horsepower to keep the texture caches stright. Unneccasary loading/decompression of animation will cause a proformance hit. This would stand to reason that the animations will have to be in memory for the duration of the battle.

2) Only two animations are ever used in seqence (This makes sence later)

3) Animation is threaded round-robin via the Kernel.

4) If the animations are compressed, and you are only using two animations at at a time, and the keyframes are uncomresssed, It could stand to reason that the whole compressed file is in memory, and when you need a particular sequence, that portion is inflated.

5) It could also stand to reason that animations are cached.

So if I were a programmer, I would set up "slots" of animation data in memory. The first slot would always have the "standing" animation. (Let's call this animation the "root" animation")

If I were to issue a command in the battle menu, let's say "attack", I would want to have that animation feched, uncompressed, placed in the next availible slot, and then executed. After Attack, I will return to the root animation,  the attack animation data will still be cached in that slot.

If I do a "throw" it will overwrite that last animation.

Now here a neat trick I picked up reversing FF3 (The NES one)....

Do you ever notice that when you do a summon, that your characters dissappear? This isn't an asthetic thing. What the game is doing is flushing out *ALL* the character animimation, model, and texture  data to make room for the summon. This leaves the caches the only real link to quickly recreate the battlefield. In the case of FF3NES. There is only so much PPU ram. The characters had to be nuked to make way for the summon graphics. When I was playing with FF6, I noticed that it was deleting all the character graphic object data before a summon as well. In the PSX VRAM. All magic tecture caches, and the "eyeball pack" (Character textures) were deleted from there.

I would suggest you look for the compressed animation data file in memory, that may be your look up table to uncompressing the animation data into each appropoite cache.

Now about compression.

How does one compress animation? Well, it can't be mathmataclly intense, as that would lag the fetch sequence. If I was going to store only a keyframe of data, and then "garbage" after it, the "garbage" in question would only be the data I changed from the previous frame. The easiest way I can think of is to give each joint an ID  and then the updated data. You could use relitive repositioning, for example [right_leg: y=+3.34, y=-2.12, z=+6.09] (Which leads to smaller numbers in the data set) or explicet new cooridinates, or a mixture of both. But I doubt that last one.

Anyway that's some food for thought.  I might just be talking out my butt again. Heaven forbid I should know what I'm talking about ^_^
 
L Spiro - That is why I said what you have doesn't entirely make sense, there is no reason for them to store anything in big endian since they store everything else in little endian.  It's an inconsistancy and thus quite suspect.

Hmm.. if I were to compress this data  I think I would use a modified LZS alogo to do it.
In LZS the first byte indicates literal stream data and compressed.
Perhaps it's compressed using something similiar only in this case the numbers included are just the ones that changed?

It's possible they used delta compression? This does limit the rate of change per frame. Maybe 1 signed byte per value? that would give 3 + 23 * 3 bytes or 72 bytes per frame.

In any case for Barret we have
The header is 5 bytes.. I know this is a FACT why?
It has nothing to do with what Spiro has done actually it's just raw data.
The first animation section is 884 bytes in Barret, it's LENGTH is 879 bytes though, this leaves 5 bytes for the header (tada).
The first KEY frame is 110 bytes in size because 6 bytes for the offsets 4.5 * 23 for the rotations.
This leaves 769 for 19 more frames or roughly 40 bytes per frame.

Examining RedXIII we have the following information.
total size is 1720 bytes and the length is 1713 (7 bytes differrence) F is 2 (that little byte that we've been so worried about!). The interesting thought is does this make the header longer (by 2 bytes?) hmmm
The translation and none of the rotation values match Spiro's
It says there are 20 frams though :)
Doing the math the first frame should be
6 + 4.5 * 47 or 218
That leaves 78 bytes per frame
So yes the data is compressed, but how.. is not obvious.  

Cyb
 
I have been a busy bee.
2 or more years ago I decided to make a program for hacking RAM since GameHack wasn’t working on Windows® XP.

Until recently it has been a fairly simple project, but it has changed from a newbie—intermediate tool to a newbie—high-end tool.

Searches and things are still simple enough for GameShark® newbies to understand, but templates are much more complicated and take time to learn how to use to their fullest.

I recommend that you (all) get this before it expires (set to one week) from the bin.

Memory Hacking Software


So, here is what I have done.
This is the Template Editor.
Template2.png


As you can see, I created a very very basic template to describe Cloud’s animation set.

In RAM, only a few of his animations are loaded at a time.  I found this out (not to my surprise) while making the template.
Not only that, but between animations, sometimes there are 8 bytes of space, sometimes there are 16 bytes of space, and sometimes there is no space.
Most likely this has to do with the number of animations that AREN’T loaded.

Total, this set occupies 24,080 bytes of RAM, as shown by the Template Editor.

The frames that are loaded into RAM have 20, 20, 12, 10, 6, 6, 2, and 4 total frames.  I hardcoded these values into the template (as well as the number of bones Cloud has) but it is possible to set them to a value in RAM (instead of always being 20 until I change it, it will find the value in RAM and use that number to resize itself instead).  If I knew where the number of bones in Cloud’s body and the number of frames per animation were stored, I could link the template to those values in RAM instead, but I don’t, so I hardcoded them into the template.


The Hex Viewer shows all of this information to me in a very easy-to-read format which can be changed to suite your needs.
HexViewer1.png


Because I can map the templates over RAM in real-time and see every value as I add them and modify them it is extremely easy to map objects in RAM.  This whole animation set took 10 minutes.
Because I defined the parameters of a valid XYZ coordinate to be a float from -360 to 360, the Hex Viewer automatically showed me where values in RAM were not meeting those parameters (shown by red) and I could see instantly that my templates were or where not aligned properly.

What is shown in the picture is the end of Cloud’s animation set.  The green line indicates the start of a new chunk.  Since Cloud’s animation ends shortly before a new chunk, most likely his is the last animation of the 3 characters in my battle party.  I have not verified this.

You will notice something strange about this picture.
Some of Cloud’s bones have -180.0f for rotations.
It seems it is possible for the bones to be rotated backwards, but why?
And how would they be stored in the file if they have a negative rotation?
Or is this some funky trick the Final Fantasy® VII engine does upon loading of the animations?  -180 is the same as 180, so what is the point?


You should be able to see already how useful the templates are.
Soon I will add the functionality of exporting.  Each data type will export as whatever you define and in any of multiple formats.
You will be able to create the final output format in the same way you use sprintf().

Currently it is possible to export that entire 24,080-byte chunk just by selecting it in the Hex Viewer and right-clicking, but when exporting a template you will have much more control over each individual data item.


This tool should help us finish off Final Fantasy® VII, and I am still working on it to make it better.
I really suggest using it.  As I mentioned, putting together that animation template and following all of Cloud’s animations took only 10 minutes and helped me gain understanding of the final format when loaded into RAM.
And you can use templates to describe any format.
In a template (let’s say A), you can assign pointers to other templates (B), and then attach that template (A) to a pointer in RAM.
When doing this, the template will automatically follow dynamic objects in RAM as they move around.  Animations are dynamic and are reallocated every time a new battle starts.
If I take the time to find the pointer to Cloud’s animation, I could attach this template to the pointer and not have to worry about finding his animation ever again.

One last word on templates.
I will also make the ability to wrap templates into libraries and send to other people.  Currently, it is very rare to get a template working on another computer, but when I have completed that step, it will be very easy for us to share our templates and help each other dig into this game.


Here is the actual software (again)!
Memory Hacking Software
Here is a tutorial on templates.
Template Tutorial
Here is the RAM dump (all in floats) of the actual area of Cloud’s animations.
Cloud’s Animations


L. Spiro
 
That's really cool.

We however need the native format that the animation is stored in on the original media. I'm not keen on "ripping" data from a game and storing it for later use. (I.E. Let's make a game using cloud's animation. I have his animation data converted into XML we can include)

The problem is it's still a copy of the original data from the original media and a no-no to redistibute.

What you have done is nothing short of amazing though.....
 
I think it will help is break the animation format. I believe ALL of the animation data is compressed somehow, that is why REDXIII rotations are odd.  I believe it's time for me to start abusing my playstation debuging software.

Cyb
 
Yes it is illegal to distribute a rip of the original and the original files.
It is not illegal for me to distribute a template library which you can use to see the animation formats as easily as I see them.

I think seeing the uncompressed format in RAM will help crack the compressed file format.

My point isn’t that we can use this to rip and then use the rips in our game(s)/tool(s).  I am definitely not suggesting that.  This is purely for help in decoding.

So, this program DOES allow ripping, but more usefully it allows very easy browsing of RAM.  Looking at RAM is not illegal, and even if it was, that doesn’t seem to stop anyone here.

Anyway, I spent two years on this and I am still spending more.  It helps a lot and I want to see it get used.


L. Spiro
 
It seems fairly simple, at least to decompress the first frame of Red XIII’s animation.

It goes by 10 bits instead of 12.


It looks like this:

Code: [Select]
Code: 
Bits from 0x1B  * 360 / 10240000000000      0.0 RXOffset0000000000      0.0 RYOffset0000000000      0.0 RZOffset0000001001      3.16406250000000000      0.00000000000      0.00000001001      3.16406250000000000      0.00000000000      0.00000000000      0.00000000000      0.00000000000      0.00001000101      24.25781251111111101      358.94531250000000000      0.01111100001      349.10156250000000011      1.05468750000000001      0.35156251101110101      311.13281251100011010      279.14062501000001111111111011101011010010010010111111101111000100110111110101110000111110111111010100000000111101110100001110101110111110010000000001001000110101 Etc.…

The rest of the first frame matches the values I found in RAM exactly.

It seems we will have heavy rewriting to do to get our engines to display these animations.

The “u1” that has caused us so much trouble is a compression key.  It can be either 0, 2, or 4, and determines whether the bitcounts are 12, 10, or 8 respectively.
Unfortunately I still can’t get past the first frame.
Using this method leaves the end at 0xCB and 4 bits.
According to RAM, the next translation is 0 266 124, which means 00 00, FE F6, 00 7C should be in RAM somewhere in that area.
But there is no FE F6.  No 00 00 either.


L. Spiro
 
The “u1” that has caused us so much trouble is a compression key. It can be either 0, 2, or 4, and determines whether the bitcounts are 12, 10, or 8 respectively.
Click to expand...
Like wowow ! I still dont get it how did you found that out ! Nice progress.
 
Heh, that isn’t all.

The first frame is never compressed, which implies all frames after use relative offsets.
After I posted I went back to the hex to verify this, and indeed what I found verifies it to the nearest 80% probability.
I began by creating a table of Cloud’s first two correct frames and then a table of the amount by which each bone changes.
The first translational offset (by file) is -466 and the next correct frame is 461, which translates to the file to be -461.  The change is 5, and with his other two translations being 0 and 0, the three displacements together are 0, 5, 0.
So I went to rtda to the first byte after his first frame, and guess what I saw? 00 05 00.
I was just about to go deeper, into the rotations, when power went out.  I just got back today and just now have time to post, so here it is.

Anyway, from what I gathered before losing power, the rotations are stored as offsets from the first frame as well, but the bitcount is different.  It isn’t 12 or 8.  I was just doing the math to check what values it could be when I was forced to stop, so now I am going to go back into it.

I don’t think this is going to take much effort, at least if everything else works as it should.  And I didn’t verify the 0 5 0 combination on any other files yet (to test THEIR translational offsets for this pattern) but I had every intention of doing so until losing power.


Well, anyway, I got the clues before losing power, so, now it is time to put them together and solve this riddle.


L. Spiro

P. S. mirex, you should really start using MemHack.exe.  And you know, it is only going to get better from here (as I am still adding more and more to it).
 
I have uploaded a new version of MemHack.exe (2.0.1.3).

Memory Hackong Software 2.0.1.3

This is because I have added a fairly helpful/important feature to it.
We can now share our templates with each other.

If anyone here maps out an area of RAM, whether it be the item list, Materia list, weapons, characters, monsters, animations, or just basic templates to help us all find information, he or she can post the template and then we can all be able to examine the RAM and either come up with solutions for solving whatever task is at hand more easily, or look over the template to see if it makes sense/describes the data accurately.

Sharing templates is similar to sharing your idea on how it “should” look, and when people put their ideas together, the final solution follows soon after.

It is true here that many things are already mapped, but in fact there are many things yet to be mapped (camera positions, anyone?).

I have not kept a tight list, but I know that animation files are still not fuly mapped, and I know that understanding how they appear in RAM is vital to unlocking them.
That is why I have made a basic (and somewhat newbie) template to describe an animation as it appears in RAM.
It can be found here.
If I had found the pointer to the animation data sets, I would have wrapped that template to the pointer and then just tell the user where the pointer is so he or she can slap this template to the pointer and let the template find the animation sets on its own.
But since I haven’t, I will have to instead describe how I find the animation sets each time.
It is very easy.
Basically, we already know that each character has 3 floats for the translational offset (0, 466, 0 for Cloud).  466 (in this case) is a fairly unique float number, so I do a float search for it.
It returns only about 12 results, and from there it is easy to check each one.
When you are viewing the list of returns, you can check each one manually, but if you right-click the address in the list it will open the RAM Watcher to that address.  You will see each of the 466’s, but only one of them is immediately followed by a float of 0.
That is the one that determines Cloud’s first frame of his first animation.
Go up 16 bytes and that is the actual start of that frame.
That is the address where you need to post this template.
Get to the Hex Viewer by hitting Alt-T, H.

Here is the other bad part of this template: I don’t know where the data is that determines how many bones and how many frames per animation, so I can’t make the template size itself to the animation.

If anyone else finds that information, please post an updated template.
Until then, you can adjust the template arrays to make it fit the animation you want, just as I have shown in my screenshot above.

Also, tomorrow I will update the .mhtl format to be able to accept additional text which will be displayed upon loading.  That way we can include installation instructions and general notes about the template, packed into the .mhtl file (this change will not affect existing .mhtl files).


I hope this template helps to visually conceive the data in RAM, and that that helps to see how the file becomes that data, and we can crack this code.
I also hope everyone starts using this.  It would be beneficial to the community as a whole, being able to share visual ideas of the data we seek.


L. Spiro
 
Sorry im kinda busy right now ... also I dont have FF7 cd's here now ... so looks like hacking PC anims its on you L Spiro.
 
Status
Not open for further replies.
Back
Top