STR, XA, FF7STR??

[G]

Hello all!

So about my progress in the ff7 development... Haven't much time to write here about what I am doing/done.
So
I decided to write some classes to play PSX FF7 videos & sound. I used various sources of info about PSX videos and now I know something about it.
In FF7 there is 2 types of movies: ordinary files with STR extension (these can be copied without problems directly from cdrom without any crc errors) & files with MOV extension. Files with MOV extension is actually a interleaved stream with video & audio frames. These sectors are written in CDXA Mode2. I wrote simple dumper to dump this sectors to files. Then I write some demuxing code (used ffmpeg sources ). For audio stream I found some sources that analyzes and plays PSX CDXA audio data and write a simple command line utility to play any channel in XA file through oss interface (for now only linux  :evil: ). For video stream I used libbs from psxdev-libs (http://www.psxdev.de). Using demuxer I split video to individual frames and try to decode it with libbs. Here I have bad news. I successfully convert many different videos from various games (tested all games I have at the moment) using only my GNU [heheh] tools EXCEPT ff7 videos. Deeper investigating in ff7 video frame shows me, that there is wierd section of data at the beginning of the video frame:
DE 0D F9 07 00 00 25 02
45 FC 96 F0 51 F8 5B 0D
B2 FB B2 FB 11 00 00 00
F8 00 00 00 62 12 00 00
E7 FF 00 00 BB 03 00 00
Of course this data is different for every frame. After this data begins "normal" frame data. Removing this data from file don't help to decode frame.
I think that problem is in "another" VLC coding method used in FF7 movies. This data may be some VLC table in Huffman Coding, or something else... I am totally confused with this...  :|
I tried some windows PSX video players which works perfectly with FF7 (of course this is closed source ). Also tried to debug pcsx emulator - it may be very funny, but I cann't find in pcsx source where VLC decoding is taking place (only mdec decoder functions which already included in libbs). Also tried official Sony's program for converting psx videos (MC32.EXE) - it works for anything except FF7 video - when I try to load FF7 bs-frame I got from my demuxer, MC32.EXE fails with message "Invalid MDEC Mode 0x7f90000" - maybe Square interpret this unknown data manual??
My sources currently are not in SVN - code is dirty, I don't want to add something, that not working yet - if anyone want, I will upload it somewhere. I also wrote some documentation about CDXA PSX video frame data.
I heard about some official Sony's PSX video format documentation - maybe someone can get this for me? So if anyone have ideas about decoding FF7 video frames, or can help somehow - post here.

 

[Cyberman]

Ergh I knew this would bite US sometime.
I've noticed a few odd anomolies in PCSX when decoding the aforementioned data (in my question to run FF7 in debugging mode).  It appears that there are bugs in PCSX emulation reguarding some of this.  (Just like ePSXe has problems with FF9's video segments).

As you said it could be decoding data.  The movies are an odd size (320x224) and can be ripped completely using PSXmc. There is another possibility it could be new matrix data for the DCT in the MDEC.  Square used specific binaries for playing movies, I suspect they made custom movie decoders.

Stephen

 

[Micky]

It looks like some games use huffman compression on top of the rle compression that the mdec decodes. That wouldn't appear in an emulator, as I expect the codec copies frames from the CD into a RAM buffer, decompresses the huffman data and then feeds the result into the mdec. I think  ffmpeg contains some code to do this. I'm sure at some point I was able to view FF7 movies with it, but not with recent versions. Maybe they broke that when they added support for another game.

 

[G]

Micky, what version of ffmpeg you was using to successfully play FF7 movies? Or, better, do you have a source of ffmpeg, that works? Or when you downloaded ffmpeg source - I will download specific SVN release of ffmpeg, and play with it. All I know is that on MPlayer users mailinglist people were complaining that FF7 movies are *entirely* different and nobody could play them.

 

[G]

Today I tried to remove this unknown data from frame and convert it with Sony's movie converter - It works! But in libbs there is another problem. Trying to understand this a bit better...

, kupo

 

[Micky]

Micky, what version of ffmpeg you was using to successfully play FF7 movies? Or, better, do you have a source of ffmpeg, that works? Or when you downloaded ffmpeg source - I will download specific SVN release of ffmpeg, and play with it. All I know is that on MPlayer users mailinglist people were complaining that FF7 movies are *entirely* different and nobody could play them.

Sorry, I've no idea... must be a few years ago, though.

 

[halkun]

I bet that the data might be camera information. A stream file [STR] can interleave more than just audio and video. it's actually a container format with many "tracks" that is up the the programmer to fill. Usually there is a flag in the header to seperate system data and user-created data. This way system data can be shuffuled to the MDEC can the user frames can be processed.

(Ok, I need to pull out my PSX notes from about 8 years ago.....)

Run down of the STR format...

When the CD-ROM is streaming data, you can bypass the BIOS and get data from point A (CD) to point B (Memory)  using DMA. Most of the time this is just a way to quickly shovel movie data directly into the MDEC decoder chip without having to lock the whole machine. A STR file is mastered into mode 2 data, meaning that you get more data per CD Sector, at the loss of some data integrity subchannels. Let's just hope that there is isn't a scratch in the mode 2 data, or you might be having a hard time.....

A STR format is basically an image of the sectors directly on the disk, I'm guessing you guys already knew that. Because each mode 2 sector is 2048 bytes long, each STR section (or "frame", I think the PSX devs called them) is also 2048 bytes long

Each frame has a header, 32 bytes long, and the the rest is data. The data can be anything, (2048-32=2016 bytes left over for whatever you want) but the header must show up at the beginning of each "sector" (or frame) so that the PSX's CD-ROM data controller doesn't go nuts trying to figure out what to do with the streaming data once it's read. (MDEC data is a little more time-crititical so it's nice to let the CD-ROM controller know what DMA path to fire the video stream down)

The header has a breakdown as follows.

The first two bytes is the STR "magic number" It starts with "0110xxxxxxxxxxxx" The other bits don't seem to be used an any real way.

The second two bytes is the data type. The only thing that is important here is the very first bit (MSB). If it's a "1" than the data in the frame is a PSX hardware format. (for example, MDEC), and if it's a "0" that means the data is user-created and what's in it is anyone's game. If this header section shows that this frame contains user-defined data, the rest of the two bytes can be filled with custion header data for the 2016 bytes of data contained in the frame.

For those following along, the rest of the 32 bit header is as follows.
-Sector number (2 bytes)
-Sector size (2 bytes)
-Frame number (4 bytes)
-Frame size (4 bytes)
-Scratchpad (16 bytes)


Now, if the frame contains MDEC data, it's header is a little different. First the MSB in the type field is set to "1" (actually, the type is 0x8001(? I think. I don't have an STR file handy to check)) and all the other frame header format is the same, save for the 16 byte scratchpad. This is changed where the first two bytes are movie width and the second two bytes are movie hight. This is where you are going to find FF7's "funny" movie resolution.

I hope this breaks down the frames for you.

Also, another thing.The user-data interspaced in FF7's movie data is most likely camera or animation data. (CLUT data for the drop in colordepth?) You will have to take that unknown and compare it to PC data related to the moves, as I'm pretty sure they couldn't jam the extra streaming data into the PC movie formats.

Also, be aware, the user-defined header may contain valueble information, such as telling the system to switch colordepth druing movieplay back, or to kick off a script event.

That should be helpful...

---EDIT---

I've got conficting notes when it comes to XA-Audio interleaving

If the sector is formatted for XA Audio data, there might be an 8 byte header before the "real" header and a trailing 280 byte padding the bottom of the frame.

Does this help?

 

[G]

Thank you, halkun for your info! About video frame format header I know, but about user data it is very helpful. Now I am 80% sure, that this user data is camera moving data. For example I was working with opening movie, and in the end we have camera moving and movie in background, so in that video frame there is user data, I checked some other files and there user data zeroed. So for now I will skip this user data and continue my work on mdec decoder (sony's utility decodes ff7 frame without this data, but my don't yet ). Thanks again for reply

, kupo

 

[G]

!!! - I HAVE DONE IT - !!!
*FF7-enabled* open source PSX video decoder ready. Now I will clean up the code (I think this will take some time, need also to rewrite sound backend to SDL) & add it to q-gears source with XA audio decoder & I think I will add also some code to play movies from q-gears main screen (can I do this?)

 

[Micky]

Cool! Is it just the extra camera data, then? And are the mdec packets modified in any way?

 

[Akari]

!!! - I HAVE DONE IT - !!!
*FF7-enabled* open source PSX video decoder ready. Now I will clean up the code (I think this will take some time, need also to rewrite sound backend to SDL) & add it to q-gears source with XA audio decoder & I think I will add also some code to play movies from q-gears main screen (can I do this?)

You can add some test module to ScreenTests module (DisplayTest and GuyTest like). And we could start adding movies to field module. At last =)

Then only field sound and music left =)

 

[G]

Cool! Is it just the extra camera data, then? And are the mdec packets modified in any way?

As halkun posted, this data is user data, so Square can incapsulate what they want there to be. I think that there can be camera information and possible sound timing for another channel played during video play (see my topic about sound system). For now I will skip this, later I will look what is there.
MDEC packets isn't modified. There is 3 slightly different variations of mdec coding (presented by Sony(tm)) [type/version field in beginning of video frame data]. This "type" seems determines a way of decoding DC coefficient (upper-left coefficient in macroblock). Final Fantasy 7 is of type 1. I haven't seen any other PSX movie of type 1. My version is that when Square begins development of FF7 (one of the first games for PSX, or I am wrong?) they only had given tools for encoding movies to type 1, later games used type 2/3 (it's only my explanation - it's just my fantasy ). Actually I don't see difference between type 1 & 2, but who knows...

You can add some test module to ScreenTests module (DisplayTest and GuyTest like). And we could start adding movies to field module. At last =)

Ok, I will do it after cleaning the code.

 

[G]

 

[halkun]

I feel so useful all of the sudden ^_^!

I knew there was a reason why I was part of this group... I just couldn't remeber what it was...

 

[Cyberman]

halkun
I know what you are saying, my mind is so scattered sometimes I feel like someone threw marbles on the floor and that's my mind all over the plac.

G
I think Type 2/3 uses different weights for the MDEC matrix for quantization.
Although the document I read had one matrix mention, hmmm really I'm not sure.
I know of another game that uses weird dimensions for an MDEC, and even more strange is that the MDEC is run from memory (Alundra 2 does this for the animated key which is just a very small MDEC), I've not seen any PS1 emulator other than the one Sony sued over (not Bleem) run this correctly.  The MDEC dimensions are 160x64 or something like that.  The game loads data while this animation plays. It is rather strange to watch to say the least.

If you don't notice a difference in anything, I guess it doesn't matter (other than you might want to hack a header on the data to test with).

For the audio this depends on the MDEC header no? It can be ADPCM in a few formats (and more than 2 channels), the primary difference is the sampling rate for this.   Using the ADPCM decoder from an emulator is likely fine (you should give credits yada yada).
As for audio, My method (which is a bit ... barbaric I suppose) is to reverse engineer the Square audio engine part of the kernel and adapt this by using the source for an SPU (Peops SPU?) as the audio generation system. The actual audio data then is handled by whatever is choosen. This should be about as platform independant a method as any.   The later only works with the PS1 music however.  I'm not sure what to do with the PC variant in the data, perhaps grumble and complain a lot?

Cyb

 

[halkun]

No, there is a software MIDI player we can take sources from, and it uses soundfonts as well. (Timidity++) That shouldn't be a problem.

 

[Cyberman]

No, there is a software MIDI player we can take sources from, and it uses soundfonts as well. (Timidity++) That shouldn't be a problem.

Apart from the interesting midi format used in the PS1? That seems to be the developement media of choice currently.  I'm familiar with Timidity. Let's see we would have to create the wave table entries from the FF7 data and then send timidity a general midi stream from the remaining FF7 data using the sound fonts (priorly created selected or what have you)?  Might require converting timidity into a 'thread on it's own' type thing.  The 'thread' gets feed midi and sound font data (from the provided game information), it spits out PCM data that gets routed by SDL to the physical sound device (if any). (Stream in stream out kind of thing).  For situations involved ADPCM and the PCM output we will have to figure out the mixing settings the game used with the Playstation hardware (PC media I believe has just the music added to the movies).  Is there a library version of timidity that is thread safe?

Cyb

 

[G]

I want also to ask, what format of saving interleaved sectors is better? I can grab full interleaved sectors for every movie from cd-image and save as is, however we don't need sync (0x00 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0xff 0x00), cd-header (4 bytes) and last 280 bytes of sector (actually I don't know what we have there, but in video data this is not needed). But on the other hand if we leave sectors as is, we will save compatibility with other players (such as ffmpeg, which wants psx data to be raw sectors dump) [need also to send patch for ffmpeg to play FF7 movies  :-D ]

 

[halkun]

If we are directly readng the CD,you are going to need to least compensate for the extra wide sectors. Ripping the full mode2 sectors as a file shouldn't exapnd the videos *that* much. In a pintch, wouldn't be awesometo write an "installer" that will rip/copy files directly to the hard drive?

For what it's worth, the last 280 bytes are padding and can be tossed.

 

[G]

I already wrote movie dumper, now it works only with image and doesn't support proper filenames (haven't much time to understand ISO-filesystem, later I will fix it) (*.MOV) - but it should not be hard to change it for work with cdrom, at least on linux
Actually I am too thinking about some installer, which by one click prepare FF7 data directory on PC to save time for users.