New ISO image access routines + Other

[Cyberman]

I've been working on these for a while.
Some notes. ISO directories are sector sized and sequentially stored. However they do not overlap sector boundries. This makes for getting the contents confusing if you aren't paying attention.  A directory size will ALWAYS be a multiple of the size of a Mode1 Form1 sector (2048 bytes).  The size of files in IS9660 reflects the size in sectors * 2048 used for Mode 2 files.  This of course means that the actual size is... relative. Furthermore a Mode2 file is never 2048 bytes in size.  (read never several times) Why is that?  Mode 2 files have subheaders which need to be read in order to know what the data is.  8 bytes are used in this subheader (actually only 4 but they are repeated againm likely to ensure that the data is read correctly).

Currently my routines are not perfect (no doh). They will only read a data sector at a time.  I will make them set up to read exactly how much data from a set of sectors you wish to read from, however, you need to be wary of the fact mode 2 sectors may return more than you bargined for (IE more than 2048 bytes per sector).   It might be good to differentiate between these special sectors by returning Sector Size for the beginning record of a file.  You can read 4096 bytes from a Mode2 file but not read a complete 2 sectors.  The size varies per sector in the file also.  Some of the movie files in FF7 are 2K sectors some aren't. The movies with audio are not and they vary throughout the file, audio is interleaved with the video sectors.

So what works? Searching for a file and finding it's directory Record information works.  Soonish I'll have the special 'file system' set up by Akari functioning with ISO images of the 2352 size per sector variety.

The other thing is .. start up file system etc.  I think it would be sane to have this a CLI option for Q-gears (and or in an XML based INI file).   The file system should be just fine.  I think in the current testing however we might have a few issues.

So should I dump the complete listing of Directory/Files and the Logical sector numbers for FF7 disk 1 so we can examine FF7's engine for sector and offset information (IE it assumes something is in a certain place instead of looking for a file).

Of note the PS1's limitation on CDROM directories appears to be 2 directory sectors. Any mroe than just 2 sectors and the PS1's normal CDROM routines loose the rest of the directory (it likely has a permanent 4K buffer to iterate through).

Cyb

 

[G]

Some of the movie files in FF7 are 2K sectors some aren't

STR movie sectors are 2K because of the ordinary STR files are multiply of 2048 - it's a concatenations of XA video frame data sectors.

 

[gigaherz]

there is 2 forms in mode2, from1 where there is 2048 bytes of data + some bytes of error correction, and form2 where the error correction isn't there, and it's just data.

nicer explanation

obviously, if you want to be able to read that data, you must be able to read the sectors in raw format, and then check the sector mode to know if it's using form1 or form2.

I don't get why do you say the file is never 2k. The file canbe anything from 1 byte to the whole CD... you only need to read the sector header to be able to tell if that sector contains 2048 or 2324 bytes, after that you can just return those bytes without ever telling the "caller" how were they stored in the disk...

to read any number of bytes from any position
you just need a "sector buffer"
2352 bytes in size
then
have some "current pointer" and "bytes left in sector" values, then just fill the buffer till you read all the data, or reached the end of the file. like:

Code: [Select]

//fast, lame and poorly named codeint read_data(char* buffer_pointer, int bytes_to_read){  int bytes_read=0;  while((bytes_to_read>0)&&(bytes_left_in_file>0))  {    if(bytes_in_sector==0) read_next_sector(&sector_pointer,&bytes_in_sector);    int bytes_to_copy=min(bytes_in_sector,bytes_to_read);    memcpy(buffer_pointer,sector_pointer,bytes_to_copy);    buffer_pointer+=bytes_to_copy;    sector_pointer+=bytes_to_copy;    bytes_left_in_file-=bytes_to_copy;    bytes_to_read-=bytes_to_copy;    bytes_read+=bytes_to_copy;    bytes_in_sector-=bytes_to_copy;  }  return bytes_read;}

This code assumes it's contained in some class and all the vars not explicitly declared are assumed to be instance variables initialized before calling.
(someone might remember I said I hate _ in code, and I feel it's really stupid that I preferred them in this case... ).

 

[Cyberman]

LOL typo sorry 2K was for the sector.  You are correct.  Mode 2 files are different than mode 2 sector.  Sorry about that.

Here is what I am talking about, it's a brief description of PS1 mixed mode sectors.  From Pixel and this Readme for his LUA CD tool.  Thus far I've found his information is 100% acurate.
Code: [Select]

------------------------------------------ GENERAL INFORMATION ABOUT A CD SECTOR ------------------------------------------   Here is the general form of a CD-Rom sector:    <--------------------------- sector: 2352 bytes ------------------------------> <- Header: 16 bytes -><---------------- Datas: 2336 bytes -------------------->   Let us move to the header description:    <--------------------------- header: 16 bytes ------------------------------> <-- sync bytes: 12 bytes --><-- localisation: 3 bytes --><-- mode: 1 byte -->** SYNC BITES **   The sync bytes are easy: it is always 00 FF FF FF FF FF FF FF FF FF FF 00   They are supposed to help the CD reader to synchronize enough to be able to   read the sector correctly.** LOCALISATION **   The localisation is the sector "position" described in time. For example,   the sector 200000 of a CD is at the "time" 44:28:50. The first is the number   of minutes, the second is the number of seconds, in the range 0-59 and the   last is the frame number, in the range 0-74. It means there is 75 frames   into a second for a CD player. Please note that the CD "begins" at 00:02:00.      Ok now that we know all this, you can feel the way the localisation is   stored. But it is not that easy...    <-------------------- localisation: 3 bytes --------------------> <-- minute: 1 byte --><-- second: 1 byte --><-- frame: 1 byte -->   That seems to be all right *BUT* the fact is that the bytes are stored   in packed BCD format. You may know what the BCD format is if you are "old"   enough for that. I will not enter into the details so if you want a more detailed   description of the BCD format, look into the net. You only have to know that:typedef unsigned char uchar;uchar from_BCD(uchar x) {return ((x & 15) + ((x & 240) >> 4) * 10));}uchar to_BCD(uchar x) {return ((x / 10) << 4) | (x % 10));}int is_valid_BCD(uchar x) {return (((x & 15) < 10) && ((x >> 4) < 10));}   Last hint: when you look at a BCD packed number, you have to read it in   hexadecimal, and then you will see a "decimal" number. So when you count   in BCD, you will have this: 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,   0x08, 0x09, 0x10, 0x11, 0x12, etc... You see? You have got a "gap": no 0x0a,   0x0b, 0x0c, etc... So the BCD is only a trick for an easy reading of hexa   dumps of various information.** MODE BYTE **   All right. This was for the localisation part. The last part is the mode   byte. It is really simple actually. It is 0 for an empty sector, 1 for a   sector in MODE1, and 2 for a sector in MODE2. Easy as hell.      Ok here we are: we know the basic form of a CD sector, and even know the   MODE of the sector. Now the data depend upon the sector mode. Here you have   the various kinds:    <-------------- MODE 1 FORM 1 Sector data: 2336 bytes ----------------------> <- data: 2048 bytes -><- EDC: 4 bytes -><- 0s: 8 bytes -><- ECC: 276 bytes ->  <---------- MODE 1 FORM 2 and also MODE 2 Sector data: 2336 bytes ----------> <----------------------------- data: 2336 bytes ----------------------------> <-------------- MODE 2 FORM 1 Sector data: 2336 bytes ----------------------> <- SH: 8 bytes -><- data: 2048 bytes -><- EDC: 4 bytes -><- ECC: 276 bytes -> <-------------- MODE 2 FORM 2 Sector data: 2336 bytes ----------------------> <- SH: 8 bytes -><---------- data: 2324 bytes ----------><- spare: 4 bytes ->   Well, I *really* do not know how to distinguish the different "FORMS" from   each others for the MODE 1. Have to look further about this.      The ECC and EDC are control blocks. The yazedc code can compute them, so   do not worry about them.** SUBHEADER **   The 'SH' (SubHeader) field is the most "complicated" one. Those eight little   bits are the only one I am really not sure of. All of that because you have to   buy the Books to find the information. This SubHeader is only found into   MODE_2_FORM_1 and MODE_2_FORM_2 sectors.      Here you have the information I have been able to gather:      -) The SubHeader has 8 bytes, but it is twice the same 4 bytes.   -) The 4 bytes are described using the following fields:     o) 1st byte: File Number (FN)     o) 2nd byte: Channel Number (CN)     o) 3rd byte: Sub Mode (SM)     o) 4rth byte: Coding Info (CI)   -) The SubHeader seems to be very important when dealing with STR files,      since it is the only way to distinguish a video sector from an audio      sector. But it seems that it does not matter when dealing with normal      files. It might be obvious since the ISO9660 format does not care about      those bits. But better try to patch them if necessary...   -) The Sub Mode byte is a bit field which seems to be described like this:       0: End of Record (EOR)       1: Video       2: Audio       3: Data       4: Trigger       5: Form 2       6: Real Time (RT)       7: End of File (EOF)   Of course, PSX CD's are recorded in MODE 2... So common files are   stored in MODE 2 FORM 1, STR/XA files are stored in "MODE 2" but   actually they are in MODE 2 FORM 1 and 2. The MOVCONV tool will in fact   produce files that contain the subheaders.      Those subheaders are very likely to vary, and seem to be very important for   stream processing. Please note that "str" video sectors are considered as   data sectors, and not as video sectors.** CN BYTE **   The CN byte indicates the channel number of the current sector. The XA   format may contain interlaced channels. So for example, if you have a   file that contains 8 channels, you will have first the first sector   of the channel 0, then the first sector of the channel 1, etc...      This is also a bit more difficult when you know that video is also   interlaced and considered as a channel itself. The common interlacement   is 7 video sectors for 1 audio sector, but this may vary. And all the   channels may be completely independant. For example, you may have a   sound-free video that contains an audio channel, this audio channel   may be used for another part in the game.      This is to optimize the reading process. Since the cd reader is a 2x cd   reader, it *HAS* to read data in full 300KBps. So, if you have a sound   free video, the reading process will be faster than the decoding process,   and everything should crash. This is about the same for the audio sectors.   The 'leap' sector function of MOVCONV adds blank sectors in order to   pad the channels that may have stopped before the others.       One "speed" of the CD reader corresponds to four times the playback speed   of a stereo audio channel at 37800Hz. So at full speed you can have eight   stereo audio channels at 37800Hz. Or you can have 32 mono audio channels   at 18900Hz.      Common video str files need 7/8 of the full speed of the CD reader.   "Common" means 320x224 videos at 15fps. So you can have a full movie   in 320x224x15fps with a stereo sound track at 37800hz. So, now, you   may understand why the common interlacement may vary.** CI BYTE **      The CI byte contain some flags about the current sector, but I am   yet unable to give a full description of them. I have only got this: for   XA audio sectors, the bit 0 is set when you have stereo sound, and the bit   2 is set when you have "half frequency", ie 18900Hz instead of 37800Hz.      The Video frames are in plain Form 1, and the Audio frames are in plain   Form 2. But it _seems_ the Video frames are not checked against ECC/EDC,   and filled with zeroes instead.      Last but not least: MODE 2 FORM 1 and MODE 2 FORM 2 are also called    XA-Mode1 and XA-Mode2 or simplier: XA-1 and XA-2.   I hope this will help you as it helped me writing this software.

I hope this helps everyone, so I don't sound completely crazy (although it wouldn't surprise me if I did).

Cyb