FIELD.BIN (exploration) thread

Lazy Bastard · Jul 22, 2012

Jenovah's Witness: As halkun said, FF7 uses raw disc sectors to reference data, rather than file names and directories. Luckily, in this case, sectors are synonymous with LBA (Logical Block Address), so all we have to do to see what file is at what sector is fire up something like ISOBuster (or one of many other apps), load the disc image, and take a look at LBA for each file.

Lazy Bastard · Jul 22, 2012

As a test, I deleted the first 8 bytes of FIELD.BIN, ungzipped it, noted the references to the three MD1STIN.* files (.DAT, .MIM, and .BSX), noted the references to the three MOGU_1.* files, and overwrote those of MD1STIN with those of MOGU_1. In other words, I told FIELD.BIN that when it wants to call MD1STIN, it should point to MOGU_1; this should make the game start at Mog House, rather than the Sector 1 train station.

I then gzipped it again, added the original 8 bytes to the beginning of the file again, renamed it to FIELD.BIN, launched CDMage, loaded a pristine copy of FF7 Disc 1 (as a Mode 2/2352-byte image), and imported FIELD.BIN directly over the top of the existing one. Probably due to different gzip implementations, my file was 2 KB smaller than the original, so CDMage asked if I'd like to pad the file with zeros, to which I responded no. It successfully imported the file (to the same LBA/sector as the original), and I saved the new image as a new file (in the same format I loaded it as, Mode 2/2352-byte image; I also told it to ignore sectors that couldn't be 'converted', as otherwise, it would immediately error out...I assume this is related to FMVs and Mode2/XA data or something). I then launched PCSX-Reloaded (a PSX emulator), loaded the ISO, and it worked fine. However, once I hit NEW GAME, the emulator crashed.

My first thought was that the new FIELD.BIN's different gzip compression could be making FF7 angry...quite possible, of course. So I tried simply loading a pristine copy of FF7 Disc 1 in CDMage, in the same way I did earlier, and then saving another copy, without modifying it. This yielded the same result. For some reason, something CDMage does when saving the image corrupts something, and this something isn't a problem until FIELD.BIN itself is loaded...even if FIELD.BIN isn't intentionally modified.

Does anyone have another program that will allow me to insert a file at a specific LBA, and won't cause the issue I'm experiencing?

Gemini · Jul 23, 2012

What do you mean by same position? If they are the same size as they originally were, if they have the original name, and if they are in the original directory?

No, I mean they must share the same LBA and size on all three disks. There's no point in retaining the original size (and whatever other crap) if you're planning on regenerating an expanded ISO.

Probably due to different gzip implementations, my file was 2 KB smaller than the original

Check if the gzip signature is correct and if the header contains any extra information like time stamps and file names (FFVII doesn't like 'em), just in case.

It successfully imported the file (to the same LBA/sector as the original), and I saved the new image as a new file (in the same format I loaded it as, Mode 2/2352-byte image;

You don't save a new image after injecting a file, it's unnecessary. The ISO is already changed as soon as you confirm the injection.

Lazy Bastard · Jul 23, 2012

You don't save a new image after injecting a file, it's unnecessary. The ISO is already changed as soon as you confirm the injection.

Negative; rather than choosing Save after making modifications, I chose Save As, and created a new image. This way, I could repeatedly test different output methods without having to use another copy of the image elsewhere on my hard drive or make copies multiple times. The "last modified" information for the original image I loaded in CDMage did not change - changes are not applied until you save them.

Bosola · Jul 23, 2012

Luckily, in this case, sectors are synonymous with LBA (Logical Block Address)

Actually, not quite - LBA values are always 150 lower than the corresponding sector IDs, because the first 150 sectors of a PSX disc are used for license data. That means if you're hunting look ups, you have to account for the numeric difference.

Gemini · Jul 24, 2012

Negative; rather than choosing Save after making modifications, I chose Save As, and created a new image.

You don't need to use Save, changes are instantly applied no matter what command you use after import. With Save As all you do is creating yet another identical copy of the ISO.

Actually, not quite - LBA values are always 150 lower than the corresponding sector IDs, because the first 150 sectors of a PSX disc are used for license data. That means if you're hunting look ups, you have to account for the numeric difference.

There's no difference to be taken into account, you were probably mislead by CDMage's sector scroller (in which sector 150=LBA 0=beginning of the ISO). Let's go by examples: LBA=0 means something is located at sector 0, byte 0; LBA=23 means the sector you need to seek to is the 23rd, byte 54188 (23*2352=54096, where 2352 is the size in bytes of each sector). Licensing data is much smaller than 150 sectors.

Lazy Bastard · Jul 25, 2012

You don't need to use Save, changes are instantly applied no matter what command you use after import. With Save As all you do is creating yet another identical copy of the ISO.

Ack, you're right. If you choose Save As, it saves both the initial copy AND the new copy...ugh. I stand corrected.

Lazy Bastard · Jul 26, 2012

Check if the gzip signature is correct and if the header contains any extra information like time stamps and file names (FFVII doesn't like 'em), just in case.

After resolving the issues I was having with importing files using CDMage, etc (and successfully importing various other files without issues), I've decompressed FIELD.BIN, recompressed it without making any modifications, then imported it back into FF7 Disc 1, and this still caused a crash after choosing NEW GAME. This leads me to believe that there indeed is an issue with the gzip header, or something else related to the gzip algorithm. I've tried using both 7Zip's gzip support and ff7dec and its bundled gzip, with the same results. Of course, I re-added the original 8 bytes that I stripped off in the first place, verbatim.

Is there anything in particular that I should be looking for? I've looked around at the gzip RFC, various articles concerning gzip headers and signatures, and general gzip documentation, but I haven't found anything that describes headers in such a way that I could hex edit and eliminate/modify them in a meaningful way...

Gemini · Jul 26, 2012

Grab the command line gzip and make sure to use the proper options (should be a couple flags, IIRC) to exclude unwanted stuff from the header.

Kranmer · Jul 26, 2012

After resolving the issues I was having with importing files using CDMage, etc (and successfully importing various other files without issues), I've decompressed FIELD.BIN, recompressed it without making any modifications, then imported it back into FF7 Disc 1, and this still caused a crash after choosing NEW GAME. This leads me to believe that there indeed is an issue with the gzip header, or something else related to the gzip algorithm. I've tried using both 7Zip's gzip support and ff7dec and its bundled gzip, with the same results. Of course, I re-added the original 8 bytes that I stripped off in the first place, verbatim.

Is there anything in particular that I should be looking for? I've looked around at the gzip RFC, various articles concerning gzip headers and signatures, and general gzip documentation, but I haven't found anything that describes headers in such a way that I could hex edit and eliminate/modify them in a meaningful way...

I edited the gzip header like this and it worked ingame (tested on a emulator)

All i basically did was copy the header of the file from the original FIELD.BIN over to the new one compressed in gzip, hope this helps.

EDIT:- just rechecked, on the re-compressed Gzip all you have to do is add the original 8 bytes back into the file (the ones removed so we can decompress) and it should work (providing you also compressed the same way i did in the video above) so just make sure that when you use the command line Gzip you use
gzip -9 -n file
to compress the file (never use the 7zip gzip for this) then add the original 8 bytes back, this works for me (also when i added it into CDmage i also said YES to zero bytes when replacing)

Lazy Bastard · Jul 26, 2012

Grab the command line gzip and make sure to use the proper options (should be a couple flags, IIRC) to exclude unwanted stuff from the header.

so just make sure that when you use the command line Gzip you use
gzip -9 -n file
to compress the file (never use the 7zip gzip for this) then add the original 8 bytes back, this works for me (also when i added it into CDmage i also said YES to zero bytes when replacing)

Yep, those gzip flags (-9 -n) did the trick

I was already adding the 8-byte header back at the end (I mean, the end of the process...beginning of the file, heh), but apparently my gzip process was adding extra crap. Thanks.

I was able to recompress and run with an emulator without issue...however, the edits I attempted resulted in my getting a black screen and a repeating FF7 Error sound, when I chose NEW GAME. No matter, this just means I need to go back and check my work. And really, I have no need for editing FIELD.BIN quite yet...I just want the capability so that when the time comes, I can do so. Also, pointing FIELD.BIN directly to the first FIELD\*.DAT file I want to load, rather than hacking the first naturally-loaded one (MD1STIN.DAT) should be slightly faster, and much more correct...not that it would really matter.

Well, off to work. Thanks again for the help

Jenova's Witness · Jul 26, 2012

Lazy Bastard · Jul 27, 2012

Jenova's Witness: Yep; I've seen the Python docs on gzip. Also, before you posted that, you and Gemini had exactly the same number of posts :-o

Just an FYI for those interested: as far as I can tell, replacing the value of one field's LBA in FIELD.BIN with that of another field doesn't allow you to 'trick' FIELD.BIN into loading the field you want. Something seems to go wrong, and you just end up with a black screen and a repeated FF7 error sound. I've also tried replacing the LBA values for all three field files per field (.BSX, .MIM, and .DAT), to no avail.

This could be because FIELD.BIN isn't just choosing a file to load, but passing it arguments, etc...some of which may not make sense relative to a different field. I haven't been able to identify the exact cause yet, but I do know this: if I'm able to cause it to error out by changing the value of a field's LBA in FIELD.BIN, and it only errors out when I edit the field that's about to be loaded, that would imply that FIELD.BIN at least references that value before loading a field. Therefore, I'm on the right track toward modifying which field is loaded. In any case, jumping straight from MD1STIN.DAT to the field I really wish to load (using Makou Reactor or Hack7) isn't a huge problem, as long as I never want to use MD1STIN.DAT legitimately...and even if I do, I can always jam in a variable and If/Else to determine whether a new game has just been launched or not.

OK, back to work on that prequel mod, unless someone has something else to add.

Micky · Jul 30, 2012

I think field.bin stores both position and size... it has been ages since I looked at it, though. You could try to match the whole table of position/size against the disc image and the directory to confirm.

lasyan3 · Jul 30, 2012

I think field.bin stores both position and size...

That's right.

Cyberman · Jul 31, 2012

That's right.

It's been eons since I've fiddled with this stuff.

Just don't have as much time as I use to.. Anyhow haven't seen you in a while.

FF7 may be doing a check to see if the data is correct when extracted. As the others have noted there is a size field as well as the LBA. If the size doesn't exactly match the real file size this will indicate it could be the uncompressed size. I don't remember which too be honest it's been quite a while.

I suggest you verify what lasyan3 and Mickey said by being sure you change the size information too match the new location.

It may be helpful to create a tool to read the table data and allow you to swap the data around with LBA and size information together.

So looking at it logically the table is 6 x N locations (6 columns N locations for rows)
IE
LBA | SIZE | LBA | SIZE | LBA SIZE

If you want to swap the data or have a row point to another location it's probably a good idea to swap the whole row or change the whole row (6 values instead of 3).

I don't remember anything about data checks such as CRC's etc. for the files, for all I remember there may be.

I can't remember where the size data is however, I'm so tired these days it's sad

Cyb

FIELD.BIN (exploration) thread

Lazy Bastard

Guest

Lazy Bastard

Guest

Gemini

Guest

Lazy Bastard

Guest

Bosola

Guest

Gemini

Guest

Lazy Bastard

Guest

Lazy Bastard

Guest

Gemini

Guest

Kranmer

Guest

Lazy Bastard

Guest

Jenova's Witness

Guest

Lazy Bastard

Guest

Micky

Guest

lasyan3

Guest

Cyberman

Guest