M
MrAdults
Guest
Thanks for putting those up, I've got them now. Will have a look later on.
[Dissidia] Models - Viewer and Extractor Tool by MrAdults (2009-04-08)
- Status
A
Aurenasek116
Guest
Take your time, nobody's rushing 
K
koral
Guest
I had a quick look at the files too, and they are abosultely nothing like the (nice and simple) CC models, so we are going to have to start from scratch with this one
The data seems to be quite well organised, and all the different components are uniquely named, which makes things a little bit easier.
I thought I saw some "xx.tga" ascii too, but I didn't think PSP could decode TGA files? They might represent some other kind of Dissidia-specific format.
Aren't GIM files suppsed to be equivalent to GIF, as in animated textures? I remember seeing them used for PS3 themes and stuff, so it feels out of character for them to be used as normal textures. Probably they are just for things like blinking and mouth moving animations.
The actual vertex data might be same as CC (probably should be too) but it wouldn't be smart to jump right in without trying to understand what else is in those files and what the file header says about offsets.
There might be no need for offsets, because we could skip to the named sections directly in most cases, they sound like they might be the same in most of the other files too.
It's a whole new can of worms :-D
The data seems to be quite well organised, and all the different components are uniquely named, which makes things a little bit easier.
I thought I saw some "xx.tga" ascii too, but I didn't think PSP could decode TGA files? They might represent some other kind of Dissidia-specific format.
Aren't GIM files suppsed to be equivalent to GIF, as in animated textures? I remember seeing them used for PS3 themes and stuff, so it feels out of character for them to be used as normal textures. Probably they are just for things like blinking and mouth moving animations.
The actual vertex data might be same as CC (probably should be too) but it wouldn't be smart to jump right in without trying to understand what else is in those files and what the file header says about offsets.
There might be no need for offsets, because we could skip to the named sections directly in most cases, they sound like they might be the same in most of the other files too.
It's a whole new can of worms :-D
A
Aurenasek116
Guest
Trust me, GIM is short for Generic Image (Or atleast that's what I was told), I'm using a simple converter written by kind people of alucard.cc which converts dissidia gim files into png/tga/dds or whatever other format you want, so we can be 100% sure those are textures.
If you want, I could put up a package with tools written by people of alucard.cc (Mainly by a guy named Vash - credits go to him) so you could use them
If you want, I could put up a package with tools written by people of alucard.cc (Mainly by a guy named Vash - credits go to him) so you could use them
B
BlitzNCS
Guest
weird, apparently they're just uncompressed tga images in the model files...perhaps he was wrong then...
but then how would you explain the .tga labels? or the tm2 ones?
A
Aurenasek116
Guest
I think it's same as with Atrac audio format, in atrac case it's simply wav but wrapped in new header to make it look like if it was new format. I don't know really, I can be wrong.
Also here are the Dissidia tools coded by Vash, it has a GIM extractor aswell as other stuff which you can have a look at, maybe it will be of any use.
http://rapidshare.com/files/213186921/Dissidia_Tools.rar.html
Rapidshare link for now since FileFront is doing some maintance.
Also here are the Dissidia tools coded by Vash, it has a GIM extractor aswell as other stuff which you can have a look at, maybe it will be of any use.
http://rapidshare.com/files/213186921/Dissidia_Tools.rar.html
Rapidshare link for now since FileFront is doing some maintance.
M
MrAdults
Guest
Just had a look myself. Looks like a nice big batch of straight-up-uncompressed bone matrices near the beginning of the file, so that's good. I would have at least expected some quat compression, but I'm not complaining. First thing to do is probably locate offset bits for the image location since it's a known piece of data. Wherever that offset is, there are probably more offsets to other meaningful mesh data.
Edit: Alright, I've been working on it over the past few hours. The file is broken up into chunks, and I've figured out how to ID the chunks, be they bone data, meshes, or textures. Mesh data is a more complicated matter. I'm using the crystal model as a simple test case.
Strangely, it draws the verts properly as a raw triangle list, but there is a data segment right before the verts that looks very much like triangle strip indices. However, trying to use that index list does not look correct in any way, so I'm not sure what the point is. It can't be a list of weight indices, because the crystal doesn't need to be weighted (since from what I can tell it only has a single root bone, and if every index were the same that would be obvious).
Vertex data is also not stored at all in native PSP GPU format. Vertex positions are offset from the beginning of the list before UV coordinates, which is not an accepted hardware format. However, vertex formats do vary wildly from model to model. I'm probbaly skipping over a format descriptor somewhere along the line.
As a side note, the very large files appear to be purely skeletal animation data with no mesh data at all. But many of the character model chunks also include some skeletal animation data.
First thing to do is probably locate offset bits for the image location since it's a known piece of data. Wherever that offset is, there are probably more offsets to other meaningful mesh data.Edit: Alright, I've been working on it over the past few hours. The file is broken up into chunks, and I've figured out how to ID the chunks, be they bone data, meshes, or textures. Mesh data is a more complicated matter. I'm using the crystal model as a simple test case.
Strangely, it draws the verts properly as a raw triangle list, but there is a data segment right before the verts that looks very much like triangle strip indices. However, trying to use that index list does not look correct in any way, so I'm not sure what the point is. It can't be a list of weight indices, because the crystal doesn't need to be weighted (since from what I can tell it only has a single root bone, and if every index were the same that would be obvious).
Vertex data is also not stored at all in native PSP GPU format. Vertex positions are offset from the beginning of the list before UV coordinates, which is not an accepted hardware format. However, vertex formats do vary wildly from model to model. I'm probbaly skipping over a format descriptor somewhere along the line.
As a side note, the very large files appear to be purely skeletal animation data with no mesh data at all. But many of the character model chunks also include some skeletal animation data.
Last edited:
A
Aurenasek116
Guest
That's a nice piece of work, but the fact about models not being in native PSP GPU format saddens me. I hope it is possible for you/koral or both to find a solution and hopefully make the models be extractable.
M
MrAdults
Guest
I seem to be able to reliably grab vertex data from all of the models now, including all surfaces and surface sub-meshes.
Next thing is to determine a reliable offset for the UV coordinates and other specific vertex data based on the vertex tag. I believe I've found the tag, it just doesn't correspond at all to the PSP hardware's encoding scheme. But it's probably still the key to determining the vertex layout. In any case, I still haven't even bothered to handle ripping the images, so I'll have to get that done sometime before I move onto testing UV's.
Next thing is to determine a reliable offset for the UV coordinates and other specific vertex data based on the vertex tag. I believe I've found the tag, it just doesn't correspond at all to the PSP hardware's encoding scheme. But it's probably still the key to determining the vertex layout. In any case, I still haven't even bothered to handle ripping the images, so I'll have to get that done sometime before I move onto testing UV's.
B
BlitzNCS
Guest
Holy crap
its been what, a day? and you've already almost figured out model data!?
Wow, you realy win dude :O
Epicness, i will keep up with this thread
its been what, a day? and you've already almost figured out model data!?
Wow, you realy win dude :O
Epicness, i will keep up with this thread
K
koral
Guest
wow, that was really quick! :-D
I haven't looked much at these files myself, so sorry I can't be of much help.
You said the vertices were not in PSP-GPU format, which is quite surprising, but what then did you mean by the "tags"?
Some sort of container mechanism for meshes in this game?
But great progress so far, I hope you will fill us in on some specifics soon!
or maybe a model viewer would be a better alternative :wink:
I haven't looked much at these files myself, so sorry I can't be of much help.
You said the vertices were not in PSP-GPU format, which is quite surprising, but what then did you mean by the "tags"?
Some sort of container mechanism for meshes in this game?
But great progress so far, I hope you will fill us in on some specifics soon!
or maybe a model viewer would be a better alternative :wink:
A
Aurenasek116
Guest
This is trully professional work, great job there. You sure work lightning fast.
S
Satoh
Guest
It looks as though they are both holding their normal and EX mode weapons from that shot, maybe we'll be able to tell once we get some UV's figured out...
K
koral
Guest
UVs without texture-maps are useless :lol:
I dont know much about TIM or GIM files, but I do know that we will have to manually parse and extract the textures for a proper kind of model-viewer.
Some further observations:
I didn't set-out to fully document these files, but I can't seem to help myself :-D
I dont know much about TIM or GIM files, but I do know that we will have to manually parse and extract the textures for a proper kind of model-viewer.
Some further observations:
- there appear to be NO obvious offsets at the begging of the files, so the files have to be parsed like X, B3D, 3DS (etc) data, reading and skipping chunks depending on thei "chunk-tags"
- there are "16-mesh" file chunks which are probably 16-bit indexed geometry
- ".tga" chunks, which are soon followed by "MIG 00 1PSP" ascii text, possibly a formal sort of header for the data.
- the Texture data itself seems to be a long list of unpalleted RGBA list (initial impressions)
- "camera" chunks are probably found only in files which are levels/locations/maps
- "arrays" chunks are plentiful and confusing
- "r-box" chunks are most likely bounding-boxes in animated models
- "lambert" chunks are probably generic material properties.
I didn't set-out to fully document these files, but I can't seem to help myself :-D
A
Aurenasek116
Guest
About that MIG stuff Koral, Vash said it's simply GIM.
M
MrAdults
Guest
Yep, I was actually going to liken them to 3DS myself. Those array chunks have vertex info in their chunk headers, and that's where I believe I've found the vertex tag. However, I'm purely guessing that that's what it is. But it seems that it changes in correlation with the vertex stride, so I'm guessing I can use it to get component offsets too.
I'm still unsure what the data in the mesh chunk is. It seems to have an offset to the corresponding array, and usually a series of what looks like triangle indices. But none of the array data is rendered with indices! And, those values always number up to numVerts in the array, so it's hard to imagine what they could possibly be other than triangle indices or weight indices (which they seemingly can't be, as I already mentioned). That data is very mysterious, although it evidently is not needed to render the models.
I have no idea what a GIM file is. But if it's straight-up RGBA, sounds easy enough. I'll get to work on that soon-ish.
Edit: Got textures loading, they're 8-bit paletted 32-bit RGBA with width/height and offset to palette in the chunk header. But it looks like that mystery data is triangle data after all, because even though the model's holes are filled in drawing it without indices, it becomes apparent that the triangle UV's are being stretched incorrectly, so the texture map on the model looks all cut up and only correctly offset from the first triangle vert.
By the way, koral, do you plan to make a viewer for these models? I'm going to write up a format spec eventually either way, but if you're planning to actively work on this, I'll make more effort to document my specific format findings and exact offset locations and so on.
Those array chunks have vertex info in their chunk headers, and that's where I believe I've found the vertex tag. However, I'm purely guessing that that's what it is. But it seems that it changes in correlation with the vertex stride, so I'm guessing I can use it to get component offsets too.I'm still unsure what the data in the mesh chunk is. It seems to have an offset to the corresponding array, and usually a series of what looks like triangle indices. But none of the array data is rendered with indices! And, those values always number up to numVerts in the array, so it's hard to imagine what they could possibly be other than triangle indices or weight indices (which they seemingly can't be, as I already mentioned). That data is very mysterious, although it evidently is not needed to render the models.
I have no idea what a GIM file is. But if it's straight-up RGBA, sounds easy enough. I'll get to work on that soon-ish.
Edit: Got textures loading, they're 8-bit paletted 32-bit RGBA with width/height and offset to palette in the chunk header. But it looks like that mystery data is triangle data after all, because even though the model's holes are filled in drawing it without indices, it becomes apparent that the triangle UV's are being stretched incorrectly, so the texture map on the model looks all cut up and only correctly offset from the first triangle vert.
By the way, koral, do you plan to make a viewer for these models? I'm going to write up a format spec eventually either way, but if you're planning to actively work on this, I'll make more effort to document my specific format findings and exact offset locations and so on.
Last edited:
A
apz freak
Guest
You guys are absolutely amazing you know that? :-D It's interesting and facinating to watch you guys decode the data. But let's not stop the conversational flow for praise just yet, carry on!
K
koral
Guest
I wasn't planning to make a new viewer/extractor for these files.
If you provided some specs and stuff then I could always include the format into the CC-Viewer sometime in the future, but if your mesh2drm tool will support these files anyway, then there would be no need for you to document your findings at all.
I am sure everyone will be happy to use your tool to view/extract the models.
Right? :?
But lets not jump the gun. :wink:
You say there are some weird issues going on with the vertices/indices. Seems like it may be somehow related to the bone weights, but that shouldn't affect UVs?
Weird
If you provided some specs and stuff then I could always include the format into the CC-Viewer sometime in the future, but if your mesh2drm tool will support these files anyway, then there would be no need for you to document your findings at all.
I am sure everyone will be happy to use your tool to view/extract the models.
Right? :?
But lets not jump the gun. :wink:
You say there are some weird issues going on with the vertices/indices. Seems like it may be somehow related to the bone weights, but that shouldn't affect UVs?
Weird
M
MrAdults
Guest
There turned out to be all kinds of issues. Meshes indirectly reference textures through materials, so I had to utilize the material chunks to get textures working. Then it turns out that there are quads as well embedded within the mesh indexing stream, and finally the part that still plagues me, the UV's are scale-and-biased, or maybe just scaled. Since the UV's are shorts, it makes sense to have a per-surface scale-and-bias value to reduce texel precision errors from the range loss. But since that factor varies from model to model (and maybe mesh to mesh, to a lesser degree), I need to find it to handle UV's properly. I've been looking for quite some time now and I just can't figure out where it is.
I had to manually multiply his UV's on U by 4.0 to get him to look correct. Squall takes a 3.0 multiply on U, while Kefka takes a 4.0 on U and 1.5 on V, but some models like the Tonberry look just fine without any scale-and-bias. Once I figure out how that scale is stored, it should be ready for a release.
In addition, you can see that the U on the buster sword's UV is flipped. This is especially interesting, because the buster sword is all one surface and all one mesh, but the hilt still looks correct! I thought this might be a material property of some sort, but it doesn't seem to be, especially since a mesh can only reference one material. This in particular is very confusing. It could mean that my scale-and-bias factor theory is wrong, and something compeltely different is going on with the UV's.
I had to manually multiply his UV's on U by 4.0 to get him to look correct.
Squall takes a 3.0 multiply on U, while Kefka takes a 4.0 on U and 1.5 on V, but some models like the Tonberry look just fine without any scale-and-bias. Once I figure out how that scale is stored, it should be ready for a release.In addition, you can see that the U on the buster sword's UV is flipped. This is especially interesting, because the buster sword is all one surface and all one mesh, but the hilt still looks correct! I thought this might be a material property of some sort, but it doesn't seem to be, especially since a mesh can only reference one material. This in particular is very confusing. It could mean that my scale-and-bias factor theory is wrong, and something compeltely different is going on with the UV's.
K
koral
Guest
Great progress :-D
So its just a case of unpredictable UVs? There should be a logical explanation somewhere, it isn't usually an issue in most cases.
The example with the buster-sword in particular: what comes to my mind is possible polygon-flipping, either the vertices are indexed incorrectly (unlikely, seeing as the mesh looks perfect) or the UVs are indexed to incorrect vertices.
I would love to help you try find the bias and scaling factors within the files themselves (if they exist at all) but I don't know which files are which.
If you can tell me the filenames for the characters you have already mentioned (Squall, Kefka, and the guy above, Bartz from FF5 isn't he?) then I will try hunt for the values.
So its just a case of unpredictable UVs? There should be a logical explanation somewhere, it isn't usually an issue in most cases.
The example with the buster-sword in particular: what comes to my mind is possible polygon-flipping, either the vertices are indexed incorrectly (unlikely, seeing as the mesh looks perfect) or the UVs are indexed to incorrect vertices.
I would love to help you try find the bias and scaling factors within the files themselves (if they exist at all) but I don't know which files are which.
If you can tell me the filenames for the characters you have already mentioned (Squall, Kefka, and the guy above, Bartz from FF5 isn't he?) then I will try hunt for the values.
- Status