[FF7PC]FF7 SYW V3, Battle, world map and fields with ERSGAN (WIP)

  • Thread starter Thread starter satsuki
  • Start date Start date
Status
Not open for further replies.
S

satsuki

Guest
Hello.
As you may have seen, i started some time ago an upscaling of FF7 using a mix of method around ESRGAN.
I posted in others post for helping and sharing but as i dont want to pollute theirs threads i start my own.

So far the filtering is complete but i'm doing a full game play to found bugs and optimise layering cut (doors, animations...) and try to find better upscaling/filtering option for screen witch are not good enouth.

Here's some ingame screens (some render better than others, as ff7 as not a consistant quality on all screen, but i tryed to find the best settings):
ff7sywv3001.jpg


ff7sywv3002.jpg


ff7sywv3003.jpg


ff7sywv3004.jpg


ff7sywv3005.jpg


ff7sywv3006.jpg


ff7sywv3007.jpg


ff7sywv3008.jpg


ff7sywv3009.jpg


ff7sywv3010.jpg


ff7sywv3011.jpg


ff7sywv3012.jpg
 
Jesus Christ!! Amazing work! Holy shit, we're gonna have a renaissance with background upscaling mods for FF7, 8 and 9! My body is ready!
 
Awesome job. This makes the world of FFVII look alive again. :) And I can see details that I din't even notice before which is great.
 
There is some things that could benefit from a touch up here and there but overall it is seriously stunning. This is looking like an actual remaster which is blowing my mind, super exciting stuff!
 
There is some things that could benefit from a touch up here and there but overall it is seriously stunning. This is looking like an actual remaster which is blowing my mind, super exciting stuff!
Click to expand...
Of course but the process is mainly automatic.
It already took me lots of time to make propers cut of layers, find the good filterings.... if i need to correct each screen to optimse it with gimp, it's take me about 15/30minutes per screen so about 200 hours to 300 hours again, + ingame tests... would be to long for the real game gain, on still picture default are clearly visible, not realy while playing game.
 
I really hope you guys who are working on upscaling projects for FF7-8 and 9 are cooperating with your techniques and findings. The user who's upscaling FF9 backgrounds is using his own data training model, trained specifically for FF9 and artwork which is similar. And the results are muuuch better than ESRGAN with Manga dataset.
 
This technology is revolutionary. Good lord.

The future is here.
 
I really hope you guys who are working on upscaling projects for FF7-8 and 9 are cooperating with your techniques and findings. The user who's upscaling FF9 backgrounds is using his own data training model, trained specifically for FF9 and artwork which is similar. And the results are muuuch better than ESRGAN with Manga dataset.
Click to expand...
I'll try is model as soon as he's release it.
But ff7 is realy different from ff9 because of the color palette is realy restrained in ff7, so ff7 quality base is lower than ff9 (and the quality is not the same on all ff7 bakground too...).
I can't train my own model because there's not realy much artwork of ff7 to train a model with.
I don't use manga109 only, i use it with a mix of esrgan, and some waifu prefilter, and layering up to 4 different filters method on a single background to alway use the best render possible.
The filter is a big part of the works but the layer cutting is another big part of the job, that's why i take long time (started this project about 3 month ago)
 
Can't you use the current remade high Res art from team avalanche jusete and jmp to train the tool? Btw the PSX has a different colour output. As far as I know the colour variation is higher in the middle while it's lower in the black and white colour space. We have a post shader (ported by sunwalker) which does correct this and the colours do blend much better together.
 
Can't you use the current remade high Res art from team avalanche jusete and jmp to train the tool? Btw the PSX has a different colour output. As far as I know the colour variation is higher in the middle while it's lower in the black and white colour space. We have a post shader (ported by sunwalker) which does correct this and the colours do blend much better together.
Click to expand...
that's the first thing i tried but the results where realy bad T_T
 
But ff7 is realy different from ff9 because of the color palette is realy restrained in ff7, so ff7 quality base is lower than ff9 (and the quality is not the same on all ff7 bakground too...).
Click to expand...
Sounds like you're really meticulous in your work, I like it! I've noticed FF8 has these color palette issues as well, my guess is that both FF7 and 8 have now bit color palettes, compared to FF9. But these are still looking as good as it gets!
 
Can't you use the current remade high Res art from team avalanche jusete and jmp to train the tool? Btw the PSX has a different colour output. As far as I know the colour variation is higher in the middle while it's lower in the black and white colour space. We have a post shader (ported by sunwalker) which does correct this and the colours do blend much better together.
Click to expand...
Maybe i can try another method.
I'll try to train a model HR art as high quality picture and instead of simply do a 25% reduction for the low res picture, i'll try ot use a color reducted + 25% reduction, maybe more logic as ff7 pictures are resolution and colors restrained.
Got some issue to train the model, but i'll try as soon as possible.
 
Can't you use the current remade high Res art from team avalanche jusete and jmp to train the tool?
Click to expand...
Good training data for a tool like this is data that is quite similar to the data you'll be scaling. The reason the FF9 stuff works so well is because the high-res images are literally the source images for the backgrounds. As such, ESRGAN can understand exactly how those images were scaled, complete with dithering, limited palettes, etc., and thus better understand how to undo that.

Team Avalanche backgrounds, though gorgeous, are often fairly heavily reimagined. Things that might seem innocuous artistically make a big difference on the technical side of things. A different wood texture here, a different cloth material there, reworked lighting... If you downscale those, you don't get the original in-game backgrounds. And ESRGAN doesn't truly understand concepts like "wood", "cloth", etc. It is looking at frequent patterns, color variations, etc. And those would be substantially different on the TA stuff.

So you could use TA backgrounds for both the high- and low-res training images and create potentially okay but not really helpful training data with similar problems to the Manga109 training (albeit in the opposite direction), or you could use the TA backgrounds for high-res and the original in-game backgrounds for low-res training images, and get awful useless training materials that aren't grounded in how scaling actually works. Neither is really desirable.
 
Good training data for a tool like this is data that is quite similar to the data you'll be scaling. The reason the FF9 stuff works so well is because the high-res images are literally the source images for the backgrounds. As such, ESRGAN can understand exactly how those images were scaled, complete with dithering, limited palettes, etc., and thus better understand how to undo that.

Team Avalanche backgrounds, though gorgeous, are often fairly heavily reimagined. Things that might seem innocuous artistically make a big difference on the technical side of things. A different wood texture here, a different cloth material there, reworked lighting... If you downscale those, you don't get the original in-game backgrounds. And ESRGAN doesn't truly understand concepts like "wood", "cloth", etc. It is looking at frequent patterns, color variations, etc. And those would be substantially different on the TA stuff.

So you could use TA backgrounds for both the high- and low-res training images and create potentially okay but not really helpful training data with similar problems to the Manga109 training (albeit in the opposite direction), or you could use the TA backgrounds for high-res and the original in-game backgrounds for low-res training images, and get awful useless training materials that aren't grounded in how scaling actually works. Neither is really desirable.
Click to expand...
Of course this is exact, that's why my first trained model was no use, but i'll try with a reducted color pallette model training as most of the upscale problems with ff7 is due to reducted colors in pictures.
Don't think it would be good.... but you know, why not test it ^^
 
Figured I'd post an update on that vapoursynth video/batch image tool here.

Though I'm not doing much original work in the first place, I'm almost done. I've made picture tutorials for installing CUDA and cuDNN from Nvidia's website, I've made a .bat file that installs and checks MXNet, I've made a .bat that downloads or updates the Super Resolution Zoo repo from Github, and I've made a python file that grabs all the neural net info files.

I just need to write some code in that python file that parses the NN info files and automatically inserts the correct syntax into a Vapoursynth script. Basically, without the script, you'd have to manually enter the path and correct arguments for the super resolution model you want, like:
Code: [Select]
Code: 
sr_args = dict(model_filename=r'D:\CustomPrograms\VapourSynth64Portable\Models\Super-Resolution-Zoo\MSRN\MSRN_4x', device_id=0, up_scale=4, is_rgb_model=True, pad=None, crop=None, pre_upscale=False)
Which is tedious if you're trying to decide between 30+ different (pretrained) models. Anyway, I have plenty of time to finish that this weekend.
 
Last edited:
Model training on its way (thanks to Ze_PilOt who help me understand why my computer wasn't processed the model).
Will take about 5 to 10 days to train it (as i'll stop it from time to time to process ff7 fields with my current method).
Let's wait and see ...
 
Model training on its way (thanks to Ze_PilOt who help me understand why my computer wasn't processed the model).
Will take about 5 to 10 days to train it (as i'll stop it from time to time to process ff7 fields with my current method).
Let's wait and see ...
Click to expand...
neato - good luck! :)

How do you go about doing something like that anyway? Do you have any good resources to link me to? I've been digging into neural networking ever since I heard about ESRGAN myself, and actually just finished coding my very own extremely basic neural network for the first time in python within the last hour (the very basics, not even using any libraries, but I understand most of it for the most part at least). But I have no idea how to even begin training a model for specific images like you're doing now and would love to know how. I'm more interested in how this can be used in different ways tbh, like adding this kind of code to a game bot (npc, whatever) but would love to play with the graphics end as well.
 
Last edited:
It's really great to see so much sharing and carrying in the community.
 
Status
Not open for further replies.
Back
Top