Comprehensive List of Natively Supported Compression Formats

[SkyHighGam3r]

(To be updated with CHD support info)

Hey guys,

So I know a few months back we had a great discussion about various compression methods. I've made a lot of progress and changes on my personal build since then, and I now feel like it's a good time to revisit the subject; and hopefully provide some kind of solid template for the 'best' formats for myself as well as others.

I'm trying to find formats that are natively supported by emulators. The standard zip formats are great (7z/Zip/Rar5, etc.), but they also require unzipping; which takes time and local hard drive space (Albeit, temporarily). This can be handled with various methods like scripts, but for the sake of simplicity, I want to focus on formats that load up natively in a given systems emulator.
 
Here is a list of what we have discussed and compiled thus far:

Playstation
Compressed Format: PBP
Original Format: BIN/CUE (single-track)
Program: PSX2PSP (v1.3)/(v1.4.2) <<<Version needs clarification
Supported by: RetroArch's PSX Mednafen Core, ePSXe, and PCSX-R
 
Notes:

• PBP Playstation games are mostly playable on soft-modded PSP hardware. However, results vary based on the game.

Important:
In order to convert PSX games to the PBP format, your game disc must bin a single track bin/cue (only one.bin file). In order to convert a multi bin-track image to single-track, there are a couple methods available to you:
 

• Mount/Rip
  Usining a program like Daemon Tools, mount the disc's cue file.
  With the disc mounted, you can then use IMGburn to 'rip' the disc into a single bin/cue.
  You can also take this opportunity to re-name your BIN/CUE when you rip the disc and it will automatically make the change inside the CUE file.

• Batch Convert
  You can use Iggy's batch file: http://www.hyperspin-fe.com/index.php?app=core&module=attach&section=attach&attach_id=29733
  This uses 7zip and ISObuster in order to function, so you will need those as well.
  You will need to compress the ISO into a 7z file before dragging it to the batch file for conversion.
  Once that's been done you must decompress it again.
  Then you can compress the BIN/Cue files to the PBP format.

 
Playstation Portable
Compressed Format: CSO
Original Format: ISO
Program: MaxCSO
Supported by: PPSSPP & PPSSPP (RetroArch)
 
Playstation 2
Compressed Format: GZ
Original Format: ISO
Program: 7zip
Supported by: PCSX2
 
Compressed Format: CSO
Original Format: ISO
Program: MaxCSO
Supported by: PCSX2
 
Notes:

• When opening a GZ file for the first time, PCSX2 will take up to a minute in order to create a gz.tmp file in the same directory. This is an index file that PCSX2 will read to access the GZ file. Everytime you open the game after that, it will open as normal.
• GZ and CSO both appear to have the same rate of compression.
• CSO will open just the same as a normal ISO.

Gamecube
Compressed Format: GCZ
Original Format: ISO
Program: Dolphin
Supported by: Dolphin
 
Wii
Compressed Format: WBFS
Original Format: ISO
Program: Wii Backup Manager
Supported by: Dolphin & Hardware (via USB Loader)

 

Compressed Format: GCZ
Original Format: ISO
Program: Dolphin
Supported by: Dolphin
 
Compressed Format: CSO
Original Format: ISO
Program: MaxCSO
Supported by: Dolphin (Requires Confirmation)

Notes:

• WBFS files are supported by various USB loader programs on 'softmodded' Wii hardware.
• GCZ files are not playable on Wii hardware.
• When compressing DL-DVD games, GCZ is recommended over WBFS.

Dreamcast
Compressed Format: CHD
Original Format: GDI
Program: GDI to CHD
Supported by: NullDC, Demul, & Reicast (RetroArch Core)

Notes:

• GDI files are direct 1:1 copies of a GD-Rom game. (The format Sega used for Dreamcast games)
• CDI files are GDI files that have been converted to fit on a standard 700mb CD-R. If a game's size was larger than 700mb (up to 1.2Gb) then files such as audio or video are downsampled. (or ripped alltogether)
• Hardware will only play CDI files burned onto CD-Rs
• Emulators will play both GDI and CDI files
• CDI files are not compatible with CHD compression.

 

GBA/NDS/3DS
Compressed Format: Same as Original
Original Format: GBA/NDS/3DS
Program: NDSTokyoTrim
Supported by: As it's the same extension, it's believed this is supported by all emulators.
(Confirmed to be working on NDS flash carts)

 

 

(As well as anything with "???" next to it)
Here are the ones needed to fill the current holes listed above:

Playstation - RetroArch's Mednafen PSX Core
.PCE  - Possible compression format?

Wii - Dolphin
Can Dolphin natively run CSO?

Dreamcast - NullDC/Reicast
CDI Games - Any compression possible?
.ELF - Possible compression format?
 

I'm looking to make this as comprehensive as possible, so please, any input you have I would love to hear.
Is there another format you are aware of? Do you think any systems should be added?

Just remember the idea is to focus on natively supported compression formats. Rather than using RocketLauncher/7zip to unzip everything. Let's see if we can make this thread worth a sticky

[Aorin]

 Do you have a link to that batch file Aorin? I did a quick search but could not find anything relevant.

 

 

So the .M3U file is basically just a text file that simply has a list of disc files.

For example I would set one up in notepad with this as the filename.extension:

"Resident Evil 2 (2 Discs).m3u"

and inside it would simply be:

Resident Evil 2 (Disc 1) - Leon.cue

Resident Evil 2 (Disc 2) - Claire.cue

This gives RetroArch a disc index so you can simply switch to disc 1, 2, etc without having to navigate to the actual folder directory.

I'm not sure if any other emulators support the feature though. So like, for ePSXe, I'd make that same .M3U file but have it say:

Resident Evil 2 (Disc 1) - Leon.pbd

Resident Evil 2 (Disc 2) - Claire.pbd

Does that make sense (I know I tend to ramble)

 

Playstation

BIN/CUE compressed to ???

Playstation Portable

ISO compressed to CSO

Playstation 2

ISO compressed to GZ

Gamecube

ISO compressed to GCZ

Wii

ISO compressed to ???

Dreamcast

CDI compressed to ???

GDI compressed to ???

 

 

 

Here are the ones needed to fill the current holes listed above:

Playstation - RetroArch's Mednafen PSX Core

I heard there is a natively supported compressed format of some kind, however that information eludes me now.

Wii - Dolphin

I am aware of the WBFS format, however there is some conflicting information on this format as being 'lossy'. I've also heard there are other formats natively supported. For instance, can you use gcz with wii? or is that only for Gamecube?

Dreamcast - ???

Dreamcast is a bit harder. Personally I am somewhere between NullDC and RetroArch's Reicast core. I also know that there are some differences when compressing CDI as compared to GDI games. I'd like to cover both as I personally feel that while GDI is a much more accurate format, the CDI files are much more easily obtained. Also, I am unsure how each emulator differs on their support for these formats, as well as any compressed formats.

 

Like you, I don't like waiting for the game to decompress every time I want to play it, for CD systems, I use some of the compression you mentioned, also I use:

 

Playstation 1= PBP (PBPs are compressed iso that were meant, at first, for playing PSX games on the PSP, today, ePSXe, PCSX-R and also PCSXR Rearmed core (Retroarch for Android) supports this compression and run it like any iso.

This compression has a batch process, we have to take care of somethings before and after the batch process, multi track games, for example, need to become single track games before entering the batch process to become then a PBP file, I'm going to post now some of the screenshots of my folders for a better understanding:

 

The results of converting/compressing PS1 games to PBP, are for the better, for me it works really well, and each game is inside a single PBP file, no need for multi bins or cue files, yeah, multi track games work perfectly, like Rayman, for example.

 

Take, for instance, the Japanese game Aconcagua, this is the size of the game before I compressed it to PBP:

 

Now, look at the size of the compressed game, basically 1/3 of its full size, surely, as any other iso, the more music the game have, the less compression it will take, but Aconcagua seems to be a game with lots of data, that's why it compresses really well, it almost rivalizes with 7z in most cases, but remember, PBP files don't need to decompress like 7z, so the advantages of PBP over 7z are great.

 

This is a screenshot of EPSXe for Android, just for you to see that PBP is getting support every day, since it's also a great advantage for mobile devices, it saves a lot of space, obviously.

 

So, now that PBP file is known, I'm going to show how to use the program that also have the batch process to turn the PSX Bins into PBPs, this is the PSX2PSP program 1.3 (Never use v1.4.2 to batch process bins to PBP), you'll see why soon:

 

Check the highlighted options:

 

More options to check:

 

This is what you'll get:

 

This is what we get, because this program was intended for use in the PSP in mind, so the PBP is inside each folder, so each folder has the game name and its serial code, then inside we have an "eboot.pbp", if we rename this eboot.pbp to something like Chrono Cross (USA) (Disc 1).pbp and put it in the PSP it won't recognize the file, but well, for our use in HS and our emulators, that's what we'll do, rename each PBP file according to our database, just like we do to any other game.

 

Now if you use PSX2PSP 1.4.2 instead of 1.3, its batch process will keep only the serial code of the game, no matter what options you check in it, it will always save like this, so Don't use 1.4.2, use 1.3!

 

You probably notice that if you decide to dive in the PBP conversion, you'll have a lot to do, just like I had/have still some work ahead, but, the loading time of the 7z and the space the PSX games requires if I decompress them to Bin+Cue are a deal breaker for me, so I chose this path, and I can say I'm pretty satisfied with its results, it saved me a lot of space and I don't have to wait for the game to load, so it's up to you if you want it.

 

Now, for Dreamcast, there's a batch file that will turn your GDI games (that ones with raw, GDI, and some other files, NOT the CDI ones as CDI are actually ripped DC games to fit in a 700mb CD-ROM, so if you want the real games, they are the GDI ones with raw files, etc.) into CHD files, the same CHDs as those of MAME, this process is far easier than the PBP process for PSX, and it will also save you over 100GB compared to a uncompressed DC set, NullDC and Demul are known to run these as any iso file.

 

PS: Don't forget that multi tracks PSX games, before becoming PBP files, should be converted to single Bin+Cue files first, there's some tutorials and tools I can find for you if you're interested, I hope this explanation helped you.

 

Torden corrected me: I was talking about Aconcagua which is this game:

This one has something like Silent Hill and Resident Evil atmosphere.

 

PS:Now, this one is called Mizurna Falls, I never played it but it sems to be a great exclusive like Aconcagua above, a Japanese game for PSX, almost a Shenmue-like game before the Shenmue.

PS(+): After sometime dealing with renaming these PBPs, which I'd like to have a batch to pull data from the database and rename each PBP correctly, I'm doing this by hand, so I rename each folder (that has an eboot.pbp inside) according to the database, after I do that, I make this keyboard cheat code to rename each of the eboot.pbp files.

 

(F2-Ctrl+C-Enter-Enter-Down Arrow-F2-Ctrl+V-Enter-Backspace), after the first repetitions you'll do all this by heart, it becomes natural, I rename over 100 games in one minute using this, as I don't have any ability to create a useful batch to do so.

[hpr]

Good topic. We even have that same discussion in the German forum.

 

It would be very nice to gather all information in Post no.1.  It would be even nicer to collect all the different tools there. A one stop place so to say!

[SkyHighGam3r]

WOW I didn't expect that much info on the first reply haha, thank you!

PBP -
So this seems like a great method of compression depending on the emulator you are using. ePSXe is pretty popular, but it doesn't appear to be supported by the Mednafen-PSX core of RetroArch I'll add it up top with the program name. It doesn't seem to complicated to convert or rename. I, for one, always do this sort of thing by hand anyway. I don't trust automated processes lol.

CHD -
So this has to be a GDI file for the compression to work? That puts a damper on things. Those are much harder to come across. How big is your typical GDI, and does the CHB compress it to less than the 700MB CDI? Also, the fact it's supported by NullDC is great, I just checked and it looks like the Reicast core of RetroArch supports this as well. Depending on how the GDI>CHD compares to the CDI would be the determining factor in which one is ideal (space and ease of use)

Do you have a name for the program that converts GDI to CHD?

[Aorin]

Yeah I know how renowned is Mednafen-PSX core, I hope soon it will support PBP files as well, so I'll also migrate to this emulator/core as I use Retroarch for almost everything else, PCSXR Rearmed core (Retroarch core for Android) runs PBP files great, so I'm crossing my fingers for that.

 

GDI to CHD compression is far, far better than a CDI file, keep in mind that CDI isn't a compression by itself, CDI DC games are, in fact, ripped DC games or down sampled CD games, you can be sure something was cut or the audio or video was down sampled to fit a CD-ROM as GD-Roms, those original medias for DC had 1 GB of space, if I remember correctly, or 1.2 GB, well I'm sure that CDI games aren't in fact, compressed games.

So, with that said, CHD compressed GDI (those complete dumps) you also save you even more space than CDI ripped games, as CHD compression rivalizes to 7z (almost that, plus you don't have to wait for it to decompress before playing it)

 

I forgot to suggest one thing in my first post, talking about CHD compression for DC games reminds me that MAME has tons of CHD games, most of them don't work.

One time I decided I didn't have to update MAME frequently, actually I don't intend updating nor the emulator or its rom set, like never, all classic games run perfectly for me, but my suggestion is about those CHD games, if you want to save 200GB+, you can check in MAME itself what games (CHD games) really work or not, I'm sure most CHD games don't work, I also remember there are a lot of weird guitar or dancing games that works actually but aren't even playable with a standard controller, so lots of wasted space with non-playable games, well, it's just a suggestion though.

[SkyHighGam3r]

Yeah I know how renowned is Mednafen-PSX core, I hope soon it will support PBP files as well, so I'll also migrate to this emulator/core as I use Retroarch for almost everything else, PCSXR Rearmed core (Retroarch core for Android) runs PBP files great, so I'm crossing my fingers for that.

 

GDI to CHD compression is far, far better than a CDI file, keep in mind that CDI isn't a compression by itself, CDI DC games are, in fact, ripped DC games or down sampled CD games, you can be sure something was cut or the audio or video was down sampled to fit a CD-ROM as GD-Roms, those original medias for DC had 1 GB of space, if I remember correctly, or 1.2 GB, well I'm sure that CDI games aren't in fact, compressed games.

So, with that said, CHD compressed GDI (those complete dumps) you also save you even more space than CDI ripped games, as CHD compression rivalizes to 7z (almost that, plus you don't have to wait for it to decompress before playing it)

 

I forgot to suggest one thing in my first post, talking about CHD compression for DC games reminds me that MAME has tons of CHD games, most of them don't work.

One time I decided I didn't have to update MAME frequently, actually I don't intend updating nor the emulator or its rom set, like never, all classic games run perfectly for me, but my suggestion is about those CHD games, if you want to save 200GB+, you can check in MAME itself what games (CHD games) really work or not, I'm sure most CHD games don't work, I also remember there are a lot of weird guitar or dancing games that works actually but aren't even playable with a standard controller, so lots of wasted space with non-playable games, well, it's just a suggestion though.

Mame is a complex beast lol I try to avoid it whenever possible. My personal build doesn't have any arcade systems actually. Unless you count the Neo Geo, which I don't lol I always run it in US Console mode via the Uni-BIOS.

I've updated the first post with the information you provided in that first response.

I'd forgotten that they downsampled audio and video for the CDI formats... come to think of it I have a ton of them from when I used to burn discs that are from DCRes and other projects that totally moved things around...

Either way, it sounds like GDIs compressed to CHD are actually smaller than CDI files? This is, of course, in addition to being accurate copies.

 

[Spawk]

Good points Aorin. Specially staying away from cdi.

I have messed around with PBP and CSO quite a bit (on a real psp) but have yet to try it in HS. Not all ISO's will convert to CSO. Others will seemingly convert fine but then not load correctly. PBP wasn't without its hiccups either although I did much less testing on these. Some of the ones i created from my own images just wouldn't work (mainly multidisc). I found myself having to get them online created by someone else. There's a good possibility that that the emulator will handle these better than the real hardware though?? I would load each one you create of either of these formats into the emulator just to make sure all is well.

Sent from my SGH-I337M using Tapatalk

[Aorin]

Good points Aorin. Specially staying away from cdi.

I have messed around with PBP and CSO quite a bit (on a real psp) but have yet to try it in HS. Not all ISO's will convert to CSO. Others will seemingly convert fine but then not load correctly. PBP wasn't without its hiccups either although I did much less testing on these. Some of the ones i created from my own images just wouldn't work (mainly multidisc). I found myself having to get them online created by someone else. There's a good possibility that that the emulator will handle these better than the real hardware though?? I would load each one you create of either of these formats into the emulator just to make sure all is well.

Sent from my SGH-I337M using Tapatalk

When we talk about CSO and PBP in the PSP hardware it's different for the PC, the SD card for PSP has a limited read/write speed compared to HDD (this we already know), if you compress a GTA VCS  to CSO and play in the real PSP , the game will lag a lot since this game needs to constantly read the ISO/UMD, a compressed GTA CSO in PPSSPP on the other hand will not lag. Now eboots for PSP, yeah not all games will work, two games I remember by heart that won't work on the PSP are Bugs Bunny - Lost in Time and Jackie Chan Stant Master, if you put these two games to run on epsxe for PC or Android they'll work, so will PCSX-R for PC and PCSX-R Rearmed core for Android Retroarch.

So I'm confident that PBPs work as well as isos, but I can only see advantages of it since it compresses really well.

So, rest assure, I tested them, even multi track games work flawlessly, Rayman and Alien Trilogy, for example, they don't even need the cue file as the PBP have all the tracks inside it, they also work on the real PSP.

Just make sure you convert the multi tracks to single tracks using iggy's batch, everything will work fine in our HS setups, and you'll have a hell more of space to use for any other things you want.

[SkyHighGam3r]

Just make sure you convert the multi tracks to single tracks using iggy's batch, everything will work fine in our HS setups, and you'll have a hell more of space to use for any other things you want.

 

Using that whatsit??

Anytime I get a multi-track file I mount it via Daemon Tools and then Rip it to Bin/Cue (single track) with IMGburn.

[Spawk]

I don't think he means multi track but rather multi disc games (like ffvii and ffviii)

Sent from my SGH-I337M using Tapatalk

[Aorin]

Actually, both cases needs some attention, multi track games, the best ripped multi track games, take Alien Trilogy for instance, it will have the main bin, with a lot of another bin files that are, in fact, music tracks, the batch Iggy gave us, uses both 7z and Isobuster programs to automatically convert these Multi bin files into a single bin+cue, so the Alien Trilogy game will turn into a standalone game+tracks inside that one BIN, only after that the game should be compressed to PBP, so you'll have it working perfectly with all the tracks.

 

Now multi Disc games, PBP files can hold up to 5 Discs inside a single PBP, an example of that is Riven, in the real PSP you can easily switch from a disc to another with the built in PSX emulator, today, epsxe emulator for android recognizes a PBP that has more than one CD inside and split it according to how many CDs it has, so you'll see 5 different CDs to choose from in the epsxe Android frontend, you can also switch between discs while in game in that emulator.

For our use in Hyperspin, each CD should be a separate PBP, so a Chrono Cross game, will have Chrono Cross (USA) (Disc 1).pbp and Chrono Cross (USA) (Disc 2).pbp, so Rocket Launcher will also recognize them.

Interesting that if you try opening a multi Disc PBP in epsxe for Windows, it will prompt you to choose what disc you want to start from, isn't it amazing? Well but for our purpose, a separate PBP will work as a separate iso, so that's what matters for us.

[SkyHighGam3r]

For our use in Hyperspin, each CD should be a separate PBP, so a Chrono Cross game, will have Chrono Cross (USA) (Disc 1).pbp and Chrono Cross (USA) (Disc 2).pbp, so Rocket Launcher will also recognize them.

 

How does this affect the .M3U files for using Multi-Disc functions? Or is that only applicable for RetroArch?

[Aorin]

How does this affect the .M3U files for using Multi-Disc functions? Or is that only applicable for RetroArch?

What functions exactly are you talking about?

I know that Retroarch can switch discs while in game as well, for Example, in Monster Rancher 1 and 2, you can go to the Shrine and it will prompt you to put a random CD, so depending on an specific data on that disc, it will generate you a monster, so you go to the emulator feature to switch the disc, MR2 will read that disc, then you put the MR2 disc back to keep playing.

I'm not sure about the .M3U file though.

[scooter1974]

Just make sure you convert the multi tracks to single tracks using iggy's batch, everything will work fine in our HS setups, and you'll have a hell more of space to use for any other things you want.

 Do you have a link to that batch file Aorin? I did a quick search but could not find anything relevant.

[SkyHighGam3r]

What functions exactly are you talking about?

I know that Retroarch can switch discs while in game as well, for Example, in Monster Rancher 1 and 2, you can go to the Shrine and it will prompt you to put a random CD, so depending on an specific data on that disc, it will generate you a monster, so you go to the emulator feature to switch the disc, MR2 will read that disc, then you put the MR2 disc back to keep playing.

I'm not sure about the .M3U file though.

So the .M3U file is basically just a text file that simply has a list of disc files.

For example I would set one up in notepad with this as the filename.extension:

"Resident Evil 2 (2 Discs).m3u"

and inside it would simply be:

Resident Evil 2 (Disc 1) - Leon.cue

Resident Evil 2 (Disc 2) - Claire.cue

This gives RetroArch a disc index so you can simply switch to disc 1, 2, etc without having to navigate to the actual folder directory.

I'm not sure if any other emulators support the feature though. So like, for ePSXe, I'd make that same .M3U file but have it say:

Resident Evil 2 (Disc 1) - Leon.pbd

Resident Evil 2 (Disc 2) - Claire.pbd

Does that make sense (I know I tend to ramble)

[Aorin]

 Do you have a link to that batch file Aorin? I did a quick search but could not find anything relevant.

Ignore any other posts in this thread and go straight to Iggy's posts:

http://www.hyperspin-fe.com/topic/11560-how-do-i-merge-multiple-ps1-bin-files/

 

So the .M3U file is basically just a text file that simply has a list of disc files.

For example I would set one up in notepad with this as the filename.extension:

"Resident Evil 2 (2 Discs).m3u"

and inside it would simply be:

Resident Evil 2 (Disc 1) - Leon.cue

Resident Evil 2 (Disc 2) - Claire.cue

This gives RetroArch a disc index so you can simply switch to disc 1, 2, etc without having to navigate to the actual folder directory.

I'm not sure if any other emulators support the feature though. So like, for ePSXe, I'd make that same .M3U file but have it say:

Resident Evil 2 (Disc 1) - Leon.pbd

Resident Evil 2 (Disc 2) - Claire.pbd

Does that make sense (I know I tend to ramble)

I really never tried that, but if it's just a simple text file that tells RA where the other discs are, it could work like it does for other files, though I'd need to make sure first.

[SkyHighGam3r]

Well the main concern is if the .M3U file will work with something like ePSXe, since RetroArch won't load the PBD files.

[Aorin]

Well the main concern is if the .M3U file will work with something like ePSXe, since RetroArch won't load the PBD files.

Yeah, it's PBP BTW, but, remember that if you want to really use Mednafen PSX because it's better than any other emulators for you, don't risk doing all this huge task, I'd recommend it for those who uses Epsxe, PCSX-R (my emulator of choice for now) or on Android.

[scooter1974]

Ignore any other posts in this thread and go straight to Iggy's posts:

http://www.hyperspin-fe.com/topic/11560-how-do-i-merge-multiple-ps1-bin-files/

 

 Appreciate that. Cheers!

[SkyHighGam3r]

Yeah, it's PBP BTW, but, remember that if you want to really use Mednafen PSX because it's better than any other emulators for you, don't risk doing all this huge task, I'd recommend it for those who uses Epsxe, PCSX-R (my emulator of choice for now) or on Android.

Oh, doi lol I will have to fix that.

I thought ePSXe was comparable to Mednafen PSX? I know a lot of people swear by it.

[Aorin]

ePSXe is a great PSX emulator and fully supports PBP files, if you want to save space and follow the PBP path, know that you have some extra work to do, but I assure you'll be satisfied with it.

[ninja2bceen]

How do you compress to pbp

[Aorin]

How do you compress to pbp

Check the first page.

[ninja2bceen]

Right I see that but can you post the app used to do it?

[Aorin]

Right I see that but can you post the app used to do it?

http://downloads.pspspot.net/psp-utilities/720-psx2psp-v1-3.html

 

I encourage you guys willing to compress your PSX games to PBP to read carefully my instructions on the first page to avoid further issues/reworks.