News:

As a consequence of the forum being updated and repaired, the chatbox has been lost.
However, you can still come say hi on our Discord server!

Main Menu

Compression Formats

Started by Daddy Poi's Oily Gorillas, 20, February, 2015, 05:29:34 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Daddy Poi's Oily Gorillas

Unlike some games that have only one or two compression formats, Golden Sun has a bunch of them.

Format 0 = - Used by World Map Tilesets, Regular Map Palettes, Map Code
My bit code table has 1 bit!
0 = Seems to match the distance-length stuff described in Format 2.
1 = 8-bit constant

Format 1 = Distance-length (LZ77 variant?) - Used by World Map Palettes, and maybe other various things.

8 flags for eight entries.
If 0: 8-bit Constant
If 1: Distance length. (aaaa.aaaa.bbbb.cccc = b is most significant bits of distance, a is least significant bits of distance, and c is length.)
If length and distance are both 0: End.
If length is 0, but distance is not, get another byte for length from compressed stream and add 0x10 to it. (Distance-length extended.)


Format 2 = Complex format... (See code at 0x08012B70) - Used by Regular Map Tilesets

The first part is distance-length data. (The bit furthest to the right is 0.) ; The number I put at the right is the length.  See next section for distance stuff.

---.----.----.--00 = 2
---.----.----.-010 = 3
---.----.----.0110 = 4
---.----.---0.1110 = 5
---.----.-001.1110 = 6
---.----.-101.1110 = 7
---.----.xx11.1110 = 8-10
xxx.xxxx.0011.1110 = 11-137
000.0000.0011.1110 = End



Each of these above (except for "End") have distance data attached to the left end as well. (Not shown in the bit code above, though. Since it might make it look more messy?)

a.aaaa.aaaa.aaa0 = a represents distance + 0x20. (The number of bits (a's) can vary here... likely depending on stream length, but it stops at a max of 12-bits.)
-.----.--aa.aaa1 = a represents distance (Always 5-bits.)


FEDCBA9876543210 part: (See Icon decompressor for the basics.)
x = For left half of byte.
y = For right half of byte. (Should be done first.)

---.--xx.yy11 = &3 ; Accesses the first 4 slots.
--x.xxyy.y101 = &7 ; Accesses the first 8 slots.
xxx.xyyy.y001 = &F ; Accesses all slots.

.

Icon decompressor =  This is a simple format that uses the FEDCBA9876543210 list like Format 2 does. When you get a number from this list, that number is moved to slot 0, and the rest of the numbers are pushed over.

Bytes can only be decompressed to 00-0F with this format.

The following table I made is formatted like this:
Binary code on left, The bits are varied in length from 1-10 depending on slot number, so that the more freqently accessed used numbers which are put in the first slots take fewer bits to reference.
Address for the code that takes the value from the value list, and puts it at the head of the list.
Slot number at end.

--.----.---0 = 08038B5C = 0
--.----.-001 = 08038d74 = 1
--.----.0101 = 08038d4c = 2
--.----.1101 = 08038d60 = 3
--.----.0011 = 08038d18 = 4
--.----.1011 = 08038d04 = 5
--.----.0111 = 08038bb4 = 6
--.--00.1111 = 08038be8 = 7
--.--10.1111 = 08038bf4 = 8
--.--01.1111 = 08038c10 = 9
--.0011.1111 = 08038c44 = A
--.1011.1111 = 08038c58 = B
--.0111.1111 = 08038c7c = C
00.1111.1111 = 08038cb0 = D
10.1111.1111 = 08038cc4 = E
01.1111.1111 = 08038ce8 = F
11.1111.1111 = End

FEDCBA9876543210

For example, if you were getting from slot 8, since the number 8 is found at slot 8, you take that number and move it to the beginning like so:

FEDCBA9765432108

If you need the number 1, you will now need to get from slot 2... which will move the number to the head of the list, again, as so:

FEDCBA9765432081

And so forth.


Battle Background = Documented in another topic ( http://forum.goldensunhacking.net/index.php?topic=2012.msg37023#msg37023 )

This format is about add/subtracting (relative difference) from last value to get the next value. This format is meant for non-tiled battle backgrounds where each decompressed byte ranges from 0x60 to 0xDF. (128 possible values because of 128 colors in the battle background palette.)

--.----.--00 = + 0
--.----.x001 = + 1,2
--.xxxx.y010 = +/- 11-26 (0x1A in hex) ; y=sign
--.--xx.x011 = + 3-10
--.----.x101 = - 1,2
xx.xxxx.x110 = = value (Relative from 0x60, aka, beginning of background's palette.)
--.--xx.x111 = - 3-10


Palette Menu graphics compression = To be documented here...

16-bits for 16-flags.
If 0: 16-bit constant
If 1: 16-bit distance-length ; 0x0000 = End? ; 0xFFE0 = Distance ; 0x001F = Length

Text compression = Huffman variant... - Each character has a tree of characters that can come next. (This is why if letter combos don't exist in the original text data, you won't be able to compress unless you rewrite these trees.) The trees are arranged by node frequency the same way huffman is normally done. (Google to learn more.) The start of a sentence starts with the tree for char 00, and char 00 also end the sentence.

08060C30 = Pointer list containing character data...
There are only two pointers. (This is one entry.) More entries may be supported, though.
Second pointer list the relative location from the first pointer... Basically each relative pointer is for each character that you last decompressed... And brings you to where the list of characters that can come after (12-bit table which appears before the calculated address.) The calculated address points to the character's tree. If you count all the ones (in binary), it should ammount to the same number of characters in the character's character list.
The characters in the tree are ordered by frequency of the same length of bits. (I mean the nodes have frequency values as well... not just the symbols...)  - You can find some documentation on building trees for Huffman compression.... It's quite likely the same thing.

080A9F54 = String Table ; Each entry is two pointers. Each being used for 0x100 strings at a time. First pointer is compressed text data, second pointer is length of each string. If length is 255+, putting in a 0xFF will also read the next byte and add to the 255.... so no 255 char limit here. Although, there might be a ~512 limit in the parser function, though.)

Demonstration:  Let us compressed the string "Teawater"

Char 0x00's list has just a "T" because this is a single line. (The only time a char can come after char 0x00.)
Char "T" 's list has an "e"... (frequency 2, but since there's not other letters to compare it to, amounts the same as above.)
Char "e" 's list has a "a" and an "r" .... equal frequency. (Forces the "text data" to now need one bit of data.)
... etc. Maybe I fix this up better a bit later / dunno, though.



Sprite Compression Formats:
Type 0 = No compression??
Type 1 = Uses Main Format 0/2.
Type 2 = ?
Type 3 = LZ77 variant - Looks like the sections are divided up. (Bytes used in distance length / Constants / Flags and Distance Length pairs. ; If value at base address is 0000, read as though it is uncompressed. ; For the compression, distance-length pair should be 0000 to end. ; Distance length as a 16-bit: xyyy ; x=Length, y=Distance
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

Misery

Wow, I had completely forgotten you had cracked the battle background decompression already. I think it's really neat that you're looking into and documenting these formats.

Quote from: Fox on 20, February, 2015, 05:29:34 PM
Icon decompressor =  This is a simple format that uses the FEDCBA9876543210 list like Format 2 does. When you get a number from this list, that number is moved to slot 0, and the rest of the numbers are pushed over.

Bytes can only be decompressed to 00-0F with this format.
So this is used for things like equipment and psynergy icons?

QuoteBinary code on left, The bits are varied in length from 1-10 depending on slot number, so that the more currently accessed used numbers which are put in the first slots take fewer bits to reference.
I assume you mean frequently?
So each slot holds a palette index number from 0-F of the global palette?

How is the slot order determined when there's an equal amount of pixels of different colors?

When is the end "bit" (%11.1111.1111) used? Is it at the end of each tile? Or the end of a full table of graphics data?

Sorry if these questions are stupid/annoying. If I can fully understand how this decompression works, I'd like to try writing a tool for converting other images into this format. Possibly even a full icon editor, but I'm not sure if I'd be able to do that.

Daddy Poi's Oily Gorillas

#2
QuoteSo this is used for things like equipment and psynergy icons?
I recorded this function as being for icons a long time ago, but I only recently figured the format out not long ago. I think it may apply to Item, Psynergy, and Status icons? (Basically 16x16 icons) I will need to check this out again, though, as well as to see if anything else uses this.

QuoteI assume you mean frequently?
Yep, frequently sounds like the right word. (Or is "recently" a better word?)

QuoteSo each slot holds a palette index number from 0-F of the global palette?
Yeah...

QuoteHow is the slot order determined when there's an equal amount of pixels of different colors?
I'm a little confused at what you mean?
At the start of the stream, it should always start with this order: FEDCBA9876543210 regardless of the data itself. For every entry that gets decompressed, the value moves to the head (slot 0) and pushes the rest down..... (That way, if you need the same color/palette index again, it just takes one more bit to get it... ("0"))

Quote
When is the end "bit" (%11.1111.1111) used? Is it at the end of each tile? Or the end of a full table of graphics data?
The latter.  "End" goes at the end of the compressed file to say that there is no more data to be decompressed.




Format 2 may follow a similar concept to the icon decompressor, but I think it assigns to both sides of the byte instead of one... and the bit format is different.
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

Misery

Quote from: Fox on 20, February, 2015, 10:25:18 PM
QuoteHow is the slot order determined when there's an equal amount of pixels of different colors?
I'm a little confused at what you mean?
At the start of the stream, it should always start with this order: FEDCBA9876543210 regardless of the data itself. For every entry that gets decompressed, the value moves to the head (slot 0) and pushes the rest down..... (That way, if you need the same color/palette index again, it just takes one more bit to get it... ("0"))
Ah, it was just me jumping to conclusions... basically I thought the order was a constant thing based on the most-to-least commonly used index. Yeah, that's completely different from how you explained it, so just don't mind that. And I guess 'frequently' still works, but not in the sense I imagined.

So as I understand it now, every time the next entry is different from the most recent entry, that new entry gets the shortest bit length assigned to it after it has been interpreted, pushing all the others 'left' one step. Sorry if that's a confusing way to phrase it, lol. I think I get it now at least.

Salanewt

Oh sweet, nice topic! I'm just going to try clarifying on a couple points for the icon decompression stuff for those who may struggle with following it though (myself included). Sprite/icon palette below for ease of clarification, though I have to be honest that your post has me a little confused in a couple areas. Should be able to figure it out after reading through it if all goes well.



Quote--.----.---0 = 08038B5C = 0
--.----.-001 = 08038d74 = 1
--.----.0101 = 08038d4c = 2
--.----.1101 = 08038d60 = 3
--.----.0011 = 08038d18 = 4
--.----.1011 = 08038d04 = 5
--.----.0111 = 08038bb4 = 6
--.--00.1111 = 08038be8 = 7
--.--10.1111 = 08038bf4 = 8
--.--01.1111 = 08038c10 = 9
--.0011.1111 = 08038c44 = A
--.1011.1111 = 08038c58 = B
--.0111.1111 = 08038c7c = C
00.1111.1111 = 08038cb0 = D
10.1111.1111 = 08038cc4 = E
01.1111.1111 = 08038ce8 = F
11.1111.1111 = End

So basically, an icon can use any of the colours in the first row, and it reads them using binary format. Additionally, it looks like any %0s that come after the last %1 will push everything else to the right by a pixel per %0 (defaults to transparency if no colour is assigned), but I may be wrong on that part. So for example...

x3 = %11, meaning that it will be yellow (fifth colour when including transparency)
x5 = %101, meaning that it will be red (second colour)
xC = %1100, meaning that it will be yellow, and everything is moved/shifted to the right by two pixels, which basically extends the length of the previous colour (I think?)
x1F = %11111, meaning pink
x20 = %100000, meaning black, and everything is shifted by five pixels

And looking at the post again, all remaining pixels will be the previously used colour until you use another one. It looks like you're saying that one can reserve colours for later so they don't have to be recalled every time they are needed, but I'm not really sure I understand how that is done from your explanation.

I don't know, does it sound like I'm getting this right so far?
Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?

Daddy Poi's Oily Gorillas

#5
Oh, I see, you're thinking about it directly from the colors point of view.  Sounds like a good way to put it for new people. (At least, before going into the bit structure and stuff. But that stuff also helps for the technical people.)

Once you use a color, it is put at the head  (slot 0)  of the 0x64 bit number (The number is really divided into two 32-bits) ... and everything else is pushed over to fill the gap that was opened. = This is the basis of that compression, since slots at the head take fewer bits to reference then at the tail.  And my guess is that if you're using one color, the chance that you'll use it again, even for just the very next pixel is probably more likely than other colors? (Well, for horizontal lines, and fillers.)
Pretty much, I just did it to explain that the 0xFEDCBA9876543210 holds indexes to palette, and the values there are rearranged. In the compression, holds variable length bit codes that are used for getting a slot in this number. (Slot 0x0 is the right side, and slot 0xF is the left side... in this 0xFEDCBA9876543210 number.)

QuoteAdditionally, it looks like any %0s that come after the last %1 will push everything else to the right by a pixel per %0 (defaults to transparency if no colour is assigned), but I may be wrong on that part. So for example...
Not sure I understand this? ; Either way, you'd get the slot for where the transparent pixel is.

Quotex3 = %11, meaning that it will be yellow (fifth colour when including transparency)
x5 = %101, meaning that it will be red (second colour)
xC = %1100, meaning that it will be yellow, and everything is moved/shifted to the right by two pixels, which basically extends the length of the previous colour (I think?)
x1F = %11111, meaning pink
x20 = %100000, meaning black, and everything is shifted by five pixels
1: Correct, but you still need the 0s attached: %0011, otherwise, it could be more confusing. Think of it like (val & 0xF = 0x3), (Although, the asm code itself shifts one bit at a time, and does conditional branching each bit that get shifted out.)
2: If this is the first grab, correct, if this is the second grab in your current list, no. (The fifth color would have moved to the first slot, and the rest pushed down to fill the gap.)
3: Wait? 1 color grab, and shift by 2 pixels? I'm not sure I follow? %1100 looks like three colors, though. Slot 0, Slot 0, and Slot 4 if you add on the 0s to the left.
5: It just looks like you are putting several bit codes into one....

QuoteIt looks like you're saying that one can reserve colours for later so they don't have to be recalled every time they are needed, but I'm not really sure I understand how that is done from your explanation.
How do you mean? All that happens when you get a color, is the color moves to the head of the (64-bit), and the rest get pushed down. (Both to make room for this color being at the head, but also to fill the gap where this color was previously....... and this happens with all bit codes.)

On the topic of using the % thing, I guess we use these?:
%, b = bits
0x, h = Hex
0o = Octal

---

Even though I know this compression format ~99%(Because I haven't looked into compressing, yet, just decompressing.), it's probably hard to explain it in a user-friendly fashion, so sorry for any inconveniences.
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

Salanewt

Aaah, okay. So the colours get rearranged as you use them?

This makes me wonder something though. How should I view the code if I want to try my hand at making one manually, 8-bit? I'm still not sure I understand how colours are shifted to slots though, and it kind of looks like 4-bit didn't work too well for me either. All I know is that I would certainly like to try this out for a bit at some point, and that I think I may be starting to get it.
Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?

Daddy Poi's Oily Gorillas

#7
QuoteAaah, okay. So the colours get rearranged as you use them?
Yes. The 64-bit value represents all 16 colors.

I was hoping that this part of my first post would clear up any confusion left behind:

QuoteFEDCBA9876543210

For example, if you were getting from slot 8, since the number 8 is found at slot 8, you take that number and move it to the beginning like so:

FEDCBA9765432108

If you need the number 1, you will now need to get from slot 2... which will move the number to the head of the list, again, as so:

FEDCBA9765432081

And so forth.
That's where I show how they're effectively moved around. And I also mentioned the 0-F in the bit code table was for slot/position number in that 64-bit number.... by this quote, anyway
QuoteSlot number at end.

---

@8-bit: Like Format 2?
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

Salanewt

Ah, okay! Yeah, I just wasn't sure because the only time you actually used either the word "color" or "palette" was in the battle background portion, and you just referred to them as slots before then. It wasn't really clear whether you were talking about that or something else, though the x00-x0F reference and Misery asking about the global palette did clear that up for me. Unfortunately, that quote just made things a tad more confusing because it didn't really specify how the colours are referenced, just how they are rearranged. It was also a bit confusing with the use of "frequently/currently," since neither of those really imply that they are arranged as you reference them. All that said, I feel rather confident that I understand it better now.

8-bit: Oh, so that's how I read it? Sweet, thanks!
Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?

Daddy Poi's Oily Gorillas

QuoteAh, okay! Yeah, I just wasn't sure because the only time you actually used either the word "color" or "palette" was in the battle background portion, and you just referred to them as slots before then. It wasn't really clear whether you were talking about that or something else, though the x00-x0F reference and Misery asking about the global palette did clear that up for me. Unfortunately, that quote just made things a tad more confusing because it didn't really specify how the colours are referenced, just how they are rearranged. It was also a bit confusing with the use of "frequently/currently," since neither of those really imply that they are arranged as you reference them. All that said, I feel rather confident that I understand it better now.
Probably should be revised/rewritten?
Well, frequent in the sense that if you keep moving something to the top of the list, it may be rarely put at the tail. Either way, do you think "recently" works best?

Quote8-bit: Oh, so that's how I read it? Sweet, thanks!
I was assuming maybe you wanted one like this that decompresses to values of 00-FF. (Although, Format 2 splits it up, still... into the left and right sides.)
I probably misunderstood, though...
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

Salanewt

Perhaps, yeah. I would suggest changing this:

Quoteso that the more freqently accessed used numbers which are put in the first slots take fewer bits to reference.

Into something more like this:

Quoterearranging the colours in a way that makes them easier to reference later in the code.

Or something like that. That way you could perhaps clarify on how this is done later in that section. Just an idea though, since I think I understand what the whole section means now.


Ah, that's what you meant! What I am hoping to attempt is to make a few custom icons for some of the spells I am incorporating into Disco Sun, as I figure that I could potentially do this now so I can get it out of the way and not have to worry about spell icons for a while. While I understand that doing this manually could be a painful experience, I am definitely willing to put the effort in; besides, I generally do assembly without assemblers or anything like that.
Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?

Daddy Poi's Oily Gorillas

#11
Quoterearranging the colours in a way that makes them easier to reference later in the code.
I think that's still too vague... Since I doubt that anything needs to be made "easier" to reference.... Instead, just less space/bits to reference. ; (And some people may even object and say it's harder... since it would take humans a bit more time than uncompressed.)

QuoteWhile I understand that doing this manually could be a painful experience, I am definitely willing to put the effort in; besides, I generally do assembly without assemblers or anything like that.
Or I, you, or somene else could code a converter.... Waaayyy easier then coding an assembler editor in comparison.
You could also consider decompressing the icons and having them in that format. (AKA: Get rid of the icon decompressor altogether.)

@Text compression: Still studying it, but Terraria had me side-tracked....

08060C30 = Character data? - Two pointers, it's actually a list of one entry. (One entry here is two pointers.) I think the code supports more entries, but they're not needed.

The second pointer has a list of relative addresses for the first pointer... And there are two segments of data that can be found through the first pointer... for each relative address in the second pointer...
One segment comes before the address (The one after relative is calculated in), (12-bits per entry), and the other segment starts at the address. (Variable-length bits...)
I think the first segment is a char list? ....
Is the second segment some type of tree?  I have a feeling it is! (Tree dimensions? And the characters get assigned to them in order?)
Might need to confirm this stuff further, though...

@Second segment
My current theory is that the tree goes as deep as how many bits of 0 you have until you reach a 1... (And repeats for the tree's branches.) ... And in the text data, you put how deep you want to go, as well into each branch... and so forth. Until you get to the end of a branch, which would be your letter.

Are there as many bits that are "1" as there are number of characters branching from a character? Looks like it! ... So 0 = branch, 1 = End of branch.....

I may have to actually use one of the character's tree data in an example when the tut comes? (I'm guessing I /might not/ need to use an text example string from GS, though?) - It'd be quite a lot of work to manually compress your own text strings, though, if you have to go and look up all the trees for each character you use. -- On second thought, the first character (Non-character?/Starter tree when you decompress the first letter of a string???) has 82 possible character connections.... so that might be a bit much. I could maybe consider Char 0x12? Or not, since it seems to have only one character in its list...?
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

approxies

Hello, everyone. Being interested in compression theme for GS, I've made some tools for graphics packing/unpacking:
What Fox calls Format 1 and Format 2 (unpacker only)

Could someone please update me if graphics compression problem is solved already and there's no need in such tools in community?

Daddy Poi's Oily Gorillas

#13
Good... Did I forget to link this to pastebin? You can find some decompression routines I wrote here... http://pastebin.com/pzhYZu7G (C#) (I forget if I made fixes to compress16 for the Icon Compressor since then, but I'll need to double check.), but you can also find some stuff in Atrius's editor source.

Porting to Python and other languages could be interesting as well.. but hmm..


--

Huh... Seems I'm still not familiar with Haskel... but haven't taken a lot of effort to learn it either. - At least byte-level code is understandable to me. (Whether it be VB, C#, GML, etc.) :) (I feel like it was a stupid mistake a long time ago when I chose to learn VB.NET... there are so many other better languages for my taste... maybe even Haskel that I could have went for... But either way, at least I chose a language at all.)
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

approxies

You did OK with .NET platform decision. Haskell has huge issues with GUI applications development and it's not popular, so a small amount of people will understand the code. But it's clean code and rapid development loop is fine when I need to write some small CLI tool for my purposes.
So, bottom line, both compression and decompression for text and graphics are supported by Atrius's editor and there's no need for separate tool?
Could you also update me where can I find GS ROM mapping for more compressed graphics offsets?

Daddy Poi's Oily Gorillas

#15
Well, it didn't fully support Text Compression, as Atrius's editor never rewrote the Char tables... It also didn't decompress battle backgrounds... etc. But the point was that it did support decompressoin/compression for some things...

I think most images are in three/four sections.
1.) The sprites tble: (0x08300000 in GS2)
2.) The Master File Table.. (0x08320000 in GS1, 0x08680000 in GS2; Yep, I have both of them memorized.)
3.) One of the code files has compressed icons.
4.) Oh, and there might be some things compressed in another code file files. (Like the palette menu graphics, I think..) but other than that, I think most graphics in those files are uncompressed(?) - Not many uncompressed images, though... but the red crystal grpahic (for convos), and that pink/purple conical selection thing used to select an enemy maybe.. some |> arrows used in debug menus....  etc. That kind of thing.)
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

approxies

OK, thanks for getting me inputs, I will investigate further. I guess, pointer for the chunk of graphics, I'm going to translate (GO and 1st match glyphs in arena) is hardcoded.

approxies

An update for my set of compression tools of Golden Sun.
Now it supports decoding/encoding for text, and two LZSS schemes for graphics, described in OP post.

TenkaiSp

Hi approxies, Fox and Lord Squirtle, I am currently looking for a way to encode my Polish translation of The Lost Age in a different way than using Atrius' Editor, as it makes the output file much bigger and is now incompatible with The Balance Age  :Sweat:
Needles to say that this thread caught my interest. Approxies, would you mind making a little tutorial to your compression tools for a non-programmer? I've played around with the tools a little bit, but can't really get them to work.
The haskell one returns "Could not find module 'Data.Bitstream.Lazy' (I just installed the GHC compiler).
Recommended translation scheme from the earlier version with .bat files works fine until the encoding (-e) part, where it shows "trees: Patch size is larger than specified. Aborted." I guess it is because my translation is larger than the original file (duh), so do I need to specify a new treeSize, Offset? How do I calculate this? :)
Here is the script, if you want to take a look.

Caledor

I'm not sure about it, but i think you can tweak gstlatoolkit itself and make it store the script later into the rom . that way you'd also gain back the compatibility with my patch.