Golden Sun Hacking Community
November 19, 2017, 12:19:25 PM *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News:
 
  Home   Forum   DC Wiki Help Search Calendar Downloads Login Register  
Pages: [1] 2   Go Down
  Print  
Author Topic: Compression Formats  (Read 4483 times)
0 Members and 1 Guest are viewing this topic.
Fox
Fox McCloud, the Hacking Doctor
Mercury Clan

Prodigy
*

Coins: 28
Offline Offline

I am: certainly not a Gallant!
Clan Position: Head Gallant
Posts: 2378

« on: February 20, 2015, 10:29:34 PM »

Unlike some games that have only one or two compression formats, Golden Sun has a bunch of them.

Format 0 = - Used by World Map Tilesets, Regular Map Palettes, Map Code
My bit code table has 1 bit!
0 = Seems to match the distance-length stuff described in Format 2.
1 = 8-bit constant

Format 1 = Distance-length (LZ77 variant?) - Used by World Map Palettes, and maybe other various things.

8 flags for eight entries.
If 0: 8-bit Constant
If 1: Distance length. (aaaa.aaaa.bbbb.cccc = b is most significant bits of distance, a is least significant bits of distance, and c is length.)
If length and distance are both 0: End.
If length is 0, but distance is not, get another byte for length from compressed stream and add 0x10 to it. (Distance-length extended.)


Format 2 = Complex format... (See code at 0x08012B70) - Used by Regular Map Tilesets

The first part is distance-length data. (The bit furthest to the right is 0.) ; The number I put at the right is the length.  See next section for distance stuff.

---.----.----.--00 = 2
---.----.----.-010 = 3
---.----.----.0110 = 4
---.----.---0.1110 = 5
---.----.-001.1110 = 6
---.----.-101.1110 = 7
---.----.xx11.1110 = 8-10
xxx.xxxx.0011.1110 = 11-137
000.0000.0011.1110 = End



Each of these above (except for "End") have distance data attached to the left end as well. (Not shown in the bit code above, though. Since it might make it look more messy?)

a.aaaa.aaaa.aaa0 = a represents distance + 0x20. (The number of bits (a's) can vary here... likely depending on stream length, but it stops at a max of 12-bits.)
-.----.--aa.aaa1 = a represents distance (Always 5-bits.)


FEDCBA9876543210 part: (See Icon decompressor for the basics.)
x = For left half of byte.
y = For right half of byte. (Should be done first.)

---.--xx.yy11 = &3 ; Accesses the first 4 slots.
--x.xxyy.y101 = &7 ; Accesses the first 8 slots.
xxx.xyyy.y001 = &F ; Accesses all slots.

.

Icon decompressor =  This is a simple format that uses the FEDCBA9876543210 list like Format 2 does. When you get a number from this list, that number is moved to slot 0, and the rest of the numbers are pushed over.

Bytes can only be decompressed to 00-0F with this format.

The following table I made is formatted like this:
Binary code on left, The bits are varied in length from 1-10 depending on slot number, so that the more freqently accessed used numbers which are put in the first slots take fewer bits to reference.
Address for the code that takes the value from the value list, and puts it at the head of the list.
Slot number at end.

--.----.---0 = 08038B5C = 0
--.----.-001 = 08038d74 = 1
--.----.0101 = 08038d4c = 2
--.----.1101 = 08038d60 = 3
--.----.0011 = 08038d18 = 4
--.----.1011 = 08038d04 = 5
--.----.0111 = 08038bb4 = 6
--.--00.1111 = 08038be8 = 7
--.--10.1111 = 08038bf4 = 8
--.--01.1111 = 08038c10 = 9
--.0011.1111 = 08038c44 = A
--.1011.1111 = 08038c58 = B
--.0111.1111 = 08038c7c = C
00.1111.1111 = 08038cb0 = D
10.1111.1111 = 08038cc4 = E
01.1111.1111 = 08038ce8 = F
11.1111.1111 = End

FEDCBA9876543210

For example, if you were getting from slot 8, since the number 8 is found at slot 8, you take that number and move it to the beginning like so:

FEDCBA9765432108

If you need the number 1, you will now need to get from slot 2... which will move the number to the head of the list, again, as so:

FEDCBA9765432081

And so forth.


Battle Background = Documented in another topic ( http://forum.goldensunhacking.net/index.php?topic=2012.msg37023#msg37023 )

This format is about add/subtracting (relative difference) from last value to get the next value. This format is meant for non-tiled battle backgrounds where each decompressed byte ranges from 0x60 to 0xDF. (128 possible values because of 128 colors in the battle background palette.)

--.----.--00 = + 0
--.----.x001 = + 1,2
--.xxxx.y010 = +/- 11-26 (0x1A in hex) ; y=sign
--.--xx.x011 = + 3-10
--.----.x101 = - 1,2
xx.xxxx.x110 = = value (Relative from 0x60, aka, beginning of background's palette.)
--.--xx.x111 = - 3-10


Palette Menu graphics compression = To be documented here...

16-bits for 16-flags.
If 0: 16-bit constant
If 1: 16-bit distance-length ; 0x0000 = End? ; 0xFFE0 = Distance ; 0x001F = Length

Text compression = Huffman variant... - Each character has a tree of characters that can come next. (This is why if letter combos don't exist in the original text data, you won't be able to compress unless you rewrite these trees.) The trees are arranged by node frequency the same way huffman is normally done. (Google to learn more.) The start of a sentence starts with the tree for char 00, and char 00 also end the sentence.

08060C30 = Pointer list containing character data...
There are only two pointers. (This is one entry.) More entries may be supported, though.
Second pointer list the relative location from the first pointer... Basically each relative pointer is for each character that you last decompressed... And brings you to where the list of characters that can come after (12-bit table which appears before the calculated address.) The calculated address points to the character's tree. If you count all the ones (in binary), it should ammount to the same number of characters in the character's character list.
The characters in the tree are ordered by frequency of the same length of bits. (I mean the nodes have frequency values as well... not just the symbols...)  - You can find some documentation on building trees for Huffman compression.... It's quite likely the same thing.

080A9F54 = String Table ; Each entry is two pointers. Each being used for 0x100 strings at a time. First pointer is compressed text data, second pointer is length of each string. If length is 255+, putting in a 0xFF will also read the next byte and add to the 255.... so no 255 char limit here. Although, there might be a ~512 limit in the parser function, though.)

Demonstration:  Let us compressed the string "Teawater"

Char 0x00's list has just a "T" because this is a single line. (The only time a char can come after char 0x00.)
Char "T" 's list has an "e"... (frequency 2, but since there's not other letters to compare it to, amounts the same as above.)
Char "e" 's list has a "a" and an "r" .... equal frequency. (Forces the "text data" to now need one bit of data.)
... etc. Maybe I fix this up better a bit later / dunno, though.



Sprite Compression Formats:
Type 0 = No compression??
Type 1 = Uses Main Format 0/2.
Type 2 = ?
Type 3 = LZ77 variant - Looks like the sections are divided up. (Bytes used in distance length / Constants / Flags and Distance Length pairs. ; If value at base address is 0000, read as though it is uncompressed. ; For the compression, distance-length pair should be 0000 to end. ; Distance length as a 16-bit: xyyy ; x=Length, y=Distance
« Last Edit: June 24, 2017, 03:13:57 PM by Fox » Logged

Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...
*Cross my fingers and hope we have Big Brother All-Stars next year... with Paulie & Cody Califorie(sp?)... and Vanessa & Tiffany Rousso.... Would be fun? :P Especially if one of the boys got evicted early since there's usually some all-guy alliance in there, right?
View Profile
Misery
Bad Luck

Great Member
***

Coins: 2
Offline Offline

Gender: Male
Clan Position: Mercury Hack Leader
Posts: 709

« Reply #1 on: February 21, 2015, 03:01:55 AM »

Wow, I had completely forgotten you had cracked the battle background decompression already. I think it's really neat that you're looking into and documenting these formats.

Icon decompressor =  This is a simple format that uses the FEDCBA9876543210 list like Format 2 does. When you get a number from this list, that number is moved to slot 0, and the rest of the numbers are pushed over.

Bytes can only be decompressed to 00-0F with this format.
So this is used for things like equipment and psynergy icons?

Quote
Binary code on left, The bits are varied in length from 1-10 depending on slot number, so that the more currently accessed used numbers which are put in the first slots take fewer bits to reference.
I assume you mean frequently?
So each slot holds a palette index number from 0-F of the global palette?

How is the slot order determined when there's an equal amount of pixels of different colors?

When is the end "bit" (%11.1111.1111) used? Is it at the end of each tile? Or the end of a full table of graphics data?

Sorry if these questions are stupid/annoying. If I can fully understand how this decompression works, I'd like to try writing a tool for converting other images into this format. Possibly even a full icon editor, but I'm not sure if I'd be able to do that.
Logged
View Profile
Fox
Fox McCloud, the Hacking Doctor
Mercury Clan

Prodigy
*

Coins: 28
Offline Offline

I am: certainly not a Gallant!
Clan Position: Head Gallant
Posts: 2378

« Reply #2 on: February 21, 2015, 03:25:18 AM »

Quote
So this is used for things like equipment and psynergy icons?
I recorded this function as being for icons a long time ago, but I only recently figured the format out not long ago. I think it may apply to Item, Psynergy, and Status icons? (Basically 16x16 icons) I will need to check this out again, though, as well as to see if anything else uses this.

Quote
I assume you mean frequently?
Yep, frequently sounds like the right word. (Or is "recently" a better word?)

Quote
So each slot holds a palette index number from 0-F of the global palette?
Yeah...

Quote
How is the slot order determined when there's an equal amount of pixels of different colors?
I'm a little confused at what you mean?
At the start of the stream, it should always start with this order: FEDCBA9876543210 regardless of the data itself. For every entry that gets decompressed, the value moves to the head (slot 0) and pushes the rest down..... (That way, if you need the same color/palette index again, it just takes one more bit to get it... ("0"))

Quote
When is the end "bit" (%11.1111.1111) used? Is it at the end of each tile? Or the end of a full table of graphics data?
The latter.  "End" goes at the end of the compressed file to say that there is no more data to be decompressed.



Format 2 may follow a similar concept to the icon decompressor, but I think it assigns to both sides of the byte instead of one... and the bit format is different.
« Last Edit: February 21, 2015, 05:07:16 AM by Fox » Logged

Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...
*Cross my fingers and hope we have Big Brother All-Stars next year... with Paulie & Cody Califorie(sp?)... and Vanessa & Tiffany Rousso.... Would be fun? :P Especially if one of the boys got evicted early since there's usually some all-guy alliance in there, right?
View Profile
Misery
Bad Luck

Great Member
***

Coins: 2
Offline Offline

Gender: Male
Clan Position: Mercury Hack Leader
Posts: 709

« Reply #3 on: February 21, 2015, 07:43:44 AM »

Quote
How is the slot order determined when there's an equal amount of pixels of different colors?
I'm a little confused at what you mean?
At the start of the stream, it should always start with this order: FEDCBA9876543210 regardless of the data itself. For every entry that gets decompressed, the value moves to the head (slot 0) and pushes the rest down..... (That way, if you need the same color/palette index again, it just takes one more bit to get it... ("0"))
Ah, it was just me jumping to conclusions... basically I thought the order was a constant thing based on the most-to-least commonly used index. Yeah, that's completely different from how you explained it, so just don't mind that. And I guess 'frequently' still works, but not in the sense I imagined.

So as I understand it now, every time the next entry is different from the most recent entry, that new entry gets the shortest bit length assigned to it after it has been interpreted, pushing all the others 'left' one step. Sorry if that's a confusing way to phrase it, lol. I think I get it now at least.
Logged
View Profile
Salanewt
His Sexiness
Mercury Clan

Oh yeah, baby!

Prodigy
*

Coins: 32
Offline Offline

I am: A part of the organization of Cool Cats, but more of a dog person in reality.
Emblems: Have a nice day.
Posts: 4565

« Reply #4 on: February 22, 2015, 04:34:08 PM »

Oh sweet, nice topic! I'm just going to try clarifying on a couple points for the icon decompression stuff for those who may struggle with following it though (myself included). Sprite/icon palette below for ease of clarification, though I have to be honest that your post has me a little confused in a couple areas. Should be able to figure it out after reading through it if all goes well.



Quote
--.----.---0 = 08038B5C = 0
--.----.-001 = 08038d74 = 1
--.----.0101 = 08038d4c = 2
--.----.1101 = 08038d60 = 3
--.----.0011 = 08038d18 = 4
--.----.1011 = 08038d04 = 5
--.----.0111 = 08038bb4 = 6
--.--00.1111 = 08038be8 = 7
--.--10.1111 = 08038bf4 = 8
--.--01.1111 = 08038c10 = 9
--.0011.1111 = 08038c44 = A
--.1011.1111 = 08038c58 = B
--.0111.1111 = 08038c7c = C
00.1111.1111 = 08038cb0 = D
10.1111.1111 = 08038cc4 = E
01.1111.1111 = 08038ce8 = F
11.1111.1111 = End

So basically, an icon can use any of the colours in the first row, and it reads them using binary format. Additionally, it looks like any %0s that come after the last %1 will push everything else to the right by a pixel per %0 (defaults to transparency if no colour is assigned), but I may be wrong on that part. So for example...

x3 = %11, meaning that it will be yellow (fifth colour when including transparency)
x5 = %101, meaning that it will be red (second colour)
xC = %1100, meaning that it will be yellow, and everything is moved/shifted to the right by two pixels, which basically extends the length of the previous colour (I think?)
x1F = %11111, meaning pink
x20 = %100000, meaning black, and everything is shifted by five pixels

And looking at the post again, all remaining pixels will be the previously used colour until you use another one. It looks like you're saying that one can reserve colours for later so they don't have to be recalled every time they are needed, but I'm not really sure I understand how that is done from your explanation.

I don't know, does it sound like I'm getting this right so far?
Logged

Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?
View Profile WWW
Fox
Fox McCloud, the Hacking Doctor
Mercury Clan

Prodigy
*

Coins: 28
Offline Offline

I am: certainly not a Gallant!
Clan Position: Head Gallant
Posts: 2378

« Reply #5 on: February 22, 2015, 11:05:26 PM »

Oh, I see, you're thinking about it directly from the colors point of view.  Sounds like a good way to put it for new people. (At least, before going into the bit structure and stuff. But that stuff also helps for the technical people.)

Once you use a color, it is put at the head  (slot 0)  of the 0x64 bit number (The number is really divided into two 32-bits) ... and everything else is pushed over to fill the gap that was opened. = This is the basis of that compression, since slots at the head take fewer bits to reference then at the tail.  And my guess is that if you're using one color, the chance that you'll use it again, even for just the very next pixel is probably more likely than other colors? (Well, for horizontal lines, and fillers.)
Pretty much, I just did it to explain that the 0xFEDCBA9876543210 holds indexes to palette, and the values there are rearranged. In the compression, holds variable length bit codes that are used for getting a slot in this number. (Slot 0x0 is the right side, and slot 0xF is the left side... in this 0xFEDCBA9876543210 number.)

Quote
Additionally, it looks like any %0s that come after the last %1 will push everything else to the right by a pixel per %0 (defaults to transparency if no colour is assigned), but I may be wrong on that part. So for example...
Not sure I understand this? ; Either way, you'd get the slot for where the transparent pixel is.

Quote
x3 = %11, meaning that it will be yellow (fifth colour when including transparency)
x5 = %101, meaning that it will be red (second colour)
xC = %1100, meaning that it will be yellow, and everything is moved/shifted to the right by two pixels, which basically extends the length of the previous colour (I think?)
x1F = %11111, meaning pink
x20 = %100000, meaning black, and everything is shifted by five pixels
1: Correct, but you still need the 0s attached: %0011, otherwise, it could be more confusing. Think of it like (val & 0xF = 0x3), (Although, the asm code itself shifts one bit at a time, and does conditional branching each bit that get shifted out.)
2: If this is the first grab, correct, if this is the second grab in your current list, no. (The fifth color would have moved to the first slot, and the rest pushed down to fill the gap.)
3: Wait? 1 color grab, and shift by 2 pixels? I'm not sure I follow? %1100 looks like three colors, though. Slot 0, Slot 0, and Slot 4 if you add on the 0s to the left.
5: It just looks like you are putting several bit codes into one....

Quote
It looks like you're saying that one can reserve colours for later so they don't have to be recalled every time they are needed, but I'm not really sure I understand how that is done from your explanation.
How do you mean? All that happens when you get a color, is the color moves to the head of the (64-bit), and the rest get pushed down. (Both to make room for this color being at the head, but also to fill the gap where this color was previously....... and this happens with all bit codes.)

On the topic of using the % thing, I guess we use these?:
%, b = bits
0x, h = Hex
0o = Octal

---

Even though I know this compression format ~99%(Because I haven't looked into compressing, yet, just decompressing.), it's probably hard to explain it in a user-friendly fashion, so sorry for any inconveniences.
« Last Edit: February 22, 2015, 11:27:10 PM by Fox » Logged

Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...
*Cross my fingers and hope we have Big Brother All-Stars next year... with Paulie & Cody Califorie(sp?)... and Vanessa & Tiffany Rousso.... Would be fun? :P Especially if one of the boys got evicted early since there's usually some all-guy alliance in there, right?
View Profile
Salanewt
His Sexiness
Mercury Clan

Oh yeah, baby!

Prodigy
*

Coins: 32
Offline Offline

I am: A part of the organization of Cool Cats, but more of a dog person in reality.
Emblems: Have a nice day.
Posts: 4565

« Reply #6 on: February 22, 2015, 11:48:59 PM »

Aaah, okay. So the colours get rearranged as you use them?

This makes me wonder something though. How should I view the code if I want to try my hand at making one manually, 8-bit? I'm still not sure I understand how colours are shifted to slots though, and it kind of looks like 4-bit didn't work too well for me either. All I know is that I would certainly like to try this out for a bit at some point, and that I think I may be starting to get it.
Logged

Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?
View Profile WWW
Fox
Fox McCloud, the Hacking Doctor
Mercury Clan

Prodigy
*

Coins: 28
Offline Offline

I am: certainly not a Gallant!
Clan Position: Head Gallant
Posts: 2378

« Reply #7 on: February 23, 2015, 12:13:15 AM »

Quote
Aaah, okay. So the colours get rearranged as you use them?
Yes. The 64-bit value represents all 16 colors.

I was hoping that this part of my first post would clear up any confusion left behind:

Quote
FEDCBA9876543210

For example, if you were getting from slot 8, since the number 8 is found at slot 8, you take that number and move it to the beginning like so:

FEDCBA9765432108

If you need the number 1, you will now need to get from slot 2... which will move the number to the head of the list, again, as so:

FEDCBA9765432081

And so forth.
That's where I show how they're effectively moved around. And I also mentioned the 0-F in the bit code table was for slot/position number in that 64-bit number.... by this quote, anyway
Quote
Slot number at end.

---

@8-bit: Like Format 2?
« Last Edit: February 23, 2015, 12:16:29 AM by Fox » Logged

Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...
*Cross my fingers and hope we have Big Brother All-Stars next year... with Paulie & Cody Califorie(sp?)... and Vanessa & Tiffany Rousso.... Would be fun? :P Especially if one of the boys got evicted early since there's usually some all-guy alliance in there, right?
View Profile
Salanewt
His Sexiness
Mercury Clan

Oh yeah, baby!

Prodigy
*

Coins: 32
Offline Offline

I am: A part of the organization of Cool Cats, but more of a dog person in reality.
Emblems: Have a nice day.
Posts: 4565

« Reply #8 on: February 23, 2015, 12:48:47 AM »

Ah, okay! Yeah, I just wasn't sure because the only time you actually used either the word "color" or "palette" was in the battle background portion, and you just referred to them as slots before then. It wasn't really clear whether you were talking about that or something else, though the x00-x0F reference and Misery asking about the global palette did clear that up for me. Unfortunately, that quote just made things a tad more confusing because it didn't really specify how the colours are referenced, just how they are rearranged. It was also a bit confusing with the use of "frequently/currently," since neither of those really imply that they are arranged as you reference them. All that said, I feel rather confident that I understand it better now.

8-bit: Oh, so that's how I read it? Sweet, thanks!
Logged

Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?
View Profile WWW
Fox
Fox McCloud, the Hacking Doctor
Mercury Clan

Prodigy
*

Coins: 28
Offline Offline

I am: certainly not a Gallant!
Clan Position: Head Gallant
Posts: 2378

« Reply #9 on: February 23, 2015, 01:04:20 AM »

Quote
Ah, okay! Yeah, I just wasn't sure because the only time you actually used either the word "color" or "palette" was in the battle background portion, and you just referred to them as slots before then. It wasn't really clear whether you were talking about that or something else, though the x00-x0F reference and Misery asking about the global palette did clear that up for me. Unfortunately, that quote just made things a tad more confusing because it didn't really specify how the colours are referenced, just how they are rearranged. It was also a bit confusing with the use of "frequently/currently," since neither of those really imply that they are arranged as you reference them. All that said, I feel rather confident that I understand it better now.
Probably should be revised/rewritten?
Well, frequent in the sense that if you keep moving something to the top of the list, it may be rarely put at the tail. Either way, do you think "recently" works best?

Quote
8-bit: Oh, so that's how I read it? Sweet, thanks!
I was assuming maybe you wanted one like this that decompresses to values of 00-FF. (Although, Format 2 splits it up, still... into the left and right sides.)
I probably misunderstood, though...
Logged

Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...
*Cross my fingers and hope we have Big Brother All-Stars next year... with Paulie & Cody Califorie(sp?)... and Vanessa & Tiffany Rousso.... Would be fun? :P Especially if one of the boys got evicted early since there's usually some all-guy alliance in there, right?
View Profile
Salanewt
His Sexiness
Mercury Clan

Oh yeah, baby!

Prodigy
*

Coins: 32
Offline Offline

I am: A part of the organization of Cool Cats, but more of a dog person in reality.
Emblems: Have a nice day.
Posts: 4565

« Reply #10 on: February 23, 2015, 04:14:03 AM »

Perhaps, yeah. I would suggest changing this:

Quote
so that the more freqently accessed used numbers which are put in the first slots take fewer bits to reference.

Into something more like this:

Quote
rearranging the colours in a way that makes them easier to reference later in the code.

Or something like that. That way you could perhaps clarify on how this is done later in that section. Just an idea though, since I think I understand what the whole section means now.


Ah, that's what you meant! What I am hoping to attempt is to make a few custom icons for some of the spells I am incorporating into Disco Sun, as I figure that I could potentially do this now so I can get it out of the way and not have to worry about spell icons for a while. While I understand that doing this manually could be a painful experience, I am definitely willing to put the effort in; besides, I generally do assembly without assemblers or anything like that.
Logged

Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?
View Profile WWW
Fox
Fox McCloud, the Hacking Doctor
Mercury Clan

Prodigy
*

Coins: 28
Offline Offline

I am: certainly not a Gallant!
Clan Position: Head Gallant
Posts: 2378

« Reply #11 on: February 23, 2015, 06:29:41 AM »

Quote
rearranging the colours in a way that makes them easier to reference later in the code.
I think that's still too vague... Since I doubt that anything needs to be made "easier" to reference.... Instead, just less space/bits to reference. ; (And some people may even object and say it's harder... since it would take humans a bit more time than uncompressed.)

Quote
While I understand that doing this manually could be a painful experience, I am definitely willing to put the effort in; besides, I generally do assembly without assemblers or anything like that.
Or I, you, or somene else could code a converter.... Waaayyy easier then coding an assembler editor in comparison.
You could also consider decompressing the icons and having them in that format. (AKA: Get rid of the icon decompressor altogether.)

@Text compression: Still studying it, but Terraria had me side-tracked....

08060C30 = Character data? - Two pointers, it's actually a list of one entry. (One entry here is two pointers.) I think the code supports more entries, but they're not needed.

The second pointer has a list of relative addresses for the first pointer... And there are two segments of data that can be found through the first pointer... for each relative address in the second pointer...
One segment comes before the address (The one after relative is calculated in), (12-bits per entry), and the other segment starts at the address. (Variable-length bits...)
I think the first segment is a char list? ....
Is the second segment some type of tree?  I have a feeling it is! (Tree dimensions? And the characters get assigned to them in order?)
Might need to confirm this stuff further, though...

@Second segment
My current theory is that the tree goes as deep as how many bits of 0 you have until you reach a 1... (And repeats for the tree's branches.) ... And in the text data, you put how deep you want to go, as well into each branch... and so forth. Until you get to the end of a branch, which would be your letter.

Are there as many bits that are "1" as there are number of characters branching from a character? Looks like it! ... So 0 = branch, 1 = End of branch.....

I may have to actually use one of the character's tree data in an example when the tut comes? (I'm guessing I /might not/ need to use an text example string from GS, though?) - It'd be quite a lot of work to manually compress your own text strings, though, if you have to go and look up all the trees for each character you use. -- On second thought, the first character (Non-character?/Starter tree when you decompress the first letter of a string???) has 82 possible character connections.... so that might be a bit much. I could maybe consider Char 0x12? Or not, since it seems to have only one character in its list...?
« Last Edit: February 23, 2015, 01:10:03 PM by Fox » Logged

Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...
*Cross my fingers and hope we have Big Brother All-Stars next year... with Paulie & Cody Califorie(sp?)... and Vanessa & Tiffany Rousso.... Would be fun? :P Especially if one of the boys got evicted early since there's usually some all-guy alliance in there, right?
View Profile

Novice Member
*

Coins: 0
Offline Offline

Posts: 15

« Reply #12 on: November 14, 2015, 06:25:31 PM »

Hello, everyone. Being interested in compression theme for GS, I've made some tools for graphics packing/unpacking:
What Fox calls Format 1 and Format 2 (unpacker only)

Could someone please update me if graphics compression problem is solved already and there's no need in such tools in community?
Logged
View Profile
Fox
Fox McCloud, the Hacking Doctor
Mercury Clan

Prodigy
*

Coins: 28
Offline Offline

I am: certainly not a Gallant!
Clan Position: Head Gallant
Posts: 2378

« Reply #13 on: November 14, 2015, 11:44:09 PM »

Good... Did I forget to link this to pastebin? You can find some decompression routines I wrote here... http://pastebin.com/pzhYZu7G (C#) (I forget if I made fixes to compress16 for the Icon Compressor since then, but I'll need to double check.), but you can also find some stuff in Atrius's editor source.

Porting to Python and other languages could be interesting as well.. but hmm..


--

Huh... Seems I'm still not familiar with Haskel... but haven't taken a lot of effort to learn it either. - At least byte-level code is understandable to me. (Whether it be VB, C#, GML, etc.) :) (I feel like it was a stupid mistake a long time ago when I chose to learn VB.NET... there are so many other better languages for my taste... maybe even Haskel that I could have went for... But either way, at least I chose a language at all.)
« Last Edit: November 15, 2015, 06:47:49 AM by Fox » Logged

Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...
*Cross my fingers and hope we have Big Brother All-Stars next year... with Paulie & Cody Califorie(sp?)... and Vanessa & Tiffany Rousso.... Would be fun? :P Especially if one of the boys got evicted early since there's usually some all-guy alliance in there, right?
View Profile

Novice Member
*

Coins: 0
Offline Offline

Posts: 15

« Reply #14 on: November 15, 2015, 10:30:03 AM »

You did OK with .NET platform decision. Haskell has huge issues with GUI applications development and it's not popular, so a small amount of people will understand the code. But it's clean code and rapid development loop is fine when I need to write some small CLI tool for my purposes.
So, bottom line, both compression and decompression for text and graphics are supported by Atrius's editor and there's no need for separate tool?
Could you also update me where can I find GS ROM mapping for more compressed graphics offsets?
Logged
View Profile
Pages: [1] 2   Go Up
  Print  
 
Jump to:  

Cbox
November 16, 2017, 09:44:44 AM
Fox: Very quiet month.
November 11, 2017, 04:16:27 AM
tino_animate: Hello everyone. Ive just recently replayed Golden sun and TLA. I got really inspired and I'm interested in creating a prequel of Golden Sun. Unfurtunatly i know bery little on how to do it bit im willing to learn how. I need a few people to help me. I'm sure we can create a very well story based on what we know of the games. Anyone interested?
October 16, 2017, 09:57:26 AM
AZVRE: Im pretty okay, what about you?
October 16, 2017, 01:17:14 AM
Fox: I think the nine is doing a cirno on me, though... Anyway, how is everyone doing today?
October 16, 2017, 01:12:20 AM
Fox: onesixtwo*
October 16, 2017, 01:12:00 AM
Fox: Okay... Meanwhile... ah... 162 being a number that has made itself known to me.... I look at the relationships of the numbers.... 162  1+6+2 = 9 ; onesizetwo = alphabetical order = 9 characters ; Distance apart analysis: os = 4 = 1[6][2] , st = 1 = [1]6[2] , ot = 5 = [1][6]2 ; ost = original sound track?
October 15, 2017, 06:31:17 PM
AZVRE: ah okay :D i thought i caused some trouble
October 14, 2017, 07:49:16 AM
Fox: Oh i know... You were messing with yours, and the description says mine. It's kind of a funny bug. :P
October 13, 2017, 08:18:31 PM
AZVRE: I did not edit your profile tho. Just to erase that confusion ^^"
October 12, 2017, 04:49:56 AM
Fox: e.g. this art = http://forum.goldensunhacking.net/Themes/Jupiter/images/catbg.gif (etc.)
October 12, 2017, 04:48:58 AM
Fox: I have a question: Where did the art for the headers come from? (Like the backgrounds to each of these things... "Affiliates", etc... Asking just incase it inspires me to do something... but if Charon made it, then that's cool too.
October 11, 2017, 04:23:02 AM
FoxWas thinking about having something for the editor anniversary, but I believe I had to change my mind....
October 10, 2017, 12:58:36 PM
Fox: That's alright. ;       @AZVRE's profile: "Current Activity:  Editing the profile of Fox." ...Eh... That looks weird.
October 07, 2017, 11:45:16 PM
Atrius: Sorry about that, the auto bot detection is still working, but it also requires an admin to visit the site regularly.
October 04, 2017, 04:17:40 AM
Fox: Yeah, it does bring a small amount of activity. Not good quality activity, but it's still activity, nonetheless.  Now if we can get them to randomly post pictures and vdeios!
October 04, 2017, 02:10:48 AM
Majora: If nothing else, I like how they're copying the content of real posts at random. Interesting to see how they adapt to seem more authentic
October 03, 2017, 08:29:38 PM
Fox: medico6601 = Most likely another bot. :(
September 26, 2017, 03:32:24 PM
Fox: by ten* Excuse me.
September 26, 2017, 03:32:12 PM
Fox: Oh yes... Also multiply all levels by two. (So actually, an adept will have levels of 54, 3, 2, and 1, rather than 5, 0, 0, 0.... if going by those.)
September 26, 2017, 03:29:07 PM
Fox: ...I mean the theory also saves space on the Status Screen.

Affiliates
Temple of Kraden Golden Sunrise
Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!
Page created in 0.089 seconds with 22 queries.