Golden Sun Hacking Community

The Editor => Golden Sun Hacking => Topic started by: Fox on February 20, 2015, 10:29:34 PM



Title: Compression Formats
Post by: Fox on February 20, 2015, 10:29:34 PM
Unlike some games that have only one or two compression formats, Golden Sun has a bunch of them.

Format 0 = - Used by World Map Tilesets, Regular Map Palettes, Map Code
My bit code table has 1 bit!
0 = Seems to match the distance-length stuff described in Format 2.
1 = 8-bit constant

Format 1 = Distance-length (LZ77 variant?) - Used by World Map Palettes, and maybe other various things.

8 flags for eight entries.
If 0: 8-bit Constant
If 1: Distance length. (aaaa.aaaa.bbbb.cccc = b is most significant bits of distance, a is least significant bits of distance, and c is length.)
If length and distance are both 0: End.
If length is 0, but distance is not, get another byte for length from compressed stream and add 0x10 to it. (Distance-length extended.)


Format 2 = Complex format... (See code at 0x08012B70) - Used by Regular Map Tilesets

The first part is distance-length data. (The bit furthest to the right is 0.) ; The number I put at the right is the length.  See next section for distance stuff.

---.----.----.--00 = 2
---.----.----.-010 = 3
---.----.----.0110 = 4
---.----.---0.1110 = 5
---.----.-001.1110 = 6
---.----.-101.1110 = 7
---.----.xx11.1110 = 8-10
xxx.xxxx.0011.1110 = 11-137
000.0000.0011.1110 = End



Each of these above (except for "End") have distance data attached to the left end as well. (Not shown in the bit code above, though. Since it might make it look more messy?)

a.aaaa.aaaa.aaa0 = a represents distance + 0x20. (The number of bits (a's) can vary here... likely depending on stream length, but it stops at a max of 12-bits.)
-.----.--aa.aaa1 = a represents distance (Always 5-bits.)


FEDCBA9876543210 part: (See Icon decompressor for the basics.)
x = For left half of byte.
y = For right half of byte. (Should be done first.)

---.--xx.yy11 = &3 ; Accesses the first 4 slots.
--x.xxyy.y101 = &7 ; Accesses the first 8 slots.
xxx.xyyy.y001 = &F ; Accesses all slots.

.

Icon decompressor =  This is a simple format that uses the FEDCBA9876543210 list like Format 2 does. When you get a number from this list, that number is moved to slot 0, and the rest of the numbers are pushed over.

Bytes can only be decompressed to 00-0F with this format.

The following table I made is formatted like this:
Binary code on left, The bits are varied in length from 1-10 depending on slot number, so that the more freqently accessed used numbers which are put in the first slots take fewer bits to reference.
Address for the code that takes the value from the value list, and puts it at the head of the list.
Slot number at end.

--.----.---0 = 08038B5C = 0
--.----.-001 = 08038d74 = 1
--.----.0101 = 08038d4c = 2
--.----.1101 = 08038d60 = 3
--.----.0011 = 08038d18 = 4
--.----.1011 = 08038d04 = 5
--.----.0111 = 08038bb4 = 6
--.--00.1111 = 08038be8 = 7
--.--10.1111 = 08038bf4 = 8
--.--01.1111 = 08038c10 = 9
--.0011.1111 = 08038c44 = A
--.1011.1111 = 08038c58 = B
--.0111.1111 = 08038c7c = C
00.1111.1111 = 08038cb0 = D
10.1111.1111 = 08038cc4 = E
01.1111.1111 = 08038ce8 = F
11.1111.1111 = End

FEDCBA9876543210

For example, if you were getting from slot 8, since the number 8 is found at slot 8, you take that number and move it to the beginning like so:

FEDCBA9765432108

If you need the number 1, you will now need to get from slot 2... which will move the number to the head of the list, again, as so:

FEDCBA9765432081

And so forth.


Battle Background = Documented in another topic ( http://forum.goldensunhacking.net/index.php?topic=2012.msg37023#msg37023 )

This format is about add/subtracting (relative difference) from last value to get the next value. This format is meant for non-tiled battle backgrounds where each decompressed byte ranges from 0x60 to 0xDF. (128 possible values because of 128 colors in the battle background palette.)

--.----.--00 = + 0
--.----.x001 = + 1,2
--.xxxx.y010 = +/- 11-26 (0x1A in hex) ; y=sign
--.--xx.x011 = + 3-10
--.----.x101 = - 1,2
xx.xxxx.x110 = = value (Relative from 0x60, aka, beginning of background's palette.)
--.--xx.x111 = - 3-10


Palette Menu graphics compression = To be documented here...

16-bits for 16-flags.
If 0: 16-bit constant
If 1: 16-bit distance-length ; 0x0000 = End? ; 0xFFE0 = Distance ; 0x001F = Length

Text compression = Huffman variant... - Each character has a tree of characters that can come next. (This is why if letter combos don't exist in the original text data, you won't be able to compress unless you rewrite these trees.) The trees are arranged by node frequency the same way huffman is normally done. (Google to learn more.) The start of a sentence starts with the tree for char 00, and char 00 also end the sentence.

08060C30 = Pointer list containing character data...
There are only two pointers. (This is one entry.) More entries may be supported, though.
Second pointer list the relative location from the first pointer... Basically each relative pointer is for each character that you last decompressed... And brings you to where the list of characters that can come after (12-bit table which appears before the calculated address.) The calculated address points to the character's tree. If you count all the ones (in binary), it should ammount to the same number of characters in the character's character list.
The characters in the tree are ordered by frequency of the same length of bits. (I mean the nodes have frequency values as well... not just the symbols...)  - You can find some documentation on building trees for Huffman compression.... It's quite likely the same thing.

080A9F54 = String Table ; Each entry is two pointers. Each being used for 0x100 strings at a time. First pointer is compressed text data, second pointer is length of each string. If length is 255+, putting in a 0xFF will also read the next byte and add to the 255.... so no 255 char limit here. Although, there might be a ~512 limit in the parser function, though.)

Demonstration:  Let us compressed the string "Teawater"

Char 0x00's list has just a "T" because this is a single line. (The only time a char can come after char 0x00.)
Char "T" 's list has an "e"... (frequency 2, but since there's not other letters to compare it to, amounts the same as above.)
Char "e" 's list has a "a" and an "r" .... equal frequency. (Forces the "text data" to now need one bit of data.)
... etc. Maybe I fix this up better a bit later / dunno, though.



Sprite Compression Formats:
Type 0 = No compression??
Type 1 = Uses Main Format 0/2.
Type 2 = ?
Type 3 = LZ77 variant - Looks like the sections are divided up. (Bytes used in distance length / Constants / Flags and Distance Length pairs. ; If value at base address is 0000, read as though it is uncompressed. ; For the compression, distance-length pair should be 0000 to end. ; Distance length as a 16-bit: xyyy ; x=Length, y=Distance


Title: Re: Compression Formats
Post by: Misery on February 21, 2015, 03:01:55 AM
Wow, I had completely forgotten you had cracked the battle background decompression already. I think it's really neat that you're looking into and documenting these formats.

Icon decompressor =  This is a simple format that uses the FEDCBA9876543210 list like Format 2 does. When you get a number from this list, that number is moved to slot 0, and the rest of the numbers are pushed over.

Bytes can only be decompressed to 00-0F with this format.
So this is used for things like equipment and psynergy icons?

Quote
Binary code on left, The bits are varied in length from 1-10 depending on slot number, so that the more currently accessed used numbers which are put in the first slots take fewer bits to reference.
I assume you mean frequently?
So each slot holds a palette index number from 0-F of the global palette?

How is the slot order determined when there's an equal amount of pixels of different colors?

When is the end "bit" (%11.1111.1111) used? Is it at the end of each tile? Or the end of a full table of graphics data?

Sorry if these questions are stupid/annoying. If I can fully understand how this decompression works, I'd like to try writing a tool for converting other images into this format. Possibly even a full icon editor, but I'm not sure if I'd be able to do that.


Title: Re: Compression Formats
Post by: Fox on February 21, 2015, 03:25:18 AM
Quote
So this is used for things like equipment and psynergy icons?
I recorded this function as being for icons a long time ago, but I only recently figured the format out not long ago. I think it may apply to Item, Psynergy, and Status icons? (Basically 16x16 icons) I will need to check this out again, though, as well as to see if anything else uses this.

Quote
I assume you mean frequently?
Yep, frequently sounds like the right word. (Or is "recently" a better word?)

Quote
So each slot holds a palette index number from 0-F of the global palette?
Yeah...

Quote
How is the slot order determined when there's an equal amount of pixels of different colors?
I'm a little confused at what you mean?
At the start of the stream, it should always start with this order: FEDCBA9876543210 regardless of the data itself. For every entry that gets decompressed, the value moves to the head (slot 0) and pushes the rest down..... (That way, if you need the same color/palette index again, it just takes one more bit to get it... ("0"))

Quote
When is the end "bit" (%11.1111.1111) used? Is it at the end of each tile? Or the end of a full table of graphics data?
The latter.  "End" goes at the end of the compressed file to say that there is no more data to be decompressed.



Format 2 may follow a similar concept to the icon decompressor, but I think it assigns to both sides of the byte instead of one... and the bit format is different.


Title: Re: Compression Formats
Post by: Misery on February 21, 2015, 07:43:44 AM
Quote
How is the slot order determined when there's an equal amount of pixels of different colors?
I'm a little confused at what you mean?
At the start of the stream, it should always start with this order: FEDCBA9876543210 regardless of the data itself. For every entry that gets decompressed, the value moves to the head (slot 0) and pushes the rest down..... (That way, if you need the same color/palette index again, it just takes one more bit to get it... ("0"))
Ah, it was just me jumping to conclusions... basically I thought the order was a constant thing based on the most-to-least commonly used index. Yeah, that's completely different from how you explained it, so just don't mind that. And I guess 'frequently' still works, but not in the sense I imagined.

So as I understand it now, every time the next entry is different from the most recent entry, that new entry gets the shortest bit length assigned to it after it has been interpreted, pushing all the others 'left' one step. Sorry if that's a confusing way to phrase it, lol. I think I get it now at least.


Title: Re: Compression Formats
Post by: Salanewt on February 22, 2015, 04:34:08 PM
Oh sweet, nice topic! I'm just going to try clarifying on a couple points for the icon decompression stuff for those who may struggle with following it though (myself included). Sprite/icon palette below for ease of clarification, though I have to be honest that your post has me a little confused in a couple areas. Should be able to figure it out after reading through it if all goes well.

(http://img.photobucket.com/albums/v698/AtriusV/Games/Golden%20Sun%20Project/SpritePalette.gif)

Quote
--.----.---0 = 08038B5C = 0
--.----.-001 = 08038d74 = 1
--.----.0101 = 08038d4c = 2
--.----.1101 = 08038d60 = 3
--.----.0011 = 08038d18 = 4
--.----.1011 = 08038d04 = 5
--.----.0111 = 08038bb4 = 6
--.--00.1111 = 08038be8 = 7
--.--10.1111 = 08038bf4 = 8
--.--01.1111 = 08038c10 = 9
--.0011.1111 = 08038c44 = A
--.1011.1111 = 08038c58 = B
--.0111.1111 = 08038c7c = C
00.1111.1111 = 08038cb0 = D
10.1111.1111 = 08038cc4 = E
01.1111.1111 = 08038ce8 = F
11.1111.1111 = End

So basically, an icon can use any of the colours in the first row, and it reads them using binary format. Additionally, it looks like any %0s that come after the last %1 will push everything else to the right by a pixel per %0 (defaults to transparency if no colour is assigned), but I may be wrong on that part. So for example...

x3 = %11, meaning that it will be yellow (fifth colour when including transparency)
x5 = %101, meaning that it will be red (second colour)
xC = %1100, meaning that it will be yellow, and everything is moved/shifted to the right by two pixels, which basically extends the length of the previous colour (I think?)
x1F = %11111, meaning pink
x20 = %100000, meaning black, and everything is shifted by five pixels

And looking at the post again, all remaining pixels will be the previously used colour until you use another one. It looks like you're saying that one can reserve colours for later so they don't have to be recalled every time they are needed, but I'm not really sure I understand how that is done from your explanation.

I don't know, does it sound like I'm getting this right so far?


Title: Re: Compression Formats
Post by: Fox on February 22, 2015, 11:05:26 PM
Oh, I see, you're thinking about it directly from the colors point of view.  Sounds like a good way to put it for new people. (At least, before going into the bit structure and stuff. But that stuff also helps for the technical people.)

Once you use a color, it is put at the head  (slot 0)  of the 0x64 bit number (The number is really divided into two 32-bits) ... and everything else is pushed over to fill the gap that was opened. = This is the basis of that compression, since slots at the head take fewer bits to reference then at the tail.  And my guess is that if you're using one color, the chance that you'll use it again, even for just the very next pixel is probably more likely than other colors? (Well, for horizontal lines, and fillers.)
Pretty much, I just did it to explain that the 0xFEDCBA9876543210 holds indexes to palette, and the values there are rearranged. In the compression, holds variable length bit codes that are used for getting a slot in this number. (Slot 0x0 is the right side, and slot 0xF is the left side... in this 0xFEDCBA9876543210 number.)

Quote
Additionally, it looks like any %0s that come after the last %1 will push everything else to the right by a pixel per %0 (defaults to transparency if no colour is assigned), but I may be wrong on that part. So for example...
Not sure I understand this? ; Either way, you'd get the slot for where the transparent pixel is.

Quote
x3 = %11, meaning that it will be yellow (fifth colour when including transparency)
x5 = %101, meaning that it will be red (second colour)
xC = %1100, meaning that it will be yellow, and everything is moved/shifted to the right by two pixels, which basically extends the length of the previous colour (I think?)
x1F = %11111, meaning pink
x20 = %100000, meaning black, and everything is shifted by five pixels
1: Correct, but you still need the 0s attached: %0011, otherwise, it could be more confusing. Think of it like (val & 0xF = 0x3), (Although, the asm code itself shifts one bit at a time, and does conditional branching each bit that get shifted out.)
2: If this is the first grab, correct, if this is the second grab in your current list, no. (The fifth color would have moved to the first slot, and the rest pushed down to fill the gap.)
3: Wait? 1 color grab, and shift by 2 pixels? I'm not sure I follow? %1100 looks like three colors, though. Slot 0, Slot 0, and Slot 4 if you add on the 0s to the left.
5: It just looks like you are putting several bit codes into one....

Quote
It looks like you're saying that one can reserve colours for later so they don't have to be recalled every time they are needed, but I'm not really sure I understand how that is done from your explanation.
How do you mean? All that happens when you get a color, is the color moves to the head of the (64-bit), and the rest get pushed down. (Both to make room for this color being at the head, but also to fill the gap where this color was previously....... and this happens with all bit codes.)

On the topic of using the % thing, I guess we use these?:
%, b = bits
0x, h = Hex
0o = Octal

---

Even though I know this compression format ~99%(Because I haven't looked into compressing, yet, just decompressing.), it's probably hard to explain it in a user-friendly fashion, so sorry for any inconveniences.


Title: Re: Compression Formats
Post by: Salanewt on February 22, 2015, 11:48:59 PM
Aaah, okay. So the colours get rearranged as you use them?

This makes me wonder something though. How should I view the code if I want to try my hand at making one manually, 8-bit? I'm still not sure I understand how colours are shifted to slots though, and it kind of looks like 4-bit didn't work too well for me either. All I know is that I would certainly like to try this out for a bit at some point, and that I think I may be starting to get it.


Title: Re: Compression Formats
Post by: Fox on February 23, 2015, 12:13:15 AM
Quote
Aaah, okay. So the colours get rearranged as you use them?
Yes. The 64-bit value represents all 16 colors.

I was hoping that this part of my first post would clear up any confusion left behind:

Quote
FEDCBA9876543210

For example, if you were getting from slot 8, since the number 8 is found at slot 8, you take that number and move it to the beginning like so:

FEDCBA9765432108

If you need the number 1, you will now need to get from slot 2... which will move the number to the head of the list, again, as so:

FEDCBA9765432081

And so forth.
That's where I show how they're effectively moved around. And I also mentioned the 0-F in the bit code table was for slot/position number in that 64-bit number.... by this quote, anyway
Quote
Slot number at end.

---

@8-bit: Like Format 2?


Title: Re: Compression Formats
Post by: Salanewt on February 23, 2015, 12:48:47 AM
Ah, okay! Yeah, I just wasn't sure because the only time you actually used either the word "color" or "palette" was in the battle background portion, and you just referred to them as slots before then. It wasn't really clear whether you were talking about that or something else, though the x00-x0F reference and Misery asking about the global palette did clear that up for me. Unfortunately, that quote just made things a tad more confusing because it didn't really specify how the colours are referenced, just how they are rearranged. It was also a bit confusing with the use of "frequently/currently," since neither of those really imply that they are arranged as you reference them. All that said, I feel rather confident that I understand it better now.

8-bit: Oh, so that's how I read it? Sweet, thanks!


Title: Re: Compression Formats
Post by: Fox on February 23, 2015, 01:04:20 AM
Quote
Ah, okay! Yeah, I just wasn't sure because the only time you actually used either the word "color" or "palette" was in the battle background portion, and you just referred to them as slots before then. It wasn't really clear whether you were talking about that or something else, though the x00-x0F reference and Misery asking about the global palette did clear that up for me. Unfortunately, that quote just made things a tad more confusing because it didn't really specify how the colours are referenced, just how they are rearranged. It was also a bit confusing with the use of "frequently/currently," since neither of those really imply that they are arranged as you reference them. All that said, I feel rather confident that I understand it better now.
Probably should be revised/rewritten?
Well, frequent in the sense that if you keep moving something to the top of the list, it may be rarely put at the tail. Either way, do you think "recently" works best?

Quote
8-bit: Oh, so that's how I read it? Sweet, thanks!
I was assuming maybe you wanted one like this that decompresses to values of 00-FF. (Although, Format 2 splits it up, still... into the left and right sides.)
I probably misunderstood, though...


Title: Re: Compression Formats
Post by: Salanewt on February 23, 2015, 04:14:03 AM
Perhaps, yeah. I would suggest changing this:

Quote
so that the more freqently accessed used numbers which are put in the first slots take fewer bits to reference.

Into something more like this:

Quote
rearranging the colours in a way that makes them easier to reference later in the code.

Or something like that. That way you could perhaps clarify on how this is done later in that section. Just an idea though, since I think I understand what the whole section means now.


Ah, that's what you meant! What I am hoping to attempt is to make a few custom icons for some of the spells I am incorporating into Disco Sun, as I figure that I could potentially do this now so I can get it out of the way and not have to worry about spell icons for a while. While I understand that doing this manually could be a painful experience, I am definitely willing to put the effort in; besides, I generally do assembly without assemblers or anything like that.


Title: Re: Compression Formats
Post by: Fox on February 23, 2015, 06:29:41 AM
Quote
rearranging the colours in a way that makes them easier to reference later in the code.
I think that's still too vague... Since I doubt that anything needs to be made "easier" to reference.... Instead, just less space/bits to reference. ; (And some people may even object and say it's harder... since it would take humans a bit more time than uncompressed.)

Quote
While I understand that doing this manually could be a painful experience, I am definitely willing to put the effort in; besides, I generally do assembly without assemblers or anything like that.
Or I, you, or somene else could code a converter.... Waaayyy easier then coding an assembler editor in comparison.
You could also consider decompressing the icons and having them in that format. (AKA: Get rid of the icon decompressor altogether.)

@Text compression: Still studying it, but Terraria had me side-tracked....

08060C30 = Character data? - Two pointers, it's actually a list of one entry. (One entry here is two pointers.) I think the code supports more entries, but they're not needed.

The second pointer has a list of relative addresses for the first pointer... And there are two segments of data that can be found through the first pointer... for each relative address in the second pointer...
One segment comes before the address (The one after relative is calculated in), (12-bits per entry), and the other segment starts at the address. (Variable-length bits...)
I think the first segment is a char list? ....
Is the second segment some type of tree?  I have a feeling it is! (Tree dimensions? And the characters get assigned to them in order?)
Might need to confirm this stuff further, though...

@Second segment
My current theory is that the tree goes as deep as how many bits of 0 you have until you reach a 1... (And repeats for the tree's branches.) ... And in the text data, you put how deep you want to go, as well into each branch... and so forth. Until you get to the end of a branch, which would be your letter.

Are there as many bits that are "1" as there are number of characters branching from a character? Looks like it! ... So 0 = branch, 1 = End of branch.....

I may have to actually use one of the character's tree data in an example when the tut comes? (I'm guessing I /might not/ need to use an text example string from GS, though?) - It'd be quite a lot of work to manually compress your own text strings, though, if you have to go and look up all the trees for each character you use. -- On second thought, the first character (Non-character?/Starter tree when you decompress the first letter of a string???) has 82 possible character connections.... so that might be a bit much. I could maybe consider Char 0x12? Or not, since it seems to have only one character in its list...?


Title: Re: Compression Formats
Post by: approxies on November 14, 2015, 06:25:31 PM
Hello, everyone. Being interested in compression theme for GS, I've made some tools for graphics packing/unpacking:
What Fox calls Format 1 (https://github.com/romhack/GoldenSunCompression/blob/master/gs-lzss/Codec.hs) and Format 2 (https://github.com/romhack/GoldenSunCompression/blob/master/Arena%20GFX/arena-lzss.hs) (unpacker only)

Could someone please update me if graphics compression problem is solved already and there's no need in such tools in community?


Title: Re: Compression Formats
Post by: Fox on November 14, 2015, 11:44:09 PM
Good... Did I forget to link this to pastebin? You can find some decompression routines I wrote here... http://pastebin.com/pzhYZu7G (C#) (I forget if I made fixes to compress16 for the Icon Compressor since then, but I'll need to double check.), but you can also find some stuff in Atrius's editor source.

Porting to Python and other languages could be interesting as well.. but hmm..


--

Huh... Seems I'm still not familiar with Haskel... but haven't taken a lot of effort to learn it either. - At least byte-level code is understandable to me. (Whether it be VB, C#, GML, etc.) :) (I feel like it was a stupid mistake a long time ago when I chose to learn VB.NET... there are so many other better languages for my taste... maybe even Haskel that I could have went for... But either way, at least I chose a language at all.)


Title: Re: Compression Formats
Post by: approxies on November 15, 2015, 10:30:03 AM
You did OK with .NET platform decision. Haskell has huge issues with GUI applications development and it's not popular, so a small amount of people will understand the code. But it's clean code and rapid development loop is fine when I need to write some small CLI tool for my purposes.
So, bottom line, both compression and decompression for text and graphics are supported by Atrius's editor and there's no need for separate tool?
Could you also update me where can I find GS ROM mapping for more compressed graphics offsets?


Title: Re: Compression Formats
Post by: Fox on November 16, 2015, 02:49:45 AM
Well, it didn't fully support Text Compression, as Atrius's editor never rewrote the Char tables... It also didn't decompress battle backgrounds... etc. But the point was that it did support decompressoin/compression for some things...

I think most images are in three/four sections.
1.) The sprites tble: (0x08300000 in GS2)
2.) The Master File Table.. (0x08320000 in GS1, 0x08680000 in GS2; Yep, I have both of them memorized.)
3.) One of the code files has compressed icons.
4.) Oh, and there might be some things compressed in another code file files. (Like the palette menu graphics, I think..) but other than that, I think most graphics in those files are uncompressed(?) - Not many uncompressed images, though... but the red crystal grpahic (for convos), and that pink/purple conical selection thing used to select an enemy maybe.. some |> arrows used in debug menus....  etc. That kind of thing.)


Title: Re: Compression Formats
Post by: approxies on November 16, 2015, 06:39:09 PM
OK, thanks for getting me inputs, I will investigate further. I guess, pointer for the chunk of graphics, I'm going to translate (GO and 1st match glyphs in arena) is hardcoded.


Title: Re: Compression Formats
Post by: approxies on February 11, 2016, 05:06:33 PM
An update (https://github.com/romhack/GoldenSunCompression/releases/latest) for my set of compression tools of Golden Sun.
Now it supports decoding/encoding for text, and two LZSS schemes for graphics, described in OP post.


Title: Re: Compression Formats
Post by: TenkaiSp on March 10, 2016, 12:51:56 PM
Hi approxies, Fox and Lord Squirtle, I am currently looking for a way to encode my Polish translation of The Lost Age in a different way than using Atrius' Editor, as it makes the output file much bigger and is now incompatible with The Balance Age  :Sweat:
Needles to say that this thread caught my interest. Approxies, would you mind making a little tutorial to your compression tools for a non-programmer? I've played around with the tools a little bit, but can't really get them to work.
The haskell one returns "Could not find module 'Data.Bitstream.Lazy' (I just installed the GHC compiler).
Recommended translation scheme from the earlier version with .bat files works fine until the encoding (-e) part, where it shows "trees: Patch size is larger than specified. Aborted." I guess it is because my translation is larger than the original file (duh), so do I need to specify a new treeSize, Offset? How do I calculate this? :)
Here (https://github.com/TenkaiSp/TLA_PL/blob/master/Golden%20Sun%20-%20The%20Lost%20Age%20%28PL%29%20Script.txt) is the script, if you want to take a look.


Title: Re: Compression Formats
Post by: Caledor on March 10, 2016, 01:16:28 PM
I'm not sure about it, but i think you can tweak gstlatoolkit itself and make it store the script later into the rom . that way you'd also gain back the compatibility with my patch.


Title: Re: Compression Formats
Post by: TenkaiSp on March 10, 2016, 04:42:51 PM
I don't really know how to do that. I mean, I know where to look, but I don't know the right values to use. The patching file in gstlatoolkit looks like this:

Quote
echo Expanding ROM
patch x tla_patched.gba $01800000

echo Assembling decompress.asm
goldroad asm\decompress.asm

echo Assembling setArgumentsTLA.asm
goldroad asm\setArgumentsTLA.asm

echo Inserting decompress.asm
patch i tla_patched.gba asm\decompress.gba $0003843C

echo Inserting setArgumentsTLA.asm
patch i tla_patched.gba asm\setArgumentsTLA.gba $00038580

echo Inserting string offset table
patch i tla_patched.gba bin\offsets.bin $00FA0000

echo Inserting strings
patch i tla_patched.gba bin\strings.bin $00FAC300


Title: Re: Compression Formats
Post by: Fox on March 10, 2016, 06:00:56 PM
Whether you choose to compress or not is up to you... Frankly, I don't think it matters unless you need the space. (Probably don't.)

I did not look at how Approxies puts the data back into the ROM... But my compressor compresses it to 0x00FA0000.... and it looks the addresses in your spoiler (gstlatoolkit) places it there too... Soo...

Quote
echo Inserting string offset table
patch i tla_patched.gba bin\offsets.bin $00FA0000

echo Inserting strings
patch i tla_patched.gba bin\strings.bin $00FAC300

With the game open in VisualBoyAdvance, you can go to Tools>Memory Viewer... in the address field... The Rom section goes from 0x08000000-0x09FFFFFF, it should match what you see when you open the file in a hex editor. So you just put 0x08FA0000 in the address field, press Go... scroll down until you find what looks like 00 data... That is unused/free data... and is the address you want to put in. (And +0xC300 for the next address.) (Assuming you're editing a hack, I recommend overwriting the text data that has already been pointed in the area if it is there. It should be easy to notice in The Balance Age since the text should be uncompressed, but I didn't check that for myself, if anything unlikely should go wrong.)

You might need to edit the same addresses in the asm files as well. (So that the game knows that the data moved.)


^^ Wrote this without checking.... but I did look into gstoolkit a long time ago, so....


Title: Re: Compression Formats
Post by: Caledor on March 10, 2016, 06:24:58 PM
Try this: open go_TLA.bat with a txt editor and change "$00FA0000" to "$01000000" and "$00FAC300" to "$0100C300" then go into the asm folder, open setArgumentsTLA.asm in the same txt editor, and change at lines 57 and 58:
"r2,=0x08FA0000" to "r2,=0x09000000" and
"r3,=0x08FAC300" to "r3,=0x0900C300"

Save both edited files and paste your translation on a clean ROM (clean as in "not already patched with gstlatoolkit).

This way you save the uncompressed text at 01000000, which is where the original ROM ends. Basically, this way the toolkit can't overlap with anything and it'd be compatible with literally every patch you can find on this forum.

EDIT: @Fox actually in my patches the text is still compressed, I made all the changes with the editor.


Title: Re: Compression Formats
Post by: TenkaiSp on March 11, 2016, 09:01:16 PM
Thanks guys, looks like saving the text at 01000000 gets rid of all compatibility problems  :happy: