News:

The forum has been updated to SMF (2.1.3)!
Please be patient as we work to polish up the place and update features as we can.

Main Menu

History and Discussion on Reinvention of the editor

Started by Daddy Poi's Oily Gorillas, 25, October, 2016, 10:46:41 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Daddy Poi's Oily Gorillas

(Sort of a placeholder topic for now,... since I don't have my materials ready... but can list all the things this should be about anyway.)

To recap on old stuff, interesting flow of events go like this, I assume: (Not necessarily in timeline order, but I can try.)
1 = Atrius fan-game development project = About making a game, and not editing a ROM, but maybe still using materials for it. (I assume that project was cancelled because of #2.)
2 = Atrius GS editor = Edits data in the ROM itself.
3 = His editor became open-sourced, a few people may have modded it, but nothing much happened with it since... And then Atrius quit the editor / gave the forums to Kain.
4 = We have huge documentation on the data.. (Not sure when I started it, but listing it as 4 since the bulk of the work may have been done since then...)
5 = I had plans to make the editor in Java... While I did get Thumb code up, it was pretty much put on hiatus. ( I guess maybe I don't really like Java that much?? D: )
Two topics made in that time was:
-http://forum.goldensunhacking.net/index.php?topic=2311.0 = "Tree" poll
-http://forum.goldensunhacking.net/index.php?topic=2335.0 = "Un-packing" poll
6 = I eventually start doing the Editor in C# with Map Editor as main form. And that editor featured fast decompression/compression of text with char tables re-written (as is supposed to be.) for best compression.
7 = Someone started on Open Golden Sun project, but I can't remember where this falls in the timeline.
8 = I start thinking about "unpacking" the ROM again with the idea of leaving the original ROM alone entirely. (After the unpack.) With a more in-depth idea on how it could possibly work.... (Haven't started it yet, but it really has me thinking the managing a ROM directly has so many weak spots in it... Or um... unnecessarily complications for maximum usage.)

1. = "Unpacking" = Not sure if it should be separates apps, or one app. "Export app" / "Build app" which could be console apps... and separate UI apps that "find labels" in your source code to use the data as necessary... - I could maybe even make it all one app... and then create shortcuts that pass an argument to "immitate" like they're separate apps.??? (Assuming that's possible.)

Thoughts. (Early versions should have less features than this.(?))
Export from ROM stuff:
-Code from a GBA emulator dissembler could be grabbed and modified... One would have to have labels for all functions and branch locations, as well as any data pointers, etc.
-Possibility to one day have files that list what those labels could be is an option, but other-wise unnecessary for a first version. (Could be dummy names like func1, func2, label1, label2, etc.)
-Further conversion/option to export as a C-related code?

FYI: In my opinion, the way I would have it done... It would be unlikely for the program (when done) to take longer than 3 seconds to export everything in a format I'd find desirable. (Most likely a second or less.) - And that's even if 100% of everything is decompressed.

Build:
-Could take code from a compiler that can already compile ASM...
-Possibility to compile both arm/thumb and C-related code?

(More to come?)

--
But yeah, this topic should be for any discussion on anyone's attempt to at least make /some/ progress on such an idea to reinvent the editor any way they want, even if trivial. It will build a history, either way.
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

charon the ferryman

#1
If you're going to rebuild the editor, please do it in something other than Game Maker 8 lol

That's the main reason why the original editor is so slow. Honestly if you tried upgrading it to studio it wouldn't even be much better.

If you need help with getting the files out of the editor, I can help extract the files using GMS if you guys need, that may help with future development in a better language. GML is a little weird though, and the objects might be hard to translate completely depending on how he wrote them. I haven't actually looked at the project file myself.

I think Open Golden Sun is a custom built engine in C# iirc. Building the editor in something like C# should make it run way faster.

Daddy Poi's Oily Gorillas

QuoteIf you're going to rebuild the editor, please do it in something other than Game Maker 8 lol
I'm not planning on using Game Maker for any of my editor projects.... so that's not even a concern.

QuoteIf you need help with getting the files out of the editor, I can help extract the files using GMS if you guys need, that may help with future development in a better language. GML is a little weird though, and the objects might be hard to translate completely depending on how he wrote them. I haven't actually looked at the project file myself.
It would be nice to have Atrius's source on a GITHUB or something, maybe... Yeah... But haven't figured out what I'd need from it, yet...

QuoteI think Open Golden Sun is a custom built engine in C# iirc. Building the editor in something like C# should make it run way faster.
Lucky me, because C# is considered pretty much my main language currently....

---

Anyway, currently, I'm looking to see if I can actually build my own disassembler and compiler...
I know I had thumb code displaying in the Java program before, but I wanted to try a somewhat different way... (Kind of for fun.)

For example, I'm looking into having a string list of each instructions format.
That could look something like this. (Incomplete, and things will be updated later.)
[spoiler]        string[] inst = {
                            "LSL R,R,O",
                            "LSR R,R,O",
                            "ASR R,R,O",
                            "ADD R,R,R",
                            "SUB R,R,R",
                            "ADD R,R,N",
                            "SUB R,R,N",
                            "MOV R8,B",
                            "CMP R8,B",
                            "ADD R8,B",
                            "SUB R8,B",
                            "AND R,R",
                            "EOR R,R",
                            "LSL R,R",
                            "LSR R,R",
                            "ASR R,R",
                            "ADC R,R",
                            "SBC R,R",
                            "ROR R,R",
                            "TST R,R",
                            "NEG R,R",
                            "CMP R,R",
                            "CMN R,R",
                            "ORR R,R",
                            "MUL R,R",
                            "BIC R,R",
                            "MVN R,R",
                            "ADD RH7,RH6",
                            "CMP RH7,RH6",
                            "MOV RH7,RH6",
                            "BX R3H6",
                            "LDR R8,W",
                            "STR R,[R,R]",
                            "STRH R,[R,R]",
                            "STRB R,[R,R]",
                            "LDSB R,[R,R]",
                            "LDR R,[R,R]",
                            "LDRH R,[R,R]",
                            "LDRB R,[R,R]",
                            "LDSH R,[R,R]",
                            "STR R,[R,W]",
                            "LDR R,[R,W]",
                            "STRB R,[R,B]",
                            "LDRB R,[R,B]",
                            "STRH R,[R,H]",
                            "LDRH R,[R,H]",
                            "STR R8,[S,W]",
                            "LDR R8,[S,W]",
                            "ADD R8,P,W",
                            "ADD R8,S,W",
                            "ADD S,W",
                            "ADD S,-W",
                            "PUSH RLL",
                            "POP RLP",
                            "STMIA RL",
                            "LDMIA RL",
                            "BKPT",
                            "BEQ H",
                            "BNE H",
                            "BCS H",
                            "BCC H",
                            "BMI H",
                            "BPL H",
                            "BVS H",
                            "BVC H",
                            "BHI H",
                            "BLS H",
                            "BGE H",
                            "BLT H",
                            "BGT H",
                            "BLE H",
                            "SWI",
                            "B H",
                            "BL H",
                            "BLH H"
        };[/spoiler]
And basically, this plan was to see if this very same list could be used for both, the disassembler, and, the compiler.

@Disassembling: Current plans are to have a second aray. (Just like in the Java program.) That will be used to calculate which instruction it is... Then when you know the instruct, I can do a scan of my string in my inst array... Parsing the necessary data... For example, after copying over the instruction name, then if "r" is found you know it is referring to a register, so you look at the bits in the machine code to assign that... The idea is that stuff like R3, etc. (Not register r3)... are more for telling the position of the register's bits... (Was thinking about having something, where if you didn't have a number, it would use a different number that I would auto-increment as necessary. Plan was that O, B, H, W, etc... were just numbers in different ways. (Since I'm sure things can be caluated strange... like word aligned addressing and such.) It is WIP, so it is definitely not perfect yet.

@Compiling: Further explains why I chose the array format as a test, since I can reuse it... Same thing as before, if the format matches, you have a valid instruction, and then you scan the second table so you know what bits represent that instruction... Etc.
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

charon the ferryman

When I get home tonight I'll download the project file and try to extract the files and get them on a GIT repository. Sound good? That should help a lot.

Luna_blade

Quote from: Fox on 26, October, 2016, 02:57:06 PM
It would be nice to have Atrius's source on a GITHUB or something, maybe... Yeah... But haven't figured out what I'd need from it, yet...
Wait, we have all source code right?
Quote from: Fox
Anyway, currently, I'm looking to see if I can actually build my own disassembler and compiler...
I know I had thumb code displaying in the Java program before, but I wanted to try a somewhat different way... (Kind of for fun.)
For example, I'm looking into having a string list of each instructions format.
That could look something like this. (Incomplete, and things will be updated later.)
And basically, this plan was to see if this very same list could be used for both, the disassembler, and, the compiler.
Wow man. That does sound good.
but even if you have that list, disassembling is a pain because of the THUMB/ARM modes right?
"Hear the sounds and melodies
Of rilets flowing down
They're the verlasting songs
Whispering all the time
As a warning that behind some rocks
There's a rigid grap even
Oreads fear the tread"

Daddy Poi's Oily Gorillas

QuoteWait, we have all source code right?
Yeah, and I know how to access them. (Don't use Studio, use GM7 or something... since Studio was/can be a bit glitchy with infinite restarts... at least back in the day when I was checking it out.)

Plus, I still think having the source code available directly from a webpage itself could make it easier to link people to code and stuff, without taking a roundabout method... (etc..)

QuoteWow man. That does sound good.
but even if you have that list, disassembling is a pain because of the THUMB/ARM modes right?
Thanks.

And yes, maybe... But for GS1/GS2, they don't really use ARM that much... (You'll see it with decompression formats, and a few other things, however.... but I'm not sure if there's anything that people would need to be editing there... if one just wants to code a simple hack...? I may look into that further, though.)

And either way, I plan to manually do a list of which is ARM, THUMB, or DATA eventually, anyway... (@GS1/GS2: However, if ARM always starts off with certain THUMB instructions, I might not need to label anything as ARM for these two games....)
I guess we get there when we're ready(?) Hmm...
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

Rolina

Oh, I'd love an editor rebuild.  Like with Kaiba, I've got quality of life request - please, please, please allow for basic keyboard shortcuts, like tab, ctrl+c/v, and many other things that make the editor easier to work with.  Doing everything with the mouse is so stupidly time consuming that it's got an annoyingly high bar of entry.

Bonus points if there's other easy fixes, like letting us swap out the sprite sheets for items/spells, patches to change multipliers to work as percentages rather than the div/10 they are now, to patch stat multipliers to allow values other than multiples of ten, etc.

Salanewt

And to add to this, an option that makes it so the class changing/separation patch doesn't make the editor save to the ROM (if applicable). It's pretty frustrating when trying to set up my own class separation system and having the editor wipe my changes.
Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?

Necroka (He/Him)

I'm down for a rebuild, though I don't think I'll be of much use as far as helping to make it is concerned.
I'm baaaaa~aaaack <3

Rolina

Quote from: Lord Squirtle on 27, October, 2016, 09:10:54 PM
And to add to this, an option that makes it so the class changing/separation patch doesn't make the editor save to the ROM (if applicable). It's pretty frustrating when trying to set up my own class separation system and having the editor wipe my changes.
That's because it's applying a patch - I had to go in and manually make those edits (though someone else had to go in and fix a few bugs).  What might work is a few buttons that provide the following:


Clear Enemies (clears all enemies)
Restore Enemies to Vanilla (Restores enemy values for abilities)
Clear Abilities (clears all abilities)
Restore Abilities to Vanilla (Restores vanilla values for abilities)
Clear Items (clears all items)
Restore Items to Vanilla (Restores vanilla values for all items)
Clear Classes (clears all classes)
Apply Split Party Template (applies a template to the class system)
Restore Classes to Vanilla (restores vanilla values for classes)

There should be more, but those are the ones I've come up with right now.

Daddy Poi's Oily Gorillas

And most of that should be easy to do if my Exporter idea comes to fruition. May need to make sure I know what I'm doing for the assembly stuff, but my idea, was that there could be a way to convert the tables to text format...(at least for one method) (That the armips assembler could still compile.) And likely aligned as one line/entry, (Probably more lines for larger tables).... (Could probably have comments for each line as well.. describing the name of that class/enemy/etc. they represent.)

So basically... "Restore any data" = Export a clean ROM, take the needed "Vanilla" tables, and replace the ones in your current project.
Where-as "Clearing" stuff.... is either "Delete the whole table" , or just manually 00 the data yourself...

I can say, that the Exporter project should hopefully allow for adding entries without getting anything out of order..! (Since my Exporter is suppose to convert all pointers to labels, so armips can convert them back into pointers.) ...  That's one reason I'm really hoping it works...!
And I assume that compiling without editing code would still mean Atrius's Editor might still work...(With my current idea of just converting straight to assembly/ignoring C/C++ stuff I was doing earlier until then...) But editing the code could break it (if it changes where the pointers Atrius's Editor grabs are.). (Since that's where Atrius's Editor looks the pointers up to find the tables.) However, I think Atrius's editor has the # of entries hard-coded into the editor, so you might still only be able to see the same number of entries?? Should check to be sure, though.

--
Of course, current project status is pretty early (still), so until then.... who knows where this is going.
However, I can show a sample result of what you could see that is basically Alpha...
http://pastebin.com/WKDwPKtm
The underscores (_) are what I'm doing to make it a "label" at the moment.
There are many things currently wrong with it at the moment (Not hard problems at all/just a little lazy is all. :P), and they're likely obvious if you already know assembly... (For example "ii" should be a number instead of ii... easy fix/but was lazy. - Although, for the ldr one,  it's possible I could convert that to a label, since it too, is basically a pointer... ldr-relative... Never an actual data table pointer since it is loading only a 32-bit into a register, but sometimes a pointer to a data table pointer.) Plus, I plan to have the Party Mechanics data tables at the end. (At least  in the early stages / maybe they could have their own file(s) at some point.)

--
And for those who are new to armips, I recommend studying what you can do with it. (It's more than plain assembly.) For example, you can also reference binary files themselves, and such.
Golden Sun Docs: Broken Seal - The Lost Age - Dark Dawn | Mario Sports Docs: Mario Golf & Mario Tennis | Misc. Docs
Refer to Yoshi's Lighthouse for any M&L hacking needs...

Sometimes I like to compare apples to oranges. (Figuratively) ... They are both fruits, but which one would you eat more? (If taken literally, I'd probably choose apples.)
Maybe it is over-analyzing, but it doesn't mean the information is useless.


The only GS Discord servers with significance are:
Golden Sun Hacking Community
GS Speedrunning
/r/Golden Sun
GS United Nations
Temple of Kraden

Can you believe how small the Golden Sun Community is?

2+2=5 Don't believe me? Those are rounded decimal numbers. Take that, flat earth theorists! :)

Salanewt

Role: Oh no, I completely understand the patch; it's just annoying when the editor also overwrites stuff when I have the patch box unchecked. I like the idea of modified/unmodified toggles combined with Fox' suggestion, where you could put a particular ROM in a template folder and then have those buttons treat that as the template.

I'd offer to look into things if not for school and other commitments, but I'm looking forward to seeing where this new editor stuff goes; good luck with it!
Oh yeah baby, £ me harder.

Fusion is just a cheap tactic to make weak Adepts stronger.

Yoshi's Lighthouse is a hacking website in progress. Why not check it out if you like Yoshi or the Mario & Luigi games?

Rolina

Oh, I wasn't informing you or anything, it was more writing out my thought process on the idea.  Basically, we need to build an algorithm that can autofill whole sections of data with some rather more useful sets.  Having a set of sets to import (as Fox put it) built into the system would be great.

We'd need some thing expanded from Atrius' editor, too - such as HP Percentile damage and other effects.