:dancing banana: (sort of)

After more than four months of chasing my own tail on this problem, I had managed to fix up the JIT compiling to let the Kickstart boot using compiled code. (See disclaimer below...)

*Phew*, there were times when I thought I am not going to write this down ever in the blog. I was this >> << close to give it up on some days. Since the world hasn't ended in 2012, I realized I have to go on, there is no escape.

I am proud to announce: the project stepped into Alpha stage on SourceForge with this update. Details of the changes are:

For the Kickstart boot these fixes were needed:

• Added compiling stop (jump) flag to instructions which might trigger interrupt for supervisor mode: OR.W #imm,SR, AND.W #imm,SR, EOR.W #imm,SR
• Temporary registers are flushed at the end of the compiling cycle, but before the code generation.
• Stop the block processing when the special flags were set in the block (an interrupt might be triggered).
• Reload the emulated program counter register when the block finishes with a supported instruction.
• Fixed wrong function epilog implementation: the return address was read from the wrong position in the stack.
• Old executed instruction pointer and emulated PC register is synchronized on PC reload.

Other fixes for the bugs I have discovered while I extensively debugged the emulation:

• Fixed missing releasing of the compiling buffer on quitting the emulator (memory leak).
• Prevent compiling of tiny blocks (less than 4 instructions in a row): the overhead of the block calling is too much.
• Fixed compiling buffer overflow checking and misleading help text for the compiling buffer unit size.
• Removed supported status for not-yet-implemented EOR.x reg,mem instruction, which was added accidentally before.

(Disclaimer) Before all of you rejoice in dancing banana overload on the user portal of your choice, there is a catch: the Kickstart is not able to start up with JIT compiled instructions, only if the original interpretive instructions are called one-by-one from the compiled code.
So, there is no practical use of the sources yet, but this was the big question: is the compiled code handling able to deal with something as complex as the Kickstart? And for a long time the answer was: no.

Some über-geeky details about the fixes (you like when I'm talking dirty, right?):

As you can see from the changes there were numerous problems around the code, all of these changes were needed for the final result. There were the usual ridiculous issues like a missing negative sign in the function Epilog (line 2308) when it tried to read back the return address from the stack - from the wrong offset.
It essentially means that the execution returned from every compiled block to the parent function instead of the block call loop. The funny familiar feeling that every coder experiences sooner or later: how on EARTH this thing ever worked? ;)

The trickiest part was finding the bug about the interrupt handling: I waded through a few hundred gigabytes of debug dump following the execution. Unfortunately, I was not able to compare the different execution sessions as I mentioned earlier in the comments for the previous post.

At last, I have found out that the OS is switching between User and Supervisor mode in the Exec.lib/Supervisor() function using the special OR immediate instruction by flipping the S flag in the Status Register. This step triggers an exception which is captured by the OS and the position of the triggering instruction identified in the ROM exactly.
The bug was: I never considered this OR instruction to be similar to a TRAP or an ILLEGAL instruction, which instructions change the Program Counter by raising an exception - which is essentially a jump. Thus the compiling hadn't stopped to give back the execution to the interpretive emulation.
As a result the compiled block contained the next instruction after the OR consecutively and later the exception was triggered separately: train-wrecking the boot completely. The only way to get out of there for the OS was to reboot, this is how the reboot-loop happened.

Promises?
Now, it gets a lot easier to fix up the different instruction implementations and implement the rest of the missing instructions. As soon as the former is done the OS will be usable and the latter can be done gradually.

I would like to thank to Toni Wilen for his hints regarding the possible ways of tracking down bugs inside UAE. His suggestions gave me ideas which eventually led to the required fixes.

Yet lot more work should be done, but at least I can see the light at the end of the tunnel. (What a cliché, man. Put yourself together!)

And finally, here is a picture of me, made by my wife to capture the moment when the freakin’ thing started up for the very first time:

OMG, who is this ork-face?

See you all soon(ish) – my holiday is over tomorrow (#sadface).

Flag yeah

I am slowly working my way through the unimplemented instructions. Another bunch is done, here are the details of the recent update:

• Implementation of flag condition "addressing" modes.
• Implementation of ST.B Dx, SF.B Dx, Scc.B Dx, ST.B mem, SF.B mem, Scc.B mem, PEA.x, MOVE.x #imm,mem, CLR.x Dy, CLR.x mem and LEA instructions.
• Fixed BTST instruction: testing bits higher than 15 was wrong.
• Merged multiple condition code-related lines in the 68k instruction descriptor table.
• Removed unnecessary parameter load for cache_miss function from the translated code PC verification code.
• Added cache miss check for normal execute handler.
• Code cleanup: removed the TODO label from immediate addressing modes used as destination and added meaningful error message instead.
• Removed ignored parameter from the unsupported opcode macroblock push function.

The highlight of this change set is the implementation of the flag checking "addressing" modes. These are not real addressing modes; more like a simple way of implementing the numerous conditional instructions which are checking the arithmetic flags.
This important change opens the gate for the conditional branching instructions (Bcc and DBcc), that are essential for any average loops and iterations. For now only the conditional set instructions are implemented, because it was much more easy to test these.

I also took some weekday nights for dealing with some simple instructions, like LEA, PEA and CLR. On the working days I am too tired, this is all I can afford.

There is still no fix for the OS booting issue. I have tried to trace it again, at least now found out it is not about the simulated cache manipulation, because there is no cache-incoherency detected. I am still puzzled by this whole problem.

Address Unknown

Finally all the normal addressing modes are done in this update. As I mentioned in the previous post: the final missing pieces were the 68020 (complex) addressing modes.
This took me lot longer than I expected. Not just because these addressing modes are complicated, but also because memory reading is involved.

Secretly I hoped that this will fix the issue with the OS, but no. I am not sure why I expected that. :P
The OS still behaves the same: reboot loop.

You can stop reading now, technical details will follow.

Why does memory access make everything much more complicated?

The answer is: every memory access has to go through the memory handling functions in the emulator. I already implemented the possibility of direct memory access, but we cannot depend on that every time (or rather most of the time).

Then why is that causing any headaches? Because the functions are outside of the translated code, coded in C and all assigned temporary registers will be gone.

Previously, for the MOVE instructions I workarounded the situation by remapping and reloading only the required temporary registers as soon as the memory operation was done. This approach is simply not working in the case of the addressing mode translation: the addressing modes are completely isolated subroutines, do not know any details of the already allocated and mapped registers (that should be restored).

For now I implemented a different workaround for the addressing modes, which is far less optimal: instead of dropping all the temporary registers, I save them on the stack then restore it when the execution returns to the translated code.

No need to say how much slower this is: saving and reloading all mapped temporary registers than dealing with the absolutely required registers only. I am not satisfied with this solution, but at least it is working. Unless the called function expects a proper stackframe, because there won't be any. (At least this is not the case for OS4, and most likely not for MorphOS either.) This solution is not strictly compatible with the SysV ABI.

Probably it would make more sense to store the temporary registers in the static context structure, there is no need to be reentrant: there are no call backs to the translated code. Maybe I can revisit this whole piece of code at a later stage.