This project is dedicated to the memory of William Morris (aka Frags), who was the main contributor to the bounty but was unable to see the final result.

Saturday, March 15, 2014

PPCJITBETA02 (The Beginning of a Beautiful Friendship)

We have just arrived to another exciting milestone on the long road: all the important instructions for the initial release are implemented under the JIT compiling.*

Lots of bugs were fixed, the emulator is much more stable now than the initial beta release.

Some new features are added too: I have merged the SAM440EP/Flex support (thanks to Soft3) and the CGX overlay for MorphOS (thanks to Thunder and Fab). See configuration documentation regarding how to set the overlay up.

As I already mentioned in the previous post: this beta release was delayed for a couple weeks due to a bug that slipped into the code base long time ago. It was discovered on Mac first, but I was able to reproduce it on MorphOS too. Took me a while to figure out what was going on, but it is fixed now.
This was a very tricky bug, it could be blamed for random crashes and endless loops also, not only on Mac, but on all supported platforms. It was triggered randomly based on the distance between the main application code and the code buffer in memory. (Thanks to Mike for discovering this right before I released the beta.)

I have spent a significant amount of time on figuring out how can I do the build for all supported platforms (AmigaOS4, MorphOS and MacOSX-PPC) using my environments. It wasn't easy, but finally I managed to do most of the release on my own.
As it seems MorphOS SDK does not support G5 yet, so I was not able to do the compiling by myself, but thanks to Fab the G5 executable is also available in the release package.
So, as of now users on all major supported platforms can grab the package and start using the right version.
(Sorry, Linux and BSD folks, you are still on your own.)

 

And the World trembled...

 ...or at least that tiny part which I am involved in when I am wearing my crazy latex suit with a huge letter "A" on my chest for my secret identity: the Amiga Software Developer.

After the first beta release forum posts, emails, news sites, blogs had risen in an enormous unmanageable thunderstorm, struck on me with insane amount of communication. (While the rest of the World barely noticed what have just happened.)
Finally I crawled through messages from every possible (and impossible) source and answered the questions to my best knowledge, accepted the good advices, kindly rejected some nonsense.

 

Aftermath

Since I received tons of feedback (good and bad), I inclined to draw some conclusions from the reaction to the very first beta release. Here is the summary for your benefit:

Some people don't understand how the JIT works and what is the exact purpose of it. All I can say is: please read the documentation... Some other (knowledgeable) folks stood up on the forums and educated the others, well done! I hope this helps, because I really don't have time to deal with it.

Many of the users have irrational expectations for how much the JIT compiling will speed up the emulator. (According to somebody: it supposed to be "ten times faster than the interpretive"... Err... Not likely. How did they come up with any number anyway?)

Well, the implementation is not finished yet, some of you guys don't really understand the concept of "beta release". Okay, I admit I was cheating a bit: technically the JIT compiler wasn't feature-complete when the first beta was done. Yet the remaining pieces were related to not too often used instructions anyway.
For the second beta the instructions are done*, yet there is clearly room for improvement regarding some bugs. Probably as soon as I will be able to fix up the optimization of the register- and flag-flow there will be a significant bump for the speed. (No, not "ten times" fold.)

It is hard to measure how much faster the programs are running and some lovely people baffled on this too. Since there is usually no obvious visual clue for the speedup and a 30%-50% increase in the processor speed is probably hard to notice while your favorite jump-and-run game is jumping and running.
Yet, you can feel that the whole emulation is more snappy than before probably even when you simply run Workbench. Except when it crashes. But even then: it crashes 30%-50% faster! :)

Too bad that some good souls are obsessed with their favourite game/program and keep saying that the JIT is worthless because it doesn't make any difference for that particular piece of software. As it seems this JIT compiling is not for you then.

There was one more interesting thing what I have noticed too late unfortunately: G5 support for MorphOS. Since I don't have a G5 machine I never considered that there is a need for that. But there were some murmur about the speed of the MorphOS version on G5 on some forums. No wonder: it needs a special version, which can be compiled from the sources for some time now. (Thanks to Tobias Netzel and to Fab for the special build.)
Probably the same applies to the PA Semi processor and the Amiga X1000, but I don't have that one either. (Donations? :)
Also the mysterious support for SAM440EP/Flex, what I have never heard of before. No wonder it was missed.

Fun fact from the Outer World: I tried to explain to my colleagues how I spent my Summer vacation. However, I am significantly older than almost any of them, so they were looking at me with confusion in their eyes mixed with a little pity. "Yea, my father loves fooling around with those old machines too!" - was one of the comments. Well put, Sir, well put.

 

Anyway...

To make you (some other geeks around the World) happy: here is the new beta...


In case you stop reading here (or you already skipped the first cheesy part):
as always, please read the README for your comfort and safety. Thanks.

Since I bought an iBook for 50 NZD, now I can produce the MorphOS and the MacOSX versions too which were also included in this release together with the AmigaOS4 version. (And by buying a Mac I broke one of my principles: no Apple product crosses the door of my house. I hope you guys are content what you were doing...)

You can find the changes since the last beta in the README file, or in the changesets at the SourceForge repository from R67 down to R53.

 

Fragmentation

I must admit I have learned a lot in the past month about the sorry state of the E-UAE project. I didn't know what is the current situation of the various binary releases until I received some references to modified AmigaOS4 and MorphOS binary versions.
I guess this is the destiny of any abandoned open source project: lots of good people is trying to improve it, but nobody is standing up and takes over the maintenance of the project.

Well, I am of the same kind, as it seems. It was never my goal to take the ownership of the E-UAE project or fork it into a new iteration.
However, as soon as I released the first beta of the JIT compiled version the watching eye of the public turned to my little scared pet project and I received lot of questions about whether this-or-that particular fix from various developers were included or not. (Mostly not.)

To satisfy at least some part of the user base I tried to gather the various fixes from every corner of the Internet and applied them on the source code. This means no way new base source repository for the E-UAE project, but at least it will help whoever wants to grab the torch and probably it will be useful for you, dear user in the meanwhile in the form of the beta releases.

 

Progress indicator

As of now I switch from batch release strategy to immediate update. This means: I will commit each change one by one to the SourceForge repository as soon as the change is ready instead of buffering up lots of changes locally and commit them in a big changeset.
So, if you look for the repository changesets and the tickets then you can watch the progress of the project closely.

I also make use of the tickets in the completion of the various fixes and tasks:

https://sourceforge.net/p/euaeppcjit/tickets

I added milestones to the tickets, so you can get a feeling of the upcoming beta and the included changes, fixes.
Open tickets are defining the majority of the outstanding work. I am currently working on the accepted ticket, while pending tickets are already committed to the repository, but not released in binary form yet. Released tickets are the closed ones.

For PPCJITBET03 you can find the planned changes here:

https://sourceforge.net/p/euaeppcjit/tickets/milestone/PPCJITBETA03

There is also a milestone named "PARKED" which is a holding box for the various bugs and problems that are not considered for this project (yet).

 

Thanks

Finally, big thanks goes to: Thunder, kas1e, MickJT, Fab, Tobias Netzel and Mike Blackburn for helping me with lots of things regarding bug finding, fixing, platform support and constantly watching out for the updates on the repository.

I am still waiting for any (detailed) bug reports, just have a good read of the README file before you jump to your email client.

Footnote
*There is a fine print here: I was struggling with CMP2 instruction and finally I gave up after a couple days. The binary code for the instruction is bundled with CHK2 and I couldn't figure out how solve the exception handling for that. So, this instruction remains unimplemented for now, not a big deal luckily.

26 comments:

  1. Ooopsie, as it seems I accidentally deleted this comment from user "Unknown":

    "the most actively developed non-windows uae these days is fs-uae, and it has a lot of winuae goodies ported over.
    that would be the ideal uae branch to squeeze this little jit emu into."

    My reply to that:
    As I mentioned above (and several other occasions, also included in the FAQ): I have no intention improving E-UAE, all I am interested in is the PPC JIT compiling. If anybody feels like taking over the maintenance of the main code base then all I can say is: go for it.

    ReplyDelete
  2. Nice work! I found the G5 version really fast and stable though I found some bugs. I will report you asap.
    One question: is the hi res graphic enabled since I didn't managed to load workbench in high resolution? And if so, am I missing something (considering I have the gfxcard_size=4 activated)?

    ReplyDelete
    Replies
    1. Thanks. I guess you are looking for super-hires (1280x512), because hires does work for me. I haven't tried that, maybe a limitation of the SDL version I used for this build. I will give it a try, but I haven't change anything related to the gfx emulation. Try the interpretive emulation, maybe it is a bug in the JIT compiling.

      Delete
  3. Super-hires works here too (though characters are displayed in a wrong way). My question (sorry if I've explained myself in a wrong way) is: is it possible to activate in any way the Picasso modes? I didn't find any uaegfx video mode in Prefs:Screenmode/ and none of them in the window log too (Fab's uae-sdl shows them).
    Thanks in advance!

    ReplyDelete
    Replies
    1. I am afraid I cannot help you with that. I have never tried to set up P96 Mac yet. As far as I know adding the gfx card memory should be enough.

      Delete
    2. Hi
      I did more tests and I discovered that P96 modes are not activated at start with E-UAE JIT. You were right: to activate graphic card is enough to assign a value to gfxcard_size= (were n is a value up to 32) but while this works on Fab's E-UAE SDL it doesn't act the same on G5 version (SysInfo shows no gfx board at all). But, well.... I obtained up to over 94 mips with this JIT version and this is amazing!
      Have a good day!

      Delete
    3. Ah, yes. There is no SDL version in the MorphOS release, but only amiwin, which does not support the P96 native modes. You can try the Mac versions, those are running on SDL.

      Delete
    4. The current E-UAE version does not have P96 support built in. Fabs SDL version is a special build, which includes this feature.
      Maybe "somebody" should take a look for implementing P96 support .... ;)

      Delete
    5. Implementing P96 support for the amiwin target is not a priority to me. Maybe once, if I had nothing better to do...

      Delete
  4. This was exactly the same bug which I was reporting from day one, with "unexplained cache miss". Great you figured it out.

    ReplyDelete
  5. On my SAM460ex if I dont' set 'ppc.use_tbc=false' timerbase is wrong calcultaed.

    WITHOUT 'ppc.use_tbc=false':
    #uaeJIT
    E-UAE 0.8.29-PPCJITBETA02
    Build date: Mar 9 2014 14:20:46
    Found 0 joysticks
    Opening cfgfile '.uaerc'...okay.
    unknown config entry: 'enable_jit=yes'
    Calibrating timebase...
    Timebase frequency: 297.392536 MHz
    ..


    WITH 'ppc.use_tbc=false':
    #uaeJIT
    E-UAE 0.8.29-PPCJITBETA02
    Build date: Mar 9 2014 14:20:46
    Found 0 joysticks
    Opening cfgfile '.uaerc'...okay.
    unknown config entry: 'enable_jit=yes'
    EClock frequency:1155.000010 MHz
    ..

    Need more data?

    ReplyDelete
    Replies
    1. Hm, interesting. Could you please check whether you can see the same problem with the Max's build?
      You can get it from here:
      http://www.soft3dev.net/pages/e-uae.php

      Thanks!

      Delete
    2. I realized that your machine is a SAM460, which is not supported at the moment in the sources.
      Maybe I can turn around the machine type checking and support all new generation OS4 machines directly.

      Delete
  6. Thanks Almos! Just realized you live in New Zealand, Iam in Christchurch :) thanks again

    ReplyDelete
    Replies
    1. Hi Dwayne, how is the Amiga-life in Chch? :)

      Delete
  7. Hi Almos i had been tested The version from Os4 from Michael euae jit SDL and the MacOsX
    The Michael version is really fast gave me the Rtg and 54 mips on my Pegasos2 1266mhz an 9800pro with jit enabled and 32 mips without jit enabled

    The strange come with the MacOSX version.
    I have a Quad G5 2.5ghz with Ati x1900gt and i have with the G5 or G4 version too only 6.7 mips with jit enabled and 3.4 mips without jit ...
    Is possible have a uae.rc config for this version just because need to check it better?
    Is this version optimized for the 10.5 xcode? i dont have the opportunity to check with old macos x version because 10.4.1 with my machine have a Kernel Crash (not compatible).

    Hope can gave an hand help, you can ask me if needed for make needed test for all the Amiga/Mos and MacOsX port

    ReplyDelete
    Replies
    1. Hi Luigi,

      Nobody complained about the G5 version, are you sure you are running the G5-optimized binary from the package?
      Unfortunately, I cannot check it, because I have a G4 Mac only. The G5-optimized version cannot be started on G4, so it is compiled for G5 target.

      There is no need to adjust anything in the config which would be specific for the host processor. The normal version is slow on G5 because of the unsupported instructions (by G5), but the specific version must be much faster.

      Delete
    2. Yes i had been test the G5 version and G4 too.
      I dont know if you had compiled it with the latest Xcode the 3.1.4 , if you need an hand i can help in this way (compiling on G5)

      Delete
    3. I compiled it using Xcode 2.4.1, that was the only version I was able to install on my 10.4.

      I doubt that it has anything to do with the compiler version, if the architecture (_ARCH_PWR4) from the compiler is set properly then the G5 mode is selected in the JIT compiler. If not then it falls back to the generic code generation. Also if you turn off the JIT compiling the interpretive must be significantly faster in the G5 version than in the generic version.

      If you have time you can give it a try. You can test whether the right architecture is set while compiling the code by searching all the defines (_ARCH_PWR4) in the code and put an #error inside that #if. When the compiling fails at that line it means the right architecture is selected.
      You can also have a look on the generated code by dumping code disassembly to the log using the logging settings (comp_log and comp_log_compiled settings), but be prepared for TONS of log output. If you see any MCRXR in the generated code then that is not executed as G5 compatible.

      But I am pretty sure that the binary I produced is compiled for G5. If it is not working properly that means something is missing somewhere. The G5 support was done by Tobias Netzel, as I mentioned earlier I cannot test it.

      Delete
    4. I can confirm that the released G5 version does emit the mcrxr instruction and hence performs as bad as the G4 version (running on a G5).
      The cause is that gcc 4.0 (latest gcc released for 10.4) doesn't define _ARCH_PWR4 nor anything else in order to recognize POWER4 or later CPUs (it does only define _ARCH_PPC64 but that means that 64 bit instructions are available).
      I'd recommend using gcc 4.2; you can get a nice tarball at http://r.research.att.com/tools/ . It seems that the latest build (gcc-4.2-5566-darwin8-all.tar.gz) got lost but build 5533 (as one big tarball) and 5531 (as three sperate tarballs) are still available. I did use those compilers myself and can confirm that they do work well.

      Delete
    5. Thanks Tobias, I will look into it for the next release.

      Delete
  8. Hello there,

    any news for the beta 3?

    ReplyDelete
    Replies
    1. When I am finished with all the tickets for beta #3:

      http://sourceforge.net/p/euaeppcjit/tickets/milestone/PPCJITBETA03/

      then there will be a release. It is not too far away, but I was on holiday for two weeks.

      Delete
  9. Can this uae_jit it be used for Basilisk II (68k mac emulator) without too much (re)work Basilisk II code (https://code.google.com/p/basilisk-ii-for-amigaos/)? TIA

    ReplyDelete
    Replies
    1. I have no idea, you have to ask somebody who knows that source code in details. There are not too many UAE-related dependencies in the JIT implementation, but it depends on what is needed by the Basilisk II implementation.

      Delete