TRAP is the right way to do it though. They're syscalls. That's the m68k way of doing syscalls and to elevate to supervisor mode etc.
From today's POV some of the stuff in the VDI and AES maybe doesn't belong in "kernel" and would be done user space. But it was the 80s, and it all had to fit in 192kb, there were no shared libraries. It was a different time.
>TRAP is the right way to do it though. They're syscalls. That's the m68k way of doing syscalls and to elevate to supervisor mode etc.
Yes, if you're implementing a UNIX. Of course, the fundamental mistake was using CP/M 68k as the base. GEM ended up effectively being just a GUI for CP/M, similar as the DOS version, rather than a modern system as AmigaOS attempted to be (minus the TripOS mistake... but at least dos.library fits on top of exec and the rest of AmigaOS, rather than being at the bottom of the stack).
The use of stack rather than registers for passing parameters is a particularly bad fit for the 68000. It effectively slows down the system considerably, as it adds significant overhead per-call.
>From today's POV some of the stuff in the VDI and AES maybe doesn't belong in "kernel" and would be done user space.
Agree 100%. I love how AmigaOS's exec.library got this right. In 2022, the world's still stuck with UNIX clones and struggling to move past that. Recently, with Haiku, Genode and Fuchsia there's finally some excitement again.
FWIW having seen the sources for both, GEMDOS isn't based directly off CP/M68k sources. There are commonalities, in particular in the names of functions, trap locations, etc. and some bits of stuff that is common (executable format for one) but they're entirely separate codebases. Most of CP/M68k is written directly in assembler. While GEMDOS is largely C.
GEM on the PC was written to run on MS/DOS. So I think to make a portable GEM for other platforms they made something with an API and filesystem closer to DOS but written by people who were clearly familiar with CP/M, and had access to CP/M68k sources.
The first platform to run GEMDOS on 68k was the Apple Lisa. That was the dev machine DR used.
And in fact EmuTOS itself now boots on the Lisa again.
What's interesting is I know Mac OS Classic used the same "push the args to the stack and call TRAP" syscall process as well. I suspect because of this that the Lisa did as well, though I couldn't tell you for sure since hardly any docs are out there for that. Digital Research was not unique in choosing these scheme. Maybe DR copied Apple, even tho the timeline seems wrong there.
FWIW you got me interested, so I'm reading the CP/M68k sources right now and from the bits I've read, I do see the stack based calling convention there, as well:
_bios2:
* For all BIOS functions with a word parameter
* Word parameter follows function number
move.w 6(sp),d1 * get 1st parameter (word)
_bios6:
_bios1:
* For all BIOS functions that have no parameter other than function number
move.w 4(sp),d0 * get function number
movem.l d3-d7/a3-a6,-(sp)
* * save C register variables
trap #3 * do BIOS call
* * returns value in d0
movem.l (sp)+,d3-d7/a3-a6
rts
There does seem to be evidence that there was a multitasking Concurrent CP/M, and/or Concurrent DOS, and/or FlexOS, from DR for 68K. It looks like it got to at least v1.2 and maybe later.
How much of that got into EmuTOS, I don't know, and I'm not sure if binaries or source of FlexOS 68k survived.
If they did, as I wrote here, they are probably arguably FOSS now:
If we go there, I understand FreeMiNT is the way to go.
But I have not played with anything beyond emuTOS for now. I think there's still a lot of good work to do there, particularly making it work, and work well, on more m68k hardware than the Atari platform.
From today's POV some of the stuff in the VDI and AES maybe doesn't belong in "kernel" and would be done user space. But it was the 80s, and it all had to fit in 192kb, there were no shared libraries. It was a different time.
That AmigaOS didn't use TRAP is a bit odd.