CI20: You say it best when you say nothing at all

I’ve always liked the song lyric in the title, in a wry sort of way, because of its apparent, and surprisingly blithe, indictment of the recipient’s ability to express themselves, and/or its suggestion that it was past time that the object of the singer’s affections shut up and got on with... well, with whatever nonverbal communication the singer was concerned with at the time. Yes, I realise it’s about how sometimes words are unnecessary and that it’s not supposed to be mean. But it’s so easy to read it as mean! I just don’t think the writer expressed himself very well, that’s all. I suspect he doesn’t take his own advice.

In any case, “you say it best when you say nothing at all" is surprisingly relevant to the CI20 bare-metal project. As we saw in the previous post about multicore, cache (in-)coherence is a real problem when communicating between cores in kseg0. There are multiple options available to us:

1. Communicate using only uncached memory. this is effective, but slow. We could take this option if communication was very infrequent.

2. Manually manage the cache. Use the MIPS cache-management instructions to effectively disable caching only for the shared variable. 

3. Communicate via other means. For example, the JZ4780 has support for inter-processor interrupts (IPIs) which are triggered by writing to a special mailbox location in CP0. This is fast and convenient, but limits communication to 32 bits — we could pass a command, or a single pointer.

4. (The country option): say nothing at all. To the extent that it’s possible, eliminate inter-core communication altogether.

Perhaps surprisingly, it seems quite feasible to eliminate inter-core communication for most things that a kernel might want to do. If you’re into OS theory, there is plenty of work in this area, for example in Barrelfish. There are several good reasons to aim for such a thing: no inter-core communication eliminates two particularly tricky sources of bugs (cache invalidation and synchronisation); it simplifies the kernel; and specifically it pushes matters relating to policy (which tends to be where we need multi-core synchronisation) out of the kernel and into user-space (more on this later); and it’s very easy to scale (because if there is no inter-core communication, then inter-core communication can’t possibly become a scalability bottleneck).

So that’s the option we’ll be taking with the CI20 bare-metal project. We will get into cache management at some point, but as far as the kernel is concerned, we will aim to keep each core as isolated as possible. When cores do need to share information — for example, when both cores are running code in the same address space — we will aim to do this sharing outside the kernel.

In this post we’ve moved away from CI20-specific code into more general areas of operating system design. This will be a theme of at least the next few posts. We will still be implementing all this stuff, though, so there will still be plenty of low-level and system-specific code to contend with!