Handling interrupts is one of the most crucial features of the CPU. It’s details are however not always very clear and trying to understand them from the official documentation can be time consuming. Especially because in all operating systems that I came across, the code that manipulates them is considered complete and changes are rare. This sadly doesn’t mean that the need to modify the handlers cannot arise. What’s worse, modifications in this code are as safe as walk through a mine field. Misconfigured IRQs may result in unstable system with weird glitches. Huntig ghosts in interrupt delivery on chipset is also one of the less pleasant experiences in an OS development.
I find that the coverage of this topic on the internet is relatively low. This document should therefore serve as a cheatsheet for both newcommers and developers who want to refresh their IRQ handling knowledge.
I do not aim to re-write the whole documentation here. I will instead try to present the topic in my own words, hoping that it helps somebody to understand the official documentation and other great sources that I reference in the code. I would also like to thank to the authors of the awesome OSdev wiki where I learned a lot of the things presented here.
The term interrupt is overloaded. I personally use to think about it as following.
The first meaning is what I percieve as actual interrupt source. This can happen via asserting pin on a real hardware, sending MSI(-X) interrupt, but also by issuing Inter Processor Interrupt (IPI). The common property of these is that no matter where they start, they end in (some) interrupt controller (further referred as IC in this text). Managing these in code usually means interacting with the interrupt controller.
The other meaning of “interrupt” is completely inside the CPU itself. It starts with delivering the IRQ to the cpu, then follows IDT lookup…
This document is about the first meaning, a.k.a. what happens before the IRQ is delivered to the processor.
So, what is the lifecycle of an IRQ? First, there must be some interrupt source. This might be timer, keyboard controller or another CPU core. The interrupt issuer delivers an interrupt using some hardware path to (one or more) interrupt controller (IC). The interrupt controller is responsible for aggregating IRQs from multiple sources and presenting them further, either to the CPU or another IC.
Over the time, multiple different ICs were added to a PC.
The PIC (Programmable Interrupt Controller) is the oldest IC still present in todays computers. It was used back with the 8086 CPU and it still lives with us.
It is (rather was, because I am not familiar enough with todays chipset physical architecture) a separate chip, having 8 pins for actual interrupts and an interface to the CPU. Communication with CPU is done via I/O ports for configuration and separate pins for the actual IRQ delivery. Detailed description can be found in some copy of an original datasheet (for example here).
In a PC compatible machine, you will find two of these controllers, the output of the second being connected to the pin 2 of the first. That said, although both controllers have 8 inputs, together they can handle only 15 and not 16 external interrupt lines.
This controller has several modes of operation, the most commonly used being called “Fully Nested Mode”. In that mode, the IC decides the priority of interrupts by the number of a pin to which the interrupt arrives. Important thing here is that THE LOWER THE NUMBER, THE HIGHER THE PRIORITY.
Both controllers have internal register where the OS writes the offset that is added to the IRQ number to produce the resulting vector. Example: Writing offset 32 to the master PIC causes, that when pin 4 is triggered on this IC, the vector delivered to the CPU will be 36. Note that you must set offset at least 32, because the first 32 vectors are reserved for internal CPU usage on x86 platform (traps and exceptions).
The PIC output is today usually connected to the LINT0 pin of LAPIC (this is called VirtualWire), but it is also possible to connect it to one of the I/O APIC pins (this is documented in the Intel MP Specification, Figure 3-2 and 3-3).
See the article on osdev wiki for further details.
The successor of the legacy PIC chip is the IO Advanced Programmable Interrupt Controller. It is a companion chip for the Local APIC and was introduced to overcome the limitations of its predecessor. Namely its inability to work effectively in multiple CPU environment and the very limited number of IRQs (15 is really low today).
As expected, the IO APIC is way more complex than the older IC. Mainly the vector assigned with each IRQ pin is configurable in a table that is called “Interrupt Redirection Table”. Each IRQ pin has its own entry in this table that specifies (among others) the vector that will be raised and destination CPU where it will be delivered.
Different implementations of IO-APICs can have different number of pins (with the maximum being 128). Single chipset can also have multiple IO-APICs.
In contrast to PIC which is accessed via well-known I/O ports, the IO-APICs registers are memory mapped. The memory address is generally not fixed and operating system should read it from the ACPI MADT (Multiple APIC Description Table).
Note that the IO-APIC does not decide the priority of IRQs. This job is left for the destination APIC. This arrangement makes sense, because each IRQ can be sent to a different APIC and each APIC can be destination for IRQs from different IO-APICs. The IO-APIC therefore doesn’t have enough information to decide the priority and so it just forwards the IRQ.
Although historically the IO-APIC was interconnected to the APIC via special bus, today it submits the interrupts to APIC as MSIs (likely, the SDM just says that the APIC bus is no longer used and the IORED table format is waaay to similar to the format of MSI message format).
The physical interrupt line can be connected to both PIC and IOAPIC. To make things more entertaining, the pin numbers may differ between these two ICs. The actual routing can be retrieved from an ACPI MADT.
Again see the osdev article for more details.
Being a separate chip in the ancient times (see this Pentium datasheet, 82489DX is described around page 600), APIC is now deeply integrated within each CPU core (logical core, meaning that if you have hyper-threading, each thread has its own APIC). Its responsibility is to receive interrupts from IO-APICs, other APICs and it is a destination for MSIs.
Each LAPIC has an ID, which is the actual address of CPU in your multiprocessor system. You can write to the LAPIC ICR (Interrupt Command Register) and send irqs to another CPUs. By sending SIPI (Startup IPI) signal, you effectively start other CPUs on the system. To send a SIPI, you obviously need to know the destination CPU LAPIC ID. This information is again present in the ACPI MADT table.
To satisfy the whole range of IRQs supported on x86 cpus, the LAPIC can dispatch 240 different IRQ vectors (256 in fact, but the first 16 are reserved and will trigger an error). It still applies that the x86 architecture uses the first 32 vectors for predefined exceptions/traps. The LAPIC however reserves only the first 16 so you can happilly send e.g. vector 18 to the LAPIC (and then pray that something reasonable happens :D).
As mentioned before, the LAPIC manages priorities of the received IRQs. If the priority of newly received IRQ is lower than the one currently being serviced (ISRV - Interrupt Serviced Register Vector), the new IRQ is inhibited until the CPU finishes handling the higher priority interrupt. The priority of interrupt is derived from the vector number. Do you remember the PIC controller and the priority being “the lower the higher” so in LAPIC this is different. Priority of the IRQ is now THE HIGHER VECTOR NUMBER, THE HIGHER THE PRIORITY.
Further, the LAPIC allows OS to specify priority of task that is currently running on the CPU. This priority is written by OS to the TPR (Task Priority Register) of the Local APIC. Now the TPR can be also compared with the interrupt vector received and the IRQ is not dispatched if the IRQ priority is lower than that of a current task.
But imagine the situation in which task with priority X runs on the CPU and an IRQ with priority Y (where Y > X) is delivered. What is now the priority threshold for the CPU? Well for exactly this reason the priority of received IRQ is not compared directly with the TPR, but with a register called PPR (Processor Priority Register). The value of PPR is read only and it is calculated by the formula given in Intel SDM. Basically it is the maximum of ISRV and TPR – except that not, we’ll come to that later..
For a reason that I don’t fully understand, it is not exacly like that. Intel decided that not the whole TPR/PPR/ISRV/IRRV number is not compared as a whole (all of them are effectively 8bits wide), but only bits [7:4]. This is called {TPR|PPR|ISRV|IRRV} priority class. The low bits (i.e. [3:0]) of the value are called {…} priority subclass. This effectively groups the 256 different IRQs to 16 different interrupt groups. It also means that IRQ with vector 30 is not going to interrupt processor running with priority 16, because the top 4 bits of this value are the same.
The existence of {…} priority subclass is not clear to me, as the SDM explicitly says that it has no impact on what IRQs are being delivered and is only used to satisfy PPR reads. An operating system can therefore read PPR value as full 8 bits, but the low 4 bits can be thrown away.
