Understanding What an Operating System Does
A computer consists of hardware, the electronic components, and software, the computer code executed by that hardware. But what may be less clear is how they work together. A computer’s most essential element is the bit or “binary digit.” Everything a computer does is represented as ones and zeroes. Different computer components represent bits in different ways. In a CPU, microscopic transistors represent ones and zeroes by either being on or off. Those transistors are arranged into logical structures, called logic gates. In electronic computer memory, bits are represented by memory cells either having a charge above or below a certain threshold. On a mechanical hard drive, bits are represented as magnetic fluctuations measured on a spinning platter. On optical discs, pits and lands that do or do not reflect laser light do the same job. No matter how the physical representation of binary code is achieved, you can eventually reduce down all consumer computer components to this raw machine code. So how do you go from the human-friendly interface of a computer to the raw, low-level processes in the computer itself? That’s where the operating system comes in. It directly controls the hardware of the computer. This software translates everything applications (and therefore the user) want into the machine code instructions that the CPU and other components understand. The most critical piece of software in this process is the kernel.
What Is the Kernel?
The kernel is, as the name suggests, the core of the operating system. The kernel is software that resides in RAM and directs everything the computer does. When something is written into memory, it’s the kernel that directs the execution. The kernel knows how to interface with hardware such as GPUs and network cards, but it may not know how to operate them to their full potential, relying on generic standards in the computer industry. The hardware drivers come into play here. Drivers tell your operating system how to work with specific components, which is why you need different drivers for Nvidia and AMD GPUs, for example. Equipped with the right drivers, the kernel is the ultimate authority within the computer, including doing things that can catastrophically destroy data.
The Role of Application Programming Interfaces (APIs)
In the days of MS-DOS, software developers had to write their software specifically for the user’s hardware. The most notorious example of this on MS-DOS systems were sound card drivers. A given video game would have to support the most popular cards (Sound Blaster, Ad-lib, Gravis Ultrasound, etc.) and hope that most players were covered. Today, things work very differently, thanks to APIs. Microsoft DirectX is a great example. If you want an in-depth explanation, check out What Is DirectX and Why Is It Important? However, the most important thing to know is that the API offers a standard way for software developers to ask for hardware resources from components like the GPU. Additionally, hardware makers must only ensure that their products comply with DirectX to ensure full compatibility with any likewise compliant software. APIs offer a layer of translation between software applications and the low-level kernel with its hardware drivers. Yes, this comes with a slight performance penalty. Still, on modern computers, this is negligible, and it comes with a variety of advantages, which is where we finally come to user mode and kernel mode.
User Mode vs. Kernel Mode
Modern operating systems run hundreds or thousands of “processes” simultaneously, dynamically giving them CPU time as needed based on their priorities and computation power requirements. When you launch an application, it generates processes, and the CPU can execute them in either user mode or kernel mode. A Windows process running in user mode only has access to its own private virtual memory address space and handle table. The software uses these tables to store data in RAM and request resources. There’s no direct access to memory or other hardware, and it’s up to the operating system to map those virtual spaces to the actual hardware of the computer. This is good for many reasons, but the most crucial benefit is that the application can’t overwrite or alter data outside its virtual memory address space. In addition, certain functions are off-limits to user-mode processes, mainly ones that could crash the system or destroy data. When a process launches or is elevated to kernel mode, it has full access to system resources, even those reserved for the operating system. So, in theory, it could overwrite crucial data that the operating system needs to run properly.
Traps and Exceptions
It’s important to understand that these two modes are enforced at the hardware level by the CPU itself. If an application running in user mode tries to do something that requires kernel-mode access, it generates a “trap” or “exception.” The operating system will then deal with the application, usually by shutting it down and generating a crash log so that the developers can see what happened in memory when things went off the rails.
The Dangers of Kernel Mode: The Blue Screen of Death
If you’ve ever experienced a Blue Screen of Death (who hasn’t?) that forced your computer to switch off or restart, there’s a good chance it was a kernel-mode process to blame. When a process in kernel mode does something it’s not supposed to, the operating system can’t recover from it, and the entire computer halts. When a user-mode process goes haywire, only the application crashes, and the rest of the software and the operating system can go on without any issues. This is one area where APIs play an essential role since it’s the API asking for kernel-mode privileges. User-mode applications essentially delegate requests that would have required kernel-mode privileges to the API. This is why kernel-mode is usually only granted to low-level system processes that need to access the computer’s hardware directly. Usually, this privilege is extended to a process because it needs more performance than user mode can provide. Some CPU instructions only work in kernel mode, so if a process needs to use those functions, it has to be elevated. If you’re having trouble with the Blue Screen of Death, be sure to read our Blue Screen of Death Troubleshooting Guide for Windows 10!
