Security for Networked Devices

Security Considerations in Embedded I/O Virtualization

As embedded hypervisors continue to grow in popularity across a variety of industries and applications, one of the biggest impacts on security and efficiency is the approach to managing I/O across virtual machines.


  • Page 1 of 1
    Bookmark and Share

Article Media

In any embedded system, there is inevitably a need for sharing a limited set of physical I/O peripherals across workloads. The embedded operating system provides abstractions, such as layer 2/3/4 sockets, for this purpose. OS abstractions free developers to focus on differentiation. 

Over time, this abstraction has grown dramatically up the stack. Instead of simply providing a TCP/IP stack and sockets, embedded operating systems must provide a full suite of Web protocols and remote management functions. Furthermore, as SoCs become more capable, application and real-time workloads are being consolidated—so the OS must provide a plethora of communications, management and HMI services while responding instantly to real-time events and protecting sensitive I/O interfaces from corruption. 

The typical OS abstractions—files, devices, network communication, graphics and threads—are proving insufficient for emerging embedded applications. In particular, developers need the flexibility of abstracting the OS itself. The networking example of control and data plane workload consolidation, with a Linux-based control plane and a real-time OS data plane, is but one use case. Applications with mixed levels of criticality are becoming the norm. A general-purpose operating system with its sophisticated middleware and open source software availability is being combined with safety, availability, security and/or real-time critical applications that need to stay isolated from the general-purpose environment. 

This evolution requires that system software, in particular the virtualization layer, adapt to its new role of securely managing I/O below and on behalf of the guest operating systems. Arguably the most difficult challenge in embedded virtualization is the task of allocating, protecting, sharing and ensuring the efficiency of I/O across the virtual machines and applications. 

Methods of Virtualization

The traditional method of I/O virtualization is emulation: all guest operating system accesses to device I/O resources are intercepted, validated and translated into hypervisor-initiated operations (Figure 1). This method maximizes reliability, security and shareability. The guest OS can never corrupt the system through the I/O device because all I/O accesses are protected via the trusted hypervisor device driver. A single device can easily be multiplexed across multiple virtual machines (VMs), and if one VM fails, the other VMs can continue to utilize the same physical I/O device, maximizing system availability. The biggest drawback is efficiency; the emulation layer causes significant overhead on all I/O operations. In addition, the hypervisor vendor must develop and maintain the device driver independent of the guest OS drivers.

Figure 1
I/O virtualization using emulation implemented with hypervisor device drivers.

In contrast, a pass-through model (Figure 2) gives a guest operating system direct access to a physical I/O device. Depending on the CPU, the guest driver can either be used without modification or with minimal paravirtualization. A single device can be shared between multiple guests by providing a virtual I/O interface between the guest that owns the physical device and any other guests that require access to that device. For network devices, this virtual interface is often called a virtual switch (layer 2) and is a common feature of most hypervisors. The pass-through model provides improved efficiency but trades off security: an improper access by the guest can take down any other guest, application, or the entire system. This model violates the primary security policy of system virtualization: isolation of virtual environments for safe coexistence of multiple OS instances on a single computer. 

Figure 2
I/O virtualization using pass-through to guest—no hypervisor drivers needed.

If present, an I/O memory management unit (IOMMU) enables a pass-through I/O virtualization model without risking direct memory accesses beyond the virtual machine’s allocated memory. IOMMUs are becoming increasingly common in embedded microprocessors, such as Intel Core, Freescale QorIQ and ARM Cortex A15. As the MMU enables the hypervisor to constrain memory accesses of virtual machines, the IOMMU constrains I/O memory accesses (especially DMA), whether they originate from software running in virtual machines or the external peripherals themselves. 

The IOMMU model enables excellent performance efficiency with increased robustness relative to a pass-through model without an IOMMU. However, IOMMUs are a relatively new concept, and a number of vulnerabilities (ways to circumvent protections) have been discovered and must be worked around carefully. In most vulnerability instances, a faulty or malicious guest is able to harm security via device, bus, or chipset-level operations other than direct memory access. Researchers at Green Hills Software, for example, have discovered ways for a guest OS to access memory beyond its VM, deny execution service to other VMs, bluepill the system hypervisor, and take down the entire computer—all via IOMMU-protected I/O devices. 

For high reliability and/or security-critical applications, the IOMMU must be applied in a different way than the traditional pass-through approach where the guest OS has unfettered access to the I/O device. Consult your hypervisor vendor to understand the myriad tradeoffs and options for use of the IOMMU and I/O virtualization in general.

A major downside of a pass-through approach (with or without the IOMMU) is that it prevents robust sharing of a single I/O device across multiple VMs. The VM that is assigned ownership of the pass-through device has exclusive access, and any other VMs must depend upon the owning VM to forward I/O. If the owning VM is compromised, all VMs shall be denied servicing for that device.

This deficiency has lead to emerging technologies that provide an ability to share a single I/O device across multiple guest operating systems using the IOMMU and hardware partitioning mechanisms built into the device I/O complex (e.g., chipset plus the peripheral itself). One example of a shareable, IOMMU-enabled, pass-through device is an Intel processor equipped with Intel VT-c (Virtualization Technology for Connectivity) coupled with PCI Express Ethernet cards implementing Single-Root I/O Virtualization (SR-IOV), a PCI-SIG standard. 

With such a system, the hardware takes care of providing independent I/O resources, such as multiple packet buffer rings, and some form of quality of execution service amongst the VMs. This mechanism lends itself well to networking devices such as Ethernet, Rapid I/O and Fibre Channel; however, other approaches are required for secure, independent sharing of peripherals such as graphics cards, keyboards and serial ports. Nevertheless, it is likely that hardware-enabled, IOMMU-protected, VM-shareable network device technology shall grow in popularity across embedded processors.

Virtual Device Drivers

Device drivers are frequently the cause of system reliability problems. For example, monolithic operating systems commonly install device drivers into the kernel where a faulty device driver can suffer a buffer overflow attack in which malware overwrites the runtime stack in order to install code into the kernel. User-mode device drivers, often called virtual device drivers, prevent these types of attacks because an infiltrated device driver can only harm the process containing the driver, not the kernel itself. In order to facilitate the development of virtual device drivers, the operating system needs to provide a flexible mechanism for I/O control to the virtual driver process. The virtual driver, however, need only be provided access to the specific device resources that the driver requires to achieve its intended function (Figure 3). 

Figure 3
User-mode virtual device driver processes.

As seen in Figure 3, the ideal virtual device driver requires very little device-specific code to reside within the supervisor-mode kernel: the interrupt service routine (ISR) and, in the case of network devices, access to direct memory access (DMA) programming registers. The ISR must be in the kernel while the interrupt vector is executed by the hardware in supervisor mode. The DMA programming is often kept in the kernel because the operation must be trusted: access to DMA registers enables the driver to overwrite any physical memory location, even the kernel itself. The user-mode approach reduces the privilege of the device driver so that it is critical only for the I/O operations pertaining to its managed device and cannot impact any other device, application, or the kernel. While a historic argument against virtual device drivers has been performance, the combination of hardware improvements and the ability of modern real-time kernels to optimize context switch and system call servicing times have demonstrably removed this barrier.

Figure 3
User-mode virtual device driver processes.

Unfortunately, the virtual driver approach still leaves a bit of device-specific code in the kernel. For improved maintainability and a cleaner architecture, it would be better if the DMA programming could reside in user-mode without increasing the driver’s privilege. An IOMMU enables the DMA programming to reside in the virtual device driver. Figure 4 depicts the combination of an MMU and IOMMU protecting accesses to memory originating from either CPU-based applications or from network devices. Perhaps most importantly, by enabling direct access to memory mapped device registers and DMA programming, the IOMMU promotes a purer form of virtual device driver architecture without sacrificing performance efficiency. 

Figure 4
IOMMU for improved device driver architecture and robustness.

Virtual device drivers are commonly employed by microkernel-style operating systems. Thus, microkernel-based hypervisors are well suited to secure virtualization. Instead of the typical monolithic approach of placing device drivers into the hypervisor itself or into a special-purpose Linux guest operating system, the microkernel-based hypervisor uses small, reduced-privilege, native processes for device drivers, I/O multiplexors, health managers, power managers, and other supervisory functions required in a virtualized environment. Each of these applications is provided only the minimum resources required to achieve its intended function, fostering secure embedded system designs.

Figure 5 shows the system-level architecture of a microkernel-based hypervisor used in a multicore networking application that must securely manage Linux control plane functionality alongside high-throughput, low-latency data plane packet processing within virtual device drivers.

Figure 5
Linux and real-time data plane partitioning using microkernel-based virtualization.

Without virtualization, the application in Figure 5 could be implemented with a dual Linux/RTOS configuration in which the control and data plane OSs are statically bound to a set of independent cores (an Asymmetric Multiprocessing, or AMP, approach). One advantage of virtualization over an AMP division of labor is the flexibility of changing the allocation of control and data plane workloads to various cores. For example, in a normal mode of operation, the architect may only want to use a single core for control and all other cores for data processing. However, the system can be placed into management mode in which Linux needs four cores (SMP) while the data processing is temporarily limited. The virtualization layer can handle the reallocation of cores seamlessly under the hood, something that a static AMP system cannot support.

Figure 5
Linux and real-time data plane partitioning using microkernel-based virtualization.

Security can also be improved by adding applications, or even separate virtual machines (called virtual appliances), which perform a dedicated security function such as anti-malware or firewalling.

Increases in software and system complexity and connectivity are driving an evolution in how embedded systems manage I/O and in the architecture of the OSs and hypervisors that are responsible for ensuring their security. The combination of a reduced-privilege, component-based design as well as intelligent I/O virtualization to enable secure consolidation without sacrificing efficiency will remain a focus of systems software suppliers in meeting the flexibility, scalability and robustness demands of next-generation embedded systems.  

Green Hills Software
Santa Barbara, CA.
(805) 965-6044.