One of the purposes of an operating system is to hide the peculiarities
of the system’s hardware devices from its users.
For example the Virtual File System presents a uniform view of the mounted
filesystems irrespective of the underlying physical devices.
This chapter describes how the Linux
The
Linux supports three types of hardware devices: character, block and network.
Character devices are read and written directly without buffering, for example
the system’s
There are many different device drivers in the Linux kernel (that is one of Linux’s strengths) but they all share some common attributes:
- Kernel code
- Device drivers are part of the kernel and, like
other code within the kernel, if they go wrong they can seriously
damage the system.
A badly written driver may even crash the system, possibly corrupting
file systems and losing data.
- Kernel interfaces
- Device drivers provide a standard interface
to the Linux kernel or to the appropriate subsystem.
For example, the terminal driver provides a file I/O interface
to the Linux kernel and a
SCSI device driver provides a SCSI device interface to the SCSI subsystem which, in turn, provides both file I/O and buffer cache interfaces to the kernel. - Kernel mechanisms and services
- Device drivers make use of
standard kernel services such as memory allocation, interrupt delivery
and wait queues to operate.
- Loadable
- Most of the Linux device drivers can be loaded on
demand as kernel modules when
they are needed and unloaded when they are no longer being used. This makes the
kernel very adaptable and efficient with the system’s resources.
- Configurable
- Linux device drivers can be built into the
kernel. Which devices are built is configurable when the kernel is compiled.
- Dynamic
- As the system boots and each device driver is initialized it looks for the hardware devices that it will control. If those devices do not exist (maybe are just not found), the device driver is simply redundant and causes no harm apart from occupying a little of the system’s memory.
Memory
Device drivers have to be careful when using memory. As they are part of the Linux kernel they cannot use virtual memory. Each time a device driver runs, maybe as an interrupt is received or as a bottom half or task queue handler is scheduled, the current process may change. The device driver cannot rely on a particular process running even if it is doing work on its behalf. Like the rest of the kernel, device drivers use data structures to keep track of the device they controll. The kernel would become unnecessarily large if these data structures were statically allocated, part of the device driver’s code. Most device drivers allocate kernel, non-paged, memory to hold their data.
Linux provides
It may be that Linux needs to do quite a lot of extra work when the
The device driver can also specify that it wants to
Interfacing Device Drivers with the Kernel
To ensure that access is always done in the correct manner, the Linux kernel must be able to interact with device drivers in standard ways. Each class of device driver, character, block and network, provides common interfaces that the kernel uses when requesting services from them. These common interfaces mean that the kernel can treat often very different devices and their device drivers absolutely the same. For example,Linux is very dynamic, every time a Linux kernel boots it may encounter different physical devices and thus need different device drivers. Linux allows you to include device drivers at kernel build time via its configuration scripts. When these drivers are initialized at boot time they may not discover any hardware to control. Other drivers can be loaded as kernel modules when they are needed. To cope with this dynamic nature of device drivers, device drivers register themselves with the kernel as they are initialized. Linux maintains tables of registered device drivers as part of its interfaces with them. These tables include pointers to routines and information that supports the interface with the device class.
Character Devices
Figure: Character Devices
Character devices, the simplest of Linux’s devices, are accessed as files,
applications use standard system calls to open them, read from them, write
to them and close them exactly as if the device were a file.
This is true even if the device is a modem being used by the PPP daemon
to connect a Linux system onto a network.
As a character device is initialized its device driver registers itself with
the Linux
Each entry in the chrdevs vector, a
device_struct data structure contains two elements: a pointer to the
name of the registered device driver and a pointer to a block of file
operations.
This block of file operations is itself the addresses of routines within the
character device driver, each of which handles specific file operations
such as open, read, write and close.
The contents of
When a character special file representing a character device (for example
Each
This has only one file operation, the open file operation. When the character special file is opened by an application the generic open file operation uses the device’s major identifier as an index into the chrdevs vector to retrieve the file operations block for this particular device. It also sets up the file data structure describing this character special file, making its file operations pointer point to those of the device driver. Thereafter all of the applications file operations will be mapped to calls to the character devices set of file operations.
Block Devices
Block devices also support being accessed like files. The mechanisms used to provide the correct set of file operations for the opened block special file are very much the same as for character devices. Linux maintains the set of registered block devices as the blkdevs vector. It, like the chrdevs vector, is indexed using the device’s major device number. Its entries are also device_struct data structures. Unlike character devices, there are classes of block devices. SCSI devices are one such class and IDE devices are another. It is the class that registers itself with the Linux kernel and provides file operations to the kernel. The device drivers for a class of block device provide class specific interfaces to the class. So, for example, a SCSI device driver has to provide interfaces to the SCSI subsystem which the SCSI subsystem uses to provide file operations for this device to the kernel.Every block device driver must provide an interface to the buffer cache as well as the normal file operations interface. Each block device driver fills in its entry in the blk_dev vector of blk_dev_struct data structures. The index into this vector is, again, the device’s major number. The blk_dev_struct data structure consists of the address of a request routine and a pointer to a list of request data structures, each one representing a request from the buffer cache for the driver to read or write a block of data.
Figure: Buffer Cache Block Device Requests
Each time the buffer cache wishes to read or write a block of data to or from a registered device it adds a request data structure onto its blk_dev_struct. The figure above shows that each request has a pointer to one or more buffer_head data structures, each one a request to read or write a block of data. The buffer_head structures are locked (by the buffer cache) and there may be a process waiting on the block operation to this buffer to complete. Each request structure is allocated from a static list, the all_requests list. If the request is being added to an empty request list, the driver’s request function is called to start processing the request queue. Otherwise the driver will simply process every request on the request list.
Once the device driver has completed a request it must remove each of the buffer_head structures from the request structure, mark them as up to date and unlock them. This unlocking of the buffer_head will wake up any process that has been sleeping waiting for the block operation to complete. An example of this would be where a file name is being resolved and the EXT2 filesystem must read the block of data that contains the next EXT2 directory entry from the block device that holds the filesystem. The process sleeps on the buffer_head that will contain the directory entry until the device driver wakes it up. The request data structure is marked as free so that it can be used in another block request.
Hard Disks
Figure: Linked list of disks
During initialization Linux maps the topology of the hard disks in the
system.It finds out how many hard disks there are and of what type.
Additionally, Linux discovers how the individual disks have been partitioned.
This is all represented by a list of gendisk data structures pointed at
by the gendisk_head list pointer. As each disk subsystem, for example
Although the disk subsystems build the gendisk entries during their initialization they are only used by Linux during partition checking. Instead, each disk subsystem maintains its own data structures which allow it to map device special major and minor device numbers to partitions within physical disks. Whenever a block device is read from or written to, either via the buffer cache or file operations, the kernel directs the operation to the appropriate device using the major device number found in its block special device file (for example /dev/sda2). It is the individual device driver or subsystem that maps the minor device number to the real physical device.
Details of the physical charactistics of hard disks can be found
IDE Disks
The most common disks used in Linux systems today are Integrated Disk Electronic
or IDE disks. IDE is a disk interface rather than an I/O bus like SCSI.
Each IDE controller can support up to two disks, one the master disk and the
other the slave disk. The master and slave functions are usually set by jumpers on the disk.
The first IDE controller in the system is known as the primary IDE controller,
the next the secondary controller and so on. IDE can manage about 3.3 Mbytes per second
of data transfer to or from the disk and the maximum
Linux names
More details of the physical charactistics of IDE drives can be found
Initializing the IDE Subsystem
IDE disks have been around for much of the IBM PC’s history. Throughout this time the interface to these devices has changed. This makes the initialization of the IDE subsystem more complex than it might at first appear.
The maximum number of IDE controllers that Linux can support is 4. Each controller is represented by an ide_hwif_t data structure in the ide_hwifs vector. Each ide_hwif_t data structure contains two ide_drive_t data structures, one per possible supported master and slave IDE drive. During the initializing of the IDE subsystem, Linux first looks to see if there is information about the disks present in the system’s CMOS memory. This is battery backed memory that does not lose its contents when the PC is powered off. This CMOS memory is actually in the system’s real time clock device which always runs no matter if your PC is on or off. The CMOS memory locations are set up by the system’s BIOS and tell Linux what IDE controllers and drives have been found. Linux retrieves the found disk’s geometry from BIOS and uses the information to set up the ide_hwif_t data structure for this drive. More modern PCs use PCI chipsets such as Intel’s 82430 VX chipset which includes a PCI EIDE controller. The IDE subsystem uses PCI BIOS callbacks to locate the PCI (E)IDE controllers in the system. It then calls PCI specific interrogation routines for those chipsets that are present.
Once each IDE interface or controller has been discovered, its ide_hwif_t is set up to reflect the controllers and attached disks. During operation the IDE driver writes commands to IDE command registers that exist in the I/O memory space. The default I/O address for the primary IDE controller’s control and status registers is 0x1F0 – 0x1F7. These addresses were set by convention in the early days of the IBM PC. The IDE driver registers each controller with the Linux block buffer cache and VFS, adding it to the blk_dev and blkdevs vectors respectively. The IDE drive will also request control of the appropriate interrupt. Again these interrupts are set by convention to be 14 for the primary IDE controller and 15 for the secondary IDE controller. However, they like all IDE details, can be overridden by command line options to the kernel. The IDE driver also adds a gendisk entry into the list of gendisk’s discovered during boot for each IDE controller found. This list will later be used to discover the partition tables of all of the hard disks found at boot time. The partition checking code understands that IDE controllers may each control two IDE disks.
SCSI Disks
The SCSI (Small Computer System Interface) bus is an efficient peer-to-peer data bus that supports up to eight devices per bus, including one or more hosts. Each device has to have a unique identifier and this is usually set by jumpers on the disks. Data can be transfered synchronously or asynchronously between any two devices on the bus and with 32 bit wide data transfers up to 40 Mbytes per second are possible. The SCSI bus transfers both data and state information between devices, and a single transaction between an initiator and a target can involve up to eight distinct phases. You can tell the current phase of a SCSI bus from five signals from the bus. The eight phases are:
- BUS FREE
- No device has control of the bus and there are no transactions currently happening,
- ARBITRATION
- A SCSI device has attempted to get control of the SCSI bus, it does this by asserting its SCSI identifer onto the address pins. The highest number SCSI identifier wins.
- SELECTION
- When a device has succeeded in getting control of the SCSI bus through arbitration it must now signal the target of this SCSI request that it wants to send a command to it. It does this by asserting the SCSI identifier of the target on the address pins.
- RESELECTION
- SCSI devices may disconnect during the processing of a request. The target may then reselect the initiator. Not all SCSI devices support this phase.
- COMMAND
- 6,10 or 12 bytes of command can be transfered from the initiator to the target,
- DATA IN, DATA OUT
- During these phases data is transfered between the initiator and the target,
- STATUS
- This phase is entered after completion of all commands and allows the target to send a status byte indicating success or failure to the initiator,
- MESSAGE IN, MESSAGE OUT
- Additional information is transfered between the initiator and the target.
The Linux SCSI subsystem is made up of two basic elements, each of which is represented by data structures:
- host
- A SCSI host is a physical piece of hardware, a SCSI
controller.
The NCR810 PCI SCSI controller is an example of a SCSI host.
If a Linux system has more than one SCSI controller of the same type, each
instance
will be represented by a separate SCSI host.
This means that a SCSI device driver may control more than one instance of its
controller.
SCSI hosts are almost always the initiator of SCSI commands.
- Device
- The most common set of SCSI device is a SCSI disk but the SCSI standard supports several more types; tape, CD-ROM and also a generic SCSI device. SCSI devices are almost always the targets of SCSI commands. These devices must be treated differently, for example with removable media such as CD-ROMs or tapes, Linux needs to detect if the media was removed. The different disk types have different major device numbers, allowing Linux to direct block device requests to the appropriate SCSI type.
Details of the physical charactistics of SCSI drives
can be found
Initializing the SCSI Subsystem
Initializing the SCSI subsystem is quite complex, reflecting the dynamic nature of SCSI buses and their devices. Linux initializes the SCSI subsystem at boot time; it finds the SCSI controllers (known as SCSI hosts) in the system and then probes each of their SCSI buses finding all of their devices. It then initializes those devices and makes them available to the rest of the Linux kernel via the normal file and buffer cache block device operations. This initialization is done in four phases:
First, Linux finds out which of the SCSI host adapters, or controllers, that were built into the kernel at kernel build time have hardware to control. Each built in SCSI host has a Scsi_Host_Template entry in the builtin_scsi_hosts vector The Scsi_Host_Template data structure contains pointers to routines that carry out SCSI host specific actions such as detecting what SCSI devices are attached to this SCSI host. These routines are called by the SCSI subsystem as it configures itself and they are part of the SCSI device driver supporting this host type. Each detected SCSI host, those for which there are real SCSI devices attached, has its Scsi_Host_Template data structure added to the scsi_hosts list of active SCSI hosts. Each instance of a detected host type is represented by a Scsi_Host data structure held in the scsi_hostlist list. For example a system with two NCR810 PCI SCSI controllers would have two Scsi_Host entries in the list, one per controller. Each Scsi_Host points at the Scsi_Host_Template representing its device driver.
Figure: SCSI Data Structures
>
Now that every SCSI host has been discovered, the SCSI subsystem must find out what SCSI devices are attached to each host’s bus. SCSI devices are numbered between 0 and 7 inclusively, each device’s number or SCSI identifier being unique on the SCSI bus to which it is attached. SCSI identifiers are usually set by jumpers on the device. The SCSI initialization code finds each SCSI device on a SCSI bus by sending it a TEST_UNIT_READY command. When a device responds, its identification is read by sending it an ENQUIRY command. This gives Linux the vendor’s name and the device’s model and revision names. SCSI commands are represented by a Scsi_Cmnd data structure and these are passed to the device driver for this SCSI host by calling the device driver routines within its Scsi_Host_Template data structure. Every SCSI device that is found is represented by a Scsi_Device data structure, each of which points to its parent Scsi_Host. All of the Scsi_Device data structures are added to the scsi_devices list. The figure above shows how the main data structures relate to one another.
There are four SCSI device types: disk, tape, CD and generic. Each of these SCSI types are
individually registered with the kernel as different major block device types. However
they will only register themselves if one or more of a given SCSI device type has been
found. Each SCSI type, for example SCSI disk, maintains its own tables of devices.
It uses these tables to direct kernel block operations (file or buffer cache) to
the correct device driver or SCSI host.
Each SCSI type is represented by a Scsi_Device_Template data structure.
This contains information about this type of SCSI device and the addresses of
routines to perform various tasks.
The SCSI subsystem uses these templates to call the SCSI type routines for each
type of SCSI device.In other words, if the SCSI subsystem wishes to attach a SCSI
disk device it will call the SCSI disk type
attach routine. The Scsi_Type_Template data structures are added to the
scsi_devicelist
list if one or more
The final phase of the SCSI subsystem initialization is to call the finish functions for each registered Scsi_Device_Template. For the SCSI disk type this spins up all of the SCSI disks that were found and then records their disk geometry. It also adds the gendisk data structure representing all SCSI disks to the linked list of disks shown in the figure above.
Delivering Block Device Requests
Once Linux has initialized the SCSI subsystem, the SCSI devices may be used. Each active SCSI device type registers itself with the kernel so that Linux can direct block device requests to it. There can be buffer cache requests via blk_dev or file operations via blkdevs. Taking a SCSI disk driver that has one or more EXT2 filesystem partitions as an example, how do kernel buffer requests get directed to the right SCSI disk when one of its EXT2 partitions is mounted?Each request to read or write a block of data to or from a SCSI disk partition results in a new request structure being added to the SCSI disks current_request list in the blk_dev vector. If the request list is being processed, the buffer cache need not do anything else; otherwise it must nudge the SCSI disk subsystem to go and process its request queue. Each SCSI disk in the system is represented by a Scsi_Disk data structure. These are kept in the rscsi_disks vector that is indexed using part of the SCSI disk partition’s minor device number. For exmaple, /dev/sdb1 has a major number of 8 and a minor number of 17; this generates an index of 1. Each Scsi_Disk data structure contains a pointer to the Scsi_Device data structure representing this device. That in turn points at the Scsi_Host data structure which “owns” it. The request data structures from the buffer cache are translated into Scsi_Cmd structures describing the SCSI command that needs to be sent to the SCSI device and this is queued onto the Scsi_Host structure representing this device. These will be processed by the individual SCSI device driver once the appropriate data blocks have been read or written.
Network Devices
A network device is, so far as Linux’s network subsystem is concerned, an entity that sends and receives packets of data. This is normally a physical device such as an ethernet card. Some network devices though are software only such as the loopback device which is used for sending data to yourself. Each network device is represented by a device data structure. Network device drivers register the devices that they control with Linux during network initialization at kernel boot time. The device data structure contains information about the device and the addresses of functions that allow the various supported network protocols to use the device’s services. These functions are mostly concerned with transmitting data using the network device. The device uses standard networking support mechanisms to pass received data up to the appropriate protocol layer. All network data (packets) transmitted and received are represented by sk_buff data structures, these are flexible data structures that allow network protocol headers to be easily added and removed. How the network protocol layers use the network devices, how they pass data back and forth using sk_buff data structures is described in detail in the Networks chapter (Chapter networks-chapter). This chapter concentrates on the device data structure and on how network devices are discovered and initialized.
The device data structure contains information about the network device:
- Name
- Unlike block and character devices which have their
device special files created
using the
mknod command, network device special files appear spontaniously as the system’s network devices are discovered and initialized. Their names are standard, each name representing the type of device that it is. Multiple devices of the same type are numbered upwards from 0. Thus the ethernet devices are known as/dev/eth0 ,/dev/eth1 ,/dev/eth2 and so on. Some common network devices are:/dev/ethN Ethernet devices /dev/slN SLIP devices /dev/pppN PPP devices /dev/lo Loopback devices - Bus Information
- This is information that the device driver
needs in order to control
the device. The irq number is the interrupt that this device is
using.
The base address is the address of any of the device’s control and
status registers
in I/O memory. The DMA channel is the DMA channel number that this
network device is using.
All of this information is set at boot time as the device is initialized.
- Interface Flags
- These describe the characteristics and
abilities of the network device:
IFF_UP Interface is up and running, IFF_BROADCAST Broadcast address in device is valid IFF_DEBUG Device debugging turned on IFF_LOOPBACK This is a loopback device IFF_POINTTOPOINT This is point to point link (SLIP and PPP) IFF_NOTRAILERS No network trailers IFF_RUNNING Resources allocated IFF_NOARP Does not support ARP protocol IFF_PROMISC Device in promiscuous receive mode, it will receive all packets no matter who they are addressed to IFF_ALLMULTI Receive all IP multicast frames IFF_MULTICAST Can receive IP multicast frames - Protocol Information
- Each device describes how it may be
used by the network protocool layers:
mtu - The size of the largest packet that this network can
transmit not including any
link layer headers that it needs to add. This maximum is used by the
protocol layers,
for example
IP , to select suitable packet sizes to send. - Family
- The family indicates the protocol family that the device can support. The family for all Linux network devices is AF_INET, the Internet address family.
- Type
- The hardware interface type describes the media that this network device is attached to. There are many different types of media that Linux network devices support. These include Ethernet, X.25, Token Ring, Slip, PPP and Apple Localtalk.
- Addresses
- The device data structure holds a number of addresses that are relevant to this network device, including its IP addresses.
- Packet Queue
- This is the queue of sk_buff packets queued waiting to be transmitted on this network device,
- Support Functions
- Each device provides a standard set of routines that protocol layers call as part of their interface to this device’s link layer. These include setup and frame transmit routines as well as routines to add standard frame headers and collect statistics. These statistics can be seen using the ifconfig command.
Initializing Network Devices
Network device drivers can, like other Linux device drivers, be built into the Linux kernel. Each potential network device is represented by a device data structure within the network device list pointed at by dev_base list pointer. The network layers call one of a number of network device service routines whose addresses are held in the device data structure if they need device specific work performing. Initially though, each device data structure holds only the address of an initialization or probe routine.
There are two problems to be solved for network device drivers.
Firstly, not all of the network device drivers built into the Linux kernel will
have devices to control.
Secondly, the ethernet devices in the system are always called
The second problem, that of dynamically assigning ethernet devices to the
standard