{"id":450,"date":"2020-08-18T19:23:47","date_gmt":"2020-08-18T20:23:47","guid":{"rendered":"http:\/\/www.linux-tutorial.info\/?page_id=77"},"modified":"2020-08-22T19:26:17","modified_gmt":"2020-08-22T20:26:17","slug":"this-is-the-page-title-toplevel-283","status":"publish","type":"page","link":"http:\/\/www.linux-tutorial.info\/?page_id=450","title":{"rendered":"PCI Devices"},"content":{"rendered":"\nPCI<\/title>\nPeripheral Component Interconnect (PCI),\nas its name implies is a standard that describes how to connect\nthe peripheral components of a system together in a structured\nand controlled way.\nThe standard describes\nthe way that the system components are electrically\nconnected and the way that they should behave.\nThis chapter looks at how the Linux kernel initializes the system’s PCI\nbuses and devices.\n<p>\n<img decoding=\"async\" src=\"pci-system.gif\">\n<p>\nFigure 6.1: Example PCI Based System\n<p>\n<p>The figure above is a logical diagram of an example PCI based system.\nThe PCI buses and PCI-PCI bridges are the glue connecting the system\ncomponents together; the CPU\nis connected to PCI bus 0, the primary PCI bus as is the video device.\nA special PCI device, a PCI-PCI bridge connects the primary\nbus to the secondary PCI bus, PCI bus 1.\nIn the jargon of the PCI specification, PCI bus 1 is described as being\n<em>downstream<\/em> of the PCI-PCI bridge and\nPCI bus 0 is <em>up-stream<\/em> of the bridge.\nConnected to the secondary PCI bus are the SCSI and ethernet\ndevices for the system.\nPhysically the bridge, secondary PCI bus and two devices\nwould all be contained on the same combination PCI card.\nThe PCI-ISA bridge in the system supports older, legacy ISA devices and the diagram\nshows a super I\/O controller chip, which controls the keyboard, mouse and\nfloppy.\n<p>\n<h2>PCI Address Spaces<\/h2>\n<p>\nThe CPU and the PCI devices need to access memory that is shared\nbetween them.\nThis memory is used by device drivers to control the PCI devices and\nto pass information between them.\nTypically the shared memory contains control and status registers for the\ndevice.\nThese registers are used to control the device and to read its status.\nFor example, the PCI SCSI device driver would read its status register\nto find out if the SCSI device was ready to write a block of information\nto the SCSI disk.\nOr it might write to the control register to start the device running after it\nhas been turned on.\n<p>\nThe CPU’s system memory could be used for this shared memory but if it were, then every time a\nPCI device accessed memory, the CPU would have to stall, waiting for\nthe PCI device to finish.\nAccess to memory is generally limited to one system component at a time.\nThis would slow the system down.\nIt is also not a good idea to allow the system’s peripheral devices to\naccess main memory in an uncontrolled way.\nThis would be very dangerous; a rogue device could make the system\nvery unstable.\n<p>\nPeripheral devices have their own memory spaces.\nThe CPU can access these spaces but access by the devices into the\nsystem’s memory is very strictly controlled using DMA (Direct Memory\nAccess) channels.\nISA devices have access to two address spaces, ISA I\/O (Input\/Output)\nand ISA memory.\nPCI has three; PCI I\/O, PCI Memory and PCI Configuration space.\nAll of these address spaces are also accessible by the CPU with the\nthe PCI I\/O and PCI Memory address spaces being used by the device\ndrivers and the PCI Configuration space being used by the PCI initialization\ncode within the Linux kernel.\n<p>\nThe Alpha AXP processor does not have natural access to addresses spaces other than\nthe system address space.\nIt uses support chipsets to access other address spaces such as PCI Configuration\nspace.\nIt uses a sparse address mapping scheme which steals part of the large virtual\naddress space and maps it to the PCI address spaces.\n<p>\n<h2>PCI Configuration Headers<\/h2>\n<p>\n<img decoding=\"async\" src=\"pci-config-header.gif\">\n<p>Figure: The PCI Configuration Header\n<p>\n<p>Every PCI device in the system, including the PCI-PCI bridges\nhas a configuration data structure that is somewhere in\nthe PCI configuration address space.\nThe PCI Configuration header allows the system to identify and\ncontrol the device.\nExactly where the header is in the PCI Configuration address space\ndepends on where in the PCI topology that device is.\nFor example, a PCI video card plugged into one PCI slot\non the PC motherboard will\nhave its configuration header at one location and if it is\nplugged into another PCI slot then its header will appear\nin another location in PCI Configuration memory.\nThis does not matter, for wherever the PCI devices and bridges\nare the system will find and configure them using the status\nand configuration registers in their configuration headers.\n<p>\nTypically, systems are designed so that every PCI slot has it’s\nPCI Configuration Header in an offset that is related to\nits slot on the board.\nSo, for example, the first slot on the board might have its\nPCI Configuration at offset 0 and the second slot at offset\n256 (all headers are the same length, 256 bytes) and so on.\nA system specific hardware mechanism is defined so that the PCI\nconfiguration code can attempt to examine all possible PCI Configuration\nHeaders for a given PCI bus and know which devices are present\nand which devices are absent simply by trying to read one of\nthe fields in the header (usually the <em>Vendor Identification<\/em>\nfield) and getting some sort of error.\nThe describes\none possible error message as returning <em>0xFFFFFFFF<\/em> when attempting to\nread the <em>Vendor Identification<\/em> and <em>Device Identification<\/em>\nfields for an empty PCI slot.\n<p>\nThe figure above shows the layout of the 256 byte PCI configuration header.\nIt contains the following fields:\n<p>\n<dl compact>\t<dt><b>Vendor Identification<\/b><\/dt><dd> A unique number\n\tdescribing the originator\n\tof the PCI device. Digital’s PCI Vendor Identification is\n\t<em>0x1011<\/em> and Intel’s is <em>0x8086<\/em>.\n\t<dt><b>Device Identification<\/b><\/dt><dd> A unique number describing\n\tthe device itself. For example, Digital’s 21141 fast\n\tethernet device has a device identification of <em>0x0009<\/em>.\n\t<dt><b>Status<\/b><\/dt><dd> This field gives the status of the\n\tdevice with the meaning of the bits of this field set\n\tby the standard. .\n\t<dt><b>Command<\/b><\/dt><dd> By writing to this field the system\n\tcontrols the device, for example allowing the device to access PCI I\/O memory,\n\t<dt><b>Class Code<\/b><\/dt><dd> This identifies the type of device that\n\tthis is. There are standard classes for every sort of device;\n\tvideo, SCSI and so on. The class code for SCSI is <em>0x0100<\/em>.\n\t<dt><b>Base Address Registers<\/b><\/dt><dd> These registers are used to\n\tdetermine and allocate the type, amount and location of\n\tPCI I\/O and PCI memory space that the device can use.\n\t<dt><b>Interrupt Pin<\/b><\/dt><dd> Four of the physical pins on the PCI card carry\n\t\tinterrupts from the card to the PCI bus. The standard labels these as\n\t\tA, B, C and D. The <em>Interrupt Pin<\/em> field describes which of these\n\t\tpins this PCI device uses. Generally it is hardwired for a pariticular\n\t\tdevice. That is, every time the system boots, the device uses the same\n\t\tinterrupt pin. This information allows the interrupt handling subsystem\n\t\tto manage interrupts from this device,\n\t<dt><b>Interrupt Line<\/b><\/dt><dd>\n\t\tThe <em>Interrupt Line<\/em> field of the device’s PCI Configuration header\n\t\tis used to pass an interrupt handle between the PCI initialisation code,\n\t\tthe device’s driver and Linux’s interrupt handling subsystem.\n\t\tThe number written there is meaningless to the the device driver but it\n\t\tallows the interrupt handler to correctly route an interrupt from the\n\t\tPCI device to the correct device driver’s interrupt handling code within\n\t\tthe Linux operating system.\n\t\tSee Chapter <a href=\"..\/dd\/interrupts.html\"\n> interrupt-chapter<\/a> on page \n\t\tfor details on how Linux handles interrupts.\n<\/dl>\n<p>\n<h2>PCI I\/O and PCI Memory Addresses<\/h2>\n<p>\nThese two address spaces are used by the devices to communicate\nwith their device drivers running in the Linux kernel on the CPU.\nFor example, the DECchip 21141 fast ethernet device maps its\ninternal registers into PCI I\/O space.\nIts Linux device driver then reads and writes those registers to control\nthe device.\nVideo drivers typically use large amounts of PCI memory space\nto contain video information.\n<p>\nUntil the PCI system has been set up and the device’s access to\nthese address spaces has been turned on using the <em>Command<\/em>\nfield in the PCI Configuration header, nothing can access them.\nIt should be noted that only the PCI configuration code reads\nand writes PCI configuration addresses; the Linux device drivers\nonly read and write PCI I\/O and PCI memory addresses.\n<p>\n<h2>PCI-ISA Bridges<\/h2>\nThese bridges support legacy ISA devices by translating PCI I\/O and PCI Memory\nspace accesses into ISA I\/O and ISA Memory accesses.\nA lot of systems now sold contain several ISA bus slots and several PCI bus slots.\nOver time the need for this backwards compatibility will dwindle and PCI only\nsystems will be sold.\nWhere in the ISA address spaces (I\/O and Memory) the ISA devices of the\nsystem have their registers was fixed in the dim mists of time by the\nearly Intel 8080 based PCs.\nEven a $5000 Alpha AXP based computer systems will have its ISA floppy controller\nat the same place in ISA I\/O space as the first IBM PC.\nThe PCI specification copes with this by reserving the lower regions\nof the PCI I\/O and PCI Memory address spaces for use by the ISA peripherals\nin the system and using a single PCI-ISA bridge to translate any PCI memory\naccesses to those regions into ISA accesses.\n<p>\n<h2>PCI-PCI Bridges<\/h2>\n<p>\nPCI-PCI bridges are special PCI devices that glue the PCI buses of the system together.\nSimple systems have a single PCI bus but there is an\nelectrical limit on the number of PCI devices that a single\nPCI bus can support.\nUsing PCI-PCI bridges to add more PCI buses allows the system\nto support many more PCI devices.\nThis is particularly important for a high performance server.\nOf course, Linux fully supports the use of PCI-PCI bridges.\n<p>\n<h3>PCI-PCI Bridges: PCI I\/O and PCI Memory Windows<\/h3>\nPCI-PCI bridges only pass a subset of PCI I\/O and PCI memory read and\nwrite requests downstream.\nFor example, in the figure above, the PCI-PCI bridge will only\npass read and write addresses from PCI bus 0 to PCI bus 1\nif they are for PCI I\/O or PCI memory addresses owned by either\nthe SCSI or ethernet device; all other PCI I\/O and memory addresses are ignored.\nThis filtering stops addresses propogating needlessly throughout the system.\nTo do this, the PCI-PCI bridges must be programmed with a base and limit for\nPCI I\/O and PCI Memory space access that they have to pass\nfrom their primary bus onto their secondary bus.\nOnce the PCI-PCI Bridges in a system have been configured\nthen so long as the Linux device drivers only access PCI I\/O and PCI\nMemory space via these windows, the PCI-PCI Bridges are invisible.\nThis is an important feature that makes life easier for\nLinux PCI device driver writers.\nHowever, it also makes PCI-PCI bridges somewhat tricky for\nLinux to configure as we shall see later on.\n<p>\n<h3>PCI-PCI Bridges: PCI Configuration Cycles and PCI Bus Numbering<\/h3>\n<p>\n<img decoding=\"async\" src=\"pci-cycle-0.gif\">\n<p>\nFigure: Type 0 PCI Configuration Cycle\n<p>\n<img decoding=\"async\" src=\"pci-cycle-1.gif\">\n<p>\nFigure: Type 1 PCI Configuration Cycle\n<p>So that the CPU’s PCI initialization code can address devices that\nare not on the main PCI bus, there has to be a mechanism\nthat allows bridges to decide whether or not to pass\nConfiguration cycles from their primary interface to\ntheir secondary interface.\nA cycle is just an address as it appears on the PCI bus.\nThe PCI specification defines two formats for the PCI Configuration addresses; Type 0 and Type 1; these\nare shown in the figures above, respectively.\nType 0 PCI Configuration cycles do not contain a bus number and these are\ninterpretted by all devices as being for PCI configuration addresses\non this PCI bus.\nBits 31:11 of the Type 0 configuraration cycles are treated as the\ndevice select field.\nOne way to design a system is to have each bit select a different device.\nIn this case bit 11 would select the PCI device in slot 0, bit 12 would\nselect the PCI device in slot 1 and so on.\nAnother way is to write the device’s slot number directly into bits 31:11.\nWhich mechanism is used in a system depends on the system’s PCI memory\ncontroller.\n<p>\nType 1 PCI Configuration cycles contain a PCI bus number and this type\nof configuration cycle is ignored by all PCI devices except the PCI-PCI\nbridges.\nAll of the PCI-PCI Bridges seeing Type 1 configuration cycles may choose\nto pass them to the PCI buses downstream of themselves.\nWhether the PCI-PCI Bridge ignores the Type 1 configuration cycle or\npasses it onto the downstream PCI bus depends on how the PCI-PCI Bridge has been configured.\nEvery PCI-PCI bridge has a primary bus interface number\nand a secondary bus interface number.\nThe primary bus interface being the one nearest the CPU and the secondary\nbus interface being the one furthest away.\nEach PCI-PCI Bridge also has a subordinate bus number and this is\nthe maximum bus number of all the PCI buses that are bridged beyond the secondary bus\ninterface.\nOr to put it another way, the subordinate bus number is\nthe highest numbered PCI bus downstream of the PCI-PCI bridge.\nWhen the PCI-PCI bridge sees a Type 1 PCI configuration cycle it does one of the\nfollowing things:\n<ul>\n<p>\n<li> Ignore it if the bus number specified is not in\n between the bridge’s secondary bus number and\n subordinate bus number (inclusive),\n<p>\n<li> Convert it to a Type 0 configuration command if\n the bus number specified matches the secondary bus\n number of the bridge,\n<p>\n<li> Pass it onto the secondary bus interface unchanged\n if the bus number specified is greater than the\n secondary bus number and less than or equal to the\n subordinate bus number.\n<\/ul>\n<p>\nSo, if we want to address Device 1 on bus 3 of the\ntopology in the figure above we must generate a Type 1 Configuration command from the CPU.\nBridge<sub>1<\/sub> passes this unchanged onto Bus 1. Bridge<sub>2<\/sub> ignores it but Bridge<sub>3<\/sub>\nconverts it into a Type 0 Configuration command and sends\nit out on Bus 3 where Device 1 responds to it.\n<p>\nIt is up to each individual operating system to allocate bus\nnumbers during PCI configuration but whatever the\nnumbering scheme used the following statement must be true for\nall of the PCI-PCI bridges in the system:\n<p>\n <em>“All PCI buses located behind a PCI-PCI bridge must\n reside between the seondary bus number and the\n subordinate bus number (inclusive).”<\/em>\n<p>\nIf this rule is broken then the PCI-PCI Bridges will not pass and\ntranslate Type 1 PCI configuration cycles correctly and the system will\nfail to find and initialise the PCI devices in the system.\nTo achieve this numbering scheme, Linux configures these special devices\nin a particular order.\nSection <a href=\"#pci-pci-bus-numbering\"\n> pci-pci-bus-numbering<\/a> on page \ndescribes Linux’s PCI bridge and bus numbering scheme in detail together\nwith a worked example.\n<p>\n<h2>Linux PCI Initialization<\/h2>\n<p>\nThe PCI initialisation code in Linux is broken into three logical parts:\n<dl compact>\n<p>\n\t<dt><b>PCI Device Driver<\/b><\/dt><dd> This pseudo-device driver searches the\n\t\tPCI system starting at Bus 0 and locates all PCI\n\t\tdevices and bridges in the system.\n\t\tIt builds a linked list of data structures describing the\n\t\ttopology of the system.\n\t\tAdditionally, it numbers all of the bridges that it finds.\n<p>\n\t<dt><b>PCI BIOS<\/b><\/dt><dd> This software layer provides the services\n\t\tdescribed in \t\tbib-pci-bios-specification. Even though Alpha AXP does\n\t\tnot have BIOS services, there is equivalent code in the Linux\n\t\tkernel providing the same functions,\n<p>\n\t<dt><b>PCI Fixup<\/b><\/dt><dd> System specific fixup code tidies up the\n\t\tsystem specific loose ends of PCI initialization.\n<p>\n<\/dl>\n<h3>The Linux Kernel PCI Data Structures<\/h3>\n<img decoding=\"async\" src=\"pci-structures.gif\">\n<p>Figure: Linux Kernel PCI Data Structures\n<p>As the Linux kernel initialises the PCI system it builds data structures mirroring\nthe real PCI topology of the system. The figure above shows the relationships of the data structures\nthat it would build for the example PCI system in the figure above.\n<p>\nEach PCI device (including the PCI-PCI Bridges) is described by a <tt>pci_dev<\/tt> data\nstructure.\nEach PCI bus is described by a <tt>pci_bus<\/tt> data structure.\nThe result is a tree structure of PCI buses each of which has a number of\nchild PCI devices attached to it.\nAs a PCI bus can only be reached using a PCI-PCI Bridge (except the primary PCI bus, bus 0),\neach <tt>pci_bus<\/tt> contains a pointer to the PCI device (the PCI-PCI Bridge) that it is\naccessed through.\nThat PCI device is a child of the the PCI Bus’s parent PCI bus.\n<p>\nNot shown in the figure above, is a pointer to\nall of the PCI devices in the system, <tt>pci_devices<\/tt>.\nAll of the PCI devices in the system have their <tt>pci_dev<\/tt> data structures queued onto\nthis queue..\nThis queue is used by the Linux kernel to quickly find all of the PCI devices in the\nsystem.\n<p>\n<h3>The PCI Device Driver<\/h3>\n<p>\nThe PCI device driver is not really a device driver at all but\na function of the operating system called at system initialisation time.\nThe PCI initialisation code must scan all of the PCI buses\nin the system looking for all PCI devices in the system (including\nPCI-PCI bridge devices).\n<p>\nIt uses the PCI BIOS code to find out if every possible slot in the\ncurrent PCI bus that it is scanning is occupied.\nIf the PCI slot is occupied, it builds a <tt>pci_dev<\/tt> data structure\ndescribing the device and links into the list of\nknown PCI devices (pointed at by <tt>pci_devices<\/tt>).\n<p>\nThe PCI initialisation code starts by scanning PCI Bus 0.\nIt tries to read the <em>Vendor Identification<\/em> and <em>Device Identification<\/em>\nfields for every possible PCI device in every possible PCI slot.\nWhen it finds an occupied slot it builds a <tt>pci_dev<\/tt> data structure\ndescribing the device.\nAll of the <tt>pci_dev<\/tt> data structures built by the PCI initialisation\ncode (including all of the PCI-PCI Bridges) are linked into a singly linked\nlist; <tt>pci_devices<\/tt>.\n<p>\nIf the PCI device that was found was a PCI-PCI bridge then a <tt>pci_bus<\/tt>\ndata structure is built and linked into the tree of <tt>pci_bus<\/tt> and\n<tt>pci_dev<\/tt> data structures pointed at by <tt>pci_root<\/tt>.\nThe PCI initialisation code can tell if the PCI device is a PCI-PCI Bridge\nbecause it has a class code of <em>0x060400<\/em>.\nThe Linux kernel then configures the PCI bus on the other (downstream) side\nof the PCI-PCI Bridge that it has just found.\nIf more PCI-PCI Bridges are found then these are also configured.\nThis process is known as a depthwise algorithm; the system’s PCI\ntopology is fully mapped depthwise before searching breadthwise.\nLooking at the figure above, Linux would configure PCI Bus 1 with its Ethernet and SCSI device before it configured the video device on PCI Bus 0.\n<p>\nAs Linux searches for downstream PCI buses it must\nalso configure the intervening PCI-PCI bridges’ secondary\nand subordinate bus numbers. This is described in detail in the section on\npci-pci-bus-numbering below.\n<p>\n<h4>\nConfiguring PCI-PCI Bridges – Assigning PCI Bus Numbers<\/h4>\n<p>\n<img decoding=\"async\" src=\"pci-pci-config-eg-1.gif\">\n<p>\nFigure: Configuring a PCI System: Part 1\n<p>\n<p>For PCI-PCI bridges to pass PCI I\/O, PCI Memory or PCI\nConfiguration address space reads and writes across them,\nthey need to know the following:\n<dl compact>\n<p>\n\t<dt><b>Primary Bus Number<\/b><\/dt><dd> The bus number immediately\n\tupstream of the PCI-PCI Bridge,\n\t<dt><b>Secondary Bus Number<\/b><\/dt><dd> The bus number immediately\n\tdownstream of the PCI-PCI Bridge,\n\t<dt><b>Subordinate Bus Number<\/b><\/dt><dd> The highest bus number of all\n\t\tof the buses that can be reached downstream of the bridge.\n\t<dt><b>PCI I\/O and PCI Memory Windows<\/b><\/dt><dd> The window base and size for PCI I\/O address space\n \tand PCI Memory address space for all addresses downstream of the PCI-PCI Bridge.\n<\/dl>\n<p>\nThe problem is that at the time when you wish to\nconfigure any given PCI-PCI bridge you do not know the\nsubordinate bus number for that bridge. You do not know\nif there are further PCI-PCI bridges downstream and if\nyou did, you do not know what numbers will be assigned to\nthem.\nThe answer is to use a depthwise recursive algorithm\nand scan each bus for any PCI-PCI bridges assigning them\nnumbers as they are found.\nAs each PCI-PCI bridge is found and its secondary bus numbered, assign it a\ntemporary subordinate number of <em>0xFF<\/em> and scan and assign\nnumbers to all PCI-PCI bridges downstream of it.\nThis all seems complicated but the worked example below makes this process\nclearer.\n<p>\n<dl compact><dt><b>PCI-PCI Bridge Numbering: Step 1<\/b><\/dt><dd>\nTaking the topology in the figure above, the first bridge\nthe scan would find is Bridge<sub>1<\/sub>.\nThe PCI bus downstream of Bridge<sub>1<\/sub> would be numbered as 1 and Bridge<sub>1<\/sub>\nassigned a secondary bus number of 1 and a temporary subordinate bus number of <em>0xFF<\/em>.\nThis means that all Type 1 PCI Configuration addresses specifying a PCI bus number\nof 1 or higher would be passed across Bridge<sub>1<\/sub> and onto PCI Bus 1.\nThey would be translated into Type 0 Configuration cycles if they have a bus number\nof 1 but left untranslated for all other bus numbers.\nThis is exactly what the Linux PCI initialisation code needs to do in order\nto go and scan PCI Bus 1.\n<p>\n<img decoding=\"async\" src=\"pci-pci-config-eg-2.gif\">\n<p>\nFigure: Configuring a PCI System: Part 2\n<p><dt><b>PCI-PCI Bridge Numbering: Step 2<\/b><\/dt><dd>\nLinux uses a depthwise algorithm and so the initialisation code goes\non to scan PCI Bus 1.\nHere it finds PCI-PCI Bridge<sub>2<\/sub>.\nThere are no further PCI-PCI bridges beyond PCI-PCI Bridge<sub>2<\/sub>,\nso it is assigned a subordinate bus number of 2 which\nmatches the number assigned to its secondary interface.\nFigure <a href=\"#pci-pci-config-eg-2\"> 6.7<\/a> shows how the buses and PCI-PCI bridges are numbered at this point.\n<p>\n<img decoding=\"async\" src=\"pci-pci-config-eg-3.gif\">\n<p>\nFigure: Configuring a PCI System: Part 3\n<p><dt><b>PCI-PCI Bridge Numbering: Step 3<\/b><\/dt><dd>\nThe PCI initialisation code returns to scanning PCI Bus 1 and finds another\nPCI-PCI bridge, Bridge<sub>3<\/sub>.\nIt is assigned 1 as its primary bus interface number, 3 as its secondary bus\ninterface number and <em>0xFF<\/em> as its subordinate bus number. The\nFigure above shows how the system is configured now.\nType 1 PCI configuration cycles with a bus number of 1, 2 or 3 wil be correctly\ndelivered to the appropriate PCI buses.\n<p>\n<img decoding=\"async\" src=\"pci-pci-config-eg-4.gif\">\n<p>\nFigure: Configuring a PCI System: Part 4\n<p>\n<p><dt><b>PCI-PCI Bridge Numbering: Step 4<\/b><\/dt><dd>\nLinux starts scanning PCI Bus 3, downstream of PCI-PCI Bridge<sub>3<\/sub>.\nPCI Bus 3 has another PCI-PCI bridge (Bridge<sub>4<\/sub>) on it, it is assigned 3 as its\nprimary bus number and 4 as its secondary bus number.\nIt is the last bridge on this branch and so it is assigned a subordinate bus\ninterface number of 4.\nThe initialisation code returns to PCI-PCI Bridge<sub>3<\/sub> and assigns it a\nsubordinate bus number of 4.\nFinally, the PCI initialisation code can assign 4 as the\nsubordinate bus number for PCI-PCI Bridge<sub>1<\/sub>.\nFigure <a href=\"#pci-pci-config-eg-4\"\n> 6.9<\/a> on page <a href=\"#pci-pci-config-eg-4\"\n>pageref<\/a>\nshows the final bus numbers.\n<\/dl>\n<p>\n<h3>PCI BIOS Functions<\/h3>\n<p>\nThe PCI BIOS functions are a series of standard routines which are common\nacross all platforms.\nFor example, they are the same for both Intel and Alpha AXP based systems.\nThey allow the CPU controlled access to all of the PCI address spaces.\n<p>\nOnly Linux kernel code and device drivers may use them.\n<p>\n<h3>PCI Fixup<\/h3>\n<p>\nThe PCI fixup code for Alpha AXP does rather more than that for Intel (which\nbasically does nothing).\n<p>\nFor Intel based systems the system BIOS, which ran at boot time, has already\nfully configured the PCI system.\nThis leaves Linux with little to do other than map that configuration.\nFor non-Intel based systems further configuration needs to happen to:\n<ul>\n<p>\n<li> Allocate PCI I\/O and PCI Memory space to each device,\n<li> Configure the PCI I\/O and PCI Memory address windows for each PCI-PCI bridge\n\tin the system,\n<li> Generate <em>Interrupt Line<\/em> values for the devices; these control interrupt\n\t\thandling for the device.\n<\/ul>\n<p>\nThe next subsections describe how that code works.\n<p>\n<h4>Finding Out How Much PCI I\/O and PCI Memory Space a\nDevice Needs<\/h4>\nEach PCI device found is queried to find out how much PCI\nI\/O and PCI Memory address space it requires.\nTo do\nthis, each Base Address Register has all 1’s written to\nit and then read.\nThe device will return 0’s in the\ndon’t-care address bits, effectively specifying the\naddress space required.\n<p>\n<img decoding=\"async\" src=\"pci-bars.gif\">\n<p>\nFigure: PCI Configuration Header: Base Address Registers\n<p>\n<p>There are two basic types of Base Address Register, the\nfirst indicates within which address space the devices\nregisters must reside; either PCI I\/O or PCI Memory\nspace. This is indicated by Bit 0 of the register.\nFigure <a href=\"#pci-bars\"\n> 6.10<\/a> shows the two forms of the Base Address Register for\nPCI Memory and for PCI I\/O.\n<p>\nTo find out just how much of each address space a given\nBase Address Register is requesting, you write all 1s\ninto the register and then read it back. The device\nwill specify zeros in the don’t care address bits,\neffectively specifying the address space required.\nThis design implies that all address spaces used are a power\nof two and are naturally aligned.\n<p>\nFor example when you initialize the DECChip 21142 PCI Fast Ethernet\ndevice, it tells you that it needs <em>0x100<\/em> bytes of\nspace of either PCI I\/O or PCI Memory. The\ninitialization code allocates it space. The moment that\nit allocates space, the 21142’s control and status registers\ncan be seen at those addresses.\n<p>\n<h4>Allocating PCI I\/O and PCI Memory to PCI-PCI Bridges and\nDevices<\/h4>\nLike all memory the PCI I\/O and PCI memory spaces are finite, and to some\nextent scarce.\nThe PCI Fixup code for non-Intel systems (and the BIOS code for Intel\nsystems) has to allocate each device the amount of memory that it is\nrequesting in an efficient manner.\nBoth PCI I\/O and PCI Memory must be allocated to a device in a naturally\naligned way. For example, if a device asks for <em>0xB0<\/em> of PCI I\/O space\nthen it must be aligned on an address that is a multiple of <em>0xB0<\/em>.\nIn addition to this, the PCI I\/O and PCI Memory bases for\nany given bridge must be aligned on 4K and on 1Mbyte\nboundaries respectively.\nGiven that the address spaces\nfor downstream devices must lie within all of the\nupstream PCI-PCI Bridge’s memory ranges for any given\ndevice, it is a somewhat difficult problem to allocate\nspace efficiently.\n<p>\nThe algorithm that Linux uses relies on each device described\nby the bus\/device tree built by the PCI Device Driver being\nallocated address space in ascending PCI I\/O memory order.\nAgain a recursive algorithm is used to walk the <tt>pci_bus<\/tt> and <tt>pci_dev<\/tt>\ndata structures built by the PCI initialisation code.\nStarting at the root PCI bus (pointed at by <tt>pci_root<\/tt>) the BIOS fixup\ncode:\n<ul>\n<p>\n<li> Aligns the current global PCI I\/O and Memory bases on 4K\n and 1 Mbyte boundaries respectively,\n<p>\n<li> For every device on the current bus (in ascending PCI I\/O\n memory needs),\n<ul>\n<li> allocates it space in PCI I\/O and\/or PCI Memory,\n<p>\n<li> moves on the global PCI I\/O and Memory bases by the\n appropriate amounts,\n<p>\n<li> enables the device’s use of PCI I\/O and PCI Memory,\n\t<\/ul>\n<p>\n<li> Allocates space recursively to all of the buses\n downstream of the current bus. Note that this will\n change the global PCI I\/O and Memory bases,\n<p>\n<li> Aligns the current global PCI I\/O and Memory bases on 4K\n and 1 Mbyte boundaries respectively and in doing so\n figure out the size and base of PCI I\/O and PCI Memory\n windows required by the current PCI-PCI bridge,\n<p>\n<li> Programs the PCI-PCI bridge that links to this bus with\n its PCI I\/O and PCI Memory bases and limits,\n<p>\n<li> Turns on bridging of PCI I\/O and PCI Memory accesses in\n\t the PCI-PCI Bridge. This means that if any PCI I\/O or PCI Memory\n\t addresses seen on the Bridge’s primary PCI bus that are within\n\tits PCI I\/O and PCI Memory address windows will be bridged onto\n\tits secondary PCI bus.\n<\/ul>\n<p>\nTaking the PCI system in the figure above as our example the\nPCI Fixup code would set up the system in the following way:\n<dl compact>\n<p>\n\t<dt><b>Align the PCI bases<\/b><\/dt><dd> PCI I\/O is <em>0x4000<\/em> and\n\t\tPCI Memory is <em>0x100000<\/em>. This allows the PCI-ISA bridges\n\t\tto translate all addresses below these into ISA address cycles,\n\t<dt><b>The Video Device<\/b><\/dt><dd> This is asking for\n\t\t<em>0x200000<\/em> of PCI Memory and so we allocate it that\n\t\tamount starting at the current PCI Memory base of\n\t\t<em>0x200000<\/em> as it has to be naturally aligned to\n\t\tthe size requested.\n\t\tThe PCI Memory base is moved to\n\t\t<em>0x400000<\/em> and the PCI I\/O base remains at <em>0x4000<\/em>.\n\t<dt><b>The PCI-PCI Bridge<\/b><\/dt><dd> We now cross the PCI-PCI Bridge and\n\t\tallocate PCI memory there, note that we do not need to\n\t\talign the bases as they are already correctly aligned:\n<dl compact>\n\t\t<dt><b>The Ethernet Device<\/b><\/dt><dd> This is asking for <em>0xB0<\/em> bytes\n\t\tof both PCI I\/O and PCI Memory space. It gets allocated PCI\n\t\tI\/O at <em>0x4000<\/em> and PCI Memory at <em>0x400000<\/em>.\n\t\tThe PCI Memory base is moved to\n\t\t<em>0x4000B0<\/em> and the PCI I\/O base to <em>0x40B0<\/em>.\n\t\t<dt><b>The SCSI Device<\/b><\/dt><dd> This is asking for <em>0x1000<\/em> PCI Memory\n\t\tand so it is allocated it at <em>0x401000<\/em> after it has been\n\t\tnaturally aligned. The PCI I\/O base is still <em>0x40B0<\/em> and\n\t\tthe PCI Memory base has been moved to <em>0x402000<\/em>.\n\t<\/dl>\n\t<dt><b>The PCI-PCI Bridge’s PCI I\/O and Memory Windows<\/b><\/dt><dd> We now return to\n\t\tthe bridge and set its PCI I\/O window at between <em>0x4000<\/em> and <em>0x40B0<\/em> and\n\t\tit’s PCI Memory window at between <em>0x400000<\/em> and <em>0x402000<\/em>.\n\t\tThis means that the PCI-PCI Bridge will ignore the PCI Memory accesses for\n\t\tthe video device and pass them on if they are for the ethernet or SCSI\n\t\tdevices.\n<\/dl>\n<p>\n","protected":false},"excerpt":{"rendered":"<p>PCI Peripheral Component Interconnect (PCI), as its name implies is a standard that describes how to connect the peripheral components of a system together in a structured and controlled way. The standard describes the way that the system components are … <a href=\"http:\/\/www.linux-tutorial.info\/?page_id=450\">Continue reading <span class=\"meta-nav\">→<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-450","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/pages\/450","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=450"}],"version-history":[{"count":1,"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/pages\/450\/revisions"}],"predecessor-version":[{"id":653,"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/pages\/450\/revisions\/653"}],"wp:attachment":[{"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=450"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}