Paging 311193 225658574 2008-07-14T19:45:11Z UltraMagnus 7361392 /* Swapping in Linux */ {{dablink|This article is about computer virtual memory. For the wireless communication devices, see [[pager]] and [[radio paging]]. [[Bank switching]] is also sometimes referred to as paging.}} In [[computer]] [[operating system]]s that have their [[main memory]] divided into [[page (computing)|pages]], '''paging''' (sometimes called '''swapping''') is a transfer of pages between main memory and an auxiliary store, such as [[hard disk drive]].<ref>{{Citation | contribution = Virtual memory systems | title = Encyclopedia of computer science and technology | editor1-last = Belzer | editor1-first = Jack | editor2-last = Holzman | editor2-first = Albert G. | editor3-last = Kent | editor3-first = Allen | publisher = CRC Press | volume=14 | pages = 32 | year = 1981 | url = http://books.google.com/books?id=KUgNGCJB4agC&printsec=frontcover | isbn = 0824722140 }} </ref> Paging is an important part of [[virtual memory]] implementation in most contemporary general-purpose operating systems, allowing them to easily use disk storage for data that does not fit into physical RAM. Paging is usually implemented as a [[Task (computers)|task]] built into the [[Kernel (computer science)|kernel]] of the operating system. ==Overview== The main functions of paging are performed when a program tries to access pages that do not currently reside in [[RAM]]. This situation is known as a [[page fault]]. The Operating System must then take control and handle the page fault, in a manner invisible to the program. Therefore, the operating system must: # Determine the location of the data in auxiliary storage. # Obtain an empty page frame in RAM to use as a container for the data. # Load the requested data into the available page frame. # Return control to the program, transparently retrying the [[Instruction (computer science)|instruction]] that caused the page fault. The need to reference memory at a particular address arises from two main sources: * [[Central processing unit|Processor]] trying to load and execute a program's instructions itself. * Data being accessed by a program's instruction. In step 2, when a page has to be loaded and all existing pages in [[RAM]] are currently in use, one of the existing pages must be swapped with the requested new page. The paging system must determine the page to swap by choosing one that is least likely to be needed within a short time. There are various [[page replacement algorithm]]s that try to answer such issue. Most operating systems use the [[least recently used]] (LRU) page replacement algorithm. The theory behind LRU is that the least recently used page is the most likely one not to be needed shortly; when a new page is needed, the least recently used page is discarded. This algorithm is most often correct but not always: e.g. a sequential process moves forward through memory and never again accesses the most recently used page. If a page chosen to be swapped has been modified since loading (if page is ''dirty''), it has to be written to auxiliary storage, otherwise it is simply discarded. In addition to swapping in pages because they are necessary, in reaction to a page fault, there are several strategies for guessing what pages might be needed, and speculatively pre-loading them. === Demand paging === {{main|Demand paging}} Demand paging refuses to guess. With demand paging, no pages are brought into RAM until necessary. In particular, with demand paging, a program usually begins execution with none of its pages pre-loaded in RAM. Pages are copied from the executable file into RAM the first time the executing code references them, usually in response to a [[page fault]]. During a particular run of a program, pages of the executable file that implement functionality not used on that particular run are never loaded. === Loader paging === <!-- I made up the name "loader paging"; surely there is some semi-official name for this? --> Loader paging{{or|date=April 2008}} guesses that the entire program will be used. Many operating systems (including those with a relocating [[loader (computing)|loader]]) load every page of a program into RAM before beginning to execute the program. === Anticipatory paging === Technique that preloads a process's nonresident pages that are likely to be referenced in the near future. Such strategies attempt to reduce the number of page faults a process experiences. === Swap prefetch === A few operating systems use anticipatory paging, also called swap prefetch. These operating systems periodically attempt to guess which pages will soon be needed, and start loading them into RAM. There are various heuristics in use, such as ''"if a program references one virtual address which causes a page fault, perhaps the next few pages' worth of virtual address space will soon be used"'' and ''"if one big program just finished execution, leaving lots of free RAM, perhaps the user will return to using some of the programs that were recently paged out"''. === Precleaning === Unix operating systems periodically use [[sync (Unix)|sync]] to pre-clean all dirty pages. This makes starting a large new program run much faster, because it can be loaded into page frames that held clean pages that were dropped, rather than being loaded into page frames that were dirty and needed to be written back to disk before they were dropped. ==Thrashing== {{main|Thrash (computer science)}} Most programs reach a steady state in their demand for memory [[locality of reference|locality]] both in terms of instructions fetched and data being accessed. This steady state is usually much less than the total memory required by the program. This steady state is sometimes referred to as the [[working set]]: the set of memory pages that are most frequently accessed. Virtual memory systems work most efficiently when the ratio of the working set to the total number of pages that can be stored in RAM is low enough to minimize the number of page faults. A program that works with huge data structures will sometimes require a working set that is too large to be efficiently managed by the page system resulting in constant page faults that drastically slow down the system. This condition is referred to as [[Thrash (computer science)|thrashing]]: pages are swapped out and then accessed causing frequent faults. An interesting characteristic of thrashing is that as the working set grows, there is very little increase in the number of faults until the critical point (when faults go up dramatically and majority of system's processing power is spent on handling them). An extreme example of this sort of situation occurred on the [[IBM System/360 Model 67]] and [[IBM System/370]] series mainframe computers, in which a particular instruction could consist of an execute instruction, which crosses a page boundary, that the instruction points to a move instruction, that itself also crosses a page boundary, targeting a move of data from a source that crosses a page boundary, to a target of data that also crosses a page boundary. The total amount of pages thus being used by this particular instruction is eight, and all eight pages must be present in memory at the same time. If the operating system will allocate less than eight pages of actual memory in this example, when it attempts to swap out some part of the instruction or data to bring in the remainder, the instruction will again page fault, and it will thrash on every attempt to restart the failing instruction. To decrease excessive paging, and thus possibly resolve thrashing problem, a user can do any of the following: * Increase the amount of RAM in the computer (generally the best long-term solution). * Decrease the number of programs being concurrently run on the computer. The term ''thrashing'' is also used in contexts other than virtual memory systems, for example to describe [[cache]] issues in computing or [[silly window syndrome]] in networking. ==Terminology differences - paging versus swapping== Historically, ''paging'' sometimes also referred simply to [[memory allocation]] scheme using fixed-length pages as opposed to variable-length [[segmentation (memory)|segments]], and without implicit suggestion that virtual memory technique is employed at all or that those pages are transferred to disk.<ref>{{Citation | first = Harvey M. | last = Deitel | title = An Introduction to Operating Systems | publisher = Addison-Wesley | pages = 181, 187 | year = 1983 | isbn = 0201144735 }}</ref> <ref>{{Citation | contribution = Operating systems | title = Encyclopedia of computer science and technology | editor1-last = Belzer | editor1-first = Jack | editor2-last = Holzman | editor2-first = Albert G. | editor3-last = Kent | editor3-first = Allen | publisher = CRC Press | volume=11 | pages = 433 | year = 1981 | url = http://books.google.com/books?id=uTFirmDlSL8C&printsec=frontcover | isbn = 0824722612 }} </ref> Such usage is rare today. Some modern systems use the term ''swapping'' along with ''paging''. Historically, ''swapping'' referred to moving from/to secondary storage a whole program at a time, in a scheme known as [[roll-in/roll-out]]. <ref>{{Citation | contribution = Operating systems | title = Encyclopedia of computer science and technology | editor1-last = Belzer | editor1-first = Jack | editor2-last = Holzman | editor2-first = Albert G. | editor3-last = Kent | editor3-first = Allen | publisher = CRC Press | volume=11 | pages = 442 | year = 1981 | url = http://books.google.com/books?id=uTFirmDlSL8C&printsec=frontcover | isbn = 0824722612 }} </ref> <ref>{{Citation | first = Harvey G. | last = Cragon | title = Memory Systems and Pipelined Processors | publisher = Jones and Bartlett Publishers | pages = 109 | year = 1996 | url = http://books.google.com/books?id=q2w3JSFD7l4C | isbn = 0867204745 }}</ref> In the 1960s, after the concept of virtual memory was introduced&mdash;in two variants, either using segments or pages&mdash;the term ''swapping'' was applied to moving, respectively, either segments or pages, between disk and memory. Today with the virtual memory mostly based on pages, not segments, ''swapping'' became a fairly close synonym of ''paging'', although with one difference. In many popular systems, there is a concept known as [[page cache]], of using the same single mechanism for ''both'' virtual memory and [[cache|disk caching]]. A page may be then transferred to or from ''any'' ordinary disk file, not necessarily a dedicated space. In some of such systems, notably [[Unix-like]] including [[Linux]], ''swapping'' only refers to virtual memory scope and ''paging'' to both. ''Page in'' is transferring a page from the disk to RAM. ''Page out'' is transferring a page from RAM to the disk. But ''swap in'' and ''out'' only refer to transferring pages between RAM and dedicated ''swap space'' or ''swap file'', and not any other place on disk. On the other hand, [[Microsoft]] systems from [[Windows NT]] line very rarely use the term ''swapping'' to differentiate from general paging, and call the dedicated secondary store just a ''page file''. ==Paging in Windows 3.0 line== Virtual memory has been a feature of [[Microsoft Windows]] since [[Windows 3.0]] in 1990. Microsoft introduced virtual memory in response to the failures of [[Windows 1.0]] and [[Windows 2.0]], attempting to slash resource requirements for the operating system. Confusion abounds about Microsoft's decision to refer to the swap file as "virtual memory". Novices unfamiliar with the concept accept this definition without question, and speak of adjusting Windows' virtual memory size. In fact ''every'' process has a fixed, unchangeable virtual memory size, usually 2 [[GiB]]. The user has only an option to change disk capacity dedicated to paging. Windows 3.x creates a [[hidden file]] named <tt>386SPART.PAR</tt> or <tt>WIN386.SWP</tt> for use as a swap file. It is generally found in the [[root directory]], but it may appear elsewhere (typically in the WINDOWS directory). Its size depends on how much swap space the system has (a setting selected by the user under [[Windows Control Panel|Control Panel]] → Enhanced under "Virtual Memory".) If the user moves or deletes this file, a [[Blue Screen of Death|blue screen]] will appear the next time Windows is started, with the [[error message]] "The permanent swap file is corrupt". The user will be prompted to choose whether or not to delete the file (whether or not it exists). [[Windows 95]] uses a similar file, and the settings for it are located under Control Panel → System → Performance tab → Virtual Memory. Windows automatically sets the size of the page file to start at 1.5&times; the size of physical memory, and expand up to 3&times; physical memory if necessary. If a user runs memory-intensive applications on a system with low physical memory, it is preferable to manually set these sizes to a value higher than default. ==Paging in Windows NT line== In NT-based versions of Windows (such as [[Windows 2000]] and [[Windows XP]]), the file used for paging is named <tt>pagefile.sys</tt>. The default location of the page file is in the root directory of the partition where Windows is installed. Windows can be configured to use free space on any available drives for page files. ===Fragmentation of the page file=== Occasionally, when the page file is gradually expanded, it can become heavily [[file system fragmentation|fragmented]] and cause performance problems. The common advice given to avoid this problem is to set a single "locked" page file size so that Windows will not resize it. However, the page file only expands when it has been filled, which, in its default configuration, is 150% the total amount of physical memory.<ref>{{cite web | url=http://support.microsoft.com/kb/889654 | title=How to determine the appropriate page file size for 64-bit versions of Windows Server 2003 or Windows XP (MSKB889654_ | work=Knowledge Base | publisher=Microsoft | date=[[November 7]] [[2007]] | accessdate=2007-12-26 }}</ref> Thus the total demand for pagefile-backed virtual memory must exceed 250% of the computer's physical memory before the page file will expand. Locking a page file's size can be problematic in the case that a Windows application requests more memory than the total size of physical memory and the page file. In this case, requests to allocate memory fail, which may cause the programs making the request (including system processes) to fail. Supporters of this view will note that the page file is rarely read or written in sequential order, so the performance advantage of having a completely sequential page file is minimal. However, it is generally agreed that a large page file will allow use of memory-heavy applications, and there is no penalty except that more disk space is used. [[Defragment]]ing the page file is also occasionally recommended to improve performance when a Windows system is chronically using much more memory than its total physical memory. Although this can help slightly, performance concerns are much more effectively dealt with by adding more physical memory. ==Swapping in Linux== In [[Linux]], as in most other [[Unix-like]] operating systems, it is common to use a whole partition of a hard disk for swapping. However, with the 2.6 Linux kernel, swap files are just as fast<ref>[http://lkml.org/lkml/2006/5/29/3 LKML: "Jesper Juhl": Re: How to send a break? - dump from frozen 64bit linux<!-- Bot generated title -->]</ref><ref>[http://lkml.org/lkml/2005/7/7/326 LKML: Andrew Morton: Re: Swap partition vs swap file]</ref> as swap partitions, although Red Hat recommends using a swap partition.<ref>http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin-guide/ch-swapspace.html</ref> The administrative flexibility of swap files outweighs that of partitions; since modern high capacity hard drives can remap physical sectors, no partition is guaranteed to be contiguous. Linux supports using a virtually unlimited number of swapping devices, each of which can be assigned a priority. When the operating system needs to swap pages out of physical memory, it uses the highest-priority device with free space. If multiple devices are assigned the same priority, they are used in a fashion similar to level 0 [[Redundant array of independent disks|RAID]] arrangements. This provides improved performance as long as the devices can be accessed efficiently in parallel. Therefore, care should be taken assigning the priorities. For example, swaps located on the same physical disk should not be used in parallel, but in order ranging from the fastest to the slowest (i.e.: the fastest having the highest priority). Recently, some experimental improvement to the 2.6 [[Linux kernel]] have been made by [[Con Kolivas]], published in his popular [[-ck patchset]]<ref>http://kernel.kolivas.org Con Kolivas' 2.6 Linux Kernel patchset</ref>. The improvement, called "swap prefetch", employs a mechanism of prefetching previously swapped pages back to physical memory even before they are actually needed, as long as the system is relatively idle (so as not to impair performance) and there is available physical memory to use. This applies to a situation when a "heavy" application has been temporarily used, causing other processes to swap out. After it is closed, both freeing large areas of memory and reducing disk load, prefetch of other processes starts, reducing their initial user response time. <ref>http://ck.wikia.com/wiki/SwapPrefetch SwapPrefetch description on ck kernel wiki. Retrieved 18-09-2007.</ref> == Tuning paging performance == The backing store for a virtual memory operating system is typically many orders of [[magnitude (mathematics)|magnitude]] slower than [[random access memory|RAM]]. Therefore it is desirable to reduce or eliminate swapping, where practical. Some operating systems offer settings to influence the kernel's decisions. # Linux offers the <code>/proc/sys/vm/swappiness</code> parameter, which changes the balance between swapping out runtime memory, as opposed to dropping pages from the system page cache. # Windows 2000, XP, and Vista offer the <code>DisablePagingExecutive</code> registry setting, which controls whether kernel-mode code and data can be eligible for paging out. In [[AIX]] to improve performance you can use more than one device as paging space, since version AIX 4.3 these paging space devices can be added or removed without rebooting the system. == Tuning swap space size == In some older virtual memory operating systems, space in swap backing store is reserved when programs allocate memory for runtime data. OS vendors typically issue guidelines about how much swap space should be allocated. Between 1.5 or 2 times the installed RAM is a typical number <ref>http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin-guide/ch-swapspace.html</ref>. With a large amount of RAM, the disk space needed for the backing store can be very large. Newer versions of these operating systems attempt to solve this problem: for example, some [[HP-UX]] kernels offer a tunable <code>swapmem_on</code> that controls whether RAM can be used for memory reservations. In systems with sufficient RAM, this significantly reduces the needed space allocation for the backing store. == See also == * [[Physical memory]], a subject of paging * [[Virtual memory]], an abstraction that paging may create * [[Demand paging]], a "lazy" paging scheme * [[Page cache]], a disk cache that utilizes virtual memory mechanism * [[Page replacement algorithm]] * [[Segmentation (memory)]] * [[Page size]] * [[Page table]] * [[Memory allocation]] == References == {{Reflist}} == External links == *[http://computer.howstuffworks.com/virtual-memory.htm How Virtual Memory Works] from HowStuffWorks.com (in fact explains only swapping concept, and not virtual memory concept) *[http://www.techarp.com/showarticle.aspx?artno=143 Guide On Optimizing Virtual Memory Speed] *[http://www.rojakpot.com/showarticle.aspx?artno=143&pgno=1 Swap File vs. Paging File] (Hint: From Windows 95 onwards, all Windows OS versions use only paging files.) *[http://am.xs4all.nl/phpwiki/index.php/swap Linux swap partition] *[http://www.faqs.org/docs/linux_admin/x1762.html Linux swap space management] *[http://www.pcnineoneone.com/howto/swpfile1.html Windows swap file management] *[http://people.msoe.edu/~mccrawt/resume/papers/CS384/mccrawt_cs384_virtual.pdf Virtual Memory Page Replacement Algorithms] *[http://support.microsoft.com/kb/308417/ Windows XP. How to manually change the size of the virtual memory paging file] *[http://support.microsoft.com/?id=312362 Windows XP. Factors that may deplete the supply of paged pool memory] *[http://www.acc.umu.se/~bosse/ SwapFs] driver that can be used to save the paging file of Windows on a swap partition of Linux. {{Operating System}} [[Category:Virtual memory]] [[Category:Memory management]] [[ca:Paginació de memòria]] [[cs:Stránkování]] [[de:Paging]] [[et:Lehekülgede saalimine]] [[eu:Orrikatze]] [[fr:Mémoire virtuelle#M.C3.A9moire_virtuelle_pagin.C3.A9e]] [[ko:페이징]] [[it:Paging]] [[ja:ページング方式]] [[pl:Stronicowanie pamięci]] [[pt:Memória paginada]] [[sk:Stránkovanie]]