Sed
27163
225578736
2008-07-14T11:55:03Z
195.176.178.209
link [[Lee E. McMahon]]
{{lowercase|sed}}
{{otheruses}}
'''sed ''' ('''S'''tream '''ED'''itor) refers to a [[Unix]] utility which (a) parses text files and (b) implements a [[programming language]] which can apply textual transformations to such files. It reads input files line by line (sequentially), applying the operation which has been specified via the [[command line]] (or a ''sed script''), and then outputs the line. It was developed from 1973 to 1974 as a [[Unix]] utility by [[Lee E. McMahon]] of [[Bell Labs]]<ref name="urlFrequently-Asked Questions about sed, the stream editor (2.1)">{{cite web |url=http://sed.sourceforge.net/sedfaq2.html#s2.1 |title=Frequently-Asked Questions about sed, the stream editor |format= |work= |accessdate=2008-05-13}}</ref>, and is available today for most operating systems.<ref name="urlFrequently-Asked Questions about sed, the stream editor (2.2)">{{cite web |url=http://sed.sourceforge.net/sedfaq2.html#s2.2 |title=Frequently-Asked Questions about sed, the stream editor |format= |work= |accessdate=2008-05-13}}</ref>
==Usage==
The following example shows a typical use of sed, where the ''-e'' option indicates that the sed expression follows:
sed -e 's/oldstuff/newstuff/g' inputFileName > outputFileName
In many versions, the ''-e'' is not required to precede the expression. The ''s'' stands for substitute. The ''g'' stands for global, which means that all matching occurrences in the line would be replaced. The [[regular expression]] (i.e. pattern) to be searched is placed after the first delimiting symbol (slash here) and the replacement follows the second symbol. Slash is the conventional symbol. Any other could be used to make syntax more readable if it does not occur in the pattern or replacement ([[Sed#The_context_address|see below]]).
Under Unix, sed is often used as a [[filter (Unix)|filter]] in a [[pipeline (Unix)|pipeline]]:
generate_data | sed -e 's/x/y/g'
That is, generate the data, and then make the small change of replacing ''x'' with ''y''.
Several substitutions or other commands can be put together in a file called, for example, ''subst.sed'' and then be applied using the ''-f'' option to read the commands from the file:
sed -f subst.sed inputFileName > outputFileName
Besides substitution, other forms of simple processing are possible. For example, the following uses the ''d'' command to delete lines that are either blank or only contain spaces:
sed -e '/^ *$/d' inputFileName
This example used some of the following [[regular expression]] [[metacharacter]]s:
* The [[caret]] (<code>^</code>) matches the beginning of the line.
* The [[dollar sign]] (<code>$</code>) matches the end of the line.
* A [[Full stop|period]] (<code>.</code>) matches any single character.
* The [[asterisk]] (<code>*</code>) matches zero or more occurrences of the previous character.
* A [[bracket]]ed expression delimited by <code>[</code> and <code>]</code> matches any of the characters inside the brackets.
Complex sed constructs are possible, allowing it to serve as a simple, but highly specialised, [[programming language]]. Flow of control, for example, can be managed by the use of a [[Label (programming language)|label]] (a colon followed by a string) and the branch instruction ''b''. An instruction ''b'' followed by a valid label name will move processing to the block following that label. If the label does not exist then the branch will end the script.
==History==
sed is one of the very early Unix commands built for command line processing of data files. It evolved as the natural successor to the popular [[grep]] command.<ref name="tur_paper">
{{cite web
| title = On the Early History and Impact of Unix
| url = http://www.columbia.edu/~rh120/ch001j.c11
}}
</ref> Cousin to the later [[AWK programming language|AWK]], sed allowed powerful and interesting data processing to be done by shell scripts.
sed and AWK are often cited as the progenitors and inspiration for [[Perl]]. The ''s///'' syntax shown above is part of Perl's syntax and originated with [[ed (UNIX)|ed]], the precursor to sed.
sed's language does not have variables and has only primitive [[GOTO]] and branching functionality; nevertheless, the language is [[Turing-complete]].<ref name="tur_paper">
{{cite web
| title = Implementation of a Turing Machine as Sed Script
| url = http://sed.sourceforge.net/grabbag/scripts/turing.txt
}}
</ref>
<ref name="tur_sed">
{{cite web
| title = turing.sed
| url = http://sed.sourceforge.net/grabbag/scripts/turing.sed
}}
</ref>
[[GNU]] sed includes several new features such as in-place editing of files (i.e., replace the original file with the result of applying the sed program). In-place editing is often used instead of [[ed (UNIX)|ed]] scripts: for example,
sed -i 's/abc/def/' file
can be used instead of
ed file
1,$ s/abc/def/
w
q
''Super-sed'' is an extended version of sed that includes regular expressions compatible with [[Perl]].
Another variant of sed is ''minised'', originally reverse-engineered from the 4.1BSD sed by [[Eric S. Raymond]] and currently maintained by René Rebe. minised was used by the [[GNU project]] until the GNU project wrote a new version of sed based on the new GNU regular expression library. The current minised contains some extensions to BSD sed but is not as feature-rich as GNU sed. Its advantage is that it is very fast and uses little memory.{{Fact|date=May 2008}} It is used on embedded systems and is the version of sed provided with [[Minix]].{{Fact|date=May 2008}}
== Samples ==
In this example, sed, which usually only works on one line, removes newlines from sentences where the second sentence starts with one space.
Consider the following text:
This is my cat
my cat's name is betty
This is my dog
my dog's name is frank
The sed script below will turn it into:
This is my cat my cat's name is betty
This is my dog my dog's name is frank
Here's the script:
sed 'N;s/\n / /;P;D;'
* (N) add the next line to the work buffer
* (s) substitute
* (/\n /) match: \n and one space
* (/ /) replace with: one space
* (P) print the top line of the work buffer
* (D) delete the top line from the work buffer and run the script again
===The context address===
Conditioned commands are possible using the "context address":
/addr-pattern/s/search-pattern/replacement/flags
or more readable variants:
/addr-pattern/s~search-pattern~replacement~flags
/addr-pattern/s,search-pattern,replacement,flags
Here, 'addr-pattern' is the address, which defines lines, where substitution of 'search-pattern' with 'replacement' will be done.
Likewise:
/addr-pattern/!s,search-pattern,replacement,flags
substitute will be executed if '''no''' 'addr-pattern' was matched.
For example, replace "world" with "mom", but only on lines which contain the word "you":
sh$ sed -e '/you/s,world,mom,g' <<" EOF"
Hello world.
Hello world. I love you.
EOF
Hello world.
Hello mom. I love you.
sh$
Negation will be:
sh$ sed -e '/you/!s,world,mom,g' <<" EOF"
Hello world.
Hello world. I love you.
EOF
Hello mom.
Hello world. I love you.
sh$
== Exotic examples ==
Despite the inherent limitations, sed scripts exist for games as [[sokoban]] or [[arkanoid]].<ref name="ark">
{{cite web
| url = http://sed.sourceforge.net/#gamez
| title = sed.sf.net - The SED $HOME
}}
</ref>
== Further reading ==
*{{cite book|title=sed & awk|edition=2nd Edition|month=March|year=1997|author=Dale Dougherty & Arnold Robbins|publisher=O'Reilly|id=ISBN 1-56592-225-5}}
*{{cite book|title=sed and awk Pocket Reference|edition=2nd Edition|month=June|year=2002|author=Arnold Robbins|publisher=[[O'Reilly and Associates]]|id=ISBN 0-596-00352-8}}
*{{cite book|title=UNIX AWK and SED Programmer's Interactive Workbook (UNIX Interactive Workbook)|author=Peter Patsis|publisher=[[Prentice Hall]]|year=[[1998-12-30]]|id=ISBN 0-13-082675-8}}
<!--*[http://www.cs.hmc.edu/qref/sed.html A very basic introduction]-->
*[http://sed.sourceforge.net/sedfaq.html The sed FAQ]
*[http://www.gnu.org/software/sed/manual/sed.html GNU sed manual]
==References==
<references/>
==See also==
*[[List of Unix programs]]
==External links==
* [http://sed.sourceforge.net Major sources for sed scripts, files, usage]
* [http://sed.sourceforge.net/sed1line.txt Handy one-line sed scripts]
* [http://sed.sourceforge.net/grabbag/scripts/ Sed script archive]
* [http://www.pement.org/sed/ A home page for sed, some focus on Windows/DOS]
* [http://www.npcguild.org/~ksb/hack/math.sed A calculator written in sed]
* [http://www.gnu.org/directory/text/editors/super-sed.html Super-sed]
* [http://www.gnulamp.com/sed.html sed Tutorial]
* [http://www.grymoire.com/Unix/Sed.html The sed tutorial from Grymoire]
* [http://unxutils.sourceforge.net GNU utilities for Win32]
* [http://users.cybercity.dk/~bse26236/batutil/help/SED.HTM#11.20 More on the address command and sub-matched replacements]
* [http://www.exactcode.de/oss/minised/ Minised homepage]
* [http://www.pcre.org/ PCRE - Perl Compatible Regular Expressions]
* [http://www.computerworld.com.au/index.php/id;1726534212;fp;4;fpid;611908207 The A-Z of Programming Languages: AWK]
* [http://www.ibm.com/developerworks/linux/library/l-sed1.html Common threads: Sed by example, Part 1] by Daniel Robbins
* [http://www.ibm.com/developerworks/linux/library/l-sed2.html Common threads: Sed by example, Part 2] by Daniel Robbins
* [http://www.ibm.com/developerworks/linux/library/l-sed3.html Common threads: Sed by example, Part 3] by Daniel Robbins
{{Unix commands}}
[[Category:Domain-specific programming languages]]
[[Category:Text-oriented programming languages]]
[[Category:Scripting languages]]
[[Category:Unix software]]
[[Category:Free compilers and interpreters]]
[[als:Sed]]
[[cs:Sed]]
[[de:Sed (Unix)]]
[[es:Sed (informática)]]
[[fr:Sed (logiciel)]]
[[nl:Stream Editor]]
[[ja:Sed (コンピュータ)]]
[[pl:Sed (program)]]
[[pt:Sed]]
[[ro:Sed]]
[[ru:Sed]]
[[fi:Sed]]