{"id":197,"date":"2020-08-18T19:23:47","date_gmt":"2020-08-18T20:23:47","guid":{"rendered":"http:\/\/www.linux-tutorial.info\/?page_id=77"},"modified":"2020-08-22T19:26:16","modified_gmt":"2020-08-22T20:26:16","slug":"this-is-the-page-title-toplevel-31","status":"publish","type":"page","link":"http:\/\/www.linux-tutorial.info\/?page_id=197","title":{"rendered":"Looking Through Files"},"content":{"rendered":"\nLooking Through Files<\/title>\n<p>\nIn the section on <tutorial id=31>looking for files<\/tutorial>, we talk about\nvarious methods for finding a particular file on your system.\nLet’s assume for a moment that we were looking for a particular file, so we used the find\ncommand to look for a specific file name, but none of the commands we issued came up with a\nmatching file. There was not a single match of any kind. This might mean that we\nremoved the file. On the other hand, we might have named it yacht.txt or\nsomething similar. What can we do to find it?\n<\/p>\n<question id=\"71\" text=\"What command would be best if you were looking for specific text within a file?\" \/>\n<concept id=\"103\" description=\"The grep command can be used to look for specific text within a file.\" \/>\n<p>\nWe could jump through the same hoops for using various spelling and letter combinations,\nsuch as we did for yacht and boat. However,\nwhat if the customer had a canoe or a junk? Are we stuck with every possible\nword for boat? Yes, unless we know something about the file, even if that\nsomething is in the file.\n<\/p>\n<p>\nThe nice thing is that grep doesn’t have to be the end of a <glossary>pipe<\/glossary>.\nOne of the arguments can be the name of a file. If you want, you can use several files,\nbecause grep will take the first <glossary>argument<\/glossary> as the pattern it should\nlook for. If we were to enter\n<\/p>\n<question id=\"72\" text=\"The grep command can use metacharacters to search for text in files.\" \/>\n<p>\n<commandexample command=\"grep\">grep [Bb]oat .\/letters\/taxes\/* <\/commandexample>\n<\/p>\n<p>\nwe would search the contents of all the files in the directory\n<directory>.\/letters\/taxes<\/directory> looking for the word Boat or boat.\n<\/p>\n<p>\nIf the file we were looking for happened to be in the directory\n<directory>.\/letters\/taxes<\/directory>, then all we would need to do is run <command>more<\/command> on the file. If things are\nlike the examples above, where we have dozens of directories to look through,\nthis is impractical. So, we turn back to find.\n<\/p>\n<p>\nOne useful option to find is <keyinput>-exec<keyinput>. When a file is found, you use -exec to\nexecute a command. We can therefore use find to find the files, then use -exec\nto run grep on them. Still, you might be asking yourself what good this is to you.\nBecause you probably don’t have dozens of files on your\nsystem related to taxes, let’s use an example from files that you most probably\nhave.\n<\/p>\n<p>\nLet’s find all the files in the <directory>\/etc<\/directory> directory containing <command>\/bin\/sh<\/command>. This would be run as\n<\/p>\n<p>\n<commandexample command=\"grep\">find .\/etc -exec grep \/bin\/sh {} \\;<\/commandexample>\n<\/p>\n<p>\nThe curly braces ({ }) are substituted for the file found by the search, so\nthe actual grep command would be something like\n<\/p>\n<p>\n<commandexample command=\"grep\">grep \/bin\/sh .\/etc\/filename<\/commandexample>\n<\/p>\n<p>\nThe “\\;” is a flag saying that this is the end of the command.\n<\/p>\n<p>\nWhat the find command does is search for all the files that match the\nspecified criteria then run grep on the criteria, searching for the pattern [Bb]oat.\n(in this case there were no criteria, so it found them all)\n<\/p>\n<p>\nDo you know what this tells us? It says that there is a file somewhere under\nthe directory .\/letters\/taxes that contains either “boat” or “Boat.” It doesn’t\ntell me what the file name is because of the way the -exec is handled. Each file\nname is handed off one at a time, replacing the {}. It is as though we had\nentered individual lines for\n<\/p>\n<p>\n<commandexample command=\"grep\">grep [Bb]oat .\/letters\/taxes\/file1<\/commandexample><br \/>\n<commandexample command=\"grep\">grep [Bb]oat .\/letters\/taxes\/file2<\/commandexample><br \/>\n<commandexample command=\"grep\">grep [Bb]oat .\/letters\/taxes\/file3<\/commandexample><br \/>\n<p>\nIf we had entered\n<\/p>\n<p>\n<commandexample command=\"grep\">grep [Bb]oat .\/letters\/taxes\/*<\/commandexample>\n<\/p>\n<p>\ngrep would have output the name of the file in front of each matching line\nit found. However, because each line is treated separately when using\n<command>find<\/command>, we don’t see the file names. We could use the <keyinput>-l<\/keyinput>\noption to\n<command>grep<\/command>, but that would\nonly give us the file name. That might be okay if there was one or two files.\nHowever, if a line in a file mentioned a “boat trip” or a “boat trailer,” these\nmight not be what we were looking for. If we used the <keyinput>-l<\/keyinput> option to grep, we\nwouldn’t see the actual line. It’s a catch-22.\n<\/p>\n<p>\nTo get what we need, we must introduce a new command: xargs. By using it as\none end of a <glossary>pipe<\/glossary>,\n you can repeat the same command on different files without\nactually having to input the command multiple times.\n<\/p>\n<p>\nIn this case, we would get what we wanted by typing\n<\/p>\n<p>\n<commandexample command=\"grep\">find .\/letters\/taxes -print | xargs grep [Bb]oat<\/commandexample>\n<\/p>\n<p>\nThe first part is the same as we talked about earlier. The find command\nsimply prints all the names it finds (all of them, in this case, because there\nwere no search criteria) and passes them to xargs. Next, xargs processes them\none at a time and creates commands using grep. However, unlike the -exec option\nto find, xargs will output the name of the file before each matching line.\n<\/p>\n<p>\nObviously, this example does not find those instances where the file we were\nlooking for contained words like “yacht” or “canoe” instead of “boat.”\nUnfortunately, the only way to catch all possibilities is to actually specify\neach one. So, that’s what we might do. Rather than listing the different possible\nsynonyms for boat, lets just take the three: boat, yacht, and canoe.\n<\/p>\n<p>\nTo do this, we need to run the <keyinput>find | xargs<\/keyinput> command three times. However,\nrather than typing in the command each time, we are going to take advantage of a\nuseful aspect of the <glossary>shell<\/glossary>.\n In some instances, the shell knows when you want to\ncontinue with a command and gives you a secondary prompt. If you are running sh\nor ksh, then this is probably denoted as “>.”\n<\/p>\n<p>\nFor example, if we typed\n<\/p>\n<p>\n<commandexample command=\"find\">find .\/letters\/taxes -print |<\/commandexample>\n<\/p>\n<p>\nthe <glossary>shell<\/glossary> knows that the <glossary>pipe<\/glossary>\n(|) cannot be at the end of the line. It then\ngives us a > or ? prompt where we can continue typing\n<\/p>\n<p>\n<commandexample>> xargs grep -i boat<\/commandexample>\n<\/p>\n<p>\nThe <glossary>shell<\/glossary>\ninterprets these two lines as if we had typed them all on the same\nline. We can use this with a <glossary>shell<\/glossary>\nconstruct that lets us do loops. This is the\nfor\/in construct for sh and ksh, and the foreach construct in csh. It would look\nlike this:\n<\/p>\n<p>\n<commandexample command=\"bash\">for j in boat ship yacht\n> do\n> find .\/letters\/taxes -print | xargs grep -i $j\n> done\n<\/commandexample>\n<p>\nIn this case, we are using the <glossary>variable<\/glossary> j, although we could have called it\nanything we wanted. When we put together quick little commands, we save\nourselves a little typing by using single letter variables.\n<\/p>\n<p>\nIn the <command>bash<\/command>\/<command>sh<\/command>\/<command>ksh<\/command> example, we need to enclose the body of the loop inside the\ndo-done pair. In the <command>csh<\/command> example, we need to include the end. In both cases,\nthis little command we have written will loop through three times. Each time,\nthe <glossary>variable<\/glossary>\n$j is replaced with one of the three words that we used. If we had\nthought up another dozen or so synonyms for boat, then we could have included\nthem all. Remember also that the <glossary>shell<\/glossary>\nknows that the <glossary>pipe<\/glossary>\n(|) is not the end of\nthe command, so this would work as well.\n<\/p>\n<p>\n<commandexample command=\"bash\">\nfor j in boat ship yacht\n> do\n> find .\/letters\/taxes -print |\n> xargs grep -i $j\n> done\n<\/commandexample>\n<p>\nDoing this from the <glossary>command line<\/glossary>\nhas a drawback. If we want to use the same\ncommand again, we need to retype everything. However, using another trick, we\ncan save the command. Remember that both the ksh and csh have history mechanisms\nto allow you to repeat and edit commands that you recently edited. However, what\nhappens tomorrow when you want to run the command again? Granted, ksh has the\n<file type=\"\">.sh_history<\/file> file, but what about sh and csh?\n<\/p>\n<p>\nWhy not save commands that we use often in a file that we have all the time?\nTo do this, you would create a basic shell script, and we have a\n<site id=33>whole section <\/site>just on that topic.\n<p>\nWhen looking through files, I am often confronted with the situation where I am\nnot just looking for a single text, but possible multiple matches. Imagine a\ndata file that contains a list of machines and their various characteristics,\neach on a separate line, which starts with that characteristic. For example: <p>\n<screenoutput>\nName: lin-db-01\nIP: 192.168.22.10\nMake: HP\nCPU: 2300\nRAM: 512\nLocation: Room 3\n<\/screenoutput>\nAll I want is the computer name, the IP address and the location, but not the\nothers. I could do three individual greps, each with a different pattern.\nHowever, it would be difficult to make the association between the separate\nentries. That is, the first time I would have a list of machine’s names, the\nsecond time a list of <glossary>IP<\/glossary> addresses and the third time a list of locations. I\nhave written scripts before that handle this kind of situation, but in this case\nit would be easier to use a standard Linux command: <command>egrep<\/command>.\n<\/p>\n<p>\nThe <command>egrep<\/command> command\nis an extension of the basic <command>grep<\/command> command. (The ‘e’ stands for extended) In\nolder versions of <command>grep<\/command>, you did not have the ability to use things like\n<keyinput>[:alpha:]<keyinput> to represent alphabetic characters, so extended grep was born. For\ndetails on representing characters like this check out the section in\n<tutorial id=19>regular expressions and metacharacters<\/tutorial>.\n<\/p>\n<p>\nOne extension is the\nability to have multiple search patterns that are checked simultaneously. That\nis, if any of the patterns are found, the line is displayed. So in the problem\nabove we might have a command like this:\n<\/p>\n<question id=\"463\" text=\"What command would you use to look for either the name Jim or Chuck in a file?\" \/>\n<p>\n<commandexample command=\"grep\">egrep “Name:|IP:|Location:” FILENAME<\/commandexample>\n<p>\nThis would then list all of the respective lines <i>in order<\/i>, making\nassociation between name and the other values a piece of cake.\n<p>\nAnother variant of grep is <command>fgrep<\/command>, which interprets the search pattern as a\nlist of <i>fixed<\/i> strings, separated by newlines, any of which is to be\nmatched. On some systems, grep, egrep and fgrep will all be a\n<glossary>hard link<\/glossary> to the same file.\n<\/p>\n<p>\nI am often confronted with files where I\nwant to filter out the “noise”. That is, there is a lot of stuff in the files\nthat I don’t want to see. A common example, is looking through large shell\nscripts or configuration files when I am not sure exactly what I am looking for.\nI know when I see it, but to simply <command>grep<\/command> for that term is impossible, as I am\nnot sure what it is. Therefore, it would be nice to ingore things like comments and\nempty lines.\n<\/p>\n<question id=\"461\" text=\"What command would you use to look for all lines in a file NOT containing a particular phrase?\" \/>\n<p>\nOnce again we could use <command>egrep<\/command> as there are two expressions we\nwant to match. However, this type we also use the <keyinput>-v<keyinput> option, which simply flips\nor inverts the meaning of the match. Let’s say there was a start-up script that\ncontained a variable you were looking for, You might have something like this:\n<p>\n<commandexample command=\"grep\">egrep -v “^$|^#” \/etc\/rc.d\/*|more<\/commandexample>\n<p>\nThe first part of the expressions says to match on the beginning of the line (^)\nfollowed immediately by the end of the line ($), which turn out to be all empty\nlines. The second part of the expression says to match on all lines that start\nwith the pound-sign (a comment). This ends up giving me all of the “interesting”\nlines in the file. The long option is easier to remember: <keyinput>–invert-match<keyinput>.\n<\/p>\n<question id=\"\" text=\"What command would you use to list the names of file containing a specific pattern?\" \/>\n<p>\nYou may also run into a case where all you are interested in is which files\ncontain a particular expression. This is where the <keyinput>-l<keyinput> option comes in (long\nversion: –<keyinput>-files-with-matches<keyinput>). For example, when I made some style changes to\nmy web site I wanted to find all of the files that contained a table. This means\nthe file had to contain the <TABLE> tag. Since this tag could contain\nsome options, I was interested in all of the file which contained “<TABLE”.\nThis could be done like this:\n<\/p>\n<p>\n<commandexample command=\"grep\">grep -l ‘<TABLE’ FILENAME<\/commandexample>\n<\/p>\n<p>\nThere is an important thing to note here. In the section on\n<tutorial id=22>interpreting the command<\/tutorial>, we learn that the shell sets up file\nredirection <i>before<\/i> it tries to execute the command. If we don’t include the\nless-than symbol in the single quotes, the shell will try to redirect the input\nfrom a file name “TABLE”. See the section on\n<tutorial id=20>quotes <\/tutorial>for details on this.\n<\/p>\n<question id=\"462\" text=\"What command would you use to list the file name of files containing a particular phrase?\" \/>\n<p>\nThe -l option (long version:\n<keyinput>–files-with-matches<keyinput>) says to simply list the file names. Using the <keyinput>-L<keyinput> option\n(long version: <keyinput>–files-without-match<keyinput>) we have the same effect as using both the\n<keyinput>-v<keyinput> and the <keyinput>-l<keyinput> options. Note that in both cases, the lines containing the matches\nare <i>not<\/i> displayed, just the file name.\n<p> Another common option is <keyinput>-q<keyinput>\n(long: <keyinput>–quiet<keyinput> or <keyinput>–silent<keyinput>). This does not display anything. So, what’s the use in that? Well, often, you simply want to know if a particular value exists in a file. Regardless of the options you use, grep will return 0 if any matches\nwere found, and 1 if no matches were found. If you check the $? variable after\nrunning <keyinput>grep -q<keyinput>. If it is 0, you found a match. Check out the section on\n<tutorial id=33>basic shell scripting<\/tutorial> for details on the $? and other\nvariables.\n<p> Keep in mind that you do not need to use grep to read through\nfiles. Instead, it can be one end of a pipe. For example, I have a number of\nscripts that look through the process list to see if a particular process is\nrunning. If so, then I know all is well. However, if the process is not running,\na message is sent to the administrators.\n","protected":false},"excerpt":{"rendered":"<p>Looking Through Files In the section on looking for files, we talk about various methods for finding a particular file on your system. Let’s assume for a moment that we were looking for a particular file, so we used the … <a href=\"http:\/\/www.linux-tutorial.info\/?page_id=197\">Continue reading <span class=\"meta-nav\">→<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-197","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/pages\/197","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=197"}],"version-history":[{"count":1,"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/pages\/197\/revisions"}],"predecessor-version":[{"id":614,"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=\/wp\/v2\/pages\/197\/revisions\/614"}],"wp:attachment":[{"href":"http:\/\/www.linux-tutorial.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=197"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}