Simple, yes, but it took me a few minutes on Google to remember the awk syntax. Unix is powerful, but it isn't intuitive.<br><br>David<br><br><div><span class="gmail_quote">On 8/11/06, <b class="gmail_sendername">maning sambale
</b> <<a href="mailto:emmanuel.sambale@gmail.com">emmanuel.sambale@gmail.com</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
David,<br><br>thank you! that's the one I need (for now). Funny how simple it is.<br>Another funny anecdote, about a year ago I passed by our GIS lab and<br>saw a girl editing very large ascii file (mouse click, edit, edit,
<br>save, next line) much the same as my files I'm manipulating right now.<br> I asked her there might be a better way in doing this. She said it's<br>the only way her instructor and the lab technician thought them. :)<br>
<br>cheers,<br><br>Maning<br><br>On 8/10/06, David Finlayson <<a href="mailto:david.p.finlayson@gmail.com">david.p.finlayson@gmail.com</a>> wrote:<br>> Try this to print column 1 and 3. I think it will work on all of your files
<br>> no matter how many spaces are in between:<br>><br>> cat file | awk '{print $1, $3}'<br>><br>> David<br>><br>><br>> On 8/9/06, maning sambale <<a href="mailto:emmanuel.sambale@gmail.com">emmanuel.sambale@gmail.com
</a>> wrote:<br>> > David & Kevin,<br>> ><br>> > Yes, python or perl would be great. But what I need right now is a<br>> > quick (maybe dirty) approach. I do intend to study python as I've
<br>> > heard a lot about it. But not this time, I'm trying to study Linux<br>> > tools the "modular way":)<br>> ><br>> > Cheers,<br>> ><br>> > Maning<br>> ><br>> > On 8/8/06, Slover, Kevin <
<a href="mailto:kslover@dewberry.com">kslover@dewberry.com</a>> wrote:<br>> > > Maning,<br>> > > As David says, python or perl are used now for manipulating text<br>> > > files. I have done several quick scripts for doing this with Perl
<br>> > > (thanks David for the python script, bout time I learn to use it). A<br>> > > basic perl script would look like this (and note, my Perl is not great,<br>> > > and am sure there are many other ways to do this) :
<br>> > ><br>> > > Explanation: Files.txt is a ls/dir listing of the wanted files to<br>> > > combine. Then, the script reads in each file, stripping any sort of<br>> > > header information from the columns, and outputting everything into one
<br>> > > file. Fairly simple, and a quick search on the web for file<br>> > > manipulation using Perl will come up with probably a better explanation.<br>> > ><br>> > > $in_file = "
Files.txt";<br>> > > $out_file = "outfile.txt";<br>> > ><br>> > > open (INFILE, $in_file) || die "INFILE";<br>> > > open (OUTFILE, ">$out_file") || die "OUTFILE";
<br>> > ><br>> > > @infiles = <INFILE>;<br>> > > close(INFILE);<br>> > ><br>> > > print OUTFILE "z,x,y\n";<br>> > ><br>> > > foreach $in_files (@infiles)
<br>> > > {<br>> > ><br>> > > open (INFILE1, $in_files) || die "Cannot open $in_files";<br>> > > while (<INFILE1>)<br>> > > {<br>> > >
<br>> > > chomp($_);<br>> > > ($x, $y, $z) = split ',',$_;<br>> > ><br>> > > if ($x != x) {<br>> > > print OUTFILE "$z,$x,$y\n"; }
<br>> > > }<br>> > ><br>> > > close(INFILE1);<br>> > > }<br>> > ><br>> > > close(OUTFILE);<br>> > ><br>> > ><br>> > > Kevin Slover
<br>> > > Coastal / GIS Specialist<br>> > > 2872 Woodcock Blvd Suite 230<br>> > > Atlanta GA 30341<br>> > > (P) 678-530-0022<br>> > > (F) 678-530-0044<br>> > ><br>> > > -----Original Message-----
<br>> > > From: <a href="mailto:grassuser-bounces@grass.itc.it">grassuser-bounces@grass.itc.it</a><br>> > > [mailto: <a href="mailto:grassuser-bounces@grass.itc.it">grassuser-bounces@grass.itc.it</a>] On Behalf Of maning sambale
<br>> > > Sent: Tuesday, August 08, 2006 12:12 AM<br>> > > To: <a href="mailto:grassuser@grass.itc.it">grassuser@grass.itc.it</a><br>> > > Subject: [GRASS-user] combine & edit multiple text files
<br>> > ><br>> > > Hi!<br>> > ><br>> > > I have a number of ascii files downloaded from ASTR fire project from<br>> > > the ESA Ionia showing monthly fire incidences from 1996-2006. I
<br>> > > intend to combine all these files, remove unwanted columns and get the<br>> > > records from my current region/study area only. All records combined<br>> > > is 929,155 records! My guess is I need to use the cat, cut, awk
<br>> > > commands.<br>> > ><br>> > > Challenge: the files have different record formating<br>> > ><br>> > > file 1 is like this (take note of the space as the delimiter):<br>> > >
<br>> > > Date Time Lat Lon NDVI Station<br>> > > 020201 032428.163 -38.379 -66.334 -.-- ESR<br>> > > 020201 032428.163 -38.375 -66.323 -.-- ESR<br>> > > 020201
032428.312 -38.378 -66.359 -.-- ESR<br>> > > 020201 032428.312 -38.374 -66.348 -.-- ESR<br>> > > 020201 032428.312 -38.371 -66.337 -.-- ESR<br>> > ><br>> > > file 2 looks like this:
<br>> > > Date Orbit Time Lat<br>> Lon<br>> > > 20030101 4384 81704.016 19.364 -155.103<br>> > > 20030101 4384 81704.164 19.373
-155.105<br>> > > 20030101 4384 81704.164 19.375 -155.096<br>> > > 20030101 4385 100833.648 56.638 161.281<br>> > > 20030101 4386 130756.352
-20.340 134.099<br>> > ><br>> > > I only need the columns for date, time, lat, lon<br>> > ><br>> > > Here's what I did:<br>> > ><br>> > > #combine all file (monthly)
<br>> > > cat 9904ESA01.FIRE 9905ESA01.FIRE 9906ESA01.FIRE 9907ESA01.FIRE<br>> > > 9908ESA01.FIRE ... > test<br>> > ><br>> > > # cut only desired columns (1_4) delimeiter is spac ' '
<br>> > > cut -d' ' -f1 test > 1<br>> > > cut -d' ' -f2 test > 2<br>> > > cut -d' ' -f3 test > 3<br>> > > cut -d' ' -f4 test > 4<br>> > ><br>> > > # combine all columns
<br>> > > paste 1 2 3 4 > test5<br>> > ><br>> > > example output:<br>> > ><br>> > > 021231 223941.761 11.035 -5.016 -.-- ESR<br>> > > 021231 224005.303 12.226
-6.243 -.-- ESR<br>> > > 20030101 4380 25934.057 -37.022 -69.589<br>> > > 20030101 4382 45951.090 33.005 -110.772<br>> > ><br>> > > The problem is for the file example 1, lat and lon columns contain
<br>> > > spaces other than the delimiter example " -38.00" while another is<br>> > > "120.00" In the file2 example, more spaces are there. I think I need<br>> > > to process different file formats separately but how do I solve the
<br>> > > problem for spaces in the lat/lon columns?<br>> > ><br>> > > One last question how do I get the records for my current region only?<br>> > ><br>> > > north: 20:00:
01.49976N<br>> > > south: 5:00:01.499767N<br>> > > west: 115:00:01.5012E<br>> > > east: 130:00:01.501193E<br>> > ><br>> > ><br>> > > I'm starting to understand awk (reading the gawk manual right now) but
<br>> > > may take a while to get do something magical.<br>> > ><br>> > > Thanks!<br>> > ><br>> > > Maning<br>> > ><br>> > > --<br>> > ><br>> |---------|----------------------------------------------------------|
<br>> > > | __.-._ |"Ohhh. Great warrior. Wars not make one great." -Yoda |<br>> > > | '-._"7' |"Freedom is still the most radical idea of all" -N.Branden|<br>> > > | /'.-c |Linux registered user #402901,
<a href="http://counter.li.org/">http://counter.li.org/</a> |<br>> > > | | /T |http://esambale.wikispaces.com|<br>> > > | _)_/LI<br>> > ><br>> |http://www.geocities.com/esambale/philbiodivmap/philbirds.html
<br>> |<br>> > ><br>> |---------|----------------------------------------------------------|<br>> > ><br>> > > _______________________________________________<br>> > > grassuser mailing list
<br>> > > <a href="mailto:grassuser@grass.itc.it">grassuser@grass.itc.it</a><br>> > > <a href="http://grass.itc.it/mailman/listinfo/grassuser">http://grass.itc.it/mailman/listinfo/grassuser</a><br>> > >
<br>> > ><br>> ><br>> ><br>> > --<br>> ><br>> |---------|----------------------------------------------------------|<br>> > | __.-._ |"Ohhh. Great warrior. Wars not make one great." -Yoda |
<br>> > | '-._"7' |"Freedom is still the most radical idea of all" - N.Branden|<br>> > | /'.-c |Linux registered user #402901, <a href="http://counter.li.org/">http://counter.li.org/</a> |
<br>> > | | /T |http://esambale.wikispaces.com|<br>> > | _)_/LI<br>> |http://www.geocities.com/esambale/philbiodivmap/philbirds.html<br>> |<br>> ><br>> |---------|----------------------------------------------------------|
<br>> ><br>> > _______________________________________________<br>> > grassuser mailing list<br>> > <a href="mailto:grassuser@grass.itc.it">grassuser@grass.itc.it</a><br>> > <a href="http://grass.itc.it/mailman/listinfo/grassuser">
http://grass.itc.it/mailman/listinfo/grassuser</a><br>> ><br>><br>><br>><br>> --<br>> David Finlayson<br><br><br>--<br>|---------|----------------------------------------------------------|<br>| __.-._ |"Ohhh. Great warrior. Wars not make one great." -Yoda |
<br>| '-._"7' |"Freedom is still the most radical idea of all" -N.Branden|<br>| /'.-c |Linux registered user #402901, <a href="http://counter.li.org/">http://counter.li.org/</a> |<br>| | /T |http://esambale.wikispaces.com|
<br>| _)_/LI |http://www.geocities.com/esambale/philbiodivmap/philbirds.html |<br>|---------|----------------------------------------------------------|<br><br>_______________________________________________<br>grassuser mailing list
<br><a href="mailto:grassuser@grass.itc.it">grassuser@grass.itc.it</a><br><a href="http://grass.itc.it/mailman/listinfo/grassuser">http://grass.itc.it/mailman/listinfo/grassuser</a><br></blockquote></div><br><br clear="all">
<br>-- <br>David Finlayson