The modern solution for problems like these is a script language like Perl or Python.<br><br>In Python a simple script for working with columns of data might like like this:<br><br>fin = open(infile)<br>for record in fin:<br>
fields = rec.split() # this part splits the fields on white space<br> date = fields[0] # pick the fields you want<br> time = fields[1]<br><br> ...<br><br> value2 = fields[9]<br><br> print "%f %f %f" % (date, time, value2) # print them to stdout or write to a file
<br><br><br>run the script and capture the output to a file<br>python script.py > bigfile.txt<br><br>I find cut, paste, sed work will for quick jobs (and they would work in your case). But as soon as I need to look up the documentation on sed I have usually reached the point where a Python script would be easier to impliment. For that reason, I never use awk any more.
<br><br>My 2 cents,<br><br>David<br><br> <br><br><br><br><br><div><span class="gmail_quote">On 8/7/06, <b class="gmail_sendername">maning sambale</b> <<a href="mailto:emmanuel.sambale@gmail.com">emmanuel.sambale@gmail.com
</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi!<br><br>I have a number of ascii files downloaded from ASTR fire project from
<br>the ESA Ionia showing monthly fire incidences from 1996-2006. I<br>intend to combine all these files, remove unwanted columns and get the<br>records from my current region/study area only. All records combined<br>is 929,155 records! My guess is I need to use the cat, cut, awk
<br>commands.<br><br>Challenge: the files have different record formating<br><br>file 1 is like this (take note of the space as the delimiter):<br><br>Date Time Lat Lon NDVI Station<br>020201 032428.163
-38.379 -66.334 -.-- ESR<br>020201 032428.163 -38.375 -66.323 -.-- ESR<br>020201 032428.312 -38.378 -66.359 -.-- ESR<br>020201 032428.312 -38.374 -66.348 -.-- ESR<br>020201 032428.312 -38.371 -66.337 -.-- ESR<br>
<br>file 2 looks like this:<br> Date Orbit Time Lat Lon<br> 20030101 4384 81704.016 19.364 -155.103<br> 20030101 4384 81704.164 19.373 -155.105<br> 20030101 4384
81704.164 19.375 -155.096<br> 20030101 4385 100833.648 56.638 161.281<br> 20030101 4386 130756.352 -20.340 134.099<br><br>I only need the columns for date, time, lat, lon<br><br>Here's what I did:
<br><br>#combine all file (monthly)<br>cat 9904ESA01.FIRE 9905ESA01.FIRE 9906ESA01.FIRE 9907ESA01.FIRE<br>9908ESA01.FIRE ... > test<br><br># cut only desired columns (1_4) delimeiter is spac ' '<br>cut -d' ' -f1 test > 1
<br>cut -d' ' -f2 test > 2<br>cut -d' ' -f3 test > 3<br>cut -d' ' -f4 test > 4<br><br># combine all columns<br>paste 1 2 3 4 > test5<br><br>example output:<br><br>021231 223941.761 11.035 -5.016 -.-- ESR<br>
021231 224005.303 12.226 -6.243 -.-- ESR<br> 20030101 4380 25934.057 -37.022 -69.589<br> 20030101 4382 45951.090 33.005 -110.772<br><br>The problem is for the file example 1, lat and lon columns contain
<br>spaces other than the delimiter example " -38.00" while another is<br>"120.00" In the file2 example, more spaces are there. I think I need<br>to process different file formats separately but how do I solve the
<br>problem for spaces in the lat/lon columns?<br><br>One last question how do I get the records for my current region only?<br><br>north: 20:00:01.49976N<br>south: 5:00:01.499767N<br>west: 115:00:01.5012E
<br>east: 130:00:01.501193E<br><br><br>I'm starting to understand awk (reading the gawk manual right now) but<br>may take a while to get do something magical.<br><br>Thanks!<br><br>Maning<br><br>--<br>|---------|----------------------------------------------------------|
<br>| __.-._ |"Ohhh. Great warrior. Wars not make one great." -Yoda |<br>| '-._"7' |"Freedom is still the most radical idea of all" -N.Branden|<br>| /'.-c |Linux registered user #402901, <a href="http://counter.li.org/">
http://counter.li.org/</a> |<br>| | /T |http://esambale.wikispaces.com|<br>| _)_/LI |http://www.geocities.com/esambale/philbiodivmap/philbirds.html |<br>|---------|----------------------------------------------------------|
<br><br>_______________________________________________<br>grassuser mailing list<br><a href="mailto:grassuser@grass.itc.it">grassuser@grass.itc.it</a><br><a href="http://grass.itc.it/mailman/listinfo/grassuser">http://grass.itc.it/mailman/listinfo/grassuser
</a><br></blockquote></div><br><br clear="all"><br>-- <br>David Finlayson