[mapserver-users] RE: [Mapserver-users] HOWTO Split Large SHP Files

[Hankley, Chip]

I could be missing something on how tileindexes work (if I am, someone
please comment)... but this is how I have done this.

Where I have a large, detailed shapefile, I chunk it up into a bunch of
smaller shapefiles, then use the TILEINDEX feature in MapServer to display
the group of shapefiles.

Basically, I first create a "tile" shapefile. There are a number of ways
that you can do this... I basically convert a grid of some specified size
into a polygon shapefile and use that as my tile.

Next, I run a script in ArcView that chunks up the input shapefile based on
the individual "tiles" in the "tile" shapefile.

I've written an avenue script (below) that does this second part, so if you
have access to AV 3.x, you could do it this way.

Chip
////////////////////////////////////////////////////////////////////////////
////////
' Name:  View.CreateTiles
'
' Title:  Creates a series of tiled shapefiles
'
' Topics:  Views
'
' Description: This script will create a 'tile' of shapefiles
'              from one shapefile based on an input shapefile
'              that defines the tile.
 
'
' Requires:    An input theme to be "tiled" and another theme
'              (polygon) that defines the tile shape.
'
' NOTE:        Derived from a cliping script developed by Pasi Hyvonen
'
' Self:     Not Applicable
'
' Returns:  Not Applicable
'---------------------------------------------------------------
' Author: Chip Hankley - GIS Coordinator
'         Graef, Anhalt, Schloemer & Associates, Inc.
'         5126 West Terrace Drive
'         Madison, Wisconsin 53718-8949
'         PHONE 608.245.1969    FAX 608.242.0787
'         chip.hankley at gasai.com 
'         http://www.gasai.com
'
' Date:   03/15/2002'
'---------------------------------------------------------------

' Get the input shapefile. This is the shapefile
' to be "tiled." It will be the active theme.
theView = av.GetActiveDoc
themesList = theView.GetThemes
theClipTheme = MsgBox.List(themesList, "Select the Theme to be TILED",
"MapServer Tiling Utility")
theClipThemeFTab = theClipTheme.GetFtab

' Get the shapefile that defines the tile shape
theTileTheme = MsgBox.List(themesList, "Select the Theme that defines the
TILE boundaries", "MapServer Tiling Utility")
theTileThemeFTab = theTileTheme.GetFtab


' Assign a uniqueID to each polygon in the tile
' shapefile
  aTheme = theTileTheme
  currentFtab = aTheme.GetFtab
  addRecno = FALSE
  hasRecno = FALSE
  updateattr = FALSE
  for each attr in currentFtab.GetFields
    if (attr.AsString.UCase = "RECNO") then
      hasRecno = TRUE
      if (MsgBox.YesNo(atheme.AsString+" contains RECNO attribute."+nl+
          "Do you want to update it?","Update",FALSE)) then
        updateattr = TRUE
        break
      else
        break
      end
    end
  end
  numRecs = currentFTab.GetNumRecords
  if (hasRecno.Not) then
    addRecno = (MsgBox.YesNo("Add Record Number Attribute "+nl+
                "to "+atheme.AsString+"?","Query",TRUE))
    if (addRecno) then
      currentFtab.SetEditable(TRUE)
      recAttr = Field.Make("Recno",#FIELD_LONG,8,0)
      av.ShowMsg("Adding Recno Attribute to "+atheme.AsString+"...")
      currentFtab.AddFields({recAttr})
      av.ClearMsg
    end
  end
  if ((updateattr) or (addRecno)) then  
    currentFtab.SetEditable(TRUE)
    recnoField = currentFtab.FindField("Recno")
    for each recNum in currentFtab
      recno = ( recNum + 1 ).SetFormat("d").AsString
      currentFtab.SetValue( recnoField, recNum, recno )
      av.ShowMsg("Populating Recno Attribute of "+atheme.AsString+"...")
      proceed = av.SetStatus((recNum / numRecs) * 100)
      if ( proceed.Not ) then
        av.ClearStatus
        av.ShowMsg( "Stopped" )
        exit
      end
    end
    currentFtab.SetEditable(FALSE)
    av.SetStatus(100)
    av.ClearMsg
    av.ClearStatus
  end

' Set up a loop that sequentially goes through each
' "tile" in the tiling shapefile and "clips" the 
' input shapefile and assigns it a unique ID.

thePath = msgbox.input("Type the path (END WITH A \) of the director to dumb
clipped shapefiles", "Specify Location", "C:\TMP\")


for each i in 1..numrecs
  theTileThemeFTab.GetSelection.ClearAll
  theTileThemeFTab.UpdateSelection
  theBitmap = theTileThemeFTab.GetSelection
  strQuery = "[Recno] = "++i.asString
  test = theTileThemeFTab.Query(strQuery, theBitmap, #VTAB_SELTYPE_NEW)
  theTileThemeFTab.SetSelection(theBitmap)
    ' def = av.GetProject.MakeFileName("LM_", "shp")
    ' def = FileDialog.Put(def, "*.shp", "Convert " + theTileTheme.getName) 
    ' anFTab = theTileThemeFTab.Export(def, Shape,
theTileThemeFTab.GetSelection.Count > 0)
  for each rec in theTileThemeFTab.GetSelection
   theShape =
theTileThemeFTab.ReturnValue(theTileThemeFTab.FindField("shape"),rec)
  end
  
  ' Select features that intersect the view extent and export the selected
ones into new shapefile.
  theClipThemeFTab.SelectByShapes({theShape}, #VTAB_SELTYPE_NEW)
  theFileNameString = thePath +"clip" + i.AsString
  theClipThemeExport =
theClipThemeFTab.ExportClean(theFileNameString.AsFileName, TRUE)

  strClass = theClipThemeFTab.GetShapeClass.GetClassName
  if ((strClass = "Polygon") or (strClass = "PolyLine")) then    
    ' Clip polygon and polyline themes
    thmTemp = FTheme.Make(theClipThemeExport)
    theView.AddTheme(thmTemp)
    theView.SetEditableTheme(thmTemp)
    if (thmTemp.GetFTab.IsEditable) then
      thmTemp.GetFTab.GetSelection.SetAll
      thmTemp.ClipSelected(theShape)
    end
    thmTemp.GetFTab.SetEditable(FALSE)
    theView.SetEditableTheme(nil)
    theView.DeleteTheme(thmTemp)
  end  
end