[QGIS Commit] [qgis/Quantum-GIS] 0d464b: Delimited text provider indexing implemented.

Sun May 12 14:17:18 PDT 2013

  Branch: refs/heads/master
  Home:   https://github.com/qgis/Quantum-GIS
  Commit: 0d464b2e1b01693a475cb35399ea0ff6c3778687
      https://github.com/qgis/Quantum-GIS/commit/0d464b2e1b01693a475cb35399ea0ff6c3778687
  Author: Chris Crook <ccrook at linz.govt.nz>
  Date:   2013-05-09 (Thu, 09 May 2013)

  Changed paths:
    M resources/context_help/QgsDelimitedTextSourceSelect-en_US
    M src/core/qgsvectorlayer.h
    M src/providers/delimitedtext/CMakeLists.txt
    M src/providers/delimitedtext/qgsdelimitedtextfeatureiterator.cpp
    M src/providers/delimitedtext/qgsdelimitedtextfeatureiterator.h
    M src/providers/delimitedtext/qgsdelimitedtextfile.cpp
    M src/providers/delimitedtext/qgsdelimitedtextfile.h
    M src/providers/delimitedtext/qgsdelimitedtextprovider.cpp
    M src/providers/delimitedtext/qgsdelimitedtextprovider.h
    M tests/src/python/test_qgsdelimitedtextprovider.py
    A tests/src/python/test_qgsdelimitedtextprovider_wanted.py

  Log Message:
  -----------
  Delimited text provider indexing implemented.

Efficiency improvements.

Adds capabilities
   SelectAtId
   CreateSpatialIndex

This uses line number as an index.  Allows jumping to a line number
without interpreting all intermediate records, so much more efficient.
Also retains the current record in a buffer, so successive requests for
the same record do not require rereading the file.

With index available have also added indexing of subsets and optional
creation of spatial index.

Note: did try using QTextStream.pos to build a random access index -
this was incredibly slow.  Have found using line based index is
reasonably fast even on quite large (50+Mb) text files.

  Commit: 799870d54ae0879173230be8d460f7e4f1dd09ca
      https://github.com/qgis/Quantum-GIS/commit/799870d54ae0879173230be8d460f7e4f1dd09ca
  Author: Chris Crook <ccrook at linz.govt.nz>
  Date:   2013-05-10 (Fri, 10 May 2013)

  Changed paths:
    M doc/TRANSLATORS
    M i18n/qgis_de.ts
    M python/console/console.py
    M src/app/composer/qgscomposer.cpp
    M src/app/qgisapp.cpp
    M src/app/qgisapp.h
    M src/app/qgsvectorlayerproperties.cpp
    M src/core/composer/qgscomposeritem.cpp
    M src/core/composer/qgscomposeritem.h
    M src/core/composer/qgscomposermap.cpp
    M src/core/composer/qgscomposition.cpp
    M src/core/composer/qgscomposition.h
    M src/core/qgsmaprenderer.cpp
    M src/core/qgspallabeling.cpp
    M src/core/qgsproject.h
    M src/core/qgsrendercontext.cpp
    M src/core/qgsrendercontext.h
    M src/core/qgsvectorlayer.cpp
    M src/core/qgsvectorlayer.h
    M src/providers/postgres/qgspostgresprovider.cpp
    M src/ui/qgisapp.ui
    M src/ui/qgsprojectpropertiesbase.ui
    M src/ui/qgssavetodbdialog.ui

  Log Message:
  -----------
  Merge resolved

  Commit: 1f7f2b34d4bb8384a0ad9789f5c006f545027d5b
      https://github.com/qgis/Quantum-GIS/commit/1f7f2b34d4bb8384a0ad9789f5c006f545027d5b
  Author: Chris Crook <ccrook at linz.govt.nz>
  Date:   2013-05-12 (Sun, 12 May 2013)

  Changed paths:
    M doc/TRANSLATORS
    M i18n/qgis_de.ts
    M i18n/qgis_gl_ES.ts
    M images/images.qrc
    A images/themes/default/console/iconSyntaxErrorConsole.png
    M python/console/console.py
    M python/console/console_editor.py
    M python/console/console_output.py
    M python/console/console_sci.py
    M python/plugins/fTools/tools/doSubsetSelect.py
    M python/plugins/sextante/algs/ftools/RandomSelection.py
    M python/plugins/sextante/algs/ftools/RandomSelectionWithinSubsets.py
    A resources/function_help/Conditionals-en_US
    A resources/function_help/format-en_US
    A resources/function_help/format_date-en_US
    A resources/function_help/title-en_US
    R resources/function_help/title_en_US
    A resources/function_help/todate-en_US
    A resources/function_help/todatetime-en_US
    A resources/function_help/tointerval-en_US
    A resources/function_help/totime-en_US
    M src/app/composer/qgscomposermapwidget.cpp
    M src/app/composer/qgscomposermapwidget.h
    M src/app/qgisapp.cpp
    M src/app/qgscustomprojectiondialog.cpp
    M src/app/qgscustomprojectiondialog.h
    M src/app/qgshandlebadlayers.cpp
    M src/app/qgshandlebadlayers.h
    M src/app/qgsoptions.cpp
    M src/core/qgscoordinatereferencesystem.cpp
    M src/core/qgscoordinatereferencesystem.h
    M src/core/raster/qgsrasterfilewriter.h
    M src/gui/qgsprojectionselector.cpp
    M src/providers/oracle/qgsoraclecolumntypethread.cpp
    M src/providers/oracle/qgsoraclecolumntypethread.h
    M src/providers/oracle/qgsoracleconn.cpp
    M src/providers/oracle/qgsoraclesourceselect.cpp
    M src/providers/oracle/qgsoraclesourceselect.h
    M src/providers/postgres/qgscolumntypethread.cpp
    M src/providers/postgres/qgscolumntypethread.h
    M src/providers/postgres/qgspgsourceselect.cpp
    M src/providers/postgres/qgspgsourceselect.h
    M src/providers/wcs/qgswcsprovider.cpp
    M src/ui/qgscustomprojectiondialogbase.ui
    M src/ui/qgsoptionsbase.ui
    M src/ui/qgsprojectionselectorbase.ui

  Log Message:
  -----------
  Merge branch 'master' into delimited_text_indexing_implemented

  Commit: 2c378964c19567d0464a66a5d215ef24042e236d
      https://github.com/qgis/Quantum-GIS/commit/2c378964c19567d0464a66a5d215ef24042e236d
  Author: Chris Crook <ccrook at linz.govt.nz>
  Date:   2013-05-12 (Sun, 12 May 2013)

  Changed paths:
    M src/core/qgsvectorlayer.h

  Log Message:
  -----------
  Documentation fix

  Commit: d8dac0cbbd662a01c943a45cda1e525bafcb2273
      https://github.com/qgis/Quantum-GIS/commit/d8dac0cbbd662a01c943a45cda1e525bafcb2273
  Author: Tim Sutton <tim at linfiniti.com>
  Date:   2013-05-12 (Sun, 12 May 2013)

  Changed paths:
    M resources/context_help/QgsDelimitedTextSourceSelect-en_US
    M src/core/qgsvectorlayer.h
    M src/providers/delimitedtext/CMakeLists.txt
    M src/providers/delimitedtext/qgsdelimitedtextfeatureiterator.cpp
    M src/providers/delimitedtext/qgsdelimitedtextfeatureiterator.h
    M src/providers/delimitedtext/qgsdelimitedtextfile.cpp
    M src/providers/delimitedtext/qgsdelimitedtextfile.h
    M src/providers/delimitedtext/qgsdelimitedtextprovider.cpp
    M src/providers/delimitedtext/qgsdelimitedtextprovider.h
    M tests/src/python/test_qgsdelimitedtextprovider.py
    A tests/src/python/test_qgsdelimitedtextprovider_wanted.py

  Log Message:
  -----------
  Merge pull request #582 from ccrook/delimited_text_indexing_implemented

Delimited text provider indexing efficiency improvements.

Compare: https://github.com/qgis/Quantum-GIS/compare/724ed7232401...d8dac0cbbd66