Gis Data in Taito

The open GIS data in Taito is stored in: /proj/ogiir-csc

Currently (June 2018) there are following datasets:

  • Paituli data. Paituli includes datasets from  Agency for rural affairs, Finnish Environment Institute (SYKE), Finnish Meteorological Institute, Institute for the languages of Finland, National Land Survey, Natural resource institute, Population Register Center, Statistics Finland, Traffic Agency and University of Helsinki.
    • Full list of Paituli datasets
    • All Paituli datasets have a readme-file with a link to Etsin dataset descriptions and terms of use. In general all datasets have an open license, but the exact terms vary a bit.
    • If in trouble finding some file, you can also use Paituli download page as help. You can see the dataset path under links (crop the beginning) or you can download the file list with "Download list of files" if the dataset has a lot of mapsheets.
  • Additionally in Taito:
    • NLS
      • Lidar data, the automatically classified version (last update summer 2017)
      • Dem 2m and 10m have virtual rasters, see section below
    • LUKE
      • Multi-source national forest inventory, 2013 and 2015

 

Readymade virtual rasters in Taito

There is a ready made structure of virtual rasters for 2m and 10m elevation model in /proj/ogiir-csc/mml/dem2m_vrt and /proj/ogiir-csc/mml/dem10m_vrt folders. There are two variants of virtual rasters for these elevation models: 

  1. The hierarchical virtual raster is mainly for viewing purposes for example with QGIS. It has a hierarchical structure where a virtual raster for each folder contains all the data stored in that folder and it's subfolders. For example if I wanted to view 2m dem from area of mapsheet M4 dem I would simply open the dem2m_vrt/etrs-tm35fin-n2000/M4/M4.vrt file which would then load virtual rasters for mapsheets M41, M42, M43 and M44 which in turn contain information about the actual tif files. If i wanted to view the whole 2m dem dataset I would simply open dem2m_vrt/etrs-tm35fin-n2000/whole_finland.vrt.

    The hierarchical file structure also contains statistics (min, max, mean, stddev) and overviews for each vrt file which enables a fairly responsive viewing of entire 2m or 10m dem for example in QGIS. A good tip is to enable the Raster toolbar through View->Toolbars which allows to easily adjust the color scale to min-max of current view extent. This way the whole dataset can be easily viewed at different zoom levels.

    You may use the lowest level virtual raster (for example M41 in the 2m DEM) also in scripts, higher level virtaul rasters may cause computational errors.
     
  2. The direct virtual rasters contain directly the source tif images without any hierarchical structure, overviews or pre-calculated statistics. The direct virtual raster is meant for using only in scripts (it should not be opened in QGIS etc).

The optimal performance for analysis will always be achieved by creating your own virtual raster that covers only your study area.

 

Creating virtual rasters for specific area

There is a ready made python script for creating virtual rasters for your study area from datasets available in Taito. The script is located at /proj/ogiir-csc/mml/karttalehtijako/vrt_creator.py. It's used in a following way:

python /proj/ogiir-csc/mml/karttalehtijako/vrt_creator.py dataset polygon_file output_directory

available dataset values are:

  • dem2m - 2m dem from mml located at /proj/ogiir-csc/mml/dem2m
  • dem10m - 10m dem from mml located at /proj/ogiir-csc/mml/dem10m
  • demCombined - dem covering whole Finland using 2m dem whenever it's available and interpolating rest of the areas to 2m resolution from 10m dem using bicubic interpolation.

with optional arguments:

  • -i: create individual vrt for each polygon, default behavior is to create one vrt covering all polygons.
  • -o: create overviews
  • -p:output name prefix

The script utilizes gdal and geopandas python module so geo-env module should be loaded prior to using the script. For more details see: https://research.csc.fi/-/geopython

Example: create one virtual raster covering my_study_area.shp polygon file from 2m dem found at /proj/ogiir-csc/mml/dem2m and save it to /wrk/USERNAME/output/2m_dem_virtual_raster.vrt. A list of files will also be saved to /wrk/USERNAME/output/2m_dem_file_list.txt as well as external overview file /wrk/USERNAME/output/2m_dem_virtual_raster.vrt.ovr

python /proj/ogiir-csc/mml/karttalehtijako/vrt_creator.py dem2m my_study_area.shp /wrk/USERNAME/output -o -p 2m_dem