RSS

Category Archives: primagis

PrimaGIS sprint report

The Plone Conference was held in Naples, Italy this year and, as expected, was a great experience. Like last year in Seattle, we had a PrimaGIS focused sprint where a group of people worked on the next version of PrimaGIS which is based purely on Plone 3.0.

We had a list of sprint topics that people were able to choose from listed at the official sprint page. Below is a breakdown of the sprint activities, what was accomplished and what is still left to do.

OpenLayers integration

One of the biggest changes in the upcoming PrimaGIS version is OpenLayers integration. I had initially planned that OpenLayers would be used as an alternative user interface for PrimaGIS, but after a thorough introduction and discussion with Josh Livni we decided to use OpenLayers as the sole interface to PrimaGIS and retire the old custom one. I believe that this decision will serve us well in the future and the resources of PrimaGIS developers can be put to better use without having to worry about the map UI. The OpenLayers developers seem to be doing an excellent job at that and we’re grateful to use their work.

Currently the OpenLayers view works fine but there is little user configurability. The next step is to refactor the code so that the UI can be configured using user friendly forms and to make it easier for power users to customize the relevant parts. We also want to hook in the more advanced OpenLayers features such as spatial feature editing.

We also implemented TileCache support for PrimaGIS, so that people can easily configure their TileCache servers to improve PrimaGIS performance. Each presentation layer is able to produce a TileCache configuration section that can be copy/pasted to your TileCache configuration file.

Sean Gillies participated remotely and implemented support in PCL for the Python Geo Interface which allowed us to pull in feature data using all geometry types (point, line, polygon) directly into OpenLayers. Thanks Sean!

Big thanks also to Josh Livni for giving a lightning talk on using OpenLayers maps (including examples on FeatureServer and WPServer), for helping me to understand OpenLayers a bit better and working on the actual integration during the sprint!

New layer classes

Partly inspired by the OpenLayers and partly by the changes made to PCL and ZCO we decided to refactor the PrimaGIS layer classes. Previously there were two types of layers: PrimaGISLayer and PrimaGISDataLayer. PrimaGISLayer was used to create a map layer from spatial data from outside of Zope and PrimaGISDataLayer for data from within the ZODB. In either case it was difficult to share layers between multiple maps.

It had always been a goal to get rid of the artificial difference between the layers and data layers and adding support for sharing layers between maps seemed a good idea also.

The new types of layers are: Layer and Presentation Layer. A Layer class is a combination of what PrimaGISLayer and PrimaGISDataLayer used to be. It is used to select the data source where spatial data is acquired and to define the styling of the spatial features. It doesn’t matter whether the data comes from within Zope or not. The difference is that Layers don’t need to be located within Map objects. They can be located anywhere on your site and shared between multiple maps. A map administration may choose to have a folder where all Layers are stored but this is a totally arbitrary choice.

A Presentation Layer is an object that is stored within a Map object and refers to one or many Layer objects. It is tied more tightly into OpenLayers concepts such as a layer type which can be either base, overlay or vector layer. Each presentation layer is available as a Web Map Service (WMS) layer and renderable by OpenLayers. Connecting multiple Layers into a Presentation Layer allows you create composite layers that can be switched on and off as one. Below is a simple diagram that shows how the different objects are tied together.

PrimaGIS object relations

Buildout for Linux distributions

Alessandro Amici took on the daunting task of refactoring the primagis.buildout into an alternative buildout configuration that can take advantage of installed system libraries. By default, primagis.buildout builds all the required components from scratch which is a process that can more than an hour even on a modern machine.

The new buildout simply assumes that a given set of packages is installed system wide and compiles the rest of the dependencies against those. Currently the buildout provides a list of debian packages that users of Debian based distributions (such as Debian or Ubuntu) should install prior to running the buildout. See README-deb.txt in the buildout source tree for more specific instructions.

Eventhough the new buildout configuration was put together for Debian based distributions it is also usable on other distributions if a similar set of dependency packages are available. Users of other distributions are invited to contribute a list of packages suitable for their distributions.

The implementation of the buildout, which was initially estimated to take less then an hour :), turned out to be much more complicated task due to packaging bugs and differences in libraries between Debian and Ubuntu. We even created a new buildout recipe to help with working with C-libraries which Alessandro released as bopen.recipe.libinc. Users of 64-bit distributions should give the buildout a try since there have been reported problems when compiling some of the dependencies by hand on these systems.

Demo product

Sune Woeller and Chris Calloway worked together on a new demo product for PrimaGIS called zgeo.primagisdemo. The idea is to provide a similar effect to the createPrimaGISDemo.py script found in the previous versions. However, instead of putting the demo map in PrimaGIS itself, a separate product was created that sets up the map using a GenericSetup profile for content import. This is still work in progress so stay tuned for a release announcement soon!

Once it’s ready you can simply copy to package into your python path and run the GenericSetup import step to get a PrimaGIS demo in your Plone site.

Remaining tasks

There are still a few items I would like to see in PrimaGIS 0.7.

  • Persistent data stores and symbolizers
  • Better OWS properties management view
  • Spatial index for ZODB data
  • Feature inspection
  • Feature editing

On most items the work has already been started and some are are a simple matter of hooking into the functionality provided by OpenLayers.

Thanks again for all the people who participated in the sprint! It’s always great to people meet you know from IRC in real life.

Advertisements
 
1 Comment

Posted by on October 19, 2007 in GIS, plone, primagis

 

PrimaGIS in Plone 3.0

Up to (and including) version 0.6 PrimaGIS has been a traditional Archetypes based project. Almost all map components (maps, layers, symbolizers, etc) were modelled and implemented as AT content types.

Having all the components be full-blown AT content types made development easy up to a certain point but also at the same time made the objects unnecessarily heavy. Also, most map components are not content-like by nature and thus, for example, having them all support Dublin Core and be indexed in the portal catalog is unnecessary.

The next major version of PrimaGIS is based on Zope 3 components.

Using the Component Architecture

The Zope 3 Component Architecture made it relatively easy to remodel the map components as light-weight domain objects . In most cases we are now able to use objects from the Python Cartographic Library (PCL) directly, where earlier versions needed to define wrappers around these PCL objects. One reason for this is that PCL uses zope.interface internally, so useful interfaces are already defined and available for use.

In Plone 3.0 map renderers, spatial data stores and feature symbolizers are registered as utilities in the Component Architecture using ZCML. The components are registered using custom ZCML directives defined under the http://namespaces.gispython.org/gis namespace. For details on the custom directive implementation you can refer to the metaconfigure.py and metadirectives.py files in the ZCO codebase.

Map renderer

A map renderer is a component that is responsible of rendering the final map image given a collection of map layers (and some metadata). The current map renderer in PrimaGIS is based on the MapServer project. It is registered using the following ZCML code:



  
      incoming="/tmp"
      fontset="/tmp/pg2/spatialdata/fonts/fontset.txt"
     />


which registers the MapServer based renderer as an unnamed utility providing cartography.styles.interfaces.IMapRenderer.

In the future we can also support other map renderers and swap them transparently by registering them instead of the MapServer renderer — without having to modify any existing map setups.

To acquire the map renderer in code we can now simply do:


>>> from cartography.styles.interfaces import IMapRenderer
>>> from zope.component import getUtility
>>> renderer = getUtility(IMapRenderer)

Spatial data stores

Data stores are components that provide the spatial data for map layers. Data stores can be divided into two categories: feature stores that provide vector data and raster stores that provide raster data.

The data stores are registered using custom ZCML directives. Each type of data store has its own parameters, but all are registered as named utilities providing cartography.data.interfaces.IDataStore. Below is an example of a Web Map Service (WMS) data store registered in ZCML.



    name="NASA Jet Propulsion Laboratory WMS"
    url="http://wms.jpl.nasa.gov/wms.cgi"
    version="1.1.1"
    incoming="/tmp"
    />

For a more comprehensive set of examples see the datastores.zcml.dist file in the ZCO codebase. Registered data stores can now be easily acquired in code:


>>> from cartography.data.interfaces import IDataStore
>>> from zope.component import getUtility
>>> datastore = getUtility(IDataStore, "Name of data store")

Symbolizers

Feature symbolizers determine how selected spatial features are rendered on the map and in concept are similar to CSS rules. In a similar manner the symbolizers are defined and registered using ZCML. The four different types of symbolizers: point, line, polygon and text symbolizers, are each registered using their respective ZCML directives.

The symbolizers get registered as named utilities providing cartography.styles.interfaces.ISLDSymbolizer. Below is an example of a line symbolizer defined in ZCML.



   


For a comprehensive set of examples see the symbolizers.zcml.dist in the ZCO codebase.

Maps and Layers

Maps are composite objects that contains one or more layers. Each layer draws data from a spatial data source and determines the styling (using rules and symbolizers) applied to its spatial features. For maps and layers there is benefit in modeling them as content types, for example to be able to apply workflow to maps or individual layers or allow the layers to be managed using the default folder management methods.

For this reason, primagis.map.Map and primagis.layer.Layer are Archetypes derived content types. However, the AT schema mechanism is not used to manage the configuration of the components themselves, but used only to manage the content like attributes such as Dublin Core metadata. This gives us a nice separation of concerns between the content related attributes and the actual mapping attributes.

The map configuration forms are implemented using Zope 3 schemas and formlib. There are some custom schema fields and widgets that make it easier to manage the mapping specific attributes, such as bounding boxes. KSS has also been used to make the editing screens more user-friendly. In the future KSS might also be used in the map view also.

Web Map Service (WMS)

The Web Map Service support makes it possible to share and re-use the imagery produced by PrimaGIS in other WMS compliant clients such as, for example, uDig, OpenLayers, and PrimaGIS itself. OpenLayers is of special interest to PrimaGIS since it allows us to provide an alternative UI for PrimaGIS itself.

OpenLayers uses a tiled approach where the map image is put together by tiling multiple smaller images requested from the WMS server. It also provides a nice Google Maps like panning mode previously not available in PrimaGIS.

The WMS support is implemented as a Zope 3 view registered as wms for primagis.interfaces.IMap. This means that you can just point your WMS client to an URL like http://domain.tld/path/to/primagis/@@wms.

For an efficient WMS implementation we will need to be able to cache the rendered tiles, possible using an existing solution like TileCache.

Future work

Below are listed some features (in no particular order) I would like to see in PrimaGIS.

  • Persistent local versions of data stores and symbolizers
  • Management UI for the local data stores and symbolizers
  • Cached WMS requests
  • GeoRSS support
  • Better OpenLayers integration / configuration support
  • Spatial indexing
  • ZODB datastores
 
5 Comments

Posted by on February 12, 2007 in plone, primagis, zope

 

Snowsprint 2007

The fourth (and my first) Snowsprint was held in Bregenzerwald, Austria during 27.03. – 3.2.2007. Altogether 56 developers attended the sprint from 14 different countries who worked on a variety of topics including calendering, multimedia, KSS, GIS, REST, textmate extensions, and caching.

The Gasthaus Hirschen where the sprint was held at proved to be everything one could hope for and the great common room was just the right environment for a week of intensive brain storming and coding.

My goals for the sprint were to learn more about the upcoming Plone 3.0 and to continue on the work on the next major version of PrimaGIS. In earlier sprints in Dublin and Seattle we had already refactored the Cartographic Objects for Zope (ZCO) and PrimaGIS using the Component Architecture of Zope 3 and now it was turn to work on the Plone UI for PrimaGIS.

I had decided to use Zope 3 technologies as much as I could, which meant I would be using Zope schemas and formlib for all forms in PrimaGIS. I also wanted to learn how to implement custom schema fields and widgets using formlib. This all turned out to work quite well, although the Zope2 publisher and security mechanism required to do some acquisition trickery to make everything play nicely together.

One exciting new feature of Plone 3.0 is the KSS (Kinetic Style Sheets) framework (found in the plone.kss and plone.app.kss packages), which makes it extremely simple to do cool AJAX style programming without having to touch Javascript at all. I took part in the KSS tutorial given by Godefroid Chapelle and Balázs Reé which helped to get started working with KSS really fast. This turned out especially useful in implementing the custom formlib widgets.

In addition to the UI work I also worked on an experimental Web Map Service (WMS) server implementation for PrimaGIS. In Zope 3 terms, this simply meant that I needed to implement a view for the primagis.map.Map object that implements the WMS specification. The immediate benefit of implementing WMS server support is that then PrimaGIS maps may be reused by any standards compliant mapping client totally independent of Zope and Plone.

For PrimaGIS itself the WMS support means that, for example, it’s now possible to chain multiple PrimaGIS maps together so that one PrimaGIS instance is able to use the data provided by another. Another cool thing was that we can now provide alternate UIs for the PrimaGIS maps. As an experiment I integrated the Open Layers Javascript UI into PrimaGIS which worked right out of the box. All this code is still very experimental and in the future we will need push most of it into OWSLib and have PrimaGIS use that to implement WMS support.

I will post more detailed entries on how the Plone 3 version of PrimaGIS works and how it differs from the current version. For now you can check out the code at the following branches:

http://svn.gispython.org/svn/zope/ZCO/branches/zco3

http://svn.gispython.org/svn/zope/PrimaGIS/branches/primagis-plone-3.0

or alternatively use the primagis.buildout system to build a development instance (using the --develop switch) which contains the latest code.

Many thanks to Lovely Systems for hosting a great sprint. See you next year!

 
1 Comment

Posted by on February 6, 2007 in plone, primagis, zope