This post identifies sections of highway more than 2 hours from a rest stop. We use a local version of OpenRouteService, an extract from OpenStreetMap maps, and rest area data from Transport for NSW to determine where there are gaps in the Rest Area network.

Rest Stop Points Data

Save a copy of the GEOJSON rest stops:

• Open https://rms-apps.carto.com/api/v2/sql?q=SELECT * FROM ra_get_rest_area_data()&format=GeoJSON”
• Save cartodb-query.json as rest-stops.geojson
• Go to Layer > Add Vector Layer
• Locate rest-stops.geojson
• Accept other defaults
• Click Add
• Click Close

To add the rest stops to QGIS:

• Go to Layer > Add Vector Layer
• Change source type to Directory
• Change type to OpenFileGDB
• Select the folder expanded above
• Click Add
• Click OK on the Vector Layers dialog
• Click Close

The Rest Stops dataset contains both permanent Rest Stops and also Driver Revivers that operate at peak periods. The number of Driver Revivers in the dataset possibly changes depending on when the data was extracted. In the extract used in this study, there were only 15 or so temporary Driver Revivers, which may have been out of date.

For the purpose of this demo, we will assume all stops are active.

Highways Data

Get major highways from NSW Data Portal. Instructions at https://opendata.transport.nsw.gov.au/dataset/road-segment-data-from-datansw

• The file is about 57MB.
• When the email arrives, save the data in the link to your local study area directory
• Expand the .zip file and a new folder for RoadNameExtent_EPSG4283.gdb is created.

To add the road network to QGIS:

• Go to Layer > Add Vector Layer
• Change source type to Directory
• Change type to OpenFileGDB
• Select the folder expanded above
• Click Add
• Click Close

The RoadNameSegment database has many MultiLineStrings. Many tools struggle with multi-lines, so let’s separate them into single parts.

• Open the Processing Toolbox, and search for Multipart
• Double click on the result for Multipart to singleparts
• Set Input Layers as RoadNameExtent_EPSG4283 RoadNameExtent
• Click Run, then Close

A new layer called Single Parts is added. Rename this to NSW Roads. There are about 117,000 segments.

For this study, we’re only interested in the major roads. These have a “functionhierarchy“ of less than or equal to 3:

• Freeways and Motorways are 1
• Highways such as Princes Highway are 2
• Major arterials such as Epping Road are 3

To filter only the roads we need:

• Right-click the NSW Roads layer in the Layers panel and choose Filter…
• Double-click on functionhierarchy, then click <= button, and type the number 3
• Click OK

Isochrones with OpenRouteService

The assumption for this study is that rest stops need to be no more than 2 hours apart on major routes.

So we will create drive time isochrones for the rest stops data.

There is a plugin for QGIS that allows us to create drive times. The plugin is called ORS Tools.

To install this plugin:

• Go to Plugins
• Search for ORS Tools
• Click Install Plugin
• Click Close

The ORS Tools plugins connects to the OpenRoutingService which offers a free API to calculate drive times. However, there are restrictions on the service in order to keep it free. One of the main restrictions is that the maximum range is 60 minutes, and the maximum distance is 120km. If we want 2-hour isochrones, then we need to find an alternative!

Install ORS

ORS can be installed in a Docker container locally, and tweaked as required.

The instructions are at https://giscience.github.io/openrouteservice/installation/Installation-and-Usage.html

The details won’t be covered here – it is very technical and depends on many inputs – operating system, CORS, Docker installs, port settings, URL parameters, OpenStreetMap extracts and more. There will be a separate blog post covering the topic. Whilst it may be a stumbling block for many, perhaps getting the theory is a good start.

When the service is up and running, it can be added as a server in the ORS configuration.

• Click the ORS Tools icon (a black hexagon with red nodes)
• Click on the gear icon
• Click the Add button
• Leave the API key blank (unless you configured it in the local install)
• In the Base URL, enter http://localhost:8080/ors

When the service was installed, a folder called conf was created. Inside that directory is a settings file called ord-config.json.

• Locate the isochrones tree
• For the maximum_range_time, change the default value of 3600 to 7200. These are seconds.
• Restart the Docker container and we’re ready to go.

Create Isochrones

• Open the ORS Tools by clicking on the ORS icon
• Change to Batch Jobs
• Choose Isochrones From Layer
• Change the Provider to your newly added local ORS Service
• Set Input Layer as rest-stops
• Leave travel Mode as driving-car
• Leave Dimension as time
• Change the Comma-Separated ranges to 120 min (which is the same as 7200 seconds)
• Click Run

Features with no ID may cause a GenericServerError. These may or may not be important? I suspect they are records that are too far from the road network to snap to it.

What is happening is the ORS tool is travelling 2 hours in every direction from every rest point and merging those polygons into one layer.

With over 800 rest stops, that is a lot of calculations! By running locally we don’t impact any other users, nor do we consume credits as we might with other network analysis tools.

For this study, the process took 12 minutes.

A new layer called Isochrones is added to the Layers panel.

• Click Close, and Close the ORS Tools dialog.

There are a lot of overlapping isochrones. Your computer may be pleading for mercy at this point as it tries to redraw the map layers.

It makes sense to merge this all into one isochrone since we’re only interested in sections of highway not within 2 hours of any rest stop.

• Open the Processing Toolbox
• Search for Dissolve
• Double click on Vector geometry > Dissolve
• Leave/change the Input layer to be Isochrones
• Click the Spanner icon of the Input Layer
• Change Invalid feature filtering to Skip (ignore) Features with Invalid Geometries
• Click the blue triangle to return to Parameters
• Click Run

This process took about 2 minutes. A new layer called Dissolved was added to the Layers panel.

The Dissolved layer is very complex. We’re going to split it up into tiles to achieve performance improvements.

Create a grid of 100km by 100km squares:

• Go to Vector > Research Tools > Create grid
• Grid type is Polygons
• Grid extent is Calculate From Layer > Dissolved
• Set Horizontal Spacing and Vertical Spacing to 100000m (100 km)
• Click Close

A new layer called Grid is dded to the Layers panel.

To intersect the Grid and the Dissolved layers:

• Search for Intersect in the Processing Toolbox
• Double-click on Vector overlay > Intersection
• Set the Input layer as Dissolved
• Set the Overlay layer as Grid
• Click Run, and when complete click Close

A new layer called Intersection is added to the Layers panel.

• Hide the Grid, Dissolved, and Isochrones layers, and watch performance go back up.

Finding Orphan Highways

We now have all the components we need to calculate sections of highway not within 2 hours of a rest stop.

The algorithm we need is called Difference. The Difference algorithm identifies all parts of the line layer that do not fall within the polygon layer. This is very different to the Split With Lines algorithm which outputs the opposite of what we’re trying to achieve.

• Search for Difference in the processing Toolbox
• Double-click on the Vector overlay > Difference
• Set the Input layer to NSW Roads
• Set the Overlay layer to Intersection
• Click Run

A new layer called Difference is added to the Layers panel.

To see where the orphaned segments are:

• Right-click on Difference and choose Zoom to Layer(s)
• Hide the Intersection layer
• Change the styling for Difference so it really stands out by changing the line thickness to 3 in the Layer Styling panel.

Results

There will be anomalies on the map, and these will usually be caused by either missing data in the OpenStreetMap data or artefacts in the polygon generated by ORS.

For example, there is a highway segment on Bells Line of Road at Bell that turns out to be overlap issue. It is only 150m long.

We’re really only interested in the longest sections of road. Before we can look at the longest sections, we need to split MultiLineStrings into Single LineStrings.

• Search for Multipart in the Processing Toolbox
• Double-click on Multipart to Singleparts
• Choose Difference as the Input Layer

A new layer called Single parts is added to the Layers panel.

We’re going to add a new virtual field for the length of each object:

• Select the Difference layer in the Layers panel
• Open the Field Calculator from the toolbar (abacus)
• Check the Create virtual field
• Name the virtual field as “segment_length” without the quotes.
• Set the Expression to $length
• Click OK

Consider sections of highway that are more than 20 kilometres long:

• Right-click on Single parts in the Layers panel
• Choose Filter…
• Double-click on segment_length at the bottom of the list of Fields
• Add the greater than operator (>), and 20000 as the value:
  • "segment_length" > 20000

Zoom the map to show all filtered Single parts.

To use the same styling from the Difference layer, perform the following:

• Right-click the Difference layer
• Go to Styles… > Copy Style > All Style Categories
• Right-click on the Single parts layer
• Go to Styles… > Paste Style > All Style Categories
• Hide the Difference layer

All highways more than 2 hours from a rest stop are located in the north-west of NSW. That said, many of these ‘highways’ are unsealed roads connecting rural centres but not necessarily on the busiest of routes.

The next stage of the study might be to analyse each of these 34 sections of highway to identify potential rest stops.

The process is not perfect – but a great starting point for discussion.

Final Map

Using QGIS2Web, I created a quick map of the final orphaned segments.

In it, the instructor shows how to answer a common GIS problem of how to calculate how many vehicles are needed to serve the customer base efficiently. The tutorial uses the awesome tools available at carto.com.

Bob’s Battery Buzzers is a road side assistance company that has had a strong presence in Melbourne. Looking to expand to WA, they bought a struggling company that has existing clients.

Now that they have completed the purchase they are looking to take their model and bring it to the West. The challenge is that CEO Bob doesn’t know how many vehicles would be needed to service the existing client base. Eager to make a good impression within budget, Bob needs to know whether seven cars would be enough for the whole city or if he needs to invest more up front.

You’ve been given the task and think that using a mapping product like CARTO would help you give Bob advice. Bob has given you the home addresses of the existing clients in Perth and wants a recommendation on how many cars to set up. Using the mapping tool (and the video instructions to guide you) produce a map that shows where the cars will be stationed and how they would cover the city.

But what if you’re a user of QGIS? Can you solve the same problem with that? Yes, of course!

If you haven’t done so already, sign up for the free mapping kit. They won’t spam you, and will provide the customer database needed for this exercise. I won’t provide the raw file here, sorry.

Instructions

Upload the CSV to QGIS

NGIS provides a CSV with data of some 38,335 data points spread over the Perth metropolitan area.

Start a new Project in QGIS, and open the customers.csv file (Ctrl-Shift-T)

Locate the Map_kit_demo_customer_locations.csv file sent to you by NGIS. Ensure that Latitude and Longitude are pre-filled, and that the projection is WGS84:

Rename the imported file to customers to make things easier by right-clicking the layer name in the Layers pane:

Add a Base Layer

To get some context, a neutral base layer is always useful. Add a new XYZ Tile Layer that references ESRI’s Light Gray basemap. The URL you will need is

https://services.arcgisonline.com/ArcGIS/rest/services/Canvas/World_Light_Gray_Base/MapServer/tile/%7Bz%7D/%7By%7D/%7Bx%7D

Reorder the layers so the basemap is at the bottom. By default, new layers are added to the top of the stack in QGIS.

Restyle the customer points so we can see what is going on. Choose a small point with no outline. Here I’ve used 0.5mm markers with fill color #d7191c with No Pen stroke style. This is what 33k markers look like!

Calculate Clusters of Points

OK it’s time to quantify the patterns lurking in the data with some analysis.

We’re going to use clustering to group our customers into areas derived from the geographic relationship between points. The NGIS tutorial uses seven areas, so we will too. Seven is the number of cars that Bob the owner thinks he will need to service the Perth area.

Open the Processing Toolbox if it is not already visible with Ctrl-Alt-T or go to Processing > Toolbox

Search for “cluster” in the Geoprocessing Toolbox, and select Vector analysis > K-Means clustering

K-means clustering: Calculates the 2D distance based k-means cluster number for each input feature.
If input geometries are lines or polygons, the clustering is based on the centroid of the feature.

• Change the Number of clusters to 7
• Leave the Cluster field name as CLUSTER_ID
• Click Run

There are at least 2 duplicate customer points, but that won’t impact results greatly.

A new layer called Clusters is added to the map. Rename the layer to *customers by clusterid” ready for some styling.

Style the *customers by clusterid” layer

• Change the method to Categorized
• Set the Value to CLUSTER_ID
• Change the Symbol to a smaller size say 1mm with no outline/stroke
• Leave the Color ramp as Random colors
• Click the Classify button.

The result should look something like this:

Calculate Cluster Centroids

At the moment we have 33 thousand colored points on a map – we don’t have any polygons or areas that define the regions defined by each cluster.

Let’s fix that by creating polygons from the data using a processing tool called Minimum bounding geometry.

Carto’s analysis tool called “Find centroids of geometries” presumably wraps the process of creating polygons and determining centroids into one tool.

Create polygons of clusters

• Go to the Processing Toolbox and seach for Minimum bounding geometry.
  
• Double-click the branch to open the dialog.
• Change optional Field dropdown that groups points into separate polygons to CLUSTER_ID
• Change *Geometry type** to **Convex Hull**
  
• Click Run
• When completed (~1 second), click Close

Back in the Layers pane a new layer has been added called Bounding geometry

• Reorder the layers so the Bounding geometry layer is below the customers layer, but above the ESRI Gray (light) layer.
• Rename the Bounding geometry layer to cluster_polygons

We now have polygons for each car in the fleet:

With our polygons ready, we can now determine the centroid, or “middle”, of each polygon.

Create cluster centroids

• In the Processing Toolbox, search for “centroid” and then double-click the branch Centroids under Vector geometry.
• Change the Input layer to our newly created cluster_polygons layer
• Check the box for Create centroid for each part, as we want all the centroids and not just the centroid of the layer as a whole. This is also known as dissolving
• Click Run
  
• Click Close

There is a new layer added called Centroids. Rename it to cluster_centroids

Style the Cluster Centroids

• Open the Layer Styling pane
• Leave the dropdown as Single Symbol, but select the “Simple marker” branch.
• Change the Symbol layer type from Simple marker to SVG Marker
• In the SVG Groups section, select shopping to filter
• Choose the car symbol
• Change the Fill color to red or similar
• (There should be No Stroke)
• Change the Size to Width 10 and Height 10 with Unit of Millimeters

Change back to the Layers pane, and un-check cluster_polygons and your map should look something like:

Create Areas of Influence

One of the project assumptions is that Bob’s cars “can only reliably cover a circle around them of about 8 kilometre radius” based on experience from Bob’s other work. We’ll use this to see what gaps we may have in our coverage of Perth.

So let’s create circle centred on each cluster centroid that have an 8 kilometre radius. But first, we need to change the Projection of our centroid layer to use Metres instead of degrees. We’ve been using EPSG:4326 up to this point which uses Degrees as the unit, so we need to Reproject to EPSG:7850 or another metres-based projection.

Reproject Centroids to a Metres-based Projection

• In the Processing Toolbox, search for reproject
• In the Vector general branch, double-click on Reproject layer
• Change the Target CRS to EPSG:7850 – GDA2020 / MGA Zone 50
  
• Click Run
• Click Close

We now have a new layer called Reprojected. Rename it to cluster_centroids_7850

Create Buffers around Centroids

• In the Processing Toolbox, search for buffer
• In the Vector geometry branch, double-click on Buffer
• Ensure the Input layer is cluster_centroids_7850
• Change the Distance to 8000 and leave the units as meters
• Leave the Segments, End cap style, Join style, and Miter limit as is.
• Check the Dissolve result so we can see where circles overlap (if they do)
• 
• Click Run
• Click Close

Optional: The map is updated and looks a bit “wonky” – change the projection of the map by clicking in the bottom right of the QGIS screen, and selecting EPSG:7850

Further enhancements:

• Set Buffered to be
  • Color: #e5e5e5 (grey)
  • Opacity: 50% so we can see underneath
• Rename Buffered to areas_of_influence

With the areas of influence circles set to 8km, there are lots of customer data points that are outside the circles and not easily reachable.

What-If Analysis

So now we see that 7 cars is not going to be enough to cover the Perth metro area. The next question is – what is the optimum number of cars required?

Rerun the above step with 12 clusters, and see what happens.

In Carto, this step is very clever in that it reruns the other required steps after defining how many clusters we want. In QGIS, this can be automated too using Python and/or Model Builder., and we will do that in another article.

To run with 12 clusters, the core steps were:

1. K-means Clustering of customers
2. Confirm it worked by styling Clusters by CLUSTER_ID
3. Create cluster polygons with Minimum bounding geometry on Clusters using CLUSTER_ID field and Convex Hull geometry type
4. Generate the centroids using Centroids on Bounding geometry with Create centroid for each part checked
5. Reproject Centroids using Reproject layer set to EPSG:7850 Zone 50
6. Create Buffer on Reprojected of 8000 metres Dissolved.

Using 12 cars, we generate the following analysis:

Summary

What percentage of customers are covered in each scenario of 7, 12 and 14 cars?

By using the “Count points in polygon” function in the Processing Toolbox we can determine how many customers can be reached by the fleet.

The percentages of customers that are within the generated Areas of Influence are:

So with 14 cars we can reach 87% of the existing customer base by staying within the 8km radius.

This analyses could be refined using time-based areas of influence, and perhaps it could be automated to rerun each step as required without input.

Housekeeping

Save the Project, if you haven’t yet done so.

We also need to save all the temporary layers. They’re indicated by a computer chip in the Layers pane implying the file is stored in memory. If we don’t, they will be deleted when we exit QGIS which is not fun for anybody! Or, use the plugin called Memory Layer Saver which will save all temporary layers in a portable binary format and reopened when the project is opened.

• Right-click the temporary layer, and choose Make Permanent from the menu:
  
• Choose Geopackage as the most flexible format

Use groups in the Layers panel to group the 7 car, 12 car and 14 car analyses.

The final project Layers may look something like this: