How to use Jupyter Notebooks for your Geospatial Workflows

Combined with the Python scripting and programming language, Jupyter Notebooks have become an indispensable tool for GIS analysts to extend their desktop GIS environment. This article describes how to use Jupyter Notebooks for Python, which popular Python packages are available for doing geospatial analysis and how to get started with Jupyter Notebooks inside a desktop GIS application.

Challenges of GIS software for doing geospatial data analysis

Today, geospatial data analysis is embedded within the larger discipline of data science that surpasses the traditional way that spatial analysis would be done. With more data, tools and approaches to interact with data, today’s GIS analyst requires a larger toolset to perform location intelligence and use data to solve a (spatial) problem.

While GIS technology offers many great tools for doing geospatial data analysis, it also has various shortcomings in today’s context of data science workflows. GIS software is still very much centered around making maps from data, which makes it easy to forget how that data is managed and analyzed before it is being presented through a map. And while GIS software is improved continuously over time with new tools to manage and analyze and present data, new tools, methods and practices are being developed everyday, which is much more than can be included in a single GIS application.

Related to this is that GIS software (or any data analysis software for that matter) comes with its own tools, practices, interfaces and related learning curve. One application may be good for one thing, but as a user you’d want to be flexible and combine best practices of multiple applications into one. Such an application faces the user with a closed environment where you are at the mercy of the application developers and their decisions on what to include in the software or not.

Application developers decide which data you can and cannot import. The data you need for your GIS analysis has to be brought into the software from the outside into the application. This has always been a challenge for GIS applications, where data format support was an issue until industry standards were developed.

However, a different problem occurred when big datasets became the norm, meaning that GIS analysts needed to process large spatial datasets locally, a task that desktop GIS was never designed for. While customized solutions were introduced over time, these did not prove to be the best approach as database technology transitioned from server-based to cloud-native solutions and market demands changed continuously.

Today, datasets are continuously being updated and are being stored, pre-processed and accessed remotely in a cloud or server environment as it is simply impossible to manage these locally. Over time, GIS application developers understood it was better to change their approach and instead of trying to integrate every possible new third-party solution into their own application, choose for an existing and proven approach that had already proved its worth in the 1980s: scripting languages, in combination with a new interactive web tool.

Geospatial Analysis + Python + Jupyter Notebooks

In the 1980s, scripting languages proved to be a handy way to automate GIS workflows. Instead of doing manual “button-pushing” operations in a GIS environment, a GIS analyst would write a script that would perform these actions when running the script in a code editor. This would not only save a lot of time and effort, it also reduced the possibility of human errors. Analytical GIS workflows lend themselve very well for this type of workflow automation, as they apply spatial algorithms to a geographical dataset on disk. For cartographic map production workflows, such an approach would not work as human interaction with a mapping interface is required.

After the introduction of the Python language for automating GIS workflows around 2004, it quickly became popular among GIS users. The Python language itself became hugely popular among the data science community, resulting in a very large ecosystem of freely available libraries for everyone to use. The GIS community was quick to realize its potential and adopted Python as the language to tap into this ecosystem and extend existing desktop GIS tools. However, what was lacking at the time was an easy-to-use tool that could replace a code editor as a working environment, or the primitive Python single-line editor inside a GIS application.

Python scripting in the QGIS Python console
Figure 1: Python scripting in the QGIS Python console

Today, the Jupyter Python Notebook is the to-go tool for GIS analysts who want to use Python to do anything from data management, to prototyping, spatial analysis, big data analytics, data visualization, among other things. Jupyter Notebooks are browser-based documents that combine code, annotations, explanations as well as links to online media. You can write, run and code there as with a code editor, but divided into individual cells instead of entire scripts. Jupyter Notebooks were an extension of scientific computational notebooks and became the de facto standard quickly after their introduction, for a large part because they provide remote access to data that might otherwise be impractical to download.

What makes Jupyter Notebooks more versatile than local programming scripts is that they can use both a local and remote backend to run code from the browser-based Notebooks. This means that you can create a Notebook from your local computer, that is run somewhere else, be it a supercomputer with a huge capacity or in the cloud, so you’re no longer dependent on your local computer’s resources for big data processing. This also takes away the necessity to download data locally and process it there.

Another benefit from Jupyter Notebooks is that you easily integrate existing Notebooks into your own workflows or collaborate with others: sharing a Notebook with others also includes the tool results (such as the output of a function), which is different from using single scripts that are run in an IDE where results are printed or returned in a console.

Python packages for geospatial analysis

Currently, there are many geospatial Python packages available that offer everything from geospatial data management to mapping capabilities inside a Jupyter Notebook. A LinkedIn post from Matt Forest from CARTO from August last year mentioned the most popular Python geospatial libraries, based on total PyPI downloads:

  • Shapely (89M): for manipulation and analysis of planar features;
  • geopy (83M): a Python client for several popular geocoding web services;
  • pyproj (48M):  performs cartographic transformations and geodetic computations;
  • Fiona (25M): for reading and writing vector data;
  • GeoPandas (18M): spatial data processing, based on pandas data objects;
  • Descartes (10.8M): enables the use of geometric objects as matplotlib paths and patches;
  • Folium (9.8M): visualizes data on an interactive Leaflet map;
  • Rasterio (9.3M): GDAL and NumPy-based library for raster data;
  • GDAL (2.8M): supports reading and writing capabilities for both vectors and rasters;
  • pysal (1.3M): for open source, cross-platform geospatial data science;
  • OSMnx (932K): download geospatial data from OpenStreetMap and model, project, visualize, and analyze real-world street networks and any other geospatial geometries;
  • ipyleaflet (922K): for creating interactive maps in the Jupyter notebook;
  • CartoPy (834K): designed for geospatial data processing in order to produce maps and other geospatial data analyses;
  • CARTOframes (583K): enables integration of CARTO maps, analysis and data services;
  • keplergl (410K): a web-based application for visual exploration of large-scale geolocation data sets;
  • GeoPlot (132K): a high-level Python geospatial plotting library.

If you’re not already using these tools as a GIS analyst or geospatial developer, it’s good to know that your skills determine your value to a company/client, and these tools will help give you an edge over the competition and keep you delivering innovative results into the future. To start using such packages, you can download the Anaconda distribution of Python, which includes the Jupyter Notebooks application as well. Anaconda includes conda, which lets you create a virtual environment for each Notebook, so you don’t have to worry about possible dependency conflicts between projects. Also, this separates the Notebooks (coding environments) from the package management environment (conda). Packages can be downloaded directly from conda, which comes with a default pip package installer for every new environment.

Jupyter Notebook functionality inside desktop GIS

To start using the Jupyter Notebook application inside a desktop GIS, note that the application is renamed ArcGIS Notebook inside ArcGIS Pro and comes with a default installation. QGIS users need to install the IPython QGIS Console plugin, which gives you access to the IPython Console inside of QGIS. The IPython Console allows you to execute commands and interact with data inside IPython interpreters.

This plugin requires you have the qtconsole Python package and the Jupyter Notebook installation installed. After installing the IPython QGIS Console plugin from the QGIS Plugin dropdown menu, you need to run two separate commands from the OSGeo4W shell as an administrator, which is explained in detail here. After this, you’ll find the IPython QGIS Console plugin listed under the installed plugin. Selecting it will open the QGIS IPython console, which gives you access to canvas, iface, app (QGIS application) objects and all qgis and PyQt core and gui modules directly from the shell.

Using the QGIS IPython Console
Figure 2: Using the QGIS IPython Console

Conclusion

Right after its introduction, Jupyter Notebooks have become an indispensable tool for GIS analysts to extend their desktop GIS environment thanks to the Python programming and scripting language. We explained how to use both Jupyter Notebooks with the Python language, and covered the most popular geospatial Python packages, and how to start using the IPython console inside QGIS and the ArcGIS Notebook application inside ArcGIS Pro. 

Python & Geospatial Resources

QGIS – what is it and why should I try it?

An open source GIS software project

When people first start to use open source GIS software, they almost always land on QGIS. It may be only for a test period but it has become so pervasive in the market and educational ecosystems that we have no less than eight QGIS books in our catalog! It is impossible to ignore.

Because there are so many new users looking at QGIS, similar questions tend to pop up when evaluating it. This blog post looks at some of the common questions that surface in online searches.

Is QGIS software free?

Part of the allure is that it is free to download, use, and share with others. It has enabled a whole generation of budding cartographers and analysts to flex their muscles without having to pay for access to their vocational tools.

Yes, QGIS is free. There is no cost to access it and you don’t need a license manager or online service to allow you to use it as much as you want.

To understand the ecosystem better one might ask, “Why is QGIS free?” as well. This gets to the heart of the open source software movement. Often, independent users and developers collaborate to create new software, share it, teach others about it, and build a community to maintain and care for it. QGIS is a great example of this kind of communal approach. It is important to know that it is free because the creator of it wanted to share it with the world.

Our own author and former publisher Gary Sherman needed a tool. So, he built it and shared it with the world. Well over a decade later dozens of developers help add features, while thousands of power users and educators help build momentum in the user community. Everyone has a way to contribute, because it is free and their personal investments of time and energy don’t go into a corporate black hole.

What is QGIS used for?

Anywhere there is a GIS need, QGIS can be used. Okay, maybe not everything, but all the basic functions of geospatial analysis, mapping, and data conversion are well supported. Connectors to many data sources and plugins with thousands of advanced functions help support a wide range of users. Earth observation users, water managers, agriculture planners, forest managers, and more, all use QGIS to design and communicate about their projects.

QGIS is not designed for just one industry or use case. At Locate Press we are being asked to produce more books and training events that target specific domains with QGIS and other open source products. For example, our QGIS for Hydrological Applications is a domain-specific title for those in the water management space. Expect more books, courses, conferences, and tutorials that will look at other domains.

Is QGIS easy to learn?

QGIS can be very easy to learn. Any GIS user could start using it in very quickly as the concepts and methods are very similar to other approaches. QGIS is being used to teach younger students as well, so age is not a barrier. The GIS education domain benefits from books like On the Way with GIS to teach young students GIS concepts.

In some cases you may just need a self-paced book to teach you. Others need a trainer or teacher, so they opt for a workshop environment. Regardless, you do not have to be a developer or super-geek to learn these tools. Just the basics of GIS and you should be fine.

How is QGIS different from ArcGIS?

As proprietary software, ESRI ArcGIS is not free (without cost). In certain circumstances, for students or developers, it can be possible to get a free version, but it is often very expensive to a small business or single user. It is also limited from a freedom perspective. You cannot copy, share, or distribute the product. And you can definitely not examine the source code or contribute to fixing bugs or adding extensions to the core product.

If the company changes their terms, increases their annual fees, or can improve the software to match your needs, then you are somewhat held hostage unless you move to an open source environment like QGIS.

Which is better, QGIS vs ArcGIS? ArGIS has many sophisticated cartographic and analysis tools and is very popular for corporate and government users. QGIS does not match all these features directly and a comparison matrix would be helpful to find. But in the end, many powerful maps and analysis are done every day using QGIS. What it may lack in polish compared to a billion dollar company, it makes up for in plugins, openness, and freedom-loving communities.

Can QGIS replace ArcGIS? It depends on your needs – you should try both. I suggest starting with QGIS because it is free, open, and won’t lock you in for profits in the future. Maybe it’s not for you and your company or school requires you to use ArcGIS – that is fine, but at least you have the option to learn and study at home if you ever want to.

How can I learn QGIS at home?

There are many trainers that teach QGIS in workshops, at conferences, both online and offline. The QGIS project also has a page of QGIS training materials that include user tutorial guides.

Locate Press also sells books (print and DRM-free e-book PDFs) that teach you about QGIS at your own pace (links at end of post). Introduction to QGIS is an entry-level introductory book. Discover QGIS 3.x looks at the latest features and dives even deeper. QGIS Map Design teaches specific cartographic techniques that GIS users will appreciate. On the Way with GIS and Open the Door to GIS use QGIS to provide a GIS education to students (including student and teacher guides). QGIS for Hydrological Applications, we mentioned above. And The PyQGIS Programmer’s Guide teaches developers how to write applications and plugins using Python. As you can see, these cover a widd range of users from cartographers to developers.

There are many other videos, blogs, books, and websites dedicated to QGIS training – a simple search finds many of them, especially watch for sessions from the FOSS4G 2021 event which was online.

Which version of QGIS should I download?

One of the biggest draws to open source is that there are free downloads of GIS software like QGIS. Downloads are available through the QGIS project website. They always have at least two versions available for download: a “latest release” and a “long term release (LTR)”.

What is the best version of QGIS? It depends on the user – for most users, the latest release will be fine. It includes all the latest improvements and features. Sometimes those new features cause bugs and you need to update when fixes are available.

On the other hand, the LTR version is focused on avoiding new bugs. There are still fixes made but new functionality is saved for other releases to keep this one as stable as possible for a longer period of time.

What computer specs do I need for GIS?

Note, I didn’t specifically call out the size of processor or how much RAM do you need to run QGIS specifically because all GIS software generally requires a modern computer to run efficiently. That said I often run QGIS on a 8 year old Macbook Pro.

Consider what you plan to do with it. Ultimately, your required specs will depend on how you want to use it. If you will be storing and analyzing data on a server using PostGIS, for example, you may not need a beefy machine just to draw the maps.

If you want to crunch a lot of data, it might still work on an older machine but just take a longer time. There are many tips, tricks, and best practises to make a GIS run more efficiently regardless of computer specs. So if you are trying something on QGIS, be sure to ask in the community forums if there are workarounds for your older or slower PC.

QGIS resources

I hope you enjoyed this quick walk through some of the more popular topics and questions about QGIS that are asked online. Here is a list of links to the various sites I referenced above. If you end up trying QGIS, let me know what you think on Twitter @locatepress. Good luck on your journey!

What is Geospatial Technology?

Geospatial technology brings tools and data together to describe, map, and analyze the world around us and worlds yet to be discovered.


The term geospatial is a relatively new invention at least in the parlance of mainstream developers. Geospatial can refer to types of data or to types of technology. The word itself is a combination of geographic and spatial – indicating an alignment between geography and the general idea of spatial/locational properties. Spatial concepts (think geometry and statistics) do not necessarily represent a place on a planet until they are combined with ideas of geography in general.

Built on the history of Geographic of Information Systems (GIS)

GIS is a technical domain, usually for geographers, that allows users to make digital maps and subject them to various types of analysis. Sources of GIS data may include satellite or aerial imagery (raster data) or line map data (vector data) delineating points, lines, or regions of interest – created by surveyors, engineers, photo interpreters, etc.

While many GIS projects output maps, their primary goal is to develop observations about a project area and overlapping properties and values. For example, land-use planning typically requires a GIS process to compare/contrast all the competing values – economic, social, environmental, etc. These are thought of as layers of spatial data that overlap one another and can be combined to show different management priorities or scenarios.

Where do deer live in the winter compared to a planned highway development in a popular tourist corridoor – many values in one location often need advanced tools to build a complete picture.

Geography made digital

While GIS helps bring geography into the digital domain, geospatial technology helps bring it to life for more people. Beyond specific GIS projects, there are many more data sources, cartographic products and ways to output maps for different consumers . Collectively, these fall into the region of geospatial data and technology.

Web-based mapping really helped propel the generalized use of geographic data into the mainstream. Before Google Maps was introduced in 2005, there were only a handful of common web-based mapping tools available for the public to use. Developers started to build their own open-source platforms to share information and collect input.

This required a whole stack of technology including geographic data, web servers, spatial databases, rendering libraries, web-interaction libraries (zoom/click/pan), and the internet itself. Geographers or GIS users may only be a small part of the overall project or not involved at all.

In the end, a handful of different technologies are needed to bring digital geospatial data to life.

Broader than just spatial analytics

Building new geospatial web-mapping tools was one part of the journey. Naturally, the more people use mapping tools, the more questions they want to answer. For example, consider how popular Google Maps became due to its driving directions. This level of spatial analytics was profoundly useful for those driving in a new location. But only a small set of built-in analytics was really ever possible with this platform – or so it seemed.

Data analysts and GIS users are used to running specific types of routines on data to get an answer. For example, calculate an optimal route from A to B. Or what is the expected water course derived from this elevation model?

However, with modern geospatial technology, the user may view and interact with the data in a more real-time approach to build understanding before they ever run an analytical routine.

They may never click a “analyze” button but can use a 3D map view to get a sense of where water will flow, or look at the streets around them to compute their own driving path in their head. In this sense, geospatial tools help them leverage geographic data in a context that is intensely personal.

Collection of mapping technology

So what tools and technology are considered geospatial in nature? As noted in the “stack” of technology above, it is a wide-ranging set of technology. It can be helpful to look at the two types of end-users that typically leverage geospatial technology: software developers and data analysts.

Geospatial developers take data of interest, depending on their domain, and create applications that allow their target audience to interact with the data in a meaningful way. This may mean taking data that is not always spatial in nature – like a list of addresses or stores running sales – and turn it into a component on a map for viewing and querying.

Location-based applications using GPS tracking on a device are also used by developers to give localized awareness of nearby data or attributes the developer wants to expose.

Geospatial analysts – often work more behind-the-scenes and provide types of data analysis outputs that get used by application developers, GIS users, or even in reports or web sites for general public consumption.

Geospatial analytics for all

Analysis with geospatial data components is not limited to one domain of analyst anymore. Data scientists or business analysts may combine data from many sources – spatial or not – to provide a common operating picture of a business or project.

Therefore, libraries and processes for analyzing geospatial data have become ubiquitous or are at least a common subset of analytical routines that many have access to. Both desktop and web-based approaches to sharing data along with analytical tools continues to grow in popularity.


Locate Press sells books for learning and applying geospatial technology, written by experts in their field:

The Need for Open Source Geo Books

Why we started the company in the first place – to expand the global knowledge base for our favorite tools and encourage new users to join in the fun.

In 2006 I was proud to help launch the Open Source Geospatial Foundation (OSGeo). My dream was to have every geospatial user around the world know about the same tools I had grown to love and use in my daily work. At the time I was primarily using MapServer, PostGIS, and GDAL. Others were using GeoServer, GRASS GIS, QGIS, and more.

But the products struggled to grow because the community, for many of them, was just getting started and there was little to no funding available to promote them. There was oftentimes not even enough funding for proper infrastructure, so spending it on marketing was not a high priority.

So I spent five years at OSGeo driving the marketing, outreach, and fundraising side of these projects and while doing so I heard many power users, academics, trainers, and professionals complain about the lack of structured training material.

In particular, professors had trouble breaking out of the proprietary toolsets because they had easier access to training materials. I wanted to see if we could fix that.

Bridging the geospatial knowledge gap

Enter Locate Press.

There is so much great information that stays locked up in the heads of developers, power users, and project managers. My plan was to work with existing experts in the field and help them share their knowledge in a useful form. Then trainers, educators, and professionals would have their knowledge more easily at hand. We used financial incentives to help them get their title into the hands of their fanbase and beyond (50% royalties is well beyond industry standard).

And there are great writers out there who also happen to be open source geospatial advocates, but these people are hard to find because they may not be so vocal in the project communities.

Finding and inviting these writers, and connecting better with broader communities, requires a different approach than we’ve had so far.

As part of a renewed strategy, I view the next 5 years of book development in two buckets: domains and technology. Every professional, every writer, and every product user fits into one of these.

Domains: building the profession

Some geospatial experts just see themselves as geographers or teachers and may never think that their knowledge or training approach is worth sharing. Others work in areas of business like forestry or cartography and consider themselves a sort of industrial niche area that only a few people care about.

These are domain professionals and warrant their own series. It’s not always about technology, but how you apply it to solve problems — in your business or on the ground. It’s not just about making maps or analyzing data, the spectrum of knowledge that is useful to share is much wider.

For example, the more entrepreneurial-minded bunch that already lead workshops, write blog series, or run podcasts already have the kinds of knowledge that others enjoy or they would have no audience. They know how to tap into the needs of a diverse set of communities and bring interesting information to light.

We need similar vigor in our professional and volunteer activities. How can the project you are working on today get into more hands? In what ways can it help others produce a product or solution faster or better?

A domain-focused book series is needed to help bridge the gap and help coalesce communities of interest around us all.

Tech: advocating for solutions

There are still many powerful, useful projects that are going unnoticed today. So marketing in general is going to help get the word out and bring in new contributors. There is an amazing amount of value in contributors who can’t code and may not even be “pros” at using a particular software product. They can still be powerful advocates for other users to learn from.

If this is you, then your advocacy and passion can still help products that you depend on to develop further. By bringing more attention to it through a book, it helps developers know their market is strong and helps professionals know the project is a serious endeavor (because it even has a book).

The end game

Ignoring for a moment that books do eventually get out of date, our work here will be done when everyone can get the training they need from a book, in-person training, or professional outreach at a conference. When they can then pull together a custom bundle of the particular products they need and have accompanying training material at a professional level, things will be amazing.

At Locate Press we’ve focused on technology titles and will continue to do so. However, as we move more and more into the different professional areas, a secondary focus domain-related topics will eventually emerge. At the end of the day there can be some very interesting outcomes.

Imagine with me if you will:

  • a box set of all the OSGeo project books
  • re-usable workshop guide books for the top 10 most popular platforms
  • how-to guides for new professionals in all major industries
  • pre-packaged training materials to support those delivering in-person training

These are all possible given the right focus and timing. I hope you’ll consider joining us in this quest.

If you have a book idea or know someone who would be a great author for us, please don’t hesitate to reach out. Even if it turns out to be mismatch, I’d love to hear from you.

In the meantime, check out our book catalog and let us know what is missing for you.

%d bloggers like this: