Console #151 -- Interview with Marcelo of Pretty Maps - Draw pretty maps from OpenStreetMap data
Featuring Duplicati, Gut, and Twitter Algorithm
🤝 Sponsor
This space is reserved for sponsors that support us to keep the newsletter going! Want to support Console? Send us a note at osh@codesee.io
🏗️ Projects
Browse through open source projects on OpenSourceHub.io, add your project to get more exposure and connect with other maintainers and contributors!
💾 Duplicati
Duplicati is a backup tool that allows you to back up your files to multiple cloud storage providers. It supports encryption, gzip compression, and incremental backups.
language: C# stars: 8626 last commit: last week
repo: github.com/duplicati/duplicati
site: duplicati.com
💻 Gut
Gut is an easy-to-use git client for Windows, macOS, and Linux
language: Go stars: 347 last commit: 3 days
repo: github.com/julien040/gut
site: gut-cli.dev
🐦 Twitter's Recommendation Algorithm
The source code for Twitter's recommendation algorithm released under AGPL license.
language: stars: 36,956 last commit: April 20, 2022
repo: github.com/twitter/the-algorithm
blog: blog.twitter.com/…/twitter-recommendation-algorithm
Join thousands of other open-source enthusiasts and developers in the Open Source Hub Discord server to continue the discussion on the projects in this week's email!
🎙️ Interview With Marcelo of Pretty Maps
Prettymaps is a small set of Python functions to draw pretty maps from OpenStreetMap data. Based on osmnx, matplotlib and shapely libraries.
language: Python stars: 9280 last commit: 16 Feb, 2023
repo: github.com/marceloprates/prettymaps
support: ko-fi.com/marceloprates_
Hey Marcelo! Thanks for joining us! Let us start with your background.
Hi! My name is Marcelo Prates and I live in Porto Alegre, Brazil. I completed my PhD in Computer Science in 2019, in which I mostly studied AI Ethics / Machine Bias and Geometrical Deep Learning / Graph Neural Networks. I have been working as a data scientist and AI researcher in the private sector since then, working for large companies (Samsung Research) as well as startups, and also as a freelance Artificial Intelligence / Data Science / Computer Vision consultant and an occasional creative coding teacher.
Although I experimented with programming languages before that, I only really became proficient in coding during my college years. My starting point for programming was the C/C++ languages, but I soon fell in love with Haskell, Python and Julia. But what really caught my attention and inspired me to develop a close connection with programming and computer science in general was the Processing programming language, meant for designers and visual artists and introduced to me by my friend Luis Mollman. Processing was my first contact with the generative art movement - creating artworks using computer code -, and I have been experimenting with creative coding ever since. My creative process now mostly revolves around Python and its ecosystem of packages. I’m interested in taking advantage of my AI and DS knowledge to help with my creative workflow, so I experiment a lot with scientific packages such as Numpy, Scikit-learn, Scikit-image, PIL, Matplotlib, Pytorch, Taichi as well as some domain-specific packages such as Geopandas / osmnx (for geospatial projects), Shapely (for constructive 2D geometry) and Vsketch (pen plotting).
What is your favorite software tool?
It’s hard to pinpoint a particular tool, but I’m fascinated by FFMPEG. I’m still very early in my process of learning how to use it to its full potential, but it’s really incredible what you can do with media using FFMPEG on the command line.
What is your favorite book and why?
I’m currently reading Walden by Henry David Thoreau and I can confidently say it’s one of the books that made me think the most about life, nature and purpose. Humboldt’s “Cosmos”, Darwin’s “The Origin of Species”, Douglas Hofstader’s “Gödel, Escher, Bach”, Wolfram’s “A New Kind of Science” and Sean Carrol’s “The Big Picture” also rank high in my list. I recently read Merlin Sheldrake’s “Entangled Life”, which sparked my curiosity by fungi, and Ed Yong’s “I Contain Multitudes”, which changed my perspective about microorganisms in general. My favorite fiction book series are probably Erico Verissimo’s “Time and the Wind” and Elena Ferrante’s “Neapolitan Novels”.
Why was Prettymaps started?
I have been interested in maps and cartography in general since childhood. Back then when I was still using Processing as my main programming language for creative coding projects, I started writing code to draw maps from OpenStreetMap data. At that time, I simply downloaded extracts from OSM manually using their web-based tool. When I finally moved to Python as my go-to language for generative art, I came to know Geoffrey Boeing’s excellent package Osmnx, which allows one to very easily fetch data using OSM’s API. I started creating maps of my home city and places I had visited using Osmnx for downloading data and Shapely and Matplotlib for drawing. My Mom and I visited France in 2018 and I decided to give her a map of Paris for her birthday. It was the most complex map I had designed yet, with captions describing touristic places, different color palettes and style parameters for each OSM geometry (buildings, parks, streets, etc.) and a mini-map drawn over the figure zooming into the aparthotel we stayed on during our trip. As my map-making abilities improved and I started experimenting more with stylistic choices, it became increasingly cumbersome to copy, paste and adapt my old scripts when I wanted to create a new map. So, in order to simplify my own creative process, I started organizing all that code into a Python module which would later become Prettymaps.
Can you give us an overview of how Prettymaps works?
Prettymaps allows you to generate a map from a query specifying a location using a very simple syntax, for example:
import prettymaps plot = prettymaps.plot(“Porto Alegre”)Behind the scenes, Prettymaps uses Osmnx to invoke OpenStreetMap’s API and download all the required geometries (buildings, water, parks, forests, streets, etc.) to draw on the map. You can specify these geometries manually when calling prettymaps.plot(), but if you don’t, as in the above example, Prettymaps will simply use a default, pre-loaded “preset” configuration saved as a JSON file in the Prettymaps installation folder. There are other preset options available in addition to the default preset, and they can be selected with the “preset” parameter of the prettymaps.plot() function, for example:
plot = prettymaps.plot(“Porto Alegre”, preset = “minimal”)If you want to specify which geometries to download from OSM manually, you can use the “layers” parameter of the prettymaps.plot() function (see example below). By default, Prettymaps will load a preset configuration and update it with whatever you specified in the “layers” parameter, but if you really want to start from scratch, you can set the “update_preset” parameter of the prettymaps.plot() function to False.
plot = prettymaps.plot( “Porto Alegre”, layers = { “building” : {“tags”: {“building”: True}}, “water” : {“tags”: {“natural”:[“water”, “bay”}}, } )The same goes for the “style” configurations. But instead of specifying things using the OpenStreetMap API syntax, which is in charge of downloading geometries, we use the Matplotlib syntax, which is in charge of drawing them (actually Matplotlib shares the credit with Shapely, a very useful tool for manipulating polygons).
plot = prettymaps.plot( “Porto Alegre”, style = { “building” : {“facecolor”: “#f00”, “linewidth”: 1}, “water” : {“facecolor”: “cyan”, “linewidth”: 2.5}, } )There are a lot of other cool features which I don’t have time to describe here, but the Prettymaps GitHub repository README contains a tutorial explaining almost all you need to know to explore Prettymaps’ map-making capabilities.
Who, or what was the biggest inspiration for Prettymaps?
Prettymaps wouldn’t exist without Geoffrey Boeing’s Osmnx package, which makes accessing OpenStreetMap data so much easier. I could also mention abey79 and his super interesting tools for creative coding with pen-plotters (Vsketch, Vpype, Lines) as inspirations for me. They made me reflect a lot about what should be my relationship with software, from a creative coding perspective. By studying Vsketch I realized that instead of creating scripts for each artwork I make, I can create generic tools to ease my creative workflow.
Are there any overarching goals of Prettymaps that drive design or implementation? If so, what trade-offs have been made in Prettymaps as a consequence of these goals?
As I was developing the first version of Prettymaps I was conflicted because on the one hand I wanted the package to be as easy to use as possible, but on the other hand, I was not willing to lose the flexibility I used to have for experimentation in my earlier scripts. So as I kept experimenting with new things in terms of map-making, I was always trying to improve Prettymaps to enable me to generate those new results with as few lines of code as possible. I’m happy with the current result as of version 1.0.0, I think I achieved a reasonable balance between flexibility and ease-to-use with some help from the new features. There are still some things you will have to do manually, sometimes you’ll have to modify the GeoDataFrames obtained from OSM, sometimes you’ll have to mess with Matplotlib configurations outside the prettymaps.plot() function, but I have tried to make these processes as easy as possible. And Prettymaps is still a work in progress, so I’m constantly paying attention to suggestions and my own intuition to make things even easier in the future.
Are there are any projects similar to Prettymaps? If so, what were they lacking that made you consider building something new?
As I mentioned before, Prettymaps is built on top of Geoffrey Boeing’s excellent package Osmnx, which allows you to download and manipulate geometries using OpenStreetMap’s API. Osmnx allows you to display these elements, but it wasn’t flexible enough to allow me to generate really colorful and beautiful maps. I built Prettymaps out of this desire; I wanted to enable people to tap into Matplotlib’s capabilities to create customizable maps from OpenStreetMap data with as few lines of code as possible.
What was the most surprising thing you learned while working on Prettymaps?
Building Prettymaps changed the way I relate to software from a creative coding perspective. Before Prettymaps I relied on tools or languages built by other people to create my art. Seeing that a lot of people were using and enjoying a tool that I had built made me realize that I can alternate between the roles of user and developer of the tools I use in my creative process. This opened up a lot of possibilities and changed my perspective: I now build software with the purpose of making my life easier in the long-term as a generative artist.
How did the project get popular?
I’m not 100% sure, but I think Prettymaps’ popularity exploded when I created a post on Reddit’s (probably in the /r/DataIsBeautiful subreddit) sharing a new map-making tool I had created. The post caught the attention of a lot of people and someone then wrote a post about Prettymaps on HackerNews, which suddenly reached the top page and shortly afterwards the top position on the top page. From that point onwards, Prettymaps’ surge in popularity never ceased to amaze me. I was probably surprised the most when GitHub’s official Twitter account wrote a Tweet advertising Prettymaps: “Build beautiful custom maps from OpenStreetMap data? Yes please, and thanks @marceloprates_!”. As of this moment, the project has 9265 stars on GitHub and is ranked the 2193 most starred GitHub repo. I’m confident it will eventually reach 10,000 stars and plan to celebrate that achievement with some new features.
Where is it being used?
A lot of people use Prettymaps to create maps of their own cities or places they visited to put on their walls or to give them as gifts, which brings me a lot of joy. I have also been contacted by people who wanted to use Prettymaps to create maps for educational, academic or professional purposes. This will become simpler when I implement some new features I have planned for making drawing text easier (that is, enabling Prettymaps to automatically provide legends for important buildings, streets, parks, lakes, beaches, etc). Unfortunately, some people also have used Prettymaps to sell unauthorized and uncredited NFT projects, which I’m 100% against. I cannot forbid anyone to do that, legally speaking, but I keep an anti-NFT message on the Prettymaps README so that people know they’re disrespecting the creator’s wishes when they do that.
Can you provide any examples of how Prettymaps has been used for educational or academic purposes?
Some people have used Prettymaps to create infographic-style maps, and I was contacted by Geography professors who study OpenStreetMap telling me that they were using Prettymaps with their students
What is your typical approach to debugging issues filed in the Prettymaps repo?
I’m not nearly as active in the Prettymaps repo as I’d like to be, but I try to solve bugs or simple or urgent issues as soon as they appear. There are several issues communicating that some things are currently impractical to do with Prettymaps without resorting to manually manipulating GeoDataFrames. Since some of these missing features require rethinking the way that Prettymaps downloads, modifies and draws data, I try to answer them while I code my own new features into Prettymaps. I tend to do this at a slower pace, because I like to experiment with my new features on my local machine a lot before deciding whether they are really useful or worth it.
What is the release process like for Prettymaps?
It didn’t really have a consistent release process until the most recent version, which is 1.0.0. From now on, I plan to release new versions at a pace I feel comfortable with. I think I performed enough refactoring and redesigned enough the way Prettymaps works to make the process of issuing new releases without breaking backwards compatibility simpler from now on.
Is Prettymaps intended to eventually be monetized if it isn’t monetized already? If so, how? If it’s already monetized, what is your main source of revenue?
It’s not monetized and I don’t intend to. I’m at a point in my life in which I’m reflecting a lot about what kind of impact I’d like my software to have in the world, and also about what type of internet I’d like to help build. OpenStreetMap is an amazing free technology built with the help from people all around the world who work for free providing data to help us build an open geographical database of the entire planet. I like to think of Prettymaps as another tool in that ecosystem, and my vision for its development is that people can use it freely to exercise their creativity. No one should be required to pay money in order to create a map of their own city.
What are you most proud of?
I was pleasantly surprised when Prettymaps reached the 1st page and afterwards the 1st place in HackerNews, shortly after it was released. I also wasn’t expecting GitHub’s official Twitter account to advertise Prettymaps. That was awesome. Prettymaps put me in touch with geography professors, urban planners, geospatial programmers, artists and a lot of regular people fascinated by maps. I wasn’t aware that so many people shared my love for them. Because of Prettymaps I got some offers to do map-related freelance work, which has been very personally rewarding. I’m interested by many different subjects, maps in particular, but my academic and professional experiences (in data science / machine learning) are not particularly related to cartography. I’m grateful to Prettymaps for opening up that possibility.
How do you balance your work on open-source with your day job and other responsibilities?
I’m at a very interesting stage in my relationship with creative coding and generative art; experimenting a lot, constantly learning new things and building tools all the time to help me with my creative process. So it’s naturally hard to find a balance between work, personal projects and leisure time. I often felt that I sacrificed my rest to work on personal projects, and this is a problem because it affects my creativity and sometimes even leads me to creative blocks. I have several projects in the making that I plan to release as open source tools, so I eventually made the decision last year to take a sabbatical period to focus on them (and on me) and free up more time for personal projects without sacrificing my own rest.
Have you ever experienced burnout? How did you deal with it?
Prettymaps grew faster and bigger than I ever anticipated it would, so I was initially very overwhelmed by all that attention and felt that I had a responsibility to fix issues and review merge requests as soon as they appeared. I was obviously very happy with the popularity, but I was also uncomfortable with the visibility that I, as the creator, was receiving on social media. This vwas one of the reasons why I decided to delete my Twitter account and chose to have a more discreet presence on the internet. I still post my artworks on Instagram, but I feel happier having less followers and visibility because this allows me to really focus on my particular vision for creative coding, generative art and Prettymaps itself. I appreciate suggestions and regularly incorporate them into the package, but I don’t really want to let other people guide my creative process too much.
What is the best way for a new developer to contribute to Prettymaps?
I’m grateful to many Github users who pointed out performance issues, bugs and limitations of Prettymaps which I wouldn’t be able to learn for myself since I’m mostly focused on other soon-to-be-released generative art projects. I think one of the main problems with Prettymaps at the moment is the lack of a more comprehensive documentation. I prepared a tutorial as a Jupyter Notebook and attached it to the README and I think it should give a nice overview on how to use the most important features, but I realize it’s not enough and welcome people to help me explain better how to use the package to its full potential.
Where do you see the project heading next?
There are some unreleased features I have developed and used to generate artworks for myself, such as the capability of importing elevation maps into Prettymaps and drawing hillshades and curve levels alongside with the OpenStreetMap geometries. I’m working on a Python package to make the process of choosing color palettes easier, and I’m very interested in integrating my different creative coding projects, making one package talk with the others in creative ways. I’m also working on a Python package I named “Easyshader” for rendering static renders and animations with a very simple syntax powered by Signed Distance Functions, and I’m thinking of ways to integrate with Prettymaps to make drawing landforms easier and intuitive.
Are there any other projects besides Prettymaps that you’re working on?
Yes!! My sabbatical has given me time to work on multiple very personally rewarding projects at the same time. I’m working on some generative art tools to help me with my creative workflow, such as a ML-powered “color palette manager”, and an experimental “versioning” tool to allow me to annotate generative art-related Python functions so that their (image) outputs are automatically written to image files with all the required information to reproduce that exact result saved into their metadata. I like to think of this as a first step towards the concept of a “computable” and reproducible generative artwork media format I’ve been developing.
I’m also working on a computer graphics library for generating static renders and (possibly interactive) animations I named “Easyshader”. It was built using raymarching and Signed Distance Functions (SDF) and allows one to build 3D shapes using a very simple syntax based on constructive solid geometry. I’m still working on some new features and on the documentation and plan to release it later this year.
Other than that, I’ve been exploring with building several tools as an effort to build my own personal “Photoshop” in the format of an ecosystem of Python packages in charge of things like: easing the manipulation of image files with a simple syntax for pixel-wise mathematical transformations, a “pixelizer” inspired by vintage gaming consoles, a sandbox for simulating acellular life forms such as the protist Physarum polycephalum and biological morphogenesis processes inspired by Alan Turing’s reaction diffusion equations, tools to help me tap into the full potential of pen-plotting machines such as AxiDraw V3, and many more.
Finally, I’m currently editing the fantastical 5-volume book series “Cosmos” by Alexander Von Humboldt and plan to release the first volume soon and request help from the open source and scientific communities to finish the series. My vision for these books (now in the public domain) is that they’re edited beautifully, adorned with high-quality pictures and diagrams either in the public domain or available as open data and, most importantly, that Humboldt, who is sadly forgotten by our society, can be once again accessible to, read by and valued by new generations of young people curious about the wonders and beauties of the natural world.
Where do you see open-source heading next?
I’m pessimistic about the effects of some new trends such as NFTs and Large Language Models in respect to open-source. I’m not comfortable with the idea of companies such as OpenAI scraping code from GitHub without compensation to open-source developers and maintainers to build AI-assistant tools for code which will be later turned into paid services. I see this as a sort of privatization of the open-source space. The same can be said about NFTs: it has become common for groups of people to use open-source tools, particularly related to generative art, to build collections of artworks and make money with them without, again, compensating or asking permission to creators.
Nevertheless, I have a vision for open-source which I intend to help make a reality. I believe that the internet is home to a vast amount of open and public data which could be made more accessible to all people. Prettymaps is a testament to that because it helps people tap into the immense potential of OpenStreetMap data and use it to create art.
Do you have any suggestions for someone trying to make their first contribution to an open-source project?
I don’t have much experience contributing to other projects, but one thing I wish I knew better as I was starting is how licenses work. I think if you’re starting your own project, particularly if it’s a generative art project, you should understand how you can protect your creations from people who intend to use them to make money. I also think that, from a generative art perspective, you should try your best to separate your artworks from the tools you’re developing to create them. This enables you to share the tool with the world but keep the artworks for yourself.
Want to join the conversation about one of the projects featured this week? Drop a comment, or see what others are saying!
Interested in sponsoring the newsletter or know of any cool projects or interesting developers you want us to interview? Reach out at osh@codesee.io or mention us @ConsoleWeekly!