OpenStreetMap's New Vector Tiles

(tech.marksblogg.com)

250 points | by marklit 6 hours ago ago

85 comments

  • maliker 5 hours ago

    It's been fascinating to watch the open source community build out vector map tile capabilities. I was doing some web GIS work back in roughly 2018, and Google/Apple's streaming vector maps performed like magic and something we would have loved to use if we could afford it. Shortly thereafter the core tech was available in open source, and then there were even free hosted solutions. Now our leaflet maps have great vector layers for free. Thanks open source!

    • NelsonMinar 28 minutes ago

      I'm a little surprised it's taken this long for OSM to get there, the basic technical pieces were available over a decade ago. I don't mean to complain about the free map service, it's excellent, and I recognize they focus more on the editing and data ownership. Serving is hard and expensive.

      Mostly I wonder how much MapBox's dominance for a few years disrupted other efforts.

      • Firefishy 15 minutes ago

        The new stack has some unique feature like the vector tiles being minutely updated directly from OSM mapping changes.

        There are still issues to fix as it is still only a technical preview.

  • zawaideh 3 hours ago

    The other issue is that the arabic font is not rendering correctly in the vector version. It's rendering left to right (instead of right to left) with characters broken up instead of linked.

    • sp8962 2 hours ago

      The whole point of vector tiles is that the rendering is local and controlled by a style configuration (except for the tile schema) that can be changed. So the brokenness you are seeing is either in the style or the library that is rendering the contents locally.

      • dnpls an hour ago

        it looks like the selected font doesn't support arabic text, producing the disconnected characters.

      • SSLy 2 hours ago

        Indeed, but that is probably not going to help end users of apps.

        • sp8962 an hour ago

          Well nobody claimed things are going to get simpler.

          It is difficult to beat raster tiles in that respect. vector tiles split up responsibility for what you get visually over multiple moving pieces with different provenance and operators.

          • astrange an hour ago

            For Arabic specifically, different users want text to render differently, so you'd want to do it on the client if you can.

            (Some countries use Roman numerals and some use Arabic numerals for numbers. And by Arabic numerals I don't mean 1-9.)

          • lxgr an hour ago

            > It is difficult to beat raster tiles in that respect.

            Intuitively, why can't the local vector rasterizer do whatever the server-side tile rasterizer does, especially if both are fully custom?

            • wiredfool an hour ago

              Partially because they're completely different stacks -- the client side is WebGL + Javascript, and the server side is whatever they've been doing for 15 years.

              They're probably missing a raqm/freebidi or something in the stack on the client side.

              • lxgr 32 minutes ago

                Ah, so the input into the client-side isn't OSM-like data (i.e. OSM XML or some high-level transform of that), but rather something closer to a vector graphics format like SVG?

                That makes sense then, thank you!

            • maxerickson an hour ago

              It can, it's just that they typically don't, because they use an off the shelf data schema for the tiles and a library to handle the rendering.

              Having the data in tiles also complicates some things (where the renderer needs to consider merged tiles to get a similar result).

  • kibwen 5 hours ago

    Does this reduce the operating costs of hosting OSM-based maps, since presumably they require less CPU spent on rendering and vectors consume less storage/bandwidth?

    • hamiltont 4 hours ago

      Yes and No.

      No because the official OSM tile layer is heavily subsidized by Fastly (€720k last I checked) and rendering by AWS (€40k)

      Yes because technically it would use fewer resources thus easier on AWS+Fastly and also easier to self-host

      In last risk assessment I read closely(1) OSM noted "If we lost [Fastly] sponsorship, we would likely cut off all third-party access to the standard tile layer and run a small number of Varnish servers."

      As I understand it, primary drivers for vectors was not cost more improving internationalization, generally enabling client-side rendering decisions, and driving a modern toolchain that would net multiple follow-on benefits

      I'm a bit behind, there is more recent info available at (2)

      1.https://operations.osmfoundation.org/2024/01/25/owg_budget.o... 2. https://osmfoundation.org/wiki/Finances

      • treyd 2 hours ago

        I wonder if it would be feasible to distribute tile data in a DHT. There is a single source of truth that could seed the DHT and users would share the load of serving the data. It'd have to be opt-in on individual nav devices for privacy and battery reasons, but anyone running their own OSM proxy would have an incentive participate.

      • Borg3 an hour ago

        Yet you forget that tiles based maps are plays very nicely even with simplest HTTP caching (even multiple layers of them). Compared to vector stuff that needs caches that are range requests aware, or some magic block storage.

        I somehow prefer to stick to tile based maps because caching, easy rendering and I also care about sat images, with cannot be vectorized.

        I think we need both of those.

        • Symbiote an hour ago

          These are tile-based maps — vector tiles, rather than raster tiles.

          Any caching you do on raster tiles also works here.

    • sp8962 3 hours ago

      No, because you've been able to self host (or have somebody host them for you) vector tiles for a long time with very little effort, and yes that will somewhat offload processing to clients, and, more importantly allow many styling decisions to be made by the client (but not all).

      Static or infrequently updated vector tiles can be generated from OSM data by a number of tools, but those most popular right now are https://github.com/systemed/tilemaker and https://github.com/onthegomap/planetiler

      The actual -new- thing is that the work Paul has done for the OSMF allows on the fly (aka in minutes) updates of the vector tiles. This is important for OSM contributors as a feedback mechanism and the main reason the OSMF operates the current raster tile service.

      What is currently a bit out in the open is which usage restrictions will apply to to using the vector tile service as, just as with the raster tile service, the intent is not to compete with or replace third party services and a vector tile service could potentially do that.

    • speedgoose 4 hours ago

      Yes, vector tiles are much easier to self host.

      You have for example https://protomaps.com/ or https://openmaptiles.org/ or https://versatiles.org/ (random order).

    • hyperknot 4 hours ago

      openfreemap.org creator here. Yes, with vector tiles you are basically hosting static files, the server has nothing to do, except HTTPS encryption. Even gzipping is already done in the tiles.

      • TrickyFoxy 3 hours ago

        For vector tiles osm.org this is not the case. They should be generated on the fly from the database to show mappers the current state of the map with minimal delay. Yes, the resulting results can be cached like static files, but much more work is done on the server.

        You can learn more about this in the blog of the developer who develops this tile server: https://www.openstreetmap.org/user/pnorman/diary/403600

        p.s. current link to the demo page: https://pnorman.github.io/tilekiln-shortbread-demo

        • hyperknot 3 hours ago

          OP asked "Does this reduce the operating costs of hosting OSM-based maps".

          openstreetmap.org has a very complex setup for real-time updates, but in general, hosting OSM-based maps is much cheaper with vector tiles.

  • Someone 4 hours ago

    > Imagery should appear much sharper and switching the language of the labels should become possible.

    I expect that to work sub-optimally. Label dimensions are far from guaranteed to stay the same if you change language, and label dimensions interact with map layout, even influencing what to show.

    If your labels grow larger, they may end up covering too much of the map or even overlapping. If they grow smaller, users may wonder why a city that was omitted before because of space constraints doesn’t show in the empty space created.

    • maxerickson 4 hours ago

      The starting case is anyway automatic positioning of the labels.

      Even on the raster tiles that have been in use for years.

    • dheera 4 hours ago

      I hate it when city and street names disappear on Google Maps as you zoom in and out. Sometimes a street or business name only appears at one particular zoom level and not higher or lower. Just show the name of every street damnit. I don't care about crowding. Standing at an intersection staring at an unlabelled map is a useless map.

      • PrismCrystal 4 hours ago

        Younger generations are much less likely to navigate by comparing a map to the street names at an intersection. Instead, people navigate by searching for a particular destination and then following the line that the router generates.

        When Google Maps is a one-size-fits-all product, you can't blame them for choosing an approach like this. Fortunately, the OSM ecosystem lets you choose (or develop for yourself) whatever approach you prefer.

        • hackmiester 4 hours ago

          That method just doesn't work in Manhattan, where due to buildings, you're lucky if your GPS is working, much less the compass to tell you which direction to face.

          • PrismCrystal 3 hours ago

            The vast majority of Google Maps users are capable of using A-GPS data, they are not reliant on a clear GPS satellite signal alone. And Google’s A-GPS data for Manhattan is extremely detailed. Again, this makes sense for a one-size-fits-all product like Google Maps.

            • astrange an hour ago

              That's not good enough, AGPS doesn't work near skyscrapers. The issue isn't that the signal isn't "clear", it's that it reflects off the buildings and the GPS receiver will get a clear but wrong signal.

              To correct this you need something like QZSS or accurate models of the buildings to compensate for it.

      • c0nsumer 3 hours ago

        On a couple recent trips I ran into this when trying to figure out what river I was near / crossing. Multiple times I basically said "lemme get a better tool for this" and flipped to OsmAnd. Google Maps just basically/barely showed there was a channel of water and didn't show the name.

        • dheera 3 hours ago

          Also not to mention while navigating Google Maps stupidly hides all the business names en route. I want to see every single business along my route, damnit, so that I know where I could possibly go to eat or stop for a rest along the way.

          The Google Assistant is also the most useless piece of crap ever. "OK Google show me all the businesses on the map" doesn't work, and neither does "OK Google zoom in" for that matter. Most useless UX ever.

      • kevincox 3 hours ago

        > Just show the name of every street damnit

        I don't know about every street. But if you zoom in all of the way then yes, every street more than slightly in the viewport should be labeled. I have the same problems with all sorts of things like lakes where you need to find the magic zoom level and map position that reveals the name.

        Some things are understandably harder, I don't necessarily need the Country, Province and City in every view of the map. But streets and lakes tend to not have much stuff inside of them so it seems obvious that when zoomed far in they should appear.

      • SoftTalker 2 hours ago

        And zoom the label font size with the map size. I'm an old guy and cannot read the text on the map. Try to zoom in... the map zooms but the text stays the same size. Maddening.

        I don't recall offhand whether it's Apple or Google maps that do this. Maybe both.

      • enriquto 4 hours ago

        > Just show the name of every street damnit.

        But why? Google Maps is not a navigation aid. Its purpose is not to help you know the name of the street you are in. It's a tool for paying customers to steer you towards their shops. If all you need to do is follow a path towards a certain shop, they don't need to show you the names of the streets.

        • astrange 3 hours ago

          Google doesn't really optimize to show you ads. They're a monopoly and the voting shares are owned by Larry/Sergey; they have no intrinsic motivation to care about anything. If you want to be optimization-brained (don't, it's bad for you) they optimize for metrics that individual people make up and use to get promoted.

          They monetize the map other ways, like with the expensive API fees, but I wonder if they really care about it. Whenever I look at it for a few seconds I see something obviously sloppy, like misspelled POI names or people adding fake businesses at their apartment.

        • LeifCarrotson 4 hours ago

          Google Maps is a navigation aid that's also used to sell ads.

          People don't open the app to look at the ads. They open it to navigate, and if it can't be used for that purpose they won't open it at all. It is infuriating to me when a Wendy's ad is given more visual importance in color and size on the app than the actual place I want to find. But it doesn't seem to bother a lot of other people. Somewhere further in the direction of more advertisements is a point where a significant number of normal people will stop using Maps as a navigation tool, and somewhere further still is a point where more advertisements will become less profitable and will be rejected by even the developers. But we're nowhere near that point yet.

          Until then, I'll keep using a combination of Garmin Explore, OsmAnd, and Maps depending on how much I care about topo data, traffic status, reviews, search results, and ads. I'm setting up an Owntracks so I can replace the Timeline data that they're removing customer access to, and contributing to OpenStreetMap, but still using Maps.

    • jayd16 2 hours ago

      Why would you expect the layout algorithm to be significantly different?

      • teraflop 2 hours ago

        It's not that the layout algorithm is different, it's that the algorithm tries to optimize the positions of text labels to get an aesthetically pleasing spacing, while preventing them from overlapping each other.

        If you keep the label positions the same, but translate the text, then the layout will have been computed with the wrong bounding boxes, and you will tend to get wrong spacing and unintended overlaps.

  • boramalper 4 hours ago

    Also see: OpenFreeMap — free OpenStreetMap vector tile hosting

    https://openfreemap.org/

    • lxgr an hour ago

      Wow, this looks incredibly polished!

      Is this based on the same vector tech stack at all, or is it a completely parallel development?

  • timwis an hour ago

    Does anyone know if vector tiles could carry adequate data to facilitate reverse geocoding (address lookup)? E.g. the polygon of a building containing its street address in its metadata. I’m surprised I haven’t seen that before, as reverse geocoding services can get expensive, though I wonder if it’s because I’ve misunderstood how vector tiles actually work.

    • lxgr an hour ago

      Possibly, but why would you want to do this client side? This seems like the type of operation that makes much more sense in "OSM space", not vector graphics space.

    • durkie an hour ago

      Yes, they could. You can attach whatever metadata you want to a vector tile feature.

  • tetris11 4 hours ago

    (Low effort comment) I wonder if this means OSMAnd and OrganicMaps will now finally team up to deliver the ultimate FOSS Map app

    • sp8962 3 hours ago

      Both apps mainly use downloaded/"offline" preprocessed map data in their own formats.

      MVT format vector tiles are not remotely suitable for navigation or search (not ruling out that something could be hobbled together, but it would be a bit of a stretch).

    • worble 4 hours ago

      At the very least I hope it means they can share map tiles between them so I don't have to use gigabytes of space on the same map

    • fsflover 4 hours ago

      There is Organic Maps already.

      • szszrk 3 hours ago

        And it works really well. The smooth zoom in is amazing.

    • sriacha 4 hours ago

      How so?

      • tetris11 3 hours ago

        The fantastic offline routing and metrics of OSMAnd placed into a lightweight and snappy OrganicMaps interface would be a huge win for me.

  • Tepix 3 hours ago

    This is a very welcome new development! I look forward to even better looking maps.

    I'm just a bit puzzled by the "my workstation" section :-). There doesn't seem to be any relation to the rest of the article, not even high system requirements to do the things discussed after that.

  • shpx an hour ago

    I can't believe they still haven't added a 2x pixel density version of the default map style. It's $CURRENTYEAR and we all have high pixel density phones. No one uses 1080p displays anymore. The default blurry version makes any page using OSM feel amateur.

    • Firefishy 30 minutes ago

      We don't offer 2x raster tiles because we simply don't have the resources to do it while keeping the tiles minutely updated and open access. We serve around 60k req/sec on a tiny donation backed budget.

      The raster tiles are primarily for our OSM mappers to see their map changes rendered quickly to improve the feedback loop.

    • Doctor_Fegg 32 minutes ago

      Raster tiles hosted at tile.openstreetmap.org are not intended for widespread production use, they’re principally for mapper feedback. If you have a business need for something that doesn’t look “amateur” you should arguably be using a third party OSM-based service or hosting your own tiles.

    • junaru 27 minutes ago

      > No one uses 1080p displays anymore.

      I live in a bubble, the post.

  • emursebrian 5 hours ago

    This will be great for custom maps.

    I am on QGIS 3.30 and many of the street labels at 1:31860 scale show up in red and are not aligned correctly.

  • Foofoobar12345 4 hours ago

    Exciting update! How does the new vector tile system handle performance on low-end devices with large datasets?

    • sp8962 3 hours ago

      ... it doesn't?

      One of the downsides of MVTs and the typical max zoom level of 14 is that that they do require more local resources than simply rendering a 256x256 bitmap.

      For OSM the question is naturally would a complete migration (aka turning off raster tile support) exclude any noticeable number of people from contributing.

      But right now that decision is still a long way off, the vector tile service hasn't even been integrated in osm.org yet.

  • gregoriol 4 hours ago

    Is there a tool to capture vector tiles to raster files? Some tools like Apple's MapKit don't support vector tiles yet, and it could be interesting to use one single map base and be able to just generate the raster, instead of having a specific raster server

    • durkie 3 hours ago

      Something like https://github.com/maptiler/tileserver-gl can serve raster tiles from a vector tile dataset

    • fingerlocks 4 hours ago

      Apple’s MapKit most definitely supports vector tiles, because you are given a CGContext to draw with. You just have to implement the MVT spec (it’s small)

      • gregoriol 2 hours ago

        The idea is to use "MapKit" for maps, not to override it and draw everything yourself... The fact that it's possible doesn't mean it's a good idea.

        It is however possible to use Maplibre, Mapbox or some other libraries which I think do support vector tiles.

    • chris_wot 4 hours ago

      I am absolutely amazed that Apple don't support this yet. I had thought mapping was a competitive advantage for them...

  • Eduard 3 hours ago

    is there already a live version of this? Because for me, there doesn't seem to be an option for activating vector tiles on https://www.openstreetmap.org/

  • ajsnigrutin 5 hours ago

    > Imagery should appear much sharper and switching the language of the labels should become possible.

    So, every eg. city-shape-vector (well.. polygon) will cary the metadata/text of the name of that city in every defined language?

    • InsertUser 9 minutes ago

      The tile schema being used for the current testing (shortbread) doesn't support many languages, but they I think those working on this want to support them once the bugs are worked out.

      For an example of how this might look see e.g. https://americanamap.org which uses OSM data, but isn't aimed to be near live like the tiles this thread is about.

    • roelschroeven 4 hours ago

      Large cities already contain the name in many languages. See Paris for example: https://www.openstreetmap.org/relation/71525

      The usefulness of being able to switch the language depends on the availability of translations of course. It might give editors an incentive to add more translations for place names.

      • sp8962 2 hours ago

        Just to nip this in the bud, OpenStreetMap in general doesn't contain "translations" it contains the exonyms that are commonly in use for geographic objects. Most of the time things only have a name in the local language so there will be no value for other languages in the OSM data. Transliterations are a bit of a grey area in this context, but are definitely more useful than actual translations which tend to be garbage.

        Further point: the data available in the vector tiles is defined by the vector tile schema and by far doesn't contain "everything".

      • M2Ys4U 3 hours ago

        If there's a Wikidata QID associated with it then one can also use that to look up names in other languages

      • maxerickson 3 hours ago

        The tiles aren't necessarily a 1:1 translation of the data.

        The official tiles will probably be close though.

      • ajsnigrutin an hour ago

        Sure, but that's contained in the database, and then the tiles are generated that only contain one text (usually) in a png form).

        Now all the metadata from the table will have to be put into SVG tile files.

    • tantalor 5 hours ago

      No but you could fetch tiles with only the text you need to patch the tiles you already have.

  • DemocracyFTW2 6 hours ago

    One clear advantage of the new SVG format is, apparently, that Arabic script is now, finally!, rendered the way it was always intended—left-to-right with unconnected letters /s

    • memsom 5 hours ago

      I wish they were SVG, that would make rendering them less of a headache.

      There is still no good library which takes in a MVT tile and spits out the appropriate PNG or JPEG for rendering in via a tile base mapping engine. There is still no good cross platform mapping engine that can render vector tiles in a way that is easy to consume. There are certainly engines on specific platforms, but unless we use something like Leaflet or OpenLayers it is hard to make it work with native APIs on, say Windows, MacOs, Linux, iOS or Android without needing to adding a whole browser engine on top of your app.

      • mxfh 2 hours ago

        If I would like to print vector-tile based maps to pdfs for handout's or something, is there any CPU renderer, server- or client-side that could output SVGs instead of rasterizing on GPU to bitmap at some point?

        I havent yet come across any renderer that would do this even partially, even before opening the whole can of worms that is text rendering.

        Closest thing I'm aware of might be ArcGIS Maps for Adobe Creative Cloud but would need something that's more like a library and preferably FOSS.

      • dvdkon 4 hours ago

        There are plenty non-web vector map renderers, but they generally provide a full GUI widget, not a simple "extent -> png" function. I don't think creating that would be too much work, though: Just set one of those libraries to render to a texture.

        Maplibre Native even seems to have a headless "render to PNG" backend: https://github.com/maplibre/maplibre-native/tree/main/platfo...

    • rob74 4 hours ago

      So while showcasing the brand-new vector tiles, they accidentally also showcased a pretty significant bug? Should have picked some city with non-latin script that's written left-to-right... er, maybe Athens?

    • SigmundA 5 hours ago

      Mapbox Vector Tiles are not SVG, its is similar to geoJSON but encoded in protobuf. The renderer is going to typically be WebGL based in the browser written in javascript.

      https://wiki.openstreetmap.org/wiki/Vector_tiles

      https://maplibre.org/maplibre-gl-js/docs/

    • rafram 5 hours ago

      Yeah, I’m curious if that problem is baked into the SVG, or if the renderer has a “don’t screw up Arabic” option that’s disabled by default. You’d think nobody would design software that way, but Photoshop has that option, and it really is turned off by default!