DEV Community

Cover image for COMTiles (Cloud Optimized Map Tiles) hosted on Amazon S3 and Visualized with MapLibre GL JS
MIERUNE profile image Yasunori Kirimoto
Yasunori Kirimoto for MIERUNE

Posted on

COMTiles (Cloud Optimized Map Tiles) hosted on Amazon S3 and Visualized with MapLibre GL JS

#foss4g #maplibre #s3 #cloudoptimized

img

I have COMTiles (Cloud Optimized Map Tiles) hosted on Amazon S3 and visualized with MapLibre GL JS 🎉

Cloud Optimized

img

Cloud Optimized is a cloud-optimized format that enables a single file to process requests for massive location data that previously required a map delivery server and tile splitting process.

The features that personally struck me are as follows.

  • Only a part of a vast file can be requested.
  • Only a single file needs to be placed in storage to process requests from the front end.
  • No need for a map distribution server
  • No pre-tiling process (vector tiles and raster tiles) is required.
  • Fast rendering

Rasters are
"COG(Cloud Optimized GeoTIFF)"
Point cloud is
"COPC(Cloud Optimized Point Cloud)"
These formats are stable.

For vector files, there are many formats, and they are not yet settled.

COMTiles(Cloud Optimized Map Tiles)

img

How COMTiles works: com-tiles-spec

This time I examined COMTiles(Cloud Optimized Map Tiles). COMTiles are often compared to PMTiles, but I have the impression that PMTiles are becoming more popular these days. The technology of COMTiles is excellent, but it is not yet easy to introduce because the library is not published in “npm,” and the documentation is incomplete. One of the reasons for PMTiles' popularity is due in part to its extensive documentation. This article will be of some help to those who are considering introducing COMTiles!

Advance Preparation

Prepare "MBTiles" to store map tiles built-in SQLite for conversion of COMTiles.

In general, map tiles (MBTiles) can be created with OpenStreetMap and proprietary data by using the following tools.

This time, I wanted to prepare data easily, so I used MapTiler DATA by MapTiler, which allows you to download map tiles (MBTiles) for a specified area.

Click DOWNLOAD DATA in OpenStreetMap basemaps.
img

Enter any region in the search window and select it → Click DOWNLOAD. In this case, I selected JAPAN.
img

Since I will use it for personal verification, click on NON-COMMERCIAL PERSONAL PROJECT → Click on FREE DOWNLOAD.
img

You can now download the MBTiles you wish to use. Rename the downloaded data to "sample.mbtiles".

Execution environment

  • node v18.1.0
  • npm v8.19.2

The following is explained in detail.

  • Building the Environment
  • Convert MBTiles to COMTiles
  • Create style.json and tiles.json
  • Host data for distribution on Amazon S3
  • Display COMTiles with MapLibre GL JS

Building the Environment

First, we will build an environment for converting MBTiles to COMTiles.

For the environment, fork "zimmermannpeter/com-tiles," a forked version of the original, or download and use it. We chose "zimmermannpeter/com-tiles" because as of December 2022, the original "mactrem/com-tiles" is still under pull request and has improved documentation and MapLibre GL JS display environment.

Overall composition
img

Runs in the root of com-tiles.



npm install
Enter fullscreen mode Exit fullscreen mode

Runs in packages/provider.



npm run build
Enter fullscreen mode Exit fullscreen mode

img
img

Runs in packages/mbtiles-converter.



npm run build
Enter fullscreen mode Exit fullscreen mode

img

Runs in the root of com-tiles.



npm audit fix --force
npm install -g ./packages/mbtiles-converter
Enter fullscreen mode Exit fullscreen mode

Now you have the commands available for conversion!

Convert MBTiles to COMTiles

Next, I will show you how to convert MBTiles to COMTiles.

Copy the MBTiles downloaded in advance to the root of com-tiles.
img

You can find the command in the help section.



convert-comtiles -h
Enter fullscreen mode Exit fullscreen mode

img

Execute the convert-comtiles command to convert them.



convert-comtiles -i sample.mbtiles -o sample.comt
Enter fullscreen mode Exit fullscreen mode

img

After executing the command, COMTiles will be created from MBTiles.
img

This completes the conversion from MBTiles to COMTiles!

Create style.json and tiles.json

Next, I will show you how to create style.json and tiles.json.

Create a bucket in Amazon S3 and set the URL in advance.
img

Modify the following style.json and tiles.json.
img

/packages/maplibre-provider/demo/assets

style.json

Specify the Amazon S3 URL to be uploaded plan in "url" and "glyphs." “url" is the URL of style.json and "glyphs" is the URL of the font.



{
  "version": 8,
  "name": "Basic",
  "sources": {
    "openmaptiles": {
      "type": "vector",
      "url": "https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/assets/tiles.json"
    }
  },
  "glyphs": "https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/assets/fonts/{fontstack}/{range}.pbf",
  "layers": [
    {
      "id": "background",
      "type": "background",
      "paint": {"background-color": "hsl(47, 26%, 88%)"}
    },
.....
Enter fullscreen mode Exit fullscreen mode

/packages/maplibre-provider/demo/assets

tiles.json

Specify the Amazon S3 URL to be uploaded plan in "tiles." “tiles" is the URL of COMTiles. “comt://" is necessary when specifying COMTiles.



{
  "tilejson": "2.0.0",
  "name": "maplibre",
  "attribution": "<a href=\"https://www.maptiler.com/copyright/\" target=\"_blank\">&copy; MapTiler</a> <a href=\"https://www.openstreetmap.org/copyright\" target=\"_blank\">&copy; OpenStreetMap contributors</a>",
.....
  "extent": [
    -20037508.342789244,
    -20037508.6269291,
    20037508.342789244,
    20037508.626929108
  ],
  "center": [
    0,
    0,
    0
  ],
  "tiles": [
    "comt://https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/data/sample.comt/{z}/{x}/{y}"
  ]
}
Enter fullscreen mode Exit fullscreen mode

This completes the creation of style.json and tiles.json!

Host data for distribution on Amazon S3

Next, host the COMTiles, style.json, tiles.json, and fonts directories on Amazon S3.

Upload the COMTiles, style.json, and tiles.json directories created in the previous step, and for the fonts directory, upload "/packages/maplibre-provider/debug/assets/fonts."
img

Set the target bucket of Amazon S3 to the public. This time, I used the public settings for the sample, but I recommend limited public settings for production use.
Trying Out Various Settings for Amazon S3 Publishing
img

This is the directory structure of the target bucket. This time, we created "assets" and "data" directories.
img

Upload style.json, tiles.json, and fonts directories to the assets directory.
img

Upload COMTiles to the data directory.
img

You are now ready to host your data for distribution on Amazon S3!

Display COMTiles with MapLibre GL JS

Finally, we will display the COMTiles in MapLibre GL JS. Modify only the "/packages/maplibre-provider/demo/index.html" of the demo.

Overall composition
img

/packages/maplibre-provider/demo

index.html

Specify the uploaded Amazon S3 URL in "style." “URL" is the URL of style.json. Change "center" and "zoom" to the area you want to display. In this case, we set them to the Tokyo area.



<html>
    <head>
        <meta charset="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
        <title>MapLibre COMT Demo</title>

        <script src="dist/maplibre-gl.js"></script>
        <link href="dist/maplibre-gl.css" rel="stylesheet" />
        <script src="dist/maplibreComtProvider.js"></script>

        <style>
            body,
            #map {
                height: 100vh;
                margin: 0px;
            }
        </style>
    </head>
    <body>
        <div id="map"></div>
        <script>
            comtiles.MapLibreComtProvider.register();
            const map = new maplibregl.Map({
                container: "map",
                style: "https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/assets/style.json",
                center: [139.767, 35.681],
                zoom: 11,
            });
            map.addControl(new maplibregl.NavigationControl({}));
        </script>
    </body>
</html>
Enter fullscreen mode Exit fullscreen mode

Load the library for displaying COMTiles.



<script src="dist/maplibreComtProvider.js"></script>
Enter fullscreen mode Exit fullscreen mode

Load MapLibreComtProvider.



comtiles.MapLibreComtProvider.register();
Enter fullscreen mode Exit fullscreen mode

Specify the style.json URL for the style.



style: "https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/assets/style.json",
Enter fullscreen mode Exit fullscreen mode

For this display environment, execute the simple local server and check it. After execution, access "http://localhost:8080".

/packages/maplibre-provider/demo



python -m http.server 8080
Enter fullscreen mode Exit fullscreen mode

img

Using COMTiles, I could visualize a huge single file (vector tiles) in MapLibre GL JS without a map distribution server! Also, it renders fast!
img
img

Related Articles

AWS Heroes

Vite Support for Various Map Library Starters

Yasunori Kirimoto for AWS Heroes ・ Jun 29 '22

#amazonlocationservice #maplibre #leaflet #vite

AWS Community Builders

Trying Out Various Settings for Amazon S3 Publishing

Yasunori Kirimoto for AWS Community Builders ・ Jan 14 '22

#aws #amazons3 #s3

References
zimmermannpeter/com-tiles

Top comments (5)

Collapse
 
nyurik profile image
Yuri Astrakhan •

Great write up, thx! AFAIK, Brandon (pmtiles) is talking to Markus (comtiles) to join their efforts and create pmtiles v4 with some/all of comtiles ideas.

Also, you may want to mention that martin-cp (part of Martin tile server) can also generate mbtiles ;) maplibre.org/martin/martin-cp.html

Collapse
 
benerain profile image
Yujia Li •

in the step of " npm install -g ./packages/mbtiles-converter ", i got a 404 error. Not Found - GET registry.npmjs.org/@com-tiles%2fpr... - Not found '@com-tiles/provider@*' is not in this registry.
and i found similar issue on github , github.com/mactrem/com-tiles/issue...

do you have any ideas?

Collapse
 
benerain profile image
Yujia Li •

i found a way: npm install without '-g' , that is run 'npm install ./packages/mbtiles-converter' and use the command './node_modules/.bin/convert-comtiles'

Collapse
 
dayjournal profile image
Yasunori Kirimoto •

COMTiles is not yet registered in npm. Therefore, please refer to this post to build it.

Collapse
 
juan_jospischetz_0af8cc profile image
Juan José Pischetz •

Hello! im having this issue when i try convert mbtiles:

Converting the MBTiles file sample.mbtiles to a COMTiles archive.
Writing the header and metadata to the COMTiles archive.
Writing the index to the COMTiles archive.
F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\tileProvider.js:35
const limits = tileMatrix.tileMatrixLimits;
^

TypeError: Cannot read properties of undefined (reading 'tileMatrixLimits')
at TileProvider.getTilesInRowMajorOrder (F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\tileProvider.js:35:39)
at getTilesInRowMajorOrder.next ()
at writeIndex (F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\index.js:117:60)
at createComTileArchive (F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\index.js:84:35)
at async F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\index.js:57:5

Node.js v18.1.0

Read next

vajravijay profile image

03. ரிலேஷனல் டேட்டாபேஸ் மாடல் என்றால் என்ன? What is Relational Database Model ? (RDBMS)

Vijayan S -

philip_zhang_854092d88473 profile image

What Does "Optional and Customizable Fields" Mean?

Philip -

vajravijay profile image

04. தரவு ஒருங்கிணைவு (Data Integrity)

Vijayan S -

hana_sato profile image

How to Implement Master Data Management: A Step-by-Step Guide

Hana Sato -