I have COMTiles (Cloud Optimized Map Tiles) hosted on Amazon S3 and visualized with MapLibre GL JS 🎉
Cloud Optimized
Cloud Optimized is a cloud-optimized format that enables a single file to process requests for massive location data that previously required a map delivery server and tile splitting process.
The features that personally struck me are as follows.
- Only a part of a vast file can be requested.
- Only a single file needs to be placed in storage to process requests from the front end.
- No need for a map distribution server
- No pre-tiling process (vector tiles and raster tiles) is required.
- Fast rendering
Rasters are
"COG(Cloud Optimized GeoTIFF)"
Point cloud is
"COPC(Cloud Optimized Point Cloud)"
These formats are stable.
For vector files, there are many formats, and they are not yet settled.
COMTiles(Cloud Optimized Map Tiles)
How COMTiles works: com-tiles-spec
This time I examined COMTiles(Cloud Optimized Map Tiles). COMTiles are often compared to PMTiles, but I have the impression that PMTiles are becoming more popular these days. The technology of COMTiles is excellent, but it is not yet easy to introduce because the library is not published in “npm,” and the documentation is incomplete. One of the reasons for PMTiles' popularity is due in part to its extensive documentation. This article will be of some help to those who are considering introducing COMTiles!
Advance Preparation
Prepare "MBTiles" to store map tiles built-in SQLite for conversion of COMTiles.
In general, map tiles (MBTiles) can be created with OpenStreetMap and proprietary data by using the following tools.
This time, I wanted to prepare data easily, so I used MapTiler DATA by MapTiler, which allows you to download map tiles (MBTiles) for a specified area.
Click DOWNLOAD DATA in OpenStreetMap basemaps.
Enter any region in the search window and select it → Click DOWNLOAD. In this case, I selected JAPAN.
Since I will use it for personal verification, click on NON-COMMERCIAL PERSONAL PROJECT → Click on FREE DOWNLOAD.
You can now download the MBTiles you wish to use. Rename the downloaded data to "sample.mbtiles".
Execution environment
- node v18.1.0
- npm v8.19.2
The following is explained in detail.
- Building the Environment
- Convert MBTiles to COMTiles
- Create style.json and tiles.json
- Host data for distribution on Amazon S3
- Display COMTiles with MapLibre GL JS
Building the Environment
First, we will build an environment for converting MBTiles to COMTiles.
For the environment, fork "zimmermannpeter/com-tiles," a forked version of the original, or download and use it. We chose "zimmermannpeter/com-tiles" because as of December 2022, the original "mactrem/com-tiles" is still under pull request and has improved documentation and MapLibre GL JS display environment.
Runs in the root of com-tiles.
npm install
Runs in packages/provider.
npm run build
Runs in packages/mbtiles-converter.
npm run build
Runs in the root of com-tiles.
npm audit fix --force
npm install -g ./packages/mbtiles-converter
Now you have the commands available for conversion!
Convert MBTiles to COMTiles
Next, I will show you how to convert MBTiles to COMTiles.
Copy the MBTiles downloaded in advance to the root of com-tiles.
You can find the command in the help section.
convert-comtiles -h
Execute the convert-comtiles command to convert them.
convert-comtiles -i sample.mbtiles -o sample.comt
After executing the command, COMTiles will be created from MBTiles.
This completes the conversion from MBTiles to COMTiles!
Create style.json and tiles.json
Next, I will show you how to create style.json and tiles.json.
Create a bucket in Amazon S3 and set the URL in advance.
Modify the following style.json and tiles.json.
/packages/maplibre-provider/demo/assets
style.json
Specify the Amazon S3 URL to be uploaded plan in "url" and "glyphs." “url" is the URL of style.json and "glyphs" is the URL of the font.
{
"version": 8,
"name": "Basic",
"sources": {
"openmaptiles": {
"type": "vector",
"url": "https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/assets/tiles.json"
}
},
"glyphs": "https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/assets/fonts/{fontstack}/{range}.pbf",
"layers": [
{
"id": "background",
"type": "background",
"paint": {"background-color": "hsl(47, 26%, 88%)"}
},
.....
/packages/maplibre-provider/demo/assets
tiles.json
Specify the Amazon S3 URL to be uploaded plan in "tiles." “tiles" is the URL of COMTiles. “comt://" is necessary when specifying COMTiles.
{
"tilejson": "2.0.0",
"name": "maplibre",
"attribution": "<a href=\"https://www.maptiler.com/copyright/\" target=\"_blank\">© MapTiler</a> <a href=\"https://www.openstreetmap.org/copyright\" target=\"_blank\">© OpenStreetMap contributors</a>",
.....
"extent": [
-20037508.342789244,
-20037508.6269291,
20037508.342789244,
20037508.626929108
],
"center": [
0,
0,
0
],
"tiles": [
"comt://https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/data/sample.comt/{z}/{x}/{y}"
]
}
This completes the creation of style.json and tiles.json!
Host data for distribution on Amazon S3
Next, host the COMTiles, style.json, tiles.json, and fonts directories on Amazon S3.
Upload the COMTiles, style.json, and tiles.json directories created in the previous step, and for the fonts directory, upload "/packages/maplibre-provider/debug/assets/fonts."
Set the target bucket of Amazon S3 to the public. This time, I used the public settings for the sample, but I recommend limited public settings for production use.
Trying Out Various Settings for Amazon S3 Publishing
This is the directory structure of the target bucket. This time, we created "assets" and "data" directories.
Upload style.json, tiles.json, and fonts directories to the assets directory.
Upload COMTiles to the data directory.
You are now ready to host your data for distribution on Amazon S3!
Display COMTiles with MapLibre GL JS
Finally, we will display the COMTiles in MapLibre GL JS. Modify only the "/packages/maplibre-provider/demo/index.html" of the demo.
/packages/maplibre-provider/demo
index.html
Specify the uploaded Amazon S3 URL in "style." “URL" is the URL of style.json. Change "center" and "zoom" to the area you want to display. In this case, we set them to the Tokyo area.
<html>
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>MapLibre COMT Demo</title>
<script src="dist/maplibre-gl.js"></script>
<link href="dist/maplibre-gl.css" rel="stylesheet" />
<script src="dist/maplibreComtProvider.js"></script>
<style>
body,
#map {
height: 100vh;
margin: 0px;
}
</style>
</head>
<body>
<div id="map"></div>
<script>
comtiles.MapLibreComtProvider.register();
const map = new maplibregl.Map({
container: "map",
style: "https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/assets/style.json",
center: [139.767, 35.681],
zoom: 11,
});
map.addControl(new maplibregl.NavigationControl({}));
</script>
</body>
</html>
Load the library for displaying COMTiles.
<script src="dist/maplibreComtProvider.js"></script>
Load MapLibreComtProvider.
comtiles.MapLibreComtProvider.register();
Specify the style.json URL for the style.
style: "https://com-tiles-202212.s3.ap-northeast-1.amazonaws.com/assets/style.json",
For this display environment, execute the simple local server and check it. After execution, access "http://localhost:8080".
/packages/maplibre-provider/demo
python -m http.server 8080
Using COMTiles, I could visualize a huge single file (vector tiles) in MapLibre GL JS without a map distribution server! Also, it renders fast!
Related Articles
Vite Support for Various Map Library Starters
Yasunori Kirimoto for AWS Heroes ・ Jun 29 '22
Trying Out Various Settings for Amazon S3 Publishing
Yasunori Kirimoto for AWS Community Builders ・ Jan 14 '22
References
zimmermannpeter/com-tiles
Top comments (5)
Great write up, thx! AFAIK, Brandon (pmtiles) is talking to Markus (comtiles) to join their efforts and create pmtiles v4 with some/all of comtiles ideas.
Also, you may want to mention that
martin-cp
(part of Martin tile server) can also generate mbtiles ;) maplibre.org/martin/martin-cp.htmlin the step of " npm install -g ./packages/mbtiles-converter ", i got a 404 error. Not Found - GET registry.npmjs.org/@com-tiles%2fpr... - Not found '@com-tiles/provider@*' is not in this registry.
and i found similar issue on github , github.com/mactrem/com-tiles/issue...
do you have any ideas?
i found a way: npm install without '-g' , that is run 'npm install ./packages/mbtiles-converter' and use the command './node_modules/.bin/convert-comtiles'
COMTiles is not yet registered in npm. Therefore, please refer to this post to build it.
Hello! im having this issue when i try convert mbtiles:
Converting the MBTiles file sample.mbtiles to a COMTiles archive.
Writing the header and metadata to the COMTiles archive.
Writing the index to the COMTiles archive.
F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\tileProvider.js:35
const limits = tileMatrix.tileMatrixLimits;
^
TypeError: Cannot read properties of undefined (reading 'tileMatrixLimits')
at TileProvider.getTilesInRowMajorOrder (F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\tileProvider.js:35:39)
at getTilesInRowMajorOrder.next ()
at writeIndex (F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\index.js:117:60)
at createComTileArchive (F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\index.js:84:35)
at async F:\comtiles\com-tiles\packages\mbtiles-converter\dist\src\index.js:57:5
Node.js v18.1.0