{ "cells": [ { "cell_type": "markdown", "id": "0f09e1c5", "metadata": {}, "source": [ "# Speedtest Data (`ScatterplotLayer`)\n", "\n", "This example will use data collected from Ookla's Speed Test application and [shared publicly in the AWS Open Data Registry](https://registry.opendata.aws/speedtest-global-performance/). From the AWS page:\n", "\n", "> Global fixed broadband and mobile (cellular) network performance, allocated to zoom level 16 web mercator tiles (approximately 610.8 meters by 610.8 meters at the equator). Data is provided in both Shapefile format as well as Apache Parquet with geometries represented in Well Known Text (WKT) projected in EPSG:4326. Download speed, upload speed, and latency are collected via the Speedtest by Ookla applications for Android and iOS and averaged for each tile.\n", "\n", "You can view a [hosted version of this notebook on Notebook Sharing Space](https://notebooksharing.space/view/ef4775b8dba37fa681b62c4719a07ea38b24f49c1a615c446e5967d7117166b6#displayOptions=) (35MB download)." ] }, { "cell_type": "markdown", "id": "7dd1dfef-3756-49d9-9480-9a4cdba22345", "metadata": {}, "source": [ "## Imports" ] }, { "cell_type": "code", "execution_count": 1, "id": "d1678764", "metadata": { "execution": { "iopub.execute_input": "2024-03-19T15:24:01.638876Z", "iopub.status.busy": "2024-03-19T15:24:01.638623Z", "iopub.status.idle": "2024-03-19T15:24:02.499502Z", "shell.execute_reply": "2024-03-19T15:24:02.499165Z" } }, "outputs": [], "source": [ "from pathlib import Path\n", "\n", "import geopandas as gpd\n", "import numpy as np\n", "import pandas as pd\n", "import shapely\n", "from palettable.colorbrewer.diverging import BrBG_10\n", "\n", "from lonboard import Map, ScatterplotLayer\n", "from lonboard.colormap import apply_continuous_cmap" ] }, { "cell_type": "markdown", "id": "d51ca576", "metadata": {}, "source": [ "## Fetch data\n", "\n" ] }, { "cell_type": "markdown", "id": "c747d8b9-94b9-421a-967a-8350bf72de9a", "metadata": {}, "source": [ "The URL for a single data file for mobile network speeds in the first quarter of 2019:" ] }, { "cell_type": "code", "execution_count": 2, "id": "34ac8eae", "metadata": { "execution": { "iopub.execute_input": "2024-03-19T15:24:02.501606Z", "iopub.status.busy": "2024-03-19T15:24:02.501415Z", "iopub.status.idle": "2024-03-19T15:24:02.503402Z", "shell.execute_reply": "2024-03-19T15:24:02.503119Z" } }, "outputs": [], "source": [ "url = \"https://ookla-open-data.s3.us-west-2.amazonaws.com/parquet/performance/type=mobile/year=2019/quarter=1/2019-01-01_performance_mobile_tiles.parquet\"" ] }, { "cell_type": "markdown", "id": "5991ef2c-5db0-4110-b6a1-b33fcbddad0d", "metadata": {}, "source": [ "The data used in this example is relatively large. In the cell below, we cache the downloading and preparation of the dataset so that it's faster to run this notebook the second time.\n", "\n", "We fetch two columns ā `avg_d_kbps` and `tile` ā from this data file directly from AWS. The `pd.read_parquet` command will perform a network request for these columns from the data file, so it may take a while on a slow network connection. `avg_d_kbps` is the average download speed for that data point in kilobits per second. `tile` is the WKT string representing a given zoom-16 Web Mercator tile.\n", "\n", "The `tile` column contains _strings_ representing WKT-formatted geometries. We need to parse those strings into geometries. Then for simplicity we'll convert into their centroids." ] }, { "cell_type": "code", "execution_count": 3, "id": "7c20cb4c-9746-486f-aef7-95dd2dedd6a5", "metadata": { "execution": { "iopub.execute_input": "2024-03-19T15:24:02.504866Z", "iopub.status.busy": "2024-03-19T15:24:02.504745Z", "iopub.status.idle": "2024-03-19T15:24:04.068872Z", "shell.execute_reply": "2024-03-19T15:24:04.068448Z" } }, "outputs": [], "source": [ "local_path = Path(\"internet-speeds.parquet\")\n", "if local_path.exists():\n", " gdf = gpd.read_parquet(local_path)\n", "else:\n", " columns = [\"avg_d_kbps\", \"tile\"]\n", " df = pd.read_parquet(url, columns=columns)\n", "\n", " tile_geometries = shapely.from_wkt(df[\"tile\"])\n", " tile_centroids = shapely.centroid(tile_geometries)\n", " gdf = gpd.GeoDataFrame(df[[\"avg_d_kbps\"]], geometry=tile_centroids, crs='EPSG:4326')\n", " gdf.to_parquet(local_path)" ] }, { "cell_type": "markdown", "id": "5852aa94-2d18-4a1b-b379-be19682d57eb", "metadata": {}, "source": [ "We can take a quick look at this data:" ] }, { "cell_type": "code", "execution_count": 4, "id": "4b27e9a4", "metadata": { "execution": { "iopub.execute_input": "2024-03-19T15:24:04.070444Z", "iopub.status.busy": "2024-03-19T15:24:04.070366Z", "iopub.status.idle": "2024-03-19T15:24:04.077585Z", "shell.execute_reply": "2024-03-19T15:24:04.077375Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", " | avg_d_kbps | \n", "geometry | \n", "
---|---|---|
0 | \n", "5983 | \n", "POINT (-160.01862 70.63722) | \n", "
1 | \n", "3748 | \n", "POINT (-160.04059 70.63357) | \n", "
2 | \n", "3364 | \n", "POINT (-160.04059 70.63175) | \n", "
3 | \n", "2381 | \n", "POINT (-160.03510 70.63357) | \n", "
4 | \n", "3047 | \n", "POINT (-160.03510 70.63175) | \n", "
\n", " | avg_d_kbps | \n", "geometry | \n", "
---|---|---|
383429 | \n", "13570 | \n", "POINT (-2.94159 58.99673) | \n", "
383430 | \n", "18108 | \n", "POINT (-3.29865 58.96276) | \n", "
383431 | \n", "5569 | \n", "POINT (-3.29315 58.97125) | \n", "
383432 | \n", "9349 | \n", "POINT (-3.29315 58.96842) | \n", "
383433 | \n", "11216 | \n", "POINT (-3.23273 58.98541) | \n", "
... | \n", "... | \n", "... | \n", "
1840929 | \n", "104723 | \n", "POINT (25.38666 35.09519) | \n", "
1840930 | \n", "62540 | \n", "POINT (25.44708 35.04124) | \n", "
1840931 | \n", "88068 | \n", "POINT (25.47455 35.04124) | \n", "
1840938 | \n", "38255 | \n", "POINT (25.38116 35.00075) | \n", "
1840939 | \n", "91750 | \n", "POINT (25.45807 34.99175) | \n", "
807221 rows Ć 2 columns
\n", "