================================================================
ML-Ready Stop-Level Bus Travel Time Dataset
P211 and P411 Bus Services, Johor Bahru, Malaysia
2025 Training Set and 2026 Cross-Year Test Set
================================================================

Version:    1.0
Date:       2025
Licence:    Creative Commons Attribution 4.0 (CC-BY 4.0)
Repository: University of Glasgow Enlighten Research Data
DOI:        10.5525/gla.researchdata.2454

================================================================
AUTHORS
================================================================

Sye Loong Keoh
School of Computing Science, University of Glasgow, UK

Corresponding author: SyeLoong.Keoh@glasgow.ac.uk
  ORCID: 0000-0003-3640-5010

================================================================
ASSOCIATED PUBLICATION
================================================================

Keoh, S.L. et al. (2026). Machine Learning-Based Bus ETA
Prediction: Cross-Year Generalisation and Training-Window
Selection in an IoT-Enabled Deployment.
[Under review]

================================================================
FUNDING AND ACKNOWLEDGEMENTS
================================================================

This dataset was collected as part of the ASEAN ICT Virtual
Organisation (IVO) project:
"An IoT-based Public Transport Data Collection and Analytic
Framework using Bluetooth Proximity Beacons"

Funding:
  UTM Grant No: R.J130000.7606.4B856
  UTM Matching Grant: Q.J130000.3006.04M36
  Supported by NICT (Japan)

Data collection conducted in partnership with:
  Johor Public Transport Corporation
  (Perbadanan Pengangkutan Awam Johor, PAJ)
  Johor Bahru City Council (MBJB)
  Iskandar Puteri City Council (MBIP)
  Kulai Municipal Council (MPKu)

================================================================
DATASET OVERVIEW
================================================================

This dataset contains machine learning-ready stop-level bus
travel time records derived from two data sources:

  (1) Bluetooth Low Energy (BLE) proximity sensing using
      Estimote iBeacon units mounted on bus dashboards,
      detected by Raspberry Pi units installed at bus stops.

  (2) Global Positioning System (GPS) fleet tracking provided
      by PAJ via a live GPS feed from the bus fleet.

Two bus services are covered:

  P211: Taman Universiti <-> Larkin Sentral
        Distance: 22.3 km  |  Stops: 26 per direction
        Frequency: ~14 return trips daily
        Typical journey time: 35-45 minutes

  P411: Kulai <-> Larkin Sentral
        Distance: 31 km  |  Stops: ~31 per direction
        Frequency: ~5 return trips daily
        Typical journey time: 45-60 minutes

Both services share a 14 km corridor between Skudai and
Larkin Sentral along Johor Bahru's main trunk road.

Data collection periods:
  2025 (Training/validation): January - December 2025
  2026 (Cross-year test):     January - June 2026

Data source by period:
  P211: BLE detection (Jan-May 2025), GPS (Jun-Dec 2025)
  P411: BLE detection (Jan-Nov 2025), GPS (Dec 2025)
  All 2026 data: GPS fleet feed (PAJ)

================================================================
FILES IN THIS DATASET
================================================================

A. ML-READY TRAINING DATASETS (2025 full year)
-----------------------------------------------
p211_ml_ready_route8_outbound.csv
  P211 Route 8 — Outbound (Taman Universiti → Larkin Sentral)
  Records: 68,744  |  Date range: 2025-01-01 to 2025-12-31

p211_ml_ready_route9_inbound.csv
  P211 Route 9 — Inbound (Larkin Sentral → Taman Universiti)
  Records: 70,900  |  Date range: 2025-01-01 to 2025-12-31

p411_ml_ready_route6_outbound.csv
  P411 Route 6 — Outbound (Kulai → Larkin Sentral)
  Records: 33,254  |  Date range: 2025-01-01 to 2025-12-31

p411_ml_ready_route7_inbound.csv
  P411 Route 7 — Inbound (Larkin Sentral → Kulai)
  Records: 39,970  |  Date range: 2025-01-01 to 2025-12-31

B. ML-READY CROSS-YEAR TEST DATASETS (2026 H1)
-----------------------------------------------
p211_2026_ml_ready_route8_outbound.csv
  P211 Route 8 — Outbound, 2026 test set
  Records: 27,292  |  Date range: 2026-01-01 to 2026-06-30

p211_2026_ml_ready_route9_inbound.csv
  P211 Route 9 — Inbound, 2026 test set
  Records: 28,878  |  Date range: 2026-01-01 to 2026-06-30

p411_2026_ml_ready_route6_outbound.csv
  P411 Route 6 — Outbound, 2026 test set
  Records: 16,186  |  Date range: 2026-01-01 to 2026-06-30

p411_2026_ml_ready_route7_inbound.csv
  P411 Route 7 — Inbound, 2026 test set
  Records: 21,658  |  Date range: 2026-01-01 to 2026-06-30

  Total ML-ready records across all 8 files: 306,882

C. SPATIAL REFERENCE FILES
---------------------------
bus_stops_reference.csv
  Geographic coordinates of all 75 unique bus stops.

route_stop_sequence.csv
  Ordered stop sequence for each of the 4 route directions.

D. ROUTE GEOMETRY FILES
-----------------------------------------------------
route_geometry_p211_r8_outbound.geojson   — P211 Route 8 road geometry
route_geometry_p211_r9_inbound.geojson    — P211 Route 9 road geometry
route_geometry_p411_r6_outbound.geojson   — P411 Route 6 road geometry
route_geometry_p411_r7_inbound.geojson    — P411 Route 7 road geometry

  These GeoJSON files contain the actual road polyline
  geometry for each route. They use
  WGS84 coordinate system (EPSG:4326).

================================================================
COLUMN DESCRIPTIONS (ML-ready CSV files)
================================================================

route_id
  Bus route identifier: 6, 7, 8, or 9

date
  Date of observation (YYYY-MM-DD format)

time
  Confirmed arrival time at destination stop (HH:MM:SS)

latitude
  GPS latitude of stop position at time of arrival
  (decimal degrees, WGS84)

longitude
  GPS longitude of stop position at time of arrival
  (decimal degrees, WGS84)

time_bin
  10-minute time slot index derived from arrival time
  Range: 0-143, where 0 = 00:00-00:10, 143 = 23:50-00:00
  Formula: time_bin = (hour * 60 + minute) // 10

distance_from_stop
  Road geometry distance between from_stop_id and cur_stop_id
  along the route polyline (metres)
  Computed using perpendicular foot projection onto route shape

cur_stop_id
  Destination stop identifier (integer, 1000-series)
  Links to bus_stops_reference.csv and route_stop_sequence.csv

route_order
  Sequential position of cur_stop_id within the route
  (1 = first stop, increases toward terminal)

time_taken  *** TARGET VARIABLE ***
  Travel time from from_stop_id to cur_stop_id (seconds)
  Includes dwell time at the origin stop (from_stop_id)
  For interpolated segments: estimated proportionally by
  road geometry distance

from_stop_id
  Origin stop identifier (integer, 1000-series)

prev_stop_time
  Confirmed departure time from origin stop (HH:MM:SS)
  For BLE data: actual detection timestamp
  For GPS data: closest GPS fix to stop coordinates

================================================================
COLUMN DESCRIPTIONS (bus_stops_reference.csv)
================================================================

stop_id     : Unique stop identifier (1000-series integer)
latitude    : Median GPS latitude across all observations
              (decimal degrees, WGS84)
longitude   : Median GPS longitude across all observations
              (decimal degrees, WGS84)

  Total stops: 80
  Geographic coverage: Johor Bahru, Malaysia
  Latitude range:  1.4868 N to 1.6626 N
  Longitude range: 103.5986 E to 103.7427 E

================================================================
COLUMN DESCRIPTIONS (route_stop_sequence.csv)
================================================================

service     : Bus service (P211 or P411)
route_id    : Route direction (6, 7, 8, or 9)
route_order : Sequential stop position within route
stop_id     : Stop identifier (links to bus_stops_reference)

  P211 Route 8 (Outbound): 25 stops
    First: 1083 (Taman Universiti area)
    Last:  1076 (Larkin Sentral)

  P211 Route 9 (Inbound): 28 stops
    First: 1077 (Larkin Sentral area)
    Last:  1084 (Taman Universiti area)

  P411 Route 6 (Outbound): 30 stops
    First: 1000 (Kulai area)
    Last:  1076 (Larkin Sentral area)

  P411 Route 7 (Inbound): 35 stops
    First: 1077 (Larkin Sentral area)
    Last:  1071 (Kulai area)

  Shared corridor (stops on both P211 and P411): 33 stops

================================================================
DATA PROCESSING PIPELINE
================================================================

The raw BLE and GPS data underwent the following processing
steps before producing the ML-ready datasets:

Step 1 — Data cleaning
  - Removed GPS jumping (coordinates outside route geometry)
  - Removed duplicate records (dwell time at stops)
  - Corrected route direction misclassifications using
    cumulative projected distance along route polyline
  - Removed records where time_taken < 0 or > 7,200 seconds

Step 2 — Stop-to-stop duration computation
  - Arrival at stop defined as GPS coordinate closest to
    stop location (Haversine distance, minimum threshold)
  - Duration = confirmed_arrival_time - confirmed_departure_time
  - For BLE: timestamps are actual BLE detection events
  - For GPS: timestamps are interpolated from nearest fix

Step 3 — Multi-stop interpolation
  Where bus confirmed at stop A and later at stop B with
  unobserved intermediate stops s_(a+1)...s_(b-1):
    estimated_duration(s_j, s_(j+1)) =
      (T_(a→b) / D_(a→b)) * d_(j→j+1)
  where T_(a→b) = observed total span duration,
        D_(a→b) = total road geometry distance,
        d_(j→j+1) = individual segment road distance

Step 4 — Outlier removal
  Per stop pair (from_stop_id, cur_stop_id):
  - Removed top 1%  of time_taken (anomaly/breakdown records)
  - Removed bottom 3% of time_taken (GPS errors/data jumping)

  Note: Interpolated records are included and not separately
  flagged. Interpolation rates vary by stop pair — see the
  associated publication for route-level statistics.

================================================================
RECOMMENDED FEATURE ENGINEERING
================================================================

The following features can be derived from the raw columns
for machine learning model training:

  day_of_week       = date.dayofweek
                      (0=Monday, 6=Sunday)

  month             = date.month (1-12)

  quarter           = date.quarter (1-4)

  prev_stop_time_s  = seconds since midnight derived from
                      prev_stop_time column
                      Formula: h*3600 + m*60 + s

  is_school_holiday = 1 if date falls within Johor school
                      holiday period (Kumpulan B schedule),
                      else 0

  is_public_holiday = 1 if date is a Malaysia federal or
                      Johor state public holiday, else 0

  is_monsoon        = 1 if month in {11, 12, 1}
                      (Northeast Monsoon season), else 0

Recommended base feature set (6 features) for cross-year
deployment — best cross-year accuracy in associated study:
  day_of_week, time_bin, from_stop_id, cur_stop_id,
  prev_stop_time_s, distance_from_stop

Target variable: time_taken (seconds)

Train/test split used in associated study:
  2025 data: 70% training, 30% held-out test
  random_state = 10

================================================================
JOHOR SCHOOL HOLIDAY CALENDAR (Kumpulan B)
================================================================

2025:
  01 Jan – 16 Jan  (Year-end break from 2024 session)
  31 Mar – 04 Apr  (Term 1 holidays + Hari Raya)
  29 May – 09 Jun  (Mid-year holidays)
  13 Sep – 21 Sep  (Term 2 holidays)
  20 Oct – 22 Oct  (Deepavali)
  20 Dec – 31 Dec  (Year-end holidays)

2026 (H1 only):
  01 Jan – 11 Jan  (Year-end break from 2025 session)
  16 Feb – 20 Feb  (Chinese New Year)
  18 Mar – 29 Mar  (Hari Raya + Term 1 holidays)
  23 May – 07 Jun  (Mid-year holidays)

================================================================
MALAYSIA PUBLIC HOLIDAYS INCLUDED
================================================================

2025:
  29-30 Jan (Chinese New Year), 11 Feb (Federal Territory Day),
  01 Mar (Israk Mikraj), 24 Mar (Johor Sultan Birthday),
  31 Mar (Hari Raya Aidilfitri), 01 Apr (Hari Raya),
  01 May (Labour Day), 02 Jun (Agong Birthday),
  07 Jun (Hari Raya Haji), 27 Jun (Awal Muharram),
  01 Aug (Johor Heritage Day), 01 Sep (Hari Kebangsaan Johor),
  05 Sep (Nuzul Al-Quran), 16 Sep (Malaysia Day),
  20 Oct (Deepavali), 25 Dec (Christmas)

2026 (H1 only):
  01-02 Feb (Thaipusam / Federal Territory Day),
  17-19 Feb (Chinese New Year),
  21-23 Mar (Hari Raya Aidilfitri),
  01 May (Labour Day), 25-27 May (Hari Raya Haji),
  01 Jun (Agong Birthday), 17 Jun (Awal Muharram)

================================================================
CITATION
================================================================

Dataset citation:
  Keoh, S.L. (2026).
  ML-Ready Stop-Level Bus Travel Time Dataset.
  University of Glasgow Enlighten
  Research Data.
  DOI: 10.5525/gla.researchdata.2454

Paper citation:
  Keoh, S.L. et al. (2026) Machine Learning-Based Bus ETA
  Prediction: Cross-Year Generalisation and Training-Window
  Selection in an IoT-Enabled Deployment. Engineering
  Applications of Artificial Intelligence (Elsevier).
  [Under review]

================================================================
CONTACT
================================================================

Principal Investigator:
  Prof. Sye Loong Keoh
  School of Computing Science
  University of Glasgow
  Glasgow G12 8RZ, United Kingdom
  Email: SyeLoong.Keoh@glasgow.ac.uk
  ORCID: 0000-0003-3640-5010

For data access queries:
  University of Glasgow Enlighten Research Data:
  https://researchdata.gla.ac.uk

================================================================
TERMS OF USE
================================================================

This dataset is made available under the Creative Commons
Attribution 4.0 International (CC-BY 4.0) licence.
You are free to share and adapt the data for any purpose,
provided appropriate credit is given.

Full licence: https://creativecommons.org/licenses/by/4.0/

The authors make no warranties regarding accuracy or
completeness. Users should refer to the associated
publication for full methodological details before
using the dataset for research.

================================================================
