Ether Data Documentation

Comprehensive documentation for the Mux-Mesh spatio-temporal data workspace

🌟 Welcome

Mux-Mesh is a comprehensive spatio-temporal data workspace designed for building scalable data processing solutions on Google Cloud Platform. This documentation provides detailed guides, API references, and architectural insights for all components within the ecosystem.

🚀 Quick Start

New to Mux-Mesh? Start here:

1. 📖 Architecture Overview - Understand the system design
2. 🔧 Core Projects - Explore the main components
3. 📚 Guides & References - Learn how to use the system
4. 🔍 Quick Navigation - Find what you need fast

📚 Documentation Index

🔧 Core Projects

🌐 API Services

Our microservices architecture provides specialized data processing capabilities:

• 🚪 APIs Gateway
  • Central API gateway for request routing and management
  • Authentication, rate limiting, and load balancing
  • 📋 Deployment Guide
• 📊 Census Data
  • US Census demographic data API with natural language querying
  • Flexible geography inputs (ZIP, DMA, city, county, coordinates, etc.)
  • AI-powered SQL generation and H3 spatial indexing
• 🗺️ Overture Maps
  • Open geographic data processing and serving
  • Overture Maps data integration and API endpoints
  • Spatial data queries and visualization support
• 🚦 TomTom Flow
  • Real-time traffic flow data processing API
  • TomTom API integration and data transformation
  • Historical and live traffic analytics

🔄 Data Pipelines

Automated data collection and processing workflows:

• 📊 TomTom Traffic Pipeline
  • Real-time traffic data collection from TomTom Intermediate Traffic API
  • OpenLR binary data processing and WKT geometry conversion
  • BigQuery integration with intelligent staging and partitioning

📖 Guides & References

Essential documentation for users and developers:

• 🌍 Geography Input Guide
  • Comprehensive guide for handling geographic input data
  • Coordinate systems, projections, and data formats
  • Best practices for spatial data processing
• 🛠️ Utilities & Scripts
  • Documentation for utility scripts and development tools
  • Automation helpers and maintenance scripts
  • Development workflow enhancements

🏗️ Architecture Overview

System Components

🌐 API Services Layer

• Gateway Service: Centralized entry point with routing, authentication, and monitoring
• Overture Service: Geographic data processing and Overture Maps integration
• TomTom Flow Service: Real-time traffic data processing and analytics

🔄 Data Processing Layer

• TomTom Traffic Pipeline: Automated data collection with intelligent scheduling
• OpenLR Processing: Binary traffic data conversion to standard formats
• BigQuery Integration: Efficient data warehousing with partitioning strategies

🛠️ Infrastructure Layer

• Shared Utilities: Common logging, monitoring, and development tools
• Cloud-Native Design: Google Cloud Platform integration and best practices
• Standardized Tooling: Consistent development patterns across all projects

🚀 Key Capabilities

📊 Data Processing

• Real-time ingestion: Live traffic flow data collection and processing
• Batch processing: Historical data analysis and reporting
• Spatial operations: Geographic data transformation and analysis
• Format conversion: OpenLR to WKT geometry processing

🔧 Infrastructure

• Scalable architecture: Auto-scaling on Google Cloud Platform
• Efficient storage: BigQuery partitioning and compression
• Automated scheduling: Cloud Run Jobs with Cloud Scheduler
• Monitoring & logging: Comprehensive observability stack

🌐 API Features

• RESTful APIs: Standard HTTP interfaces for all services
• Authentication: Secure access control and API key management
• Rate limiting: Protection against abuse and overuse
• Documentation: Interactive API documentation and examples

🔍 Quick Navigation

🎯 I want to…	📍 Go here
Understand the system	Architecture Overview
Set up API gateway	Gateway Documentation
Query census data	Census API Guide
Work with maps data	Overture Maps Guide
Process traffic data	TomTom Flow API
Set up data pipeline	Traffic Pipeline Guide
Handle geographic data	Geography Input Guide
Use utility scripts	Utilities Documentation
Deploy to production	Gateway Deployment

🔗 External Resources

📖 Documentation & Guides

• Main Repository - Source code and development setup
• Google Cloud Documentation - Platform documentation
• BigQuery Documentation - Data warehouse setup

🌐 Data Sources & APIs

• TomTom Traffic API - Real-time traffic data
• Overture Maps - Open geographic data
• OpenLR Specification - Location referencing standard

🛠️ Development Tools

• uv Package Manager - Python dependency management
• FastAPI Documentation - API framework
• Google Cloud SDK - Cloud deployment tools