Set up Docker and 24-hour monitoring #1

Merged
NyxiumYuuki merged 1 commit from claude/docker-setup-monitoring-01XxBdb5R99jUqtqbAp6p61j into master 2025-11-29 06:00:52 +01:00
7 changed files with 529 additions and 11 deletions

70
.dockerignore Normal file
View file

@ -0,0 +1,70 @@
# Python
__pycache__/
*.py[cod]
*$py.class
*.so
.Python
*.egg-info/
dist/
build/
*.egg
# Virtual environments
venv/
env/
ENV/
.venv
# IDE
.vscode/
.idea/
*.swp
*.swo
*~
# Git
.git/
.gitignore
.gitattributes
# Documentation
*.md
!pyproject.toml
docs/
# Tests
tests/
.pytest_cache/
.coverage
htmlcov/
.tox/
.mypy_cache/
# CI/CD
.github/
.gitlab-ci.yml
.travis.yml
# Environment
.env
.env.local
.env.*.local
# Logs
*.log
# OS
.DS_Store
Thumbs.db
# Docker
Dockerfile*
docker-compose*.yml
.dockerignore
# Nginx
nginx/
# Data directories (will be mounted as volumes)
data/
schemas/

View file

@ -57,3 +57,12 @@ REQUEST_TIMEOUT=30.0
# Maximum retry attempts for failed requests # Maximum retry attempts for failed requests
MAX_RETRIES=3 MAX_RETRIES=3
# =============================================================================
# Docker Configuration (for Docker Compose deployments)
# =============================================================================
# Your Docker Hub username (used when pulling pre-built images)
DOCKER_USERNAME=yourusername
# Docker image tag to use (e.g., latest, v1.0.0, dev)
IMAGE_TAG=latest

337
CLAUDE.md Normal file
View file

@ -0,0 +1,337 @@
# CLAUDE.md - Development Guide for AI Assistants
This document provides context for AI assistants (like Claude) working on the GeoGuessr MCP Server project.
## Project Overview
**GeoGuessr MCP Server** is a Model Context Protocol (MCP) server that provides Claude with tools to analyze GeoGuessr game statistics. The server features automatic API monitoring and dynamic schema adaptation to handle API changes gracefully.
### Key Architecture Components
```
geoguessr-mcp/
├── src/geoguessr_mcp/
│ ├── api/ # GeoGuessr API client and response handling
│ ├── auth/ # Authentication management
│ ├── models/ # Data models (Profile, Stats, Games, etc.)
│ ├── monitoring/ # Dynamic schema detection and API monitoring
│ │ ├── endpoint/ # Endpoint monitoring logic
│ │ └── schema/ # Schema detection and registry
│ ├── services/ # Business logic services
│ ├── tools/ # MCP tool definitions
│ ├── config.py # Configuration and settings
│ └── main.py # Application entry point
├── tests/ # Unit and integration tests
├── Dockerfile # Container definition
├── docker-compose.yml # Development deployment
└── docker-compose.prod.yml # Production deployment with nginx
```
## Core Concepts
### 1. Dynamic Schema System
The server automatically adapts to GeoGuessr API changes:
- **Schema Detection**: Analyzes API responses to detect field types, structure, and nesting
- **Change Detection**: Uses hash-based comparison to identify schema modifications
- **Persistence**: Caches schemas to disk for survival across restarts
- **Monitoring**: Runs periodic checks (every 24 hours by default) on all known endpoints
**Key Files**:
- `src/geoguessr_mcp/monitoring/schema/schema_detector.py` - Analyzes response structure
- `src/geoguessr_mcp/monitoring/schema/schema_registry.py` - Manages schema storage
- `src/geoguessr_mcp/monitoring/endpoint/endpoint_monitor.py` - Periodic monitoring
### 2. MCP Tool Architecture
Tools are organized by domain:
- **auth_tools.py** - Authentication (login, logout, cookie management)
- **profile_tools.py** - User profiles and statistics
- **game_tools.py** - Game history and details
- **analysis_tools.py** - Performance analysis and recommendations
- **monitoring_tools.py** - API health and endpoint exploration
Each tool returns a `DynamicResponse` which includes both the data and schema information.
### 3. Authentication Flow
The server supports three authentication methods:
1. **Environment variable**: `GEOGUESSR_NCFA_COOKIE` in .env
2. **Login tool**: Email/password authentication via MCP
3. **Manual cookie**: Direct cookie setting via tool
Session state is managed in `src/geoguessr_mcp/auth/session.py`.
## Docker Deployment
### Build Process
The Dockerfile uses a multi-stage approach:
1. Install dependencies from `pyproject.toml` using `uv` for speed
2. Copy source code
3. Run the server via `python -m geoguessr_mcp.main`
**Important**: The Dockerfile does NOT use a separate `requirements.txt`. It installs directly from `pyproject.toml`.
### Deployment Options
1. **Local Build**: `docker compose up -d --build`
2. **Docker Hub**: Build, tag, push, then pull on VPS
3. **Production**: Use `docker-compose.prod.yml` with nginx reverse proxy
### Monitoring Configuration
Monitoring runs every 24 hours by default:
- Set `MONITORING_ENABLED=true` to enable
- Set `MONITORING_INTERVAL_HOURS=24` for interval
- Schemas are persisted in `/app/data/schemas` volume
## Development Workflow
### Running Tests
```bash
# Unit tests
pytest tests/unit/ -v
# Integration tests (requires auth)
pytest tests/integration/ -v
# With coverage
pytest --cov=src/geoguessr_mcp tests/
```
### Local Development
```bash
# Install dependencies
pip install -e ".[dev]"
# Run server locally
python -m geoguessr_mcp.main
# Or use the entry point
geoguessr-mcp
```
### Code Quality
- **Formatter**: Black (line length 100)
- **Linter**: Ruff
- **Type Checker**: MyPy (strict mode)
Run checks:
```bash
black src/ tests/
ruff check src/ tests/
mypy src/
```
## Common Tasks
### Adding a New Tool
1. Create tool function in appropriate `tools/*.py` file
2. Use `@mcp.tool()` decorator
3. Return `DynamicResponse` or structured data
4. Add to tool registration in `__init__.py`
5. Write unit tests
Example:
```python
@mcp.tool()
async def my_new_tool(param: str) -> str:
"""Tool description for Claude."""
# Implementation
return result
```
### Adding a New Endpoint to Monitor
1. Add endpoint definition to `MONITORED_ENDPOINTS` in `endpoint_monitor.py`
2. Specify path, method, params, and description
3. Schema detection happens automatically
Example:
```python
EndpointDefinition(
path="/v3/profiles/new-data",
description="New profile data endpoint",
)
```
### Updating Configuration
Configuration is managed via environment variables in `config.py`:
- Add new setting to `Settings` dataclass
- Use `field(default_factory=lambda: os.getenv(...))` pattern
- Document in README.md environment variables section
## Testing Strategy
### Unit Tests
- Mock external API calls using `respx`
- Test each component in isolation
- Located in `tests/unit/`
### Integration Tests
- Use fixtures from `tests/conftest.py`
- Mock authentication for CI/CD
- Test full request/response flows
- Located in `tests/integration/`
### Fixtures
- `mock_auth_session` - Provides authenticated session
- `mock_http_client` - Mocked httpx client
- `sample_*` - Sample API responses
## Key Dependencies
- **mcp[cli]** - Model Context Protocol framework
- **httpx** - Async HTTP client for API calls
- **uvicorn** - ASGI server
- **starlette** - Web framework (used by FastMCP)
- **python-dotenv** - Environment variable management
## Troubleshooting
### Common Issues
1. **"server.py not found"**: The entry point is `main.py`, not `server.py`
2. **Schema not persisting**: Check volume mount for `/app/data/schemas`
3. **Monitoring not running**: Verify `MONITORING_ENABLED=true` and check logs
4. **Auth failures**: Ensure valid NCFA cookie or use login tool
### Debugging
Enable debug logging:
```bash
LOG_LEVEL=DEBUG docker compose up
```
Check monitoring status via tool:
```python
# In Claude
check_api_status()
```
## CI/CD Considerations
### Building for Production
```bash
# Multi-architecture build
docker buildx build \
--platform linux/amd64,linux/arm64 \
-t username/geoguessr-mcp:latest \
--push .
# Single architecture
docker build -t username/geoguessr-mcp:latest .
docker push username/geoguessr-mcp:latest
```
### Environment Variables for Deployment
Required:
- `GEOGUESSR_NCFA_COOKIE` - For authenticated endpoints
Optional:
- `MONITORING_INTERVAL_HOURS` - Default: 24
- `LOG_LEVEL` - Default: INFO
- `MCP_PORT` - Default: 8000
## Schema Registry Details
### Schema Hash Computation
Schemas are identified by a hash of:
- Field names and types
- Nested structure
- Array element types
- Null/undefined handling
### Schema Change Detection
When a schema changes:
1. New hash is computed
2. Comparison with previous hash
3. If different, `schema_changed` flag is set
4. New schema is persisted to disk
5. Monitoring result includes change notification
### Schema Persistence Format
Schemas are stored as JSON in `SCHEMA_CACHE_DIR`:
```json
{
"endpoint": "/v3/profiles",
"schema_hash": "abc123...",
"fields": [...],
"last_updated": "2025-11-29T...",
"http_status": 200
}
```
## API Monitoring System
### Monitored Endpoints
Currently monitoring ~15 endpoints:
- Profile & Stats (v3, v4)
- Games & Social (feed, friends, badges)
- Competitive (seasons)
- Explorer & Objectives
- Challenges (daily)
### Monitoring Loop
1. Every 24 hours (configurable)
2. Checks each endpoint sequentially
3. 0.5s delay between requests (rate limiting)
4. Updates schema registry on changes
5. Logs availability and response times
6. Generates monitoring report
### Monitoring Tools
Users can:
- `check_api_status()` - Get current status of all endpoints
- `get_endpoint_schema(path)` - Get schema for specific endpoint
- `list_available_endpoints()` - List all known endpoints
- `explore_endpoint(path)` - Manually check a new endpoint
## Best Practices
1. **Always use DynamicResponse** for tools that return API data
2. **Mock authentication** in tests using fixtures
3. **Keep schemas generic** - don't hardcode field expectations
4. **Log schema changes** at INFO level for visibility
5. **Use async/await** consistently throughout the codebase
6. **Handle rate limiting** with delays between requests
7. **Validate configuration** in Settings.__post_init__
8. **Document tools** with clear docstrings for Claude
## Future Enhancements
Potential areas for improvement:
- [ ] Add metrics collection (Prometheus/Grafana)
- [ ] Implement rate limiting middleware
- [ ] Add caching layer for frequently accessed data
- [ ] Support for webhooks/real-time updates
- [ ] Advanced analytics and ML-based recommendations
- [ ] Multi-user support with session management
- [ ] GraphQL endpoint for flexible queries
## Resources
- [MCP Documentation](https://modelcontextprotocol.io/)
- [FastMCP Guide](https://github.com/jlowin/fastmcp)
- [GeoGuessr API (unofficial)](https://www.geoguessr.com/api)
- [Docker Best Practices](https://docs.docker.com/develop/dev-best-practices/)
---
**Last Updated**: 2025-11-29
**Maintainer**: See pyproject.toml for author information

View file

@ -7,6 +7,7 @@ WORKDIR /app
# Set environment variables # Set environment variables
ENV PYTHONDONTWRITEBYTECODE=1 ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1 ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH=/app
# Install system dependencies # Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \ RUN apt-get update && apt-get install -y --no-install-recommends \
@ -16,14 +17,17 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
# Install uv for faster package installation # Install uv for faster package installation
RUN pip install --no-cache-dir uv RUN pip install --no-cache-dir uv
# Copy requirements first for better caching # Copy only dependency files for better layer caching
COPY requirements.txt . COPY pyproject.toml .
# Install Python dependencies # Install Python dependencies from pyproject.toml
RUN uv pip install --system --no-cache -r requirements.txt RUN uv pip install --system --no-cache -e .
# Copy application code # Copy application source code
COPY server.py . COPY src/ ./src/
# Create data directory for schema cache
RUN mkdir -p /app/data/schemas
# Expose the port # Expose the port
EXPOSE 8000 EXPOSE 8000
@ -32,5 +36,5 @@ EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \ HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8000/health || exit 1 CMD curl -f http://localhost:8000/health || exit 1
# Run the server # Run the server using the installed package entry point
CMD ["python", "server.py"] CMD ["python", "-m", "geoguessr_mcp.main"]

View file

@ -166,7 +166,45 @@ Claude uses explore_endpoint tool:
## 🏭 Production Deployment ## 🏭 Production Deployment
### With SSL (Recommended) ### Method 1: Build and Push Docker Image
For VPS deployment, it's recommended to build and push your image to Docker Hub:
```bash
# 1. Build the image
docker build -t yourusername/geoguessr-mcp:latest .
# 2. Login to Docker Hub
docker login
# 3. Push the image
docker push yourusername/geoguessr-mcp:latest
# 4. On your VPS, pull and run
docker pull yourusername/geoguessr-mcp:latest
```
### Method 2: Deploy with Docker Compose on VPS
#### Development/Testing Setup
```bash
# Clone repository on VPS
git clone https://github.com/yourusername/geoguessr-mcp.git
cd geoguessr-mcp
# Create .env file
cat > .env << EOF
GEOGUESSR_NCFA_COOKIE=your_cookie_here
DOCKER_USERNAME=yourusername
IMAGE_TAG=latest
EOF
# Deploy
docker compose up -d
```
#### Production Setup with SSL
1. Create SSL certificates: 1. Create SSL certificates:
```bash ```bash
@ -176,11 +214,57 @@ mkdir -p nginx/ssl
# nginx/ssl/privkey.pem # nginx/ssl/privkey.pem
``` ```
2. Deploy with production compose: 2. Update docker-compose.prod.yml to use your Docker image:
```bash
# Edit docker-compose.prod.yml and uncomment the image line:
# image: ${DOCKER_USERNAME:-yourusername}/geoguessr-mcp:${IMAGE_TAG:-latest}
# Then comment out the build section
```
3. Deploy with production compose:
```bash ```bash
docker compose -f docker-compose.prod.yml up -d docker compose -f docker-compose.prod.yml up -d
``` ```
### Method 3: Direct Docker Run
If you prefer not to use Docker Compose:
```bash
# Pull the image
docker pull yourusername/geoguessr-mcp:latest
# Create a volume for schema cache
docker volume create geoguessr-schemas
# Run the container
docker run -d \
--name geoguessr-mcp \
--restart unless-stopped \
-p 8000:8000 \
-e GEOGUESSR_NCFA_COOKIE=your_cookie \
-e MONITORING_ENABLED=true \
-e MONITORING_INTERVAL_HOURS=24 \
-e LOG_LEVEL=INFO \
-v geoguessr-schemas:/app/data/schemas \
yourusername/geoguessr-mcp:latest
```
### Building Multi-Architecture Images
For deployment on different CPU architectures (ARM64, AMD64):
```bash
# Enable buildx
docker buildx create --use
# Build and push multi-arch image
docker buildx build \
--platform linux/amd64,linux/arm64 \
-t yourusername/geoguessr-mcp:latest \
--push .
```
### Environment Variables ### Environment Variables
| Variable | Default | Description | | Variable | Default | Description |
@ -189,8 +273,11 @@ docker compose -f docker-compose.prod.yml up -d
| `MCP_PORT` | 8000 | Server port | | `MCP_PORT` | 8000 | Server port |
| `MCP_TRANSPORT` | streamable-http | MCP transport protocol | | `MCP_TRANSPORT` | streamable-http | MCP transport protocol |
| `MONITORING_ENABLED` | true | Enable API monitoring | | `MONITORING_ENABLED` | true | Enable API monitoring |
| `MONITORING_INTERVAL_HOURS` | 24 | Monitoring check interval | | `MONITORING_INTERVAL_HOURS` | 24 | Monitoring check interval (runs every 24h) |
| `SCHEMA_CACHE_DIR` | /app/data/schemas | Directory for schema persistence |
| `LOG_LEVEL` | INFO | Logging verbosity | | `LOG_LEVEL` | INFO | Logging verbosity |
| `DOCKER_USERNAME` | yourusername | Your Docker Hub username (for compose files) |
| `IMAGE_TAG` | latest | Docker image tag |
## 🧪 Development ## 🧪 Development

View file

@ -4,9 +4,15 @@ version: '3.8'
services: services:
geoguessr-mcp: geoguessr-mcp:
# Option 1: Build locally
build: build:
context: . context: .
dockerfile: Dockerfile dockerfile: Dockerfile
# Option 2: Use pre-built image from Docker Hub (recommended for VPS)
# Uncomment the line below and comment out the build section above
# image: ${DOCKER_USERNAME:-yourusername}/geoguessr-mcp:${IMAGE_TAG:-latest}
container_name: geoguessr-mcp-server container_name: geoguessr-mcp-server
restart: unless-stopped restart: unless-stopped
expose: expose:

View file

@ -2,9 +2,14 @@ version: '3.8'
services: services:
geoguessr-mcp: geoguessr-mcp:
# Option 1: Build locally
build: build:
context: . context: .
dockerfile: Dockerfile dockerfile: Dockerfile
# Option 2: Use pre-built image from Docker Hub (uncomment to use)
# image: ${DOCKER_USERNAME:-yourusername}/geoguessr-mcp:${IMAGE_TAG:-latest}
container_name: geoguessr-mcp-server container_name: geoguessr-mcp-server
restart: unless-stopped restart: unless-stopped
ports: ports: