FFBridge Postmortem Analyzer - Bridge Tournament Data Extraction and Analysis
This project provides a comprehensive toolkit for extracting, analyzing, and post-mortem analysis of bridge tournament data from the French Bridge Federation (FFBridge) system. It includes web scraping capabilities, data processing tools, and interactive analysis interfaces.
- Web Scraping: Automated extraction of bridge tournament data from FFBridge and BridgePlus websites
- Data Processing: Convert raw tournament data into structured formats for analysis
- Interactive Analysis: Streamlit-based web interface for post-mortem game analysis
- Authentication: Automated bearer token management for FFBridge API access
- Bridge Library: Comprehensive library for bridge deal analysis and augmentation
- Enhanced HTML Parsing: Robust Unicode-aware parsing with improved French-to-English translation
- Type Safety: Comprehensive type definitions and static analysis support
- Tournament discovery and club extraction
- Team and board results scraping
- Individual board/deal extraction with PBN format
- Parallel processing for high-performance data extraction
- All async functions follow
_asyncnaming convention - Enhanced HTML parsing with dynamic board number extraction
- Robust opponent information extraction using specific HTML structure
- NEW: Improved Unicode handling for French suit symbols (♠♥♦♣)
- NEW: Enhanced contract extraction with comprehensive French-to-English strain mapping
- NEW: Robust vulnerability translation with error handling
- Web-based interface for game analysis
- Chat-based postmortem analysis with AI assistance
- Support for both simultaneous and RRN tournaments
- Real-time data visualization and statistics
- Automated bearer token extraction using Playwright
- Environment variable management for credentials
- Support for both Lancelot and EASI token types
- Deal analysis and augmentation tools
- Double dummy analysis integration
- Bidding and play analysis
- Machine learning utilities for bridge AI
# Install core dependencies
pip install -r requirements.txt
# Install Playwright browsers
playwright install chromium
# Optional: Install advanced features (machine learning, AI, etc.)
pip install -r requirements-optional.txt
# Set up environment variables
cp env_template.txt .env
# Edit .env with your FFBridge credentials
# Run tests to verify installation
python test_type_definitions.py
python test_bridge_library.py
# Optional: Run type checking
mypy ffbridge_streamlit.pyimport asyncio
from mlBridgeLib.mlBridgeBPLib import get_all_boards_for_team_async
async def extract_tournament_data():
# Extract all boards for a team
result = await get_all_boards_for_team_async(
tr="S202602", # Tournament ID
cl="5802079", # Club ID
sc="A", # Section
eq="212" # Team number
)
boards_df = result['boards']
frequency_df = result['score_frequency']
print(f"Extracted {len(boards_df)} boards")
print(f"Opponent info: {boards_df['Opponent_Pair_Names'][0]} vs {boards_df['Team_Name'][0]}")
return result
# Run extraction
data = asyncio.run(extract_tournament_data())# Launch Streamlit interface
streamlit run ffbridge_streamlit.pyffbridge_postmortem_generator.py drives the live app in a headless browser, captures
the generated PDF, and emails it. Combined with the included PowerShell
helper this becomes a daily "email me the report for session X" job.
One-time setup:
pip install -r requirements.txt
playwright install chromiumThen put SMTP credentials in a .env file next to ffbridge_streamlit.py.
Gmail with an App Password is the tested-and-working path:
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=you@gmail.com
SMTP_PASSWORD=xxxxxxxxxxxxxxxx # 16-char Google app password, no spaces
SMTP_FROM=you@gmail.comImportant Gmail gotchas:
SMTP_USERandSMTP_FROMmust match the Google account that owns the app password. You cannot use a Yahoo (or other) address as the sender on a Gmail-authenticated connection -- Gmail will reject the message.- 2-Step Verification must be enabled on that Google account; otherwise
the "App passwords" page won't be available and any password you paste
will fail with
535-5.7.8 Username and Password not accepted. - The app password is 16 characters with no spaces (Google shows it
with spaces just for readability -- strip them when saving to
.env).
Email the player's most recent game right now (SMTP creds come from
.env):
python ffbridge_postmortem_generator.py `
--url "https://ffbridge.postmortem.chat/?player_id=9500754" `
--email coach@example.comPick a specific session (rarely needed — usually you want auto-latest):
python ffbridge_postmortem_generator.py `
--url "https://ffbridge.postmortem.chat/?player_id=9500754&session_id=107118" `
--email coach@example.comForce a re-send even if the cache says it already went out, and print the full SMTP conversation if something looks wrong:
python ffbridge_postmortem_generator.py `
--url "https://ffbridge.postmortem.chat/?player_id=9500754" `
--email coach@example.com `
--force-email `
--smtp-debugGenerate the PDF locally without sending email (handy for testing the browser side in isolation):
python ffbridge_postmortem_generator.py `
--url "https://ffbridge.postmortem.chat/?player_id=9500754" `
--email me@example.com `
--output report.pdf `
--no-emailPass SMTP settings on the command line instead of via .env:
python ffbridge_postmortem_generator.py `
--url "https://ffbridge.postmortem.chat/?player_id=9500754" `
--email coach@example.com `
--from-email you@gmail.com `
--smtp-host smtp.gmail.com `
--smtp-port 587 `
--smtp-user you@gmail.com `
--smtp-password "xxxxxxxxxxxxxxxx"Register a daily 6 PM scheduled task that always picks the player's most recent game:
.\schedule_daily_report.ps1 `
-Url "https://ffbridge.postmortem.chat/?player_id=9500754" `
-Email coach@example.com `
-Time 18:00What the scheduled run does each evening:
- Purges any cached PDFs in
report_cache\whose mtime is older than the TTL (default 168 h / 1 week). The TTL is a sliding window: every cache hit refreshes the file's mtime, so an entry survives indefinitely as long as the player hasn't played a newer game. - Opens the URL in headless Chromium and waits for the report metadata.
- Always loads the player's most recent game (no date filter).
- Skips silently if
report_cache\{player_id}_{game_date}.pdfalready exists — the cache assumes the report was already emailed for that game. - Otherwise downloads the PDF, stores it in the cache, and emails it.
Net effect: each game produces exactly one email, no matter how often the scheduler runs in between, and no matter how many days pass between games.
Remove the scheduled task:
.\schedule_daily_report.ps1 -UnregisterIf a real run fails at the email step, run test_smtp.py to bisect
whether the problem is in the browser/PDF side or in SMTP itself. By
default it reads SMTP_* from your .env:
# Just connect, EHLO/STARTTLS, and authenticate -- don't actually send:
python test_smtp.py --no-send
# Send a tiny test message to yourself:
python test_smtp.pyThe main requirements are in requirements.txt and include:
- Streamlit: Web interface and data visualization
- Polars/Pandas: Data processing and analysis
- Playwright: Web scraping and automation
- Endplay: Bridge game analysis and double dummy calculations
- BeautifulSoup: HTML parsing
- Matplotlib/Seaborn: Data visualization
- ReportLab: PDF report generation
Advanced features require additional packages in requirements-optional.txt:
- PyTorch/FastAI: Machine learning capabilities (mlBridgeAiLib.py)
- Scikit-learn: Machine learning utilities
- SQLAlchemy: Database support
- OpenAI: AI integration for chat features
- Library Usage Guide: Comprehensive guide for using the bridge scraping library
- Authentication Guide: Setup and usage of automated authentication
- Test Suite: Run
python test_bridge_library.pyfor comprehensive testing - Type Definitions Test: Run
python test_type_definitions.pyfor type validation - Type Checking: Run
mypy ffbridge_streamlit.pyfor static type analysis
- Streamlined Dependencies: Reorganized requirements.txt with clear separation of core vs optional dependencies
- Optional Features: Created requirements-optional.txt for advanced features like machine learning
- Better Documentation: Updated installation instructions and dependency documentation
- Cleaner Setup: Removed unused dependencies and improved version constraints
- Enhanced HTML Parsing: Improved Unicode handling for French suit symbols (♠♥♦♣)
- Robust Contract Extraction: Comprehensive French-to-English strain mapping with error handling
- Improved Vulnerability Translation: Enhanced error handling for French vulnerability terms
- Better Regex Patterns: Updated regex patterns to handle Unicode characters and complex HTML structures
- Type Safety Improvements: Enhanced type hints and validation throughout the codebase
- Documentation Updates: Comprehensive updates to reflect latest capabilities
- Comprehensive Documentation: Created detailed LIBRARY_USAGE_GUIDE.md with complete API documentation
- Enhanced Type Safety: Improved TypedDict definitions for API configurations and data structures
- Streamlit Interface: Added comprehensive chat-based postmortem analysis with AI assistance
- API Integration: Enhanced FFBridge API integration with dual token authentication (Lancelot + EASI)
- Error Handling: Improved error handling and validation throughout the application
- Testing: Updated test suite with comprehensive coverage for new functionality
- Comprehensive Type Hints: Added full type annotations throughout the codebase for better IDE support and code quality
- Enhanced Documentation: Improved function docstrings with detailed parameter and return type descriptions
- Type Safety: Added TypedDict definitions for complex data structures and API configurations
- Development Tools: Added mypy and typing-extensions for static type checking
- Code Quality: Better organized requirements.txt with version constraints and categorized dependencies
- Enhanced HTML Parsing: Dynamic board number extraction from HTML instead of hardcoded assumptions
- Improved Opponent Extraction: Uses specific HTML div structure (
<div class="col">contre <span class="paires">...) for reliable opponent information - Better Error Handling: Graduated warnings instead of hard failures, with detailed diagnostic information
- Data Correspondence: Board numbers, scores, and matchpoints are extracted from same HTML rows to ensure accuracy
- Robust Validation: Enhanced data validation with detailed troubleshooting information
- Fixed Board Parsing: Removed hardcoded board skip patterns (19, 20) that were tournament-specific
- New Data Fields: Added
Opponent_Pair_Sectionfield for better opponent tracking - Enhanced Validation: Automatic team existence checking with detailed diagnostics
- Improved Error Handling: Better error messages and graceful handling of missing data
- Section Validation: Automatic detection of opponent section mismatches
- Async Naming Convention: All async functions now use
_asyncsuffix - New Functions: Added
get_board_for_player_async(),get_board_for_team(), and optimized player-specific extraction - Enhanced Error Handling: Improved robustness in web scraping operations
- Authentication Improvements: Better token management and environment variable handling
- Dynamic Board Discovery: Automatically detects board numbers from HTML content
- Opponent Information: Comprehensive opponent tracking with direction, names, number, and section
- Data Integrity: Maintains correspondence between board numbers, scores, and matchpoints
- Error Diagnostics: Detailed troubleshooting information for common issues
# OLD: Hardcoded assumptions
for i in range(1, 27):
if i not in [19, 20]: # Skip boards 19 and 20 (not played)
board_nums.append(i)
# NEW: Dynamic extraction from HTML
board_data = []
data_rows = await page.query_selector_all('div.row')
for row in data_rows:
# Extract board number, score, and matchpoints from same row
# Ensures correspondence between related data# NEW: Comprehensive strain mapping including Unicode symbols
FRENCH_TO_ENGLISH_STRAIN_MAP = {
'P': 'S', # Spades (Piques)
'C': 'H', # Hearts (Cœurs)
'K': 'D', # Diamonds (Carreau)
'T': 'C', # Clubs (Trèfle)
'sa': 'N', # No Trump (Sans Atout) - lowercase
'SA': 'N', # No Trump (Sans Atout) - uppercase
'♠': 'S', # Unicode spade symbol
'♥': 'H', # Unicode heart symbol
'♦': 'D', # Unicode diamond symbol
'♣': 'C', # Unicode club symbol
'pique': 'S', # French class names
'coeur': 'H', # French class names
'carreau': 'D', # French class names
'trefle': 'C', # French class names
}# NEW: Enhanced regex pattern for contract extraction
suit_span_match = re.search(
r'Contrat.*?(?:<span class="(pique|coeur|carreau|trefle)">[^<]*</span>|(SA))',
contract_span_html
)
if suit_span_match and suit_span_match.group(1): # Suit class found
suit_class = suit_span_match.group(1)
elif suit_span_match and suit_span_match.group(2): # SA match found
suit_class = suit_span_match.group(2)
else:
raise ValueError(f"Could not find suit in contract span: {contract_span_html}")
strain = FRENCH_TO_ENGLISH_STRAIN_MAP[suit_class]# Uses specific HTML structure for reliable extraction
opponent_div = await page.query_selector('div.col:has-text("contre")')
if opponent_div:
paires_span = await opponent_div.query_selector('span.paires')
if paires_span:
# Parse opponent text like "NS : SININGE - KRIEF (113 A)"
paires_text = paires_span.text_content()
# Extract direction, names, number, and sectionffbridge-postmortem/
├── mlBridgeLib/ # Core bridge analysis library
│ ├── mlBridgeBPLib.py # BridgePlus scraping library (main)
│ ├── mlBridgeLib.py # Core bridge utilities
│ ├── mlBridgeAugmentLib.py # Deal augmentation tools
│ └── ... # Other bridge analysis modules
├── streamlitlib/ # Streamlit utilities
├── cache/ # Cached API responses
├── competitions/ # Tournament data storage
├── results/ # Analysis results
├── old/ # Legacy code and documentation
├── api-*.bat # API testing scripts
├── ffbridge_streamlit.py # Main Streamlit application
├── test_bridge_library.py # Comprehensive test suite
├── LIBRARY_USAGE_GUIDE.md # Detailed usage documentation
└── requirements.txt # Python dependencies
Board,Score,Percentage,Contract,Declarer,LeadPair_Names,Pair_Direction,Pair_Number,SectionOpponent_Pair_Names,Opponent_Pair_Direction,Opponent_Pair_Number,Opponent_Pair_Section
Board,PBN,Top,Dealer,Vul,Contract,Result,ScoreTeam_Name,Pair_Number,Section,Tournament_ID,Club_IDOpponent_Pair_Names,Opponent_Pair_Direction,Opponent_Pair_Number,Opponent_Pair_Section
Board,Score,Frequency,Matchpoints_NS,Matchpoints_EW
- Parallel Processing: Concurrent extraction of multiple boards
- Shared Browser Context: Reuse browser instances for multiple requests
- Intelligent Waiting: Adaptive delays based on page load times
- Robust Error Recovery: Graceful handling of network issues and page changes
- Efficient Data Structures: Polars DataFrames for high-performance data operations
- Type Safety: Comprehensive type hints throughout the codebase using Python's typing system
- Static Analysis: Support for mypy static type checking
- Documentation: Detailed docstrings with parameter and return type descriptions
- Structured Data: TypedDict definitions for complex API configurations and data structures
- IDE Support: Enhanced autocomplete and error detection in modern IDEs
The library provides comprehensive error handling with detailed diagnostics:
# Enhanced error messages with suggestions
if len(route_data) < 5:
logger.warning(f"Very few route records extracted: only {len(route_data)} records found")
logger.warning("This might indicate:")
logger.warning(" 1. Team played few boards")
logger.warning(" 2. HTML parsing issues")
logger.warning(" 3. Page structure changed")This project is actively developed and maintained. Key areas for contribution:
- Additional bridge analysis algorithms
- UI/UX improvements for the Streamlit interface
- Performance optimizations for data extraction
- Support for additional tournament formats
- Enhanced error handling and diagnostics
- Type safety improvements and additional type hints
- Documentation enhancements
The project includes comprehensive test suites to ensure code quality and functionality:
python test_type_definitions.pyTests the core type definitions and basic functionality without external dependencies.
python test_bridge_library.pyComprehensive test suite for bridge library functionality, API integration, and data processing.
- Type Definitions: ApiUrlConfig, ApiUrlsDict, DataFramesDict
- API Integration: Authentication, request handling, error management
- Data Processing: DataFrame operations, validation, augmentation
- Bridge Library: Tournament discovery, data extraction, board analysis
- Streamlit Interface: UI components, session state, report generation
This tests all major functions and provides detailed performance metrics.
For static type analysis:
# Check all main files
python check_types.py
# Check specific file
mypy ffbridge_streamlit.py
# Check with custom configuration
mypy ffbridge_streamlit.py --config-file mypy.iniThis will check for type errors and provide suggestions for improving type safety.
MIT License - See LICENSE file for details.
For issues and questions:
- Check the Library Usage Guide for detailed documentation
- Run the test suite to verify installation
- Review error messages for diagnostic information
- Check the
old/directory for legacy documentation