Benchmark de Performance des Bases de Données Graphe
Suite de tests mesurant la latence, la mémoire et le débit de requêtes entre moteurs relationnels et orientés graphe.
Problème et solution
Déterminer la solution de base de données optimale pour des graphes sociaux et des systèmes de recommandation.
Fonctionnalités
- Tests automatisés de traversée sur plusieurs niveaux de profondeur
- Comparaison des centiles de latence (p50, p95, p99)
- Suivi détaillé de l'utilisation de la mémoire et du CPU
- Environnements de test reproductibles avec Docker
Stack technique
| Technologie | Version | Objectif | Pourquoi ce choix |
|---|---|---|---|
| Go | 1.25.1 | Primary language | Strong concurrency primitives, fast compilation, good driver ecosystem |
| neo4j-go-driver/v6 | v6.2.0 | Bolt protocol client for CognoDB, Neo4j, Memgraph | Official driver, supports Bolt 5.x, parameterized queries |
| go-echarts/v2 | v2.7.2 | HTML chart generation | Go-native, generates standalone HTML |
| godotenv | v1.5.1 | Environment configuration | Standard .env loading |
| net/http | stdlib | REST client for TigerGraph | No external dependency needed |
Architecture
Benchmark Engine
- Adapter pattern with common GraphDatabase interface (16 methods)
- 5 database adapters (4 active, FalkorDB disabled due to OOM)
- Sequential workload executor with warm-up and measured iterations
- Concurrent workload executor with N goroutines and sync.WaitGroup
- Factory function for adapter creation from configuration
CLI Pipeline
- 10 CLI entry points: benchmark, captest, charts, cleanup, dataset, load, runall, summary, verify, verify-load
- JSONL append-only result output for crash resilience
- CSV summary aggregation with percentile calculation
- HTML chart generation via go-echarts
- Pipeline automation via runall (verify → load → benchmark → report)
Structure du projet
Points d'API
Query Workloads (port 0)
| Méthode | Route | Objectif |
|---|---|---|
| Cypher | MATCH (n:Person {id: $id}) RETURN n.id AS id | Point lookup — single node by ID |
| Cypher | MATCH (n:Person {region: $val}) RETURN n.id AS id LIMIT 10 | Indexed lookup — nodes by region |
| Cypher | MATCH (n:Person {id: $id})-[:FRIEND]->(m:Person) RETURN collect(m.id) | 1-hop traversal — direct friends |
| Cypher | MATCH (n:Person {id: $id})-[:FRIEND]->()-[:FRIEND]->(m:Person) RETURN collect(DISTINCT m.id) | 2-hop traversal — friends of friends |
| Cypher | MATCH (n:Person {id: $id})-[:FRIEND]->()-[:FRIEND]->()-[:FRIEND]->(m:Person) RETURN collect(DISTINCT m.id) | 3-hop traversal — extended network |
| Cypher | MATCH (n:Person) RETURN n.region AS region, count(n) AS cnt ORDER BY cnt DESC LIMIT 10 | Aggregation — region distribution |
| Cypher | CREATE (n:Person {id: $id, region: $region, ...}) | Write — create new node |
Conception de la base de données
Person (node label)
| Colonne | Type | Notes |
|---|---|---|
| idPK | STRING | Unique constraint, indexed |
| region | STRING | Indexed |
| gender | STRING | Not indexed |
| age | STRING | Indexed (CognoDB, Neo4j, Memgraph only) |
| completion_percentage | STRING | Not indexed |
49,683 nodes extracted from SNAP soc-Pokec dataset. All properties are strings.
FRIEND (relationship type)
| Colonne | Type | Notes |
|---|---|---|
| source_id | STRING | Person.id (from) |
| target_id | STRING | Person.id (to) |
100,000 directed relationships. No properties on edges. Many-to-many cardinality.
Directed graph: Person --[FRIEND]--> Person. Many-to-many via graph relationships (no junction table). Relationships are first-class citizens in the graph model.
Décisions de conception clés
Single Bolt Driver for 3 Databases
Décision : Use neo4j-go-driver/v6 for CognoDB, Neo4j, and Memgraph
Pourquoi : All three support Bolt protocol; Cypher is logically equivalent
Avantages : Code reuse, consistent behavior across databases, adapter pattern isolates DB-specific behavior
Compromis : Cannot test database-specific optimizations; Memgraph has its own Go driver
MaxPoolSize = 1
Décision : Single connection per benchmark run despite constant defining 10
Pourquoi : Measure single-connection latency; avoid connection pool effects
Avantages : Clean latency measurement without pool contention
Compromis : Doesn't test connection pooling behavior
JSONL for Raw Results
Décision : One JSON object per line, append-only output
Pourquoi : Stream results without loading all into memory; easy to append across runs
Avantages : Resilient to crashes (partial results preserved)
Compromis : Requires full file scan for aggregation
No Retries on Measured Queries
Décision : Zero retries for benchmark queries to capture true latency
Pourquoi : Network blips and failures are part of real-world performance
Avantages : Honest performance measurement including failure modes
Compromis : Network blips cause false negatives
Goroutine-Per-Client Concurrency
Décision : Each concurrent client creates its own database connection
Pourquoi : Simulates real-world multi-client scenarios; avoids shared connection contention
Avantages : Realistic concurrency testing with independent connections
Compromis : Higher resource usage; connection limits become the bottleneck
MERGE for Idempotent Node Loading
Décision : Use MERGE (upsert) instead of CREATE for node loading
Pourquoi : Makes loading idempotent — running multiple times doesn't create duplicates
Avantages : Safe re-runs, consistent dataset state
Compromis : MERGE is slightly slower than CREATE due to existence checks
Tests
Ce qui est testé
- Capability test matrix (cmd/captest/) — tests all 8 workload types against all databases with small 10-node graph
- Connection verification (cmd/verify/) — tests basic connectivity and parameterized queries
- Data integrity verification (cmd/verify-load/) — verifies node/relationship counts (49,683 nodes, 100,000 relationships)
Couverture manquante
- No unit tests for individual functions (_test.go files do not exist)
- No tests for statistical functions (percentile, mean, stddev calculation)
- No tests for CSV parsing edge cases (empty datasets, malformed CSVs)
- No tests for concurrent behavior
- No tests for chart generation correctness
- No test framework installed (no testify, gomock)
Défis clés
- Assurer des conditions d'évaluation équitables entre des modèles hétérogènes
- Éviter les saturations mémoire lors de traversées profondes
Apprentissages clés
- Compromis concrets entre moteurs de graphes natifs et RDBMS
- Méthodologie d'analyse des centiles de latence sous forte charge