Display metric units in web UI, make seed idempotent, simplify code
- Load m.unit in get_domain() so MetricBound carries units from DB - Add Unit column to domains list template - Make load_transport_seed() idempotent with IntegrityError handling and metric unit backfill for existing DBs - Remove unused imports (json, sqlite3, Entity) - Simplify combinator loop to list comprehension - Merge duplicate conditional/valid branches in pipeline - Consolidate duplicated SQL in get_all_results() - Expand CLAUDE.md with fuller architecture docs and conventions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
26
CLAUDE.md
26
CLAUDE.md
@@ -1,12 +1,20 @@
|
|||||||
# PhysCom — Physical Combinatorics
|
# CLAUDE.md
|
||||||
|
|
||||||
Innovation discovery engine: generate entity combinations, filter by physical constraints, score against domain-specific metrics, rank results.
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
## What this project is
|
||||||
|
|
||||||
|
PhysCom (Physical Combinatorics) — innovation discovery engine that generates entity combinations across dimensions (e.g. platform × power_source), filters by physical constraints, scores against domain-specific metrics, and ranks results. Includes a CLI, a Flask/HTMX web UI, and a 5-pass pipeline (constraints → estimation → scoring → LLM review → human review).
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
- **Tests**: `python -m pytest tests/ -q` (48 tests, ~3s). Run after every change.
|
- **Install**: `pip install -e ".[dev,web]"` (editable install with test and web deps)
|
||||||
|
- **Tests (all)**: `python -m pytest tests/ -q` (48 tests, ~5s). Run after every change.
|
||||||
|
- **Single test file**: `python -m pytest tests/test_scorer.py -q`
|
||||||
|
- **Single test**: `python -m pytest tests/test_scorer.py::test_score_combination -q`
|
||||||
- **Web dev server**: `python -m physcom_web`
|
- **Web dev server**: `python -m physcom_web`
|
||||||
- **CLI**: `python -m physcom`
|
- **CLI**: `python -m physcom` (or `physcom` if installed)
|
||||||
|
- **Docker**: `docker compose up web` / `docker compose run cli physcom seed`
|
||||||
- **Seed data**: loaded automatically on first DB init (SQLite, `physcom.db` or `$PHYSCOM_DB`)
|
- **Seed data**: loaded automatically on first DB init (SQLite, `physcom.db` or `$PHYSCOM_DB`)
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
@@ -53,13 +61,19 @@ tests/ # pytest, uses seeded_repo fixture from conftest.py
|
|||||||
1. **Pass 1 — Constraints**: `ConstraintResolver.resolve()` → blocked/conditional/valid. Blocked combos get a result row and `continue`.
|
1. **Pass 1 — Constraints**: `ConstraintResolver.resolve()` → blocked/conditional/valid. Blocked combos get a result row and `continue`.
|
||||||
2. **Pass 2 — Estimation**: LLM or `_stub_estimate()` → raw metric values. Saved immediately via `save_raw_estimates()` (normalized_score=NULL).
|
2. **Pass 2 — Estimation**: LLM or `_stub_estimate()` → raw metric values. Saved immediately via `save_raw_estimates()` (normalized_score=NULL).
|
||||||
3. **Pass 3 — Scoring**: `Scorer.score_combination()` → log-normalized scores + weighted geometric mean composite. Saves via `save_scores()` + `save_result()`.
|
3. **Pass 3 — Scoring**: `Scorer.score_combination()` → log-normalized scores + weighted geometric mean composite. Saves via `save_scores()` + `save_result()`.
|
||||||
4. **Pass 4 — LLM Review**: Only for above-threshold combos with an LLM provider.
|
4. **Pass 4 — LLM Review**: Only for above-threshold combos with an LLM provider. No real provider yet (only `MockLLMProvider`).
|
||||||
5. **Pass 5 — Human Review**: Manual via web UI results page.
|
5. **Pass 5 — Human Review**: Manual via web UI results page.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- Tests use `seeded_repo` fixture (in-memory SQLite with transport seed data: 9 platforms, 9 power sources, 2 domains). There's also a bare `repo` fixture for tests that seed their own data.
|
||||||
|
- Individual entity fixtures (walking, bicycle, spaceship, solar_sail, etc.) are defined in `conftest.py`.
|
||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
- Python 3.11+, `from __future__ import annotations` everywhere.
|
- Python 3.11+, `from __future__ import annotations` everywhere.
|
||||||
- Dataclasses for models, no ORM.
|
- Dataclasses for models, no ORM.
|
||||||
- Tests use `seeded_repo` fixture (in-memory SQLite with transport seed data).
|
|
||||||
- Don't use `cd` in Bash commands — run from the working directory so pre-approved permission patterns match.
|
- Don't use `cd` in Bash commands — run from the working directory so pre-approved permission patterns match.
|
||||||
- Don't add docstrings/comments/type annotations to code you didn't change.
|
- Don't add docstrings/comments/type annotations to code you didn't change.
|
||||||
|
- `INSERT OR IGNORE` won't update existing rows — if adding a new column/field to seed data, also add an UPDATE for backfill.
|
||||||
|
- Jinja2 `0.0` is falsy — use `is not none` not `if value` when displaying scores that can legitimately be zero.
|
||||||
|
|||||||
@@ -2,7 +2,6 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import click
|
import click
|
||||||
|
|||||||
@@ -204,7 +204,7 @@ class Repository:
|
|||||||
if not row:
|
if not row:
|
||||||
return None
|
return None
|
||||||
weights = self.conn.execute(
|
weights = self.conn.execute(
|
||||||
"""SELECT m.name, dmw.weight, dmw.norm_min, dmw.norm_max, dmw.metric_id
|
"""SELECT m.name, m.unit, dmw.weight, dmw.norm_min, dmw.norm_max, dmw.metric_id
|
||||||
FROM domain_metric_weights dmw
|
FROM domain_metric_weights dmw
|
||||||
JOIN metrics m ON dmw.metric_id = m.id
|
JOIN metrics m ON dmw.metric_id = m.id
|
||||||
WHERE dmw.domain_id = ?""",
|
WHERE dmw.domain_id = ?""",
|
||||||
@@ -218,7 +218,7 @@ class Repository:
|
|||||||
MetricBound(
|
MetricBound(
|
||||||
metric_name=w["name"], weight=w["weight"],
|
metric_name=w["name"], weight=w["weight"],
|
||||||
norm_min=w["norm_min"], norm_max=w["norm_max"],
|
norm_min=w["norm_min"], norm_max=w["norm_max"],
|
||||||
metric_id=w["metric_id"],
|
metric_id=w["metric_id"], unit=w["unit"] or "",
|
||||||
)
|
)
|
||||||
for w in weights
|
for w in weights
|
||||||
],
|
],
|
||||||
@@ -411,26 +411,17 @@ class Repository:
|
|||||||
|
|
||||||
def get_all_results(self, domain_name: str, status: str | None = None) -> list[dict]:
|
def get_all_results(self, domain_name: str, status: str | None = None) -> list[dict]:
|
||||||
"""Return all results for a domain, optionally filtered by combo status."""
|
"""Return all results for a domain, optionally filtered by combo status."""
|
||||||
|
query = """SELECT cr.*, c.hash, c.status as combo_status, d.name as domain_name
|
||||||
|
FROM combination_results cr
|
||||||
|
JOIN combinations c ON cr.combination_id = c.id
|
||||||
|
JOIN domains d ON cr.domain_id = d.id
|
||||||
|
WHERE d.name = ?"""
|
||||||
|
params: list = [domain_name]
|
||||||
if status:
|
if status:
|
||||||
rows = self.conn.execute(
|
query += " AND c.status = ?"
|
||||||
"""SELECT cr.*, c.hash, c.status as combo_status, d.name as domain_name
|
params.append(status)
|
||||||
FROM combination_results cr
|
query += " ORDER BY cr.composite_score DESC"
|
||||||
JOIN combinations c ON cr.combination_id = c.id
|
rows = self.conn.execute(query, params).fetchall()
|
||||||
JOIN domains d ON cr.domain_id = d.id
|
|
||||||
WHERE d.name = ? AND c.status = ?
|
|
||||||
ORDER BY cr.composite_score DESC""",
|
|
||||||
(domain_name, status),
|
|
||||||
).fetchall()
|
|
||||||
else:
|
|
||||||
rows = self.conn.execute(
|
|
||||||
"""SELECT cr.*, c.hash, c.status as combo_status, d.name as domain_name
|
|
||||||
FROM combination_results cr
|
|
||||||
JOIN combinations c ON cr.combination_id = c.id
|
|
||||||
JOIN domains d ON cr.domain_id = d.id
|
|
||||||
WHERE d.name = ?
|
|
||||||
ORDER BY cr.composite_score DESC""",
|
|
||||||
(domain_name,),
|
|
||||||
).fetchall()
|
|
||||||
results = []
|
results = []
|
||||||
for r in rows:
|
for r in rows:
|
||||||
combo = self.get_combination(r["combination_id"])
|
combo = self.get_combination(r["combination_id"])
|
||||||
|
|||||||
@@ -25,9 +25,7 @@ def generate_combinations(
|
|||||||
raise ValueError(f"No entities found for dimension '{dim}'")
|
raise ValueError(f"No entities found for dimension '{dim}'")
|
||||||
entity_groups.append(entities)
|
entity_groups.append(entities)
|
||||||
|
|
||||||
combinations = []
|
return [
|
||||||
for entity_tuple in itertools.product(*entity_groups):
|
Combination(entities=list(entity_tuple))
|
||||||
combo = Combination(entities=list(entity_tuple))
|
for entity_tuple in itertools.product(*entity_groups)
|
||||||
combinations.append(combo)
|
]
|
||||||
|
|
||||||
return combinations
|
|
||||||
|
|||||||
@@ -157,14 +157,13 @@ class Pipeline:
|
|||||||
result.pass1_blocked += 1
|
result.pass1_blocked += 1
|
||||||
self._update_run_counters(run_id, result, current_pass=1)
|
self._update_run_counters(run_id, result, current_pass=1)
|
||||||
continue # blocked — skip remaining passes
|
continue # blocked — skip remaining passes
|
||||||
elif cr.status == "conditional":
|
|
||||||
combo.status = "valid"
|
|
||||||
self.repo.update_combination_status(combo.id, "valid")
|
|
||||||
result.pass1_conditional += 1
|
|
||||||
else:
|
else:
|
||||||
combo.status = "valid"
|
combo.status = "valid"
|
||||||
self.repo.update_combination_status(combo.id, "valid")
|
self.repo.update_combination_status(combo.id, "valid")
|
||||||
result.pass1_valid += 1
|
if cr.status == "conditional":
|
||||||
|
result.pass1_conditional += 1
|
||||||
|
else:
|
||||||
|
result.pass1_valid += 1
|
||||||
|
|
||||||
self._update_run_counters(run_id, result, current_pass=1)
|
self._update_run_counters(run_id, result, current_pass=1)
|
||||||
elif 1 in passes:
|
elif 1 in passes:
|
||||||
|
|||||||
@@ -265,22 +265,35 @@ INTERPLANETARY = Domain(
|
|||||||
|
|
||||||
|
|
||||||
def load_transport_seed(repo) -> dict:
|
def load_transport_seed(repo) -> dict:
|
||||||
"""Load all transport seed data into the repository. Returns counts."""
|
"""Load all transport seed data into the repository. Idempotent — safe to re-run."""
|
||||||
|
import sqlite3
|
||||||
from physcom.db.repository import Repository
|
from physcom.db.repository import Repository
|
||||||
repo: Repository
|
repo: Repository
|
||||||
|
|
||||||
counts = {"platforms": 0, "power_sources": 0, "domains": 0}
|
counts = {"platforms": 0, "power_sources": 0, "domains": 0}
|
||||||
|
|
||||||
for entity in PLATFORMS:
|
for entity in PLATFORMS:
|
||||||
repo.add_entity(entity)
|
try:
|
||||||
counts["platforms"] += 1
|
repo.add_entity(entity)
|
||||||
|
counts["platforms"] += 1
|
||||||
|
except sqlite3.IntegrityError:
|
||||||
|
pass
|
||||||
|
|
||||||
for entity in POWER_SOURCES:
|
for entity in POWER_SOURCES:
|
||||||
repo.add_entity(entity)
|
try:
|
||||||
counts["power_sources"] += 1
|
repo.add_entity(entity)
|
||||||
|
counts["power_sources"] += 1
|
||||||
|
except sqlite3.IntegrityError:
|
||||||
|
pass
|
||||||
|
|
||||||
repo.add_domain(URBAN_COMMUTING)
|
for domain in (URBAN_COMMUTING, INTERPLANETARY):
|
||||||
repo.add_domain(INTERPLANETARY)
|
try:
|
||||||
counts["domains"] = 2
|
repo.add_domain(domain)
|
||||||
|
counts["domains"] += 1
|
||||||
|
except sqlite3.IntegrityError:
|
||||||
|
pass
|
||||||
|
# Backfill metric units on existing DBs (ensure_metric is idempotent).
|
||||||
|
for mb in domain.metric_bounds:
|
||||||
|
repo.ensure_metric(mb.metric_name, unit=mb.unit)
|
||||||
|
|
||||||
return counts
|
return counts
|
||||||
|
|||||||
@@ -13,12 +13,13 @@
|
|||||||
<p>{{ d.description }}</p>
|
<p>{{ d.description }}</p>
|
||||||
<table>
|
<table>
|
||||||
<thead>
|
<thead>
|
||||||
<tr><th>Metric</th><th>Weight</th><th>Norm Min</th><th>Norm Max</th></tr>
|
<tr><th>Metric</th><th>Unit</th><th>Weight</th><th>Norm Min</th><th>Norm Max</th></tr>
|
||||||
</thead>
|
</thead>
|
||||||
<tbody>
|
<tbody>
|
||||||
{% for mb in d.metric_bounds %}
|
{% for mb in d.metric_bounds %}
|
||||||
<tr>
|
<tr>
|
||||||
<td>{{ mb.metric_name }}</td>
|
<td>{{ mb.metric_name }}</td>
|
||||||
|
<td>{{ mb.unit }}</td>
|
||||||
<td>{{ mb.weight }}</td>
|
<td>{{ mb.weight }}</td>
|
||||||
<td>{{ mb.norm_min }}</td>
|
<td>{{ mb.norm_min }}</td>
|
||||||
<td>{{ mb.norm_max }}</td>
|
<td>{{ mb.norm_max }}</td>
|
||||||
|
|||||||
@@ -2,8 +2,6 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import sqlite3
|
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from physcom.db.schema import init_db
|
from physcom.db.schema import init_db
|
||||||
|
|||||||
@@ -1,7 +1,5 @@
|
|||||||
"""Tests for async pipeline: resume, cancellation, status guard, run lifecycle."""
|
"""Tests for async pipeline: resume, cancellation, status guard, run lifecycle."""
|
||||||
|
|
||||||
import json
|
|
||||||
|
|
||||||
from physcom.engine.constraint_resolver import ConstraintResolver
|
from physcom.engine.constraint_resolver import ConstraintResolver
|
||||||
from physcom.engine.scorer import Scorer
|
from physcom.engine.scorer import Scorer
|
||||||
from physcom.engine.pipeline import Pipeline, CancelledError
|
from physcom.engine.pipeline import Pipeline, CancelledError
|
||||||
@@ -287,7 +285,6 @@ def test_save_combination_loads_existing_status(seeded_repo):
|
|||||||
"""save_combination should load the status of an existing combo from DB."""
|
"""save_combination should load the status of an existing combo from DB."""
|
||||||
repo = seeded_repo
|
repo = seeded_repo
|
||||||
from physcom.models.combination import Combination
|
from physcom.models.combination import Combination
|
||||||
from physcom.models.entity import Entity
|
|
||||||
|
|
||||||
entities = repo.list_entities(dimension="platform")[:1] + repo.list_entities(dimension="power_source")[:1]
|
entities = repo.list_entities(dimension="platform")[:1] + repo.list_entities(dimension="power_source")[:1]
|
||||||
combo = Combination(entities=entities)
|
combo = Combination(entities=entities)
|
||||||
|
|||||||
Reference in New Issue
Block a user