close guardrail gaps and fix the scoring pipeline top to bottom
Constraint resolver: aggregate mass/footprint across a combo instead of pairwise-only checks, treat medium/atmosphere as agreement not supply/demand, reduce multi-provider checks by best/sum instead of AND-ing every provider, fail closed on unrecognized mutex values, add a propulsion-viability (thrust-to-weight) rule. Seed data updated to match (nuclear/solar-sail footprint floors, water-medium exclusions, explicit ground/gravity providers). Domain metric units were stored globally per metric name instead of per-domain, silently corrupting cost_efficiency for every domain but the first one seeded — fixed with a schema migration. Stub estimator's cost_efficiency/safety/availability/reliability were a backwards formula and flat constants; replaced with heuristics grounded in each entity's thrust_profile/energy_form/infrastructure. LLM estimate_physics() now receives each metric's unit and expected range instead of a bare name, fixing wildly miscalibrated estimates traced back to the prompt's own hardcoded example anchoring the model to the wrong order of magnitude. Sharpened the safety-estimation and plausibility-review prompts. Deduped provider parsing logic into llm/parsing.py. Web pipeline form can now pick an LLM provider per run instead of only via server env var. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -21,6 +21,9 @@ def _run_pipeline_in_background(
|
||||
passes: list[int],
|
||||
threshold: float,
|
||||
run_id: int,
|
||||
llm_provider: str | None = None,
|
||||
llm_model: str | None = None,
|
||||
llm_host: str | None = None,
|
||||
) -> None:
|
||||
"""Run the pipeline in a background thread with its own DB connection."""
|
||||
from physcom.db.schema import init_db
|
||||
@@ -45,7 +48,8 @@ def _run_pipeline_in_background(
|
||||
from physcom.llm.registry import build_llm_provider
|
||||
resolver = ConstraintResolver()
|
||||
scorer = Scorer(domain)
|
||||
pipeline = Pipeline(repo, resolver, scorer, llm=build_llm_provider())
|
||||
llm = build_llm_provider(provider=llm_provider, model=llm_model, host=llm_host)
|
||||
pipeline = Pipeline(repo, resolver, scorer, llm=llm)
|
||||
|
||||
pipeline.run(
|
||||
domain, dim_list,
|
||||
@@ -108,11 +112,17 @@ def pipeline_run():
|
||||
flash("Select at least one dimension.", "error")
|
||||
return redirect(url_for("pipeline.pipeline_form"))
|
||||
|
||||
llm_provider = request.form.get("llm_provider", "").strip() or None
|
||||
llm_model = request.form.get("llm_model", "").strip() or None
|
||||
llm_host = request.form.get("llm_host", "").strip() or None
|
||||
|
||||
# Create pipeline_run record
|
||||
config = {
|
||||
"passes": passes,
|
||||
"threshold": threshold,
|
||||
"dimensions": dim_list,
|
||||
"llm_provider": llm_provider,
|
||||
"llm_model": llm_model,
|
||||
}
|
||||
run_id = repo.create_pipeline_run(domain.id, config)
|
||||
|
||||
@@ -123,7 +133,8 @@ def pipeline_run():
|
||||
# Start background thread
|
||||
t = threading.Thread(
|
||||
target=_run_pipeline_in_background,
|
||||
args=(db_path, domain_name, dim_list, passes, threshold, run_id),
|
||||
args=(db_path, domain_name, dim_list, passes, threshold, run_id,
|
||||
llm_provider, llm_model, llm_host),
|
||||
daemon=True,
|
||||
)
|
||||
t.start()
|
||||
|
||||
@@ -49,6 +49,29 @@
|
||||
</div>
|
||||
</fieldset>
|
||||
|
||||
<fieldset>
|
||||
<legend>LLM Provider</legend>
|
||||
<p class="form-hint">Used for Pass 2 estimation and Pass 4 review. Leave on "server default" to use whatever LLM_PROVIDER is configured in the server environment (or the physics stub if none).</p>
|
||||
<div class="form-group">
|
||||
<select name="llm_provider" id="llm_provider">
|
||||
<option value="">— server default —</option>
|
||||
<option value="stub">Stub (fast, no LLM)</option>
|
||||
<option value="ollama">Ollama (local)</option>
|
||||
<option value="gemini">Gemini (cloud, requires server-side GEMINI_API_KEY)</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="llm_model">Model</label>
|
||||
<p class="form-hint">Leave blank to use the provider's default model.</p>
|
||||
<input type="text" name="llm_model" id="llm_model" placeholder="e.g. qwen2.5:7b or gemini-2.0-flash">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label for="llm_host">Ollama host</label>
|
||||
<p class="form-hint">Only used when Ollama is selected. Leave blank for http://localhost:11434.</p>
|
||||
<input type="text" name="llm_host" id="llm_host" placeholder="http://localhost:11434">
|
||||
</div>
|
||||
</fieldset>
|
||||
|
||||
<div class="form-group">
|
||||
<label for="threshold">Score Threshold</label>
|
||||
<p class="form-hint">Minimum composite score (0–1) for a combination to pass scoring. Lower values keep more results; higher values are more selective.</p>
|
||||
|
||||
Reference in New Issue
Block a user