Backtest Analysis

Use an LLM to evaluate backtest results — get a performance score (1-10), strengths, weaknesses, risk assessment, and actionable improvement suggestions.

Quick Usage

import vibetrading.strategy
import vibetrading.backtest

results = vibetrading.backtest.run(strategy_code, interval="1h", data=data)

report = vibetrading.strategy.analyze(
    results,
    strategy_code=strategy_code,
    model="gpt-4o",
)

print(f"Score: {report.score}/10")
print(report.summary)

BacktestAnalysisResult

The analysis returns a BacktestAnalysisResult with the following fields:

Field	Type	Description
score	int	Overall score (1-10)
summary	str	2-3 sentence assessment
strengths	list[str]	What the strategy does well
weaknesses	list[str]	Problems to address
risk_assessment	str	Risk evaluation
suggestions	list[str]	Actionable improvement recommendations
detailed_analysis	str	Multi-paragraph deep analysis
raw_metrics	dict	The metrics dict that was analyzed

Scoring Guidelines

Score	Meaning	Characteristics
9-10	Exceptional	Sharpe > 2.0, drawdown < 10%, strong win rate
7-8	Good	Positive risk-adjusted returns, manageable drawdowns
5-6	Mediocre	Marginal returns or concerning risk metrics
3-4	Poor	Negative returns or extreme drawdowns
1-2	Failing	Liquidation, massive losses, or non-functional

Reading the Report

report = vibetrading.strategy.analyze(results, strategy_code=code)

# Score & summary
print(f"Score: {report.score}/10")
print(report.summary)

# Strengths
for s in report.strengths:
    print(f"  + {s}")

# Weaknesses
for w in report.weaknesses:
    print(f"  - {w}")

# Risk assessment
print(report.risk_assessment)

# Actionable suggestions
for i, s in enumerate(report.suggestions, 1):
    print(f"  {i}. {s}")

# Full analysis text
print(report.detailed_analysis)

Feeding Analysis Back to the Generator

The analysis result includes format_for_llm() — a method that converts the report into structured feedback suitable for a follow-up generation call:

import vibetrading.strategy

# First generate and backtest
code = vibetrading.strategy.generate("BTC momentum with RSI", model="gpt-4o")
results = vibetrading.backtest.run(code, interval="1h", data=data)

# Analyze
report = vibetrading.strategy.analyze(results, strategy_code=code)

# Feed back into generator for improvement
feedback = report.format_for_llm()
improved_code = vibetrading.strategy.generate(
    "BTC momentum with RSI",
    model="gpt-4o",
    feedback=feedback,
)

Or use vibetrading.strategy.generate() and vibetrading.strategy.analyze() in a loop to iterate manually.

Detail Levels

Control how much data is sent to the LLM:

# Metrics only (fast, cheap)
report = vibetrading.strategy.analyze(results, detail_level="brief")

# Metrics + trade summary (default)
report = vibetrading.strategy.analyze(results, detail_level="standard")

# Metrics + trades + equity curve (most thorough)
report = vibetrading.strategy.analyze(results, detail_level="detailed")

Using BacktestAnalyzer Directly

For repeated analysis or custom configuration:

from vibetrading.strategy import BacktestAnalyzer

analyzer = BacktestAnalyzer(
    model="gpt-4o",
    temperature=0.3,
)

report1 = analyzer.analyze(results1, strategy_code=code1)
report2 = analyzer.analyze(results2, strategy_code=code2)