ML-Powered Intelligence

Chuchu embeds lightweight ML models for instant decision-making with zero external dependencies.

Overview

Two ML models power Chuchu’s intelligence:

  1. Complexity Classifier – Automatically triggers Guided Mode for complex tasks
  2. Intent Classifier – Routes user requests 500x faster than LLM calls

Both models:

1. Complexity Classifier

Purpose

Analyzes task descriptions and classifies them as:

Auto-Activation

When you run chu chat, the complexity classifier decides if Guided Mode should activate:

chu chat "fix typo in readme"
# → Simple task, stays in chat mode

chu chat "implement oauth2 with jwt"
# → Complex task, automatically switches to Guided Mode

Configuration

# View current threshold (default: 0.55)
chu config get defaults.ml_complex_threshold

# Increase threshold (fewer Guided Mode triggers)
chu config set defaults.ml_complex_threshold 0.7

# Decrease threshold (more Guided Mode triggers)
chu config set defaults.ml_complex_threshold 0.4

Threshold Guide:

Training Data

Located at ml/complexity_detection/data/:

Example entries:

message,label,notes
"fix typo in readme",0,Simple single file edit
"implement oauth2 authentication",1,Complex multi-file feature
"first add tests then refactor api",2,Explicit sequence

Labels: 0=simple, 1=complex, 2=multistep

2. Intent Classifier

Purpose

Classifies user intent for routing to specialized agents:

Performance vs LLM

Metric ML Classifier LLM Router
Latency ~1ms ~500ms
Cost $0 $0.0001-0.001
Accuracy 85-90% 95%+
Fallback Yes (LLM) N/A

Smart Fallback

If confidence < threshold, falls back to LLM:

// Internal logic
confidence := mlPredict(userMessage)
if confidence >= threshold {
    return mlResult  // Fast path: 1ms
} else {
    return llmCall()  // Slow path: 500ms, more accurate
}

Configuration

# View current threshold (default: 0.7)
chu config get defaults.ml_intent_threshold

# Higher = more LLM fallbacks (slower but safer)
chu config set defaults.ml_intent_threshold 0.85

# Lower = more ML predictions (faster but riskier)
chu config set defaults.ml_intent_threshold 0.6

Threshold Guide:

Training Data

Located at ml/intent/data/:

Example entries:

message,label
"explain this code",query
"add error handling",editor
"how to implement oauth",research
"check for bugs",review

CLI Commands

List Models

chu ml list

Shows available models and their status.

Train Models

# Train complexity classifier
chu ml train complexity

# Train intent classifier
chu ml train intent

Automatically:

  1. Creates Python venv
  2. Installs dependencies
  3. Trains model
  4. Exports to JSON

Test Models

# Interactive testing
chu ml test complexity
chu ml test intent

# Single prediction
chu ml test complexity "implement oauth"
chu ml test intent "explain this code"

Evaluate Models

# Use default eval.csv
chu ml eval complexity
chu ml eval intent

# Use custom dataset
chu ml eval intent -f my_eval.csv

Shows:

Predict (Go Runtime)

# Default: complexity
chu ml predict "implement oauth"

# Explicit model
chu ml predict complexity "fix typo"
chu ml predict intent "explain this code"

Uses embedded Go model – no Python required.

Model Architecture

Both models use the same architecture:

Input Text
    ↓
Tokenize + Clean
    ↓
TF-IDF Vectorization (1-3 grams)
    ↓
Logistic Regression
    ↓
Softmax → Probabilities

TF-IDF Features

Classifier

Export Format

Models export to JSON with:

Example structure:

{
  "metadata": {
    "version": "1.0.0",
    "model_type": "intent",
    "trained_at": "2025-11-22T12:00:00Z",
    "accuracy": 0.891
  },
  "tfidf": {
    "vocabulary": {"implement": 42, "oauth": 215, ...},
    "idf_weights": [2.3, 1.8, ...]
  },
  "classifier": {
    "coefficients": [[0.5, -0.2, ...], ...],
    "intercepts": [0.1, -0.3, ...],
    "classes": ["query", "editor", "research", "review"]
  }
}

Customizing Models

Add Training Examples

Edit ml/{model}/data/training_data.csv:

message,label
"your new example",appropriate_label

Then retrain:

chu ml train {model}

Adjust Hyperparameters

Edit ml/{model}/scripts/train.py:

vectorizer = TfidfVectorizer(
    max_features=1000,     # Vocabulary size
    ngram_range=(1, 3),    # Unigrams to trigrams
    min_df=1,              # Minimum document frequency
    max_df=0.9             # Maximum document frequency
)

clf = LogisticRegression(
    C=1.0,                 # Regularization strength
    max_iter=1000,         # Training iterations
    class_weight='balanced' # Handle class imbalance
)

Export to Go

After training, copy model to assets:

cp ml/{model}/models/{model}_model.json internal/ml/assets/

Then rebuild:

go build -o bin/chu cmd/chu/main.go

Performance Metrics

Complexity Classifier

Intent Classifier

ROI Analysis

For 1000 requests/day:

Without ML (LLM only):

With ML (80% ML, 20% LLM):

Savings:

Troubleshooting

Model not found

# Train the model first
chu ml train complexity
chu ml train intent

Python dependencies

Models require Python 3.8+ with:

Install manually:

cd ml/{model}
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Low accuracy

  1. Add more training examples
  2. Balance classes (equal examples per label)
  3. Increase max_features in TF-IDF
  4. Adjust regularization C parameter

Embedded model outdated

After retraining, copy new model:

cp ml/{model}/models/{model}_model.json internal/ml/assets/
go build ./cmd/chu

Next Steps