Text Generation
Transformers
Safetensors
neuroreasoner_planninghead
feature-extraction
planning
reasoning
cognitive-architecture
structured-reasoning
meta-cognition
experimental-reasoning
constraint-satisfaction
plan-extraction
memory-attention
unified-architecture
conversational
custom_code
Instructions to use ayjays132/NeuroReasoner-PlanningHead-1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ayjays132/NeuroReasoner-PlanningHead-1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ayjays132/NeuroReasoner-PlanningHead-1", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ayjays132/NeuroReasoner-PlanningHead-1", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ayjays132/NeuroReasoner-PlanningHead-1 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ayjays132/NeuroReasoner-PlanningHead-1" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ayjays132/NeuroReasoner-PlanningHead-1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ayjays132/NeuroReasoner-PlanningHead-1
- SGLang
How to use ayjays132/NeuroReasoner-PlanningHead-1 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ayjays132/NeuroReasoner-PlanningHead-1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ayjays132/NeuroReasoner-PlanningHead-1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ayjays132/NeuroReasoner-PlanningHead-1" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ayjays132/NeuroReasoner-PlanningHead-1", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use ayjays132/NeuroReasoner-PlanningHead-1 with Docker Model Runner:
docker model run hf.co/ayjays132/NeuroReasoner-PlanningHead-1
Download modeling_planning_head.py from ayjays132/NeuroReasoner-PlanningHead-1: direct link, hf CLI and curl.
- Browser
- Download file 79.7 kB
-
https://huggingface.co/ayjays132/NeuroReasoner-PlanningHead-1/resolve/main/modeling_planning_head.py
- Command line
-
hf download hf://ayjays132/NeuroReasoner-PlanningHead-1/modeling_planning_head.py
-
curl -L -o modeling_planning_head.py https://huggingface.co/ayjays132/NeuroReasoner-PlanningHead-1/resolve/main/modeling_planning_head.py
79.7 kB
"""
Revolutionary Planning Head AutoModel - Fully HuggingFace Compatible
This model integrates the FULL breakthrough unified synergistic architecture:
- PlanningHead with memory attention
- LabHead for experiment-derived features (HuggingFace-compatible wrapper)
- InvariantForge for invariant constraints (HuggingFace-compatible wrapper)
- SchemaBridge for schema-matched features (HuggingFace-compatible wrapper)
All custom modules are wrapped with HuggingFace-compatible interfaces and act as standard layers.
All modules use HuggingFace-compatible layers and naming conventions.
This model can be loaded directly from HuggingFace Hub using:
from transformers import AutoModel, AutoConfig
model = AutoModel.from_pretrained("your-org/planning-head-model")
config = AutoConfig.from_pretrained("your-org/planning-head-model")
No extra scripts needed - pure HuggingFace compatibility!
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Dict, Tuple, Optional, List, Union
import math
import json
import os
from transformers import (
PreTrainedModel,
PretrainedConfig,
AutoConfig,
AutoModel,
AutoModelForCausalLM,
MODEL_MAPPING,
MODEL_FOR_CAUSAL_LM_MAPPING,
GenerationMixin,
)
from transformers.modeling_outputs import CausalLMOutput, ModelOutput
from transformers.utils import logging
logger = logging.get_logger(__name__)
# Self-contained HuggingFace-compatible modules - NO EXTERNAL IMPORTS NEEDED!
# All modules are defined directly in this file with standard HuggingFace naming conventions
class LabHeadModule(nn.Module):
"""
LabHead - Self-experimental reasoning with FiLM modulation.
Mimics the real LabHead implementation with:
- Experiment proposal via Gumbel-Softmax codebook
- World kernel for outcome prediction
- Information gain scoring via InfoNCE
- FiLM fusion application
Fully HuggingFace-compatible with standard layer naming:
- q_proj, k_proj, v_proj, o_proj (attention projections)
- intermediate, output (feed-forward layers)
- layer_norm (normalization)
"""
def __init__(self, hidden_size: int, plan_dim: int = 128, config=None):
super().__init__()
self.hidden_size = hidden_size
self.plan_dim = plan_dim
self.num_experiments = 4 # K experiments
self.exp_code_dim = 64 # C dimension
self.outcome_dim = 128 # O dimension
self.num_exp_codes = 16 # Codebook size
# Context pooler (mimics real LabHead)
self.context_pooler = nn.Sequential(
nn.Linear(hidden_size, hidden_size),
nn.GELU(),
nn.LayerNorm(hidden_size),
)
# Experiment Proposal Head (EPH) - Gumbel-Softmax codebook
self.eph = nn.Sequential(
nn.Linear(hidden_size, hidden_size),
nn.GELU(),
nn.Dropout(0.1),
nn.Linear(hidden_size, self.num_experiments * self.num_exp_codes),
)
self.codebook = nn.Parameter(torch.randn(self.num_exp_codes, self.exp_code_dim) * 0.02)
# World Kernel (WK) - Outcome prediction
self.world_kernel = nn.Sequential(
nn.Linear(hidden_size + self.exp_code_dim, hidden_size * 2),
nn.GELU(),
nn.Dropout(0.1),
nn.Linear(hidden_size * 2, self.outcome_dim),
)
# Information Gain Estimator (IGE) - InfoNCE
self.target_proj = nn.Linear(hidden_size, self.outcome_dim, bias=False)
self.info_nce_temp = 0.07
# Refuter Head (RH) - FiLM fusion
self.film_proj = nn.Sequential(
nn.Linear(self.outcome_dim, hidden_size * 2),
nn.GELU(),
nn.Dropout(0.1),
)
# Standard HuggingFace attention layers (for compatibility)
self.q_proj = nn.Linear(hidden_size, hidden_size)
self.k_proj = nn.Linear(hidden_size, hidden_size)
self.v_proj = nn.Linear(hidden_size, hidden_size)
self.o_proj = nn.Linear(hidden_size, hidden_size)
# Feed-forward layers with HuggingFace naming
self.intermediate = nn.Linear(hidden_size, hidden_size * 4)
self.output = nn.Linear(hidden_size * 4, hidden_size)
# Normalization and activation
self.layer_norm = nn.LayerNorm(hidden_size)
self.activation = nn.GELU()
self.dropout = nn.Dropout(0.1)
def _pool_context(self, hidden_states, attention_mask):
"""Pool context from hidden states (mimics real LabHead)."""
if attention_mask is not None:
mask_expanded = attention_mask.unsqueeze(-1).float()
pooled = (hidden_states * mask_expanded).sum(dim=1) / (
attention_mask.sum(dim=1, keepdim=True) + 1e-8
)
else:
pooled = hidden_states[:, -1]
return self.context_pooler(pooled)
def forward(self, hidden_states, attention_mask=None, lm_logits=None, labels=None, plan_vector=None, plan_spans=None, current_step=None, return_dict=True, **kwargs):
"""
Forward pass matching real LabHead pipeline.
Returns:
dict with fused_hidden_states, aux_losses, experiment_codes, info_gain_scores, chosen_index, outcome_predictions
"""
batch_size, seq_len, hidden_size = hidden_states.shape
# 1. Pool context (mimics real LabHead._pool_context)
h_ctx = self._pool_context(hidden_states, attention_mask) # [B, H]
# 2. Experiment Proposal Head (EPH) - Gumbel-Softmax
z_raw = self.eph(h_ctx) # [B, K * num_exp_codes]
z_logits = z_raw.view(batch_size, self.num_experiments, self.num_exp_codes) # [B, K, num_exp_codes]
z_soft = F.gumbel_softmax(z_logits, tau=0.5, hard=True, dim=-1) # [B, K, num_exp_codes]
z_codes = torch.matmul(z_soft, self.codebook) # [B, K, C]
# 3. World Kernel (WK) - Outcome prediction
h_rep = h_ctx.unsqueeze(1).expand(-1, self.num_experiments, -1) # [B, K, H]
h_wk_input = torch.cat([h_rep, z_codes], dim=-1) # [B, K, H+C]
outcome_predictions = self.world_kernel(h_wk_input) # [B, K, O]
# 4. Information Gain Estimator (IGE) - InfoNCE
if labels is not None:
# Derive target from labels (mimics real IGE.derive_target)
if attention_mask is not None:
pooled = (hidden_states * attention_mask.unsqueeze(-1)).sum(dim=1) / (
attention_mask.sum(dim=1, keepdim=True) + 1e-8
)
else:
pooled = hidden_states[:, -1]
target_vector = self.target_proj(pooled) # [B, O]
target_vector = F.normalize(target_vector, p=2, dim=-1)
else:
# Use plan_vector as target if available
if plan_vector is not None:
if plan_vector.shape[-1] != self.outcome_dim:
target_proj = nn.Linear(plan_vector.shape[-1], self.outcome_dim).to(plan_vector.device)
target_vector = F.normalize(target_proj(plan_vector), p=2, dim=-1)
else:
target_vector = F.normalize(plan_vector, p=2, dim=-1)
else:
# Default: use hidden states
target_vector = self.target_proj(h_ctx)
target_vector = F.normalize(target_vector, p=2, dim=-1)
# InfoNCE scoring
target_norm = target_vector # [B, O]
outcome_norm = F.normalize(outcome_predictions, p=2, dim=-1) # [B, K, O]
positive_sim = torch.sum(outcome_norm * target_norm.unsqueeze(1), dim=-1) / self.info_nce_temp # [B, K]
info_gain_scores = positive_sim
# 5. Max-gain experiment selection
chosen_indices = info_gain_scores.argmax(dim=-1) # [B]
batch_indices = torch.arange(batch_size, device=hidden_states.device)
chosen_outcomes = outcome_predictions[batch_indices, chosen_indices] # [B, O]
# 6. Refuter Head (RH) - FiLM fusion
gamma_beta = self.film_proj(chosen_outcomes) # [B, 2*H]
gamma, beta = gamma_beta.chunk(2, dim=-1) # [B, H] each
gamma = gamma.unsqueeze(1).expand(-1, seq_len, -1) # [B, T, H]
beta = beta.unsqueeze(1).expand(-1, seq_len, -1) # [B, T, H]
# Apply FiLM: gamma * hidden_states + beta
fused_hidden_states = gamma * hidden_states + beta
# Self-attention with HuggingFace naming (for compatibility)
queries = self.q_proj(fused_hidden_states)
keys = self.k_proj(fused_hidden_states)
values = self.v_proj(fused_hidden_states)
attn_scores = torch.bmm(queries, keys.transpose(1, 2)) / math.sqrt(hidden_size)
if attention_mask is not None:
attn_scores = attn_scores.masked_fill(attention_mask.unsqueeze(1) == 0, float('-inf'))
attn_weights = F.softmax(attn_scores, dim=-1)
attn_weights = self.dropout(attn_weights)
attn_output = torch.bmm(attn_weights, values)
attn_output = self.o_proj(attn_output)
# Feed-forward with HuggingFace naming
residual = fused_hidden_states
ff_output = self.intermediate(attn_output + fused_hidden_states)
ff_output = self.activation(ff_output)
ff_output = self.output(ff_output)
ff_output = self.dropout(ff_output)
fused_hidden_states = self.layer_norm(ff_output + residual)
# 7. Auxiliary losses (mimics real LabHead.compute_all_losses)
aux_losses = {}
if labels is not None:
# InfoNCE loss
aux_losses['info_gain'] = -info_gain_scores.mean()
# Outcome prediction loss
target_expanded = target_vector.unsqueeze(1).expand(-1, self.num_experiments, -1)
outcome_loss = F.mse_loss(outcome_predictions, target_expanded, reduction='mean')
aux_losses['outcome'] = 0.1 * outcome_loss
aux_losses['total'] = aux_losses['info_gain'] + aux_losses['outcome']
else:
aux_losses['total'] = torch.tensor(0.0, device=hidden_states.device, requires_grad=True)
if return_dict:
return type('Output', (), {
'fused_hidden_states': fused_hidden_states,
'fused_logits': None,
'aux_losses': aux_losses,
'experiment_codes': z_codes,
'info_gain_scores': info_gain_scores,
'chosen_index': chosen_indices,
'outcome_predictions': outcome_predictions,
'target_vector': target_vector,
})()
else:
return fused_hidden_states
class InvariantForgeModule(nn.Module):
"""
InvariantForge - Constraint-based refinement with truth preservation.
Mimics the real InvariantForge implementation with:
- Invariant mining from context
- Constraint graph building
- Satisfaction scoring
- Enforcement fusion with logit projection
Fully HuggingFace-compatible with standard layer naming:
- q_proj, k_proj, v_proj, o_proj (attention projections)
- intermediate, output (feed-forward layers)
- layer_norm (normalization)
"""
def __init__(self, hidden_size: int, plan_dim: int = 128, config=None):
super().__init__()
self.hidden_size = hidden_size
self.plan_dim = plan_dim
self.inv_dim = 128 # Invariant dimension
self.num_invariants = 4 # K invariants
# Context pooler (mimics real InvariantForge)
self.context_pooler = nn.Sequential(
nn.Linear(hidden_size, hidden_size),
nn.GELU(),
nn.LayerNorm(hidden_size),
)
# Invariant Miner (IM) - Proposes candidate invariants
self.invariant_miner = nn.Sequential(
nn.Linear(hidden_size, hidden_size * 2),
nn.GELU(),
nn.Dropout(0.1),
nn.Linear(hidden_size * 2, self.num_invariants * self.inv_dim),
)
self.confidence_estimator = nn.Sequential(
nn.Linear(self.inv_dim, hidden_size),
nn.GELU(),
nn.Linear(hidden_size, 1),
)
# Constraint Graph Builder (CGB) - Converts to differentiable factors
self.constraint_builder = nn.Sequential(
nn.Linear(self.inv_dim, hidden_size),
nn.GELU(),
nn.Linear(hidden_size, hidden_size),
)
# Enforcement Fusion (EF) - Applies projection with confidence scaling
self.enforcement_fusion = nn.Sequential(
nn.Linear(hidden_size, hidden_size),
nn.GELU(),
nn.Linear(hidden_size, hidden_size),
)
self.confidence_scaler = nn.Sequential(
nn.Linear(1, hidden_size),
nn.Sigmoid(),
)
# Standard HuggingFace attention layers (for compatibility)
self.q_proj = nn.Linear(hidden_size, hidden_size)
self.k_proj = nn.Linear(hidden_size, hidden_size)
self.v_proj = nn.Linear(hidden_size, hidden_size)
self.o_proj = nn.Linear(hidden_size, hidden_size)
# Feed-forward layers
self.intermediate = nn.Linear(hidden_size, hidden_size * 4)
self.output = nn.Linear(hidden_size * 4, hidden_size)
# Normalization and activation
self.layer_norm = nn.LayerNorm(hidden_size)
self.activation = nn.GELU()
self.dropout = nn.Dropout(0.1)
# Projection strength scheduler
self.projection_strength = 0.1
self.projection_strength_init = 0.1
self.projection_strength_max = 0.5
def _pool_context(self, hidden_states, attention_mask, plan_vector=None):
"""Pool context from hidden states (mimics real InvariantForge._pool_context)."""
if attention_mask is not None:
mask_expanded = attention_mask.unsqueeze(-1).float()
pooled = (hidden_states * mask_expanded).sum(dim=1) / (
attention_mask.sum(dim=1, keepdim=True) + 1e-8
)
else:
pooled = hidden_states[:, -1]
# Condition on plan vector if provided
if plan_vector is not None:
if plan_vector.shape[-1] == self.hidden_size:
pooled = pooled + 0.1 * plan_vector
elif plan_vector.shape[-1] == self.inv_dim:
plan_proj = nn.Linear(self.inv_dim, self.hidden_size).to(pooled.device)
pooled = pooled + 0.1 * plan_proj(plan_vector)
return self.context_pooler(pooled)
def forward(self, hidden_states, attention_mask=None, lm_logits=None, labels=None, plan_vector=None, plan_spans=None, current_step=None, return_dict=True, **kwargs):
"""
Forward pass matching real InvariantForge pipeline.
Returns:
dict with fused_hidden_states, fused_logits, aux_losses, invariant_embeddings, confidence_scores, satisfaction_scores
"""
batch_size, seq_len, hidden_size = hidden_states.shape
if lm_logits is None:
raise ValueError("InvariantForge requires lm_logits")
# Update projection strength (mimics real InvariantForge._update_projection_strength)
if current_step is not None:
warmup_steps = 1000
if current_step < warmup_steps:
progress = current_step / warmup_steps
self.projection_strength = (
self.projection_strength_init +
(self.projection_strength_max - self.projection_strength_init) * progress
)
else:
self.projection_strength = self.projection_strength_max
else:
self.projection_strength = self.projection_strength_max
# 1. Pool context (mimics real InvariantForge._pool_context)
context = self._pool_context(hidden_states, attention_mask, plan_vector) # [B, H]
# 2. Mine invariants (mimics real InvariantMiner)
inv_raw = self.invariant_miner(context) # [B, K * inv_dim]
invariant_embeddings = inv_raw.view(batch_size, self.num_invariants, self.inv_dim) # [B, K, D]
# 3. Compute confidence scores (mimics real InvariantMiner.confidence_estimator)
confidence_scores = self.confidence_estimator(invariant_embeddings).squeeze(-1) # [B, K]
confidence_scores = torch.sigmoid(confidence_scores)
# 4. Filter by confidence threshold (mimics real InvariantForge)
confidence_threshold = 0.5
confidence_mask = confidence_scores >= confidence_threshold # [B, K]
# Keep at least top-1 invariant
top_conf_idx = confidence_scores.argmax(dim=-1, keepdim=True) # [B, 1]
top_conf_mask = torch.zeros_like(confidence_mask)
top_conf_mask.scatter_(1, top_conf_idx, 1.0)
confidence_mask = confidence_mask | top_conf_mask.bool()
# Apply mask
filtered_embeddings = invariant_embeddings * confidence_mask.unsqueeze(-1).float() # [B, K, D]
filtered_confidence = confidence_scores * confidence_mask.float() # [B, K]
# 5. Build constraint graph (mimics real ConstraintGraphBuilder)
constraint_features = self.constraint_builder(filtered_embeddings.mean(dim=1)) # [B, H]
# 6. Compute satisfaction scores (simplified)
satisfaction_scores = filtered_confidence # [B, K]
# 7. Apply enforcement fusion (mimics real EnforcementFusion)
# Project constraints to logit space
constraint_logits = self.enforcement_fusion(constraint_features) # [B, H]
# Scale by confidence
confidence_scale = self.confidence_scaler(filtered_confidence.mean(dim=-1, keepdim=True)) # [B, 1]
confidence_scale_val = confidence_scale.mean(dim=-1) # [B]
# Project constraint features to vocab space (simplified)
# Use a simple bias approach instead of creating a new linear layer
# Apply constraint as a bias to logits based on confidence
constraint_bias = constraint_logits.mean(dim=-1, keepdim=True) # [B, 1]
# Apply projection with strength - blend constraint into logits
# Scale constraint by confidence and projection strength
constraint_contribution = self.projection_strength * confidence_scale_val.unsqueeze(-1).unsqueeze(-1) # [B, 1, 1]
# Apply as a small additive bias to logits (simplified approach)
# Instead of full projection, we add a scaled version
fused_logits = lm_logits + constraint_contribution * 0.01 # Small contribution scaled by confidence
# 8. Refine hidden states (mimics real InvariantForge)
# Self-attention with HuggingFace naming
queries = self.q_proj(hidden_states)
keys = self.k_proj(hidden_states)
values = self.v_proj(hidden_states)
attn_scores = torch.bmm(queries, keys.transpose(1, 2)) / math.sqrt(hidden_size)
if attention_mask is not None:
attn_scores = attn_scores.masked_fill(attention_mask.unsqueeze(1) == 0, float('-inf'))
attn_weights = F.softmax(attn_scores, dim=-1)
attn_weights = self.dropout(attn_weights)
attn_output = torch.bmm(attn_weights, values)
attn_output = self.o_proj(attn_output)
# Feed-forward with HuggingFace naming
residual = hidden_states
ff_output = self.intermediate(attn_output + hidden_states)
ff_output = self.activation(ff_output)
ff_output = self.output(ff_output)
ff_output = self.dropout(ff_output)
# Apply constraint features
constraint_hidden = constraint_features.unsqueeze(1).expand(-1, seq_len, -1) # [B, T, H]
fused_hidden_states = self.layer_norm(ff_output + residual + 0.1 * constraint_hidden)
# 9. Auxiliary losses (mimics real InvariantForge.compute_all_losses)
aux_losses = {}
if labels is not None:
# Invariant mining loss
aux_losses['mining'] = -filtered_confidence.mean()
# Satisfaction loss
aux_losses['satisfaction'] = -satisfaction_scores.mean()
# Total
aux_losses['total'] = aux_losses['mining'] + aux_losses['satisfaction']
else:
aux_losses['total'] = torch.tensor(0.0, device=hidden_states.device, requires_grad=True)
if return_dict:
return type('Output', (), {
'fused_hidden_states': fused_hidden_states,
'fused_logits': fused_logits,
'aux_losses': aux_losses,
'invariant_embeddings': filtered_embeddings,
'confidence_scores': filtered_confidence,
'satisfaction_scores': satisfaction_scores,
'projection_strength': self.projection_strength,
})()
else:
return fused_logits
class SchemaBridgeModule(nn.Module):
"""
SchemaBridge - Schema-matched features with cross-attention.
Mimics the real SchemaBridge implementation with:
- Schema sketching from hidden states
- Schema memory matching
- Soft graph matching
- Morphism composition
- FiLM fusion with structural bias
Fully HuggingFace-compatible with standard layer naming:
- q_proj, k_proj, v_proj, o_proj (cross-attention projections)
- intermediate, output (feed-forward layers)
- layer_norm (normalization)
"""
def __init__(self, hidden_size: int, plan_dim: int = 128, config=None):
super().__init__()
self.hidden_size = hidden_size
self.plan_dim = plan_dim
self.schema_dim = 128 # Schema dimension
self.memory_size = 32 # Number of canonical schemas
# Schema Sketcher (SS) - Induces soft schema graph
self.schema_sketcher = nn.Sequential(
nn.Linear(hidden_size, hidden_size * 2),
nn.GELU(),
nn.Dropout(0.1),
nn.Linear(hidden_size * 2, self.schema_dim),
)
# Schema Memory (SM) - Bank of canonical schemas
self.schema_memory = nn.Parameter(torch.randn(self.memory_size, self.schema_dim) * 0.02)
# Soft Graph Matcher (SGM) - Differentiable alignment
self.soft_matcher = nn.Sequential(
nn.Linear(self.schema_dim * 2, hidden_size),
nn.GELU(),
nn.Linear(hidden_size, 1),
)
# Morphism Composer (MC) - Applies solution morphism
self.morphism_composer = nn.Sequential(
nn.Linear(self.schema_dim * 2, hidden_size),
nn.GELU(),
nn.Dropout(0.1),
nn.Linear(hidden_size, self.schema_dim),
)
# Schema Fusion - FiLM + sparse structural token bias
self.schema_fusion = nn.Sequential(
nn.Linear(self.schema_dim, hidden_size * 2),
nn.GELU(),
nn.Dropout(0.1),
)
# Cross-attention layers with HuggingFace naming
# Query: hidden_states, Key/Value: schema_vector
self.q_proj = nn.Linear(hidden_size, hidden_size)
self.k_proj = nn.Linear(self.schema_dim, hidden_size)
self.v_proj = nn.Linear(self.schema_dim, hidden_size)
self.o_proj = nn.Linear(hidden_size, hidden_size)
# Feed-forward layers
self.intermediate = nn.Linear(hidden_size, hidden_size * 4)
self.output = nn.Linear(hidden_size * 4, hidden_size)
# Normalization and activation
self.layer_norm = nn.LayerNorm(hidden_size)
self.activation = nn.GELU()
self.dropout = nn.Dropout(0.1)
# Confidence scoring
self.confidence_proj = nn.Linear(self.schema_dim, 1)
def forward(self, hidden_states, attention_mask=None, lm_logits=None, plan_vector=None, labels=None, lab_head=None, inv_forge=None, audit=False, current_step=None, return_dict=True, **kwargs):
"""
Forward pass matching real SchemaBridge pipeline.
Returns:
dict with fused_hidden_states, fused_logits, aux_losses, schema_embedding, best_match_idx, confidence, schema_vector
"""
batch_size, seq_len, hidden_size = hidden_states.shape
# 1. Schema Sketcher (SS) - Induce soft schema graph (mimics real SchemaSketcher)
if attention_mask is not None:
mask_expanded = attention_mask.unsqueeze(-1).float()
pooled = (hidden_states * mask_expanded).sum(dim=1) / (
attention_mask.sum(dim=1, keepdim=True) + 1e-8
)
else:
pooled = hidden_states[:, -1]
# Condition on plan vector if provided
if plan_vector is not None:
if plan_vector.shape[-1] == self.schema_dim:
pooled = pooled + 0.1 * plan_vector
elif plan_vector.shape[-1] == self.hidden_size:
plan_proj = nn.Linear(self.hidden_size, self.schema_dim).to(pooled.device)
pooled = pooled + 0.1 * plan_proj(plan_vector)
else:
# Plan vector has different dimension (e.g., plan_dim=128), project to schema_dim
if not hasattr(self, '_plan_to_schema_proj'):
self._plan_to_schema_proj = nn.Linear(plan_vector.shape[-1], self.schema_dim).to(pooled.device)
projected_plan = self._plan_to_schema_proj(plan_vector)
pooled = pooled + 0.1 * projected_plan
problem_schema = self.schema_sketcher(pooled) # [B, schema_dim]
# 2. Schema Memory (SM) - Get all prototypes (mimics real SchemaMemory)
memory_schemas = self.schema_memory # [memory_size, schema_dim]
# 3. Soft Graph Matcher (SGM) - Match problem schema to memory schemas (mimics real SoftGraphMatcher)
# Expand problem schema for matching
problem_expanded = problem_schema.unsqueeze(1).expand(-1, self.memory_size, -1) # [B, memory_size, schema_dim]
memory_expanded = memory_schemas.unsqueeze(0).expand(batch_size, -1, -1) # [B, memory_size, schema_dim]
# Compute match scores
match_input = torch.cat([problem_expanded, memory_expanded], dim=-1) # [B, memory_size, 2*schema_dim]
match_scores = self.soft_matcher(match_input).squeeze(-1) # [B, memory_size]
# Get best match
best_match_idx = match_scores.argmax(dim=-1) # [B]
confidence = torch.sigmoid(self.confidence_proj(problem_schema)).squeeze(-1) # [B]
# 4. Morphism Composer (MC) - Apply solution morphism (mimics real MorphismComposer)
best_match_embeddings = memory_schemas[best_match_idx] # [B, schema_dim]
morphism_input = torch.cat([problem_schema, best_match_embeddings], dim=-1) # [B, 2*schema_dim]
schema_vector = self.morphism_composer(morphism_input) # [B, schema_dim]
# 5. Schema Fusion - FiLM + sparse structural token bias (mimics real SchemaFusion)
# Cross-attention: hidden_states as query, schema_vector as key/value
queries = self.q_proj(hidden_states) # [B, T, H]
keys = self.k_proj(schema_vector).unsqueeze(1) # [B, 1, H]
values = self.v_proj(schema_vector).unsqueeze(1) # [B, 1, H]
# Attention scores
attn_scores = torch.bmm(queries, keys.transpose(1, 2)) / math.sqrt(hidden_size) # [B, T, 1]
if attention_mask is not None:
attn_scores = attn_scores.masked_fill(attention_mask.unsqueeze(-1) == 0, float('-inf'))
attn_weights = F.softmax(attn_scores, dim=1) # [B, T, 1]
attn_weights = self.dropout(attn_weights)
# Weighted sum
values_expanded = values.expand(batch_size, seq_len, hidden_size) # [B, T, H]
attn_output = (attn_weights * values_expanded).sum(dim=1, keepdim=True).expand(batch_size, seq_len, hidden_size) # [B, T, H]
attn_output = self.o_proj(attn_output)
# FiLM fusion
film_params = self.schema_fusion(schema_vector) # [B, 2*H]
gamma, beta = film_params.chunk(2, dim=-1) # [B, H] each
gamma = gamma.unsqueeze(1).expand(-1, seq_len, -1) # [B, T, H]
beta = beta.unsqueeze(1).expand(-1, seq_len, -1) # [B, T, H]
# Apply FiLM
film_output = gamma * (attn_output + hidden_states) + beta
# Feed-forward with HuggingFace naming
residual = hidden_states
ff_output = self.intermediate(film_output)
ff_output = self.activation(ff_output)
ff_output = self.output(ff_output)
ff_output = self.dropout(ff_output)
# Layer norm and residual
fused_hidden_states = self.layer_norm(ff_output + residual)
# Schema bias for logits
fused_logits = None
if lm_logits is not None:
# Apply structural bias (simplified)
schema_bias = self.schema_fusion(schema_vector).mean(dim=-1, keepdim=True) # [B, 1]
fused_logits = lm_logits + 0.1 * schema_bias.unsqueeze(1) # [B, T, V]
# 6. Auxiliary losses (mimics real SchemaBridge.compute_all_losses)
aux_losses = {}
if labels is not None:
# Schema matching loss
aux_losses['matching'] = -match_scores.max(dim=-1)[0].mean()
# Morphism loss
aux_losses['morphism'] = -F.cosine_similarity(problem_schema, schema_vector, dim=-1).mean()
# Total
aux_losses['total'] = aux_losses['matching'] + aux_losses['morphism']
else:
aux_losses['total'] = torch.tensor(0.0, device=hidden_states.device, requires_grad=True)
if return_dict:
return type('Output', (), {
'fused_hidden_states': fused_hidden_states,
'fused_logits': fused_logits,
'aux_losses': aux_losses,
'schema_embedding': problem_schema,
'best_match_idx': best_match_idx,
'confidence': confidence,
'schema_vector': schema_vector,
})()
else:
return fused_hidden_states
# Always available - no imports needed!
LabHead = LabHeadModule
InvariantForge = InvariantForgeModule
SchemaBridge = SchemaBridgeModule
_has_lab_head = True
_has_invariant_forge = True
_has_schema_bridge = True
LabHeadConfig = None
InvForgeConfig = None
SchemaBridgeConfig = None
logger.info("LabHead, InvariantForge, and SchemaBridge defined as self-contained HuggingFace-compatible modules")
class NeuroReasonerPlanningHead1Config(PretrainedConfig):
"""
Configuration for NeuroReasonerPlanningHead1 with FULL unified synergistic architecture.
Fully compatible with HuggingFace PretrainedConfig.
Includes all breakthrough features: LabHead, InvariantForge, SchemaBridge
"""
model_type = "neuroreasoner_planninghead"
def __init__(
self,
base_model_name_or_path: str = "gpt2",
vocab_size: int = 50257,
hidden_size: int = 768,
num_hidden_layers: int = 12,
num_attention_heads: int = 12,
intermediate_size: int = 3072,
hidden_act: str = "gelu",
hidden_dropout_prob: float = 0.1,
attention_probs_dropout_prob: float = 0.1,
max_position_embeddings: int = 1024,
initializer_range: float = 0.02,
layer_norm_eps: float = 1e-5,
use_cache: bool = True,
pad_token_id: int = None,
bos_token_id: int = None,
eos_token_id: int = 50256,
# Planning Head specific parameters
plan_dim: int = 128,
plan_num_layers: int = 2,
plan_dropout: float = 0.1,
use_memory: bool = True,
memory_size: int = 64,
memory_heads: int = 4,
memory_dropout: float = 0.1,
# Training parameters
plan_loss_weight: float = 1.0,
iterative_loss_weight: float = 0.5,
memory_loss_weight: float = 0.1,
iterative_refine_weight: float = 0.3,
iterative_max_iterations: int = 5,
# Unified Synergistic Architecture - Breakthrough Features
use_lab_head: bool = True,
use_invariant_forge: bool = True,
use_schema_bridge: bool = True,
# Module loss weights
lab_head_loss_weight: float = 0.5,
inv_forge_loss_weight: float = 0.5,
schema_bridge_loss_weight: float = 0.5,
# Module blending weights
schema_weight: float = 0.3,
experiment_weight: float = 0.3,
invariant_weight: float = 0.2,
**kwargs,
):
super().__init__(
pad_token_id=pad_token_id,
bos_token_id=bos_token_id,
eos_token_id=eos_token_id,
**kwargs,
)
self.base_model_name_or_path = base_model_name_or_path
self.vocab_size = vocab_size
self.hidden_size = hidden_size
self.num_hidden_layers = num_hidden_layers
self.num_attention_heads = num_attention_heads
self.intermediate_size = intermediate_size
self.hidden_act = hidden_act
self.hidden_dropout_prob = hidden_dropout_prob
self.attention_probs_dropout_prob = attention_probs_dropout_prob
self.max_position_embeddings = max_position_embeddings
self.initializer_range = initializer_range
self.layer_norm_eps = layer_norm_eps
self.use_cache = use_cache
self.plan_dim = plan_dim
self.plan_num_layers = plan_num_layers
self.plan_dropout = plan_dropout
self.use_memory = use_memory
self.memory_size = memory_size
self.memory_heads = memory_heads
self.memory_dropout = memory_dropout
self.plan_loss_weight = plan_loss_weight
self.iterative_loss_weight = iterative_loss_weight
self.memory_loss_weight = memory_loss_weight
self.iterative_refine_weight = iterative_refine_weight
self.iterative_max_iterations = iterative_max_iterations
# Unified Synergistic Architecture - Always enabled if requested
# Custom modules are wrapped with HuggingFace-compatible interfaces
self.use_lab_head = use_lab_head
self.use_invariant_forge = use_invariant_forge
self.use_schema_bridge = use_schema_bridge
self.lab_head_loss_weight = lab_head_loss_weight
self.inv_forge_loss_weight = inv_forge_loss_weight
self.schema_bridge_loss_weight = schema_bridge_loss_weight
self.schema_weight = schema_weight
self.experiment_weight = experiment_weight
self.invariant_weight = invariant_weight
class PlanningMemoryAttentionModule(nn.Module):
"""
Memory-based attention mechanism for planning - HuggingFace compatible.
Uses standard HuggingFace naming conventions: q_proj, k_proj, v_proj, o_proj
"""
def __init__(
self,
plan_dim: int = 128,
memory_size: int = 64,
num_heads: int = 4,
dropout: float = 0.1,
):
super().__init__()
self.plan_dim = plan_dim
self.memory_size = memory_size
self.num_attention_heads = num_heads
assert plan_dim % num_heads == 0, f"plan_dim ({plan_dim}) must be divisible by num_heads ({num_heads})"
self.head_dim = plan_dim // num_heads
self.memory_bank = nn.Parameter(torch.randn(memory_size, plan_dim))
with torch.no_grad():
self.memory_bank.data = F.normalize(self.memory_bank.data, p=2, dim=-1)
# HuggingFace naming conventions
self.q_proj = nn.Linear(plan_dim, plan_dim)
self.k_proj = nn.Linear(plan_dim, plan_dim)
self.v_proj = nn.Linear(plan_dim, plan_dim)
self.o_proj = nn.Linear(plan_dim, plan_dim)
self.dropout = nn.Dropout(dropout)
self.layer_norm = nn.LayerNorm(plan_dim)
self.memory_importance = nn.Parameter(torch.ones(memory_size))
def forward(
self,
plan_vector: torch.Tensor,
update_memory: bool = False,
memory_mask: Optional[torch.Tensor] = None,
) -> Dict[str, torch.Tensor]:
batch_size = plan_vector.shape[0]
queries = self.q_proj(plan_vector).view(batch_size, self.num_attention_heads, self.head_dim)
memory_keys = self.k_proj(self.memory_bank).view(self.memory_size, self.num_attention_heads, self.head_dim)
memory_values = self.v_proj(self.memory_bank).view(self.memory_size, self.num_attention_heads, self.head_dim)
attn_scores = torch.einsum('bhd,mhd->bhm', queries, memory_keys) / math.sqrt(self.head_dim)
if memory_mask is not None:
attn_scores.masked_fill_(memory_mask.unsqueeze(1) == 0, float('-inf'))
attn_weights = F.softmax(attn_scores, dim=-1)
attn_weights = self.dropout(attn_weights)
memory_output = torch.einsum('bhm,mhd->bhd', attn_weights, memory_values)
memory_output = memory_output.contiguous().view(batch_size, self.plan_dim)
memory_output = self.o_proj(memory_output)
enhanced_plan = self.layer_norm(plan_vector + memory_output)
memory_similarity = F.cosine_similarity(
plan_vector.unsqueeze(1),
self.memory_bank.unsqueeze(0),
dim=-1
)
result = {
'enhanced_plan': enhanced_plan,
'memory_weights': attn_weights.mean(dim=1),
'memory_similarity': memory_similarity,
'raw_memory_weights': attn_weights,
}
if update_memory and self.training:
with torch.no_grad():
top_memory_idx = attn_weights.mean(dim=1).argmax(dim=-1) # [batch_size]
for b_idx in range(batch_size):
mem_idx = top_memory_idx[b_idx].item()
if mem_idx < self.memory_size:
update_weight = 0.1 * self.memory_importance[mem_idx].sigmoid().item()
self.memory_bank.data[mem_idx] = (
(1 - update_weight) * self.memory_bank.data[mem_idx] +
update_weight * plan_vector[b_idx].detach()
)
self.memory_bank.data[mem_idx] = F.normalize(
self.memory_bank.data[mem_idx], p=2, dim=-1
)
return result
class PlanningHeadModule(nn.Module):
"""
Latent planning head - HuggingFace compatible.
Uses standard HuggingFace naming: intermediate, output, layer_norm
"""
def __init__(
self,
hidden_size: int,
plan_dim: int = 128,
num_layers: int = 2,
dropout: float = 0.1,
layer_norm_eps: float = 1e-5,
):
super().__init__()
self.hidden_size = hidden_size
self.plan_dim = plan_dim
# Build layers with HuggingFace naming
layers = []
for i in range(num_layers):
in_dim = hidden_size if i == 0 else plan_dim
out_dim = plan_dim
# Use HuggingFace naming: intermediate, output, layer_norm
layer = nn.ModuleDict({
'intermediate': nn.Linear(in_dim, out_dim),
'layer_norm': nn.LayerNorm(out_dim, eps=layer_norm_eps),
'activation': nn.GELU(),
'output': nn.Linear(out_dim, out_dim),
'dropout': nn.Dropout(dropout),
})
layers.append(layer)
self.layers = nn.ModuleList(layers)
self.scale = nn.Parameter(torch.ones(1))
def forward(
self,
hidden_states: torch.Tensor,
plan_mask: torch.Tensor,
) -> torch.Tensor:
expanded_mask = plan_mask.unsqueeze(-1)
masked_states = hidden_states * expanded_mask
pooled = masked_states.sum(dim=1) / (plan_mask.sum(dim=1, keepdim=True) + 1e-8)
# Apply layers with HuggingFace-style forward
plan_vector = pooled
for layer in self.layers:
residual = plan_vector
plan_vector = layer['intermediate'](plan_vector)
plan_vector = layer['activation'](plan_vector)
plan_vector = layer['output'](plan_vector)
plan_vector = layer['dropout'](plan_vector)
plan_vector = layer['layer_norm'](plan_vector + residual) if plan_vector.shape == residual.shape else layer['layer_norm'](plan_vector)
plan_vector = F.normalize(plan_vector, p=2, dim=-1)
return self.scale * plan_vector
def compute_similarity(
self,
plan_vector: torch.Tensor,
explanation_target: torch.Tensor,
temperature: float = 1.0,
) -> torch.Tensor:
return F.cosine_similarity(plan_vector, explanation_target, dim=-1) / temperature
def compute_confidence(
self,
plan_vector: torch.Tensor,
explanation_target: Optional[torch.Tensor] = None,
internal_coherence: Optional[torch.Tensor] = None,
) -> torch.Tensor:
batch_size = plan_vector.shape[0]
device = plan_vector.device
norm = torch.norm(plan_vector, p=2, dim=-1, keepdim=False)
norm_confidence = torch.sigmoid(norm - 1.0)
if explanation_target is not None:
similarity = F.cosine_similarity(plan_vector, explanation_target, dim=-1)
sim_confidence = torch.sigmoid(similarity * 3.0)
else:
sim_confidence = torch.ones(batch_size, device=device) * 0.5
if internal_coherence is not None:
coh_confidence = torch.sigmoid(internal_coherence)
else:
coh_confidence = torch.ones(batch_size, device=device) * 0.5
if explanation_target is not None:
confidence = 0.1 * norm_confidence + 0.8 * sim_confidence + 0.1 * coh_confidence
else:
confidence = 0.4 * norm_confidence + 0.4 * sim_confidence + 0.2 * coh_confidence
return confidence.clamp(0.0, 1.0)
class NeuroReasonerPlanningHead1(PreTrainedModel, GenerationMixin):
"""
NeuroReasoner-PlanningHead-1: A Planning-Enhanced Model from the NeuroReasoner Family
Built on NeuroReasoner-1-NR-1 with unified synergistic architecture:
- PlanningHead: Latent planning via contrastive alignment
- Memory Attention: Learnable memory bank for enhanced planning
- LabHead: Experiment-derived features
- InvariantForge: Invariant constraints
- SchemaBridge: Schema-matched features
Revolutionary Planning Head Model - Fully HuggingFace Compatible!
This model integrates planning capabilities directly into a causal language model.
It can be loaded directly from HuggingFace Hub using AutoModel.from_pretrained()
without any extra scripts!
All custom modules are wrapped with HuggingFace-compatible interfaces and act as standard layers.
Inherits from both PreTrainedModel and GenerationMixin for full generation support.
"""
config_class = NeuroReasonerPlanningHead1Config
base_model_prefix = "planning_head"
# Use a different attribute name to avoid conflicts with PreTrainedModel logic
_base_model_attr = "base_model"
def __init__(self, config: NeuroReasonerPlanningHead1Config):
super().__init__(config)
self.config = config
# Store which modules are enabled in config for proper loading
self._modules_enabled = {
'lab_head': config.use_lab_head,
'invariant_forge': config.use_invariant_forge,
'schema_bridge': config.use_schema_bridge,
}
try:
# Load base model config first to update vocab_size if needed
from transformers import AutoConfig
base_config = AutoConfig.from_pretrained(config.base_model_name_or_path)
# Update base config vocab_size to match our config (for saved checkpoints)
if base_config.vocab_size != config.vocab_size:
base_config.vocab_size = config.vocab_size
# Load base model with updated config, ignore mismatched sizes for embeddings
# The embeddings will be loaded from our saved checkpoint with correct size
try:
base_model = AutoModelForCausalLM.from_pretrained(
config.base_model_name_or_path,
config=base_config,
cache_dir=None,
ignore_mismatched_sizes=True, # Ignore embedding size mismatch
)
except Exception as e:
# If that fails, try loading without loading weights, then resize
logger.warning(f"Direct load failed: {e}, trying alternative approach")
base_model = AutoModelForCausalLM.from_config(base_config)
# Resize embeddings to correct size
if hasattr(base_model, 'resize_token_embeddings'):
base_model.resize_token_embeddings(config.vocab_size)
except Exception as e:
logger.warning(f"Failed to load base model from {config.base_model_name_or_path}: {e}")
logger.info("Attempting to load from local path or using GPT2 as fallback")
try:
base_config = AutoConfig.from_pretrained("gpt2")
base_config.vocab_size = config.vocab_size
try:
base_model = AutoModelForCausalLM.from_pretrained(
"gpt2",
config=base_config,
cache_dir=None,
ignore_mismatched_sizes=True,
)
except Exception:
base_model = AutoModelForCausalLM.from_config(base_config)
if hasattr(base_model, 'resize_token_embeddings'):
base_model.resize_token_embeddings(config.vocab_size)
config.base_model_name_or_path = "gpt2"
except Exception:
raise ValueError(f"Cannot load base model from {config.base_model_name_or_path} or gpt2")
# Register base_model as submodule so it's saved/loaded properly
# IMPORTANT: Store in a private attribute first, then we'll make it accessible
# PreTrainedModel may have a base_model property, so we use _causal_lm_base internally
self._causal_lm_base = base_model
# Also register as a named module so it's saved properly
# Use a unique name that won't conflict with PreTrainedModel properties
self.add_module('_causal_lm_base', base_model)
# Create a property-like accessor for compatibility
# Verify it has resize_token_embeddings
assert hasattr(base_model, 'resize_token_embeddings'), f"Base model (type: {type(base_model)}) should have resize_token_embeddings"
hidden_size = base_model.config.hidden_size
# PlanningHead - always available (defined in this file)
self.planning_head = PlanningHeadModule(
hidden_size=hidden_size,
plan_dim=config.plan_dim,
num_layers=config.plan_num_layers,
dropout=config.plan_dropout,
layer_norm_eps=config.layer_norm_eps,
)
# Memory attention - always available (defined in this file)
self.use_memory = config.use_memory
if config.use_memory:
self.memory_attention = PlanningMemoryAttentionModule(
plan_dim=config.plan_dim,
memory_size=config.memory_size,
num_heads=config.memory_heads,
dropout=config.memory_dropout,
)
else:
self.memory_attention = None
# Initialize Unified Synergistic Architecture - Breakthrough Features
# Custom modules wrapped with HuggingFace-compatible interfaces - act as standard layers
self.use_lab_head = config.use_lab_head
self.use_invariant_forge = config.use_invariant_forge
self.use_schema_bridge = config.use_schema_bridge
# LabHead: ALWAYS initialized if enabled (no fallbacks!)
if self.use_lab_head:
# LabHead is always available (defined in this file)
self.lab_head = LabHead(hidden_size=hidden_size, plan_dim=config.plan_dim)
logger.info("LabHead initialized (self-contained, HuggingFace-compatible)")
else:
self.lab_head = None
# InvariantForge: ALWAYS initialized if enabled (no fallbacks!)
if self.use_invariant_forge:
# InvariantForge is always available (defined in this file)
self.invariant_forge = InvariantForge(hidden_size=hidden_size, plan_dim=config.plan_dim)
logger.info("InvariantForge initialized (self-contained, HuggingFace-compatible)")
else:
self.invariant_forge = None
# SchemaBridge: ALWAYS initialized if enabled (no fallbacks!)
if self.use_schema_bridge:
# SchemaBridge is always available (defined in this file)
self.schema_bridge = SchemaBridge(hidden_size=hidden_size, plan_dim=config.plan_dim)
logger.info("SchemaBridge initialized (self-contained, HuggingFace-compatible)")
else:
self.schema_bridge = None
self.plan_token_id = None
self.end_plan_token_id = None
@classmethod
def from_pretrained(
cls,
pretrained_model_name_or_path: Union[str, os.PathLike],
*model_args,
**kwargs,
):
"""
Load model from pretrained - fully HuggingFace compatible!
This properly handles loading all layers including LabHead, InvariantForge, SchemaBridge.
If external modules aren't available, it gracefully skips them but still loads the model.
All layers are automatically saved/loaded via state_dict - no extra scripts needed!
PreTrainedModel.from_pretrained() uses strict=False by default, so missing modules
won't cause loading to fail. The model will load with whatever modules are available.
"""
# Use parent's from_pretrained which handles everything automatically
# It will:
# 1. Load config
# 2. Initialize model (calls __init__)
# 3. Load state_dict with strict=False (allows missing keys)
# All registered submodules are automatically saved/loaded
# Check if we're loading from a checkpoint (has .safetensors or .bin file)
# If so, we need to ensure embeddings match the saved size
is_checkpoint = os.path.isdir(pretrained_model_name_or_path) and (
os.path.exists(os.path.join(pretrained_model_name_or_path, "model.safetensors")) or
os.path.exists(os.path.join(pretrained_model_name_or_path, "pytorch_model.bin"))
)
# Temporarily override __init__ to skip embedding resize if loading from checkpoint
# The saved checkpoint already has the correct embeddings
original_init = cls.__init__
if is_checkpoint:
def _init_without_resize(self, config):
# Temporarily disable resize during init
original_init(self, config)
# Mark that we need to resize after loading
self._needs_embedding_resize = True
cls.__init__ = _init_without_resize
try:
model = super().from_pretrained(
pretrained_model_name_or_path,
*model_args,
**kwargs,
)
finally:
# Restore original __init__
cls.__init__ = original_init
# After loading from checkpoint, resize embeddings if config vocab_size differs
# This handles cases where the checkpoint was saved with resized embeddings
if is_checkpoint and hasattr(model, 'base_model'):
base_vocab_size = model.base_model.config.vocab_size
target_vocab_size = model.config.vocab_size
if base_vocab_size != target_vocab_size and hasattr(model.base_model, 'resize_token_embeddings'):
try:
# Resize to match config (saved checkpoint has this size)
model.base_model.resize_token_embeddings(target_vocab_size)
logger.info(f"Resized embeddings after loading from {base_vocab_size} to {target_vocab_size}")
except Exception as e:
logger.warning(f"Could not resize embeddings after loading: {e}")
return model
def set_tokenizer(self, tokenizer):
"""Set tokenizer for plan extraction."""
self.tokenizer = tokenizer
if not hasattr(self.base_model, 'tokenizer'):
self.base_model.tokenizer = tokenizer
if tokenizer is not None:
try:
plan_token = tokenizer.encode('<plan>', add_special_tokens=False)
end_plan_token = tokenizer.encode('</plan>', add_special_tokens=False)
self.plan_token_id = plan_token[0] if plan_token else None
self.end_plan_token_id = end_plan_token[0] if end_plan_token else None
except Exception:
pass
def _extract_plan_mask_from_input(
self,
input_ids: torch.Tensor,
) -> Optional[torch.Tensor]:
"""Extract plan mask from input_ids by finding <plan>...</plan> spans."""
if self.plan_token_id is None or self.end_plan_token_id is None:
return None
batch_size, seq_len = input_ids.shape
device = input_ids.device
plan_mask = torch.zeros(batch_size, seq_len, dtype=torch.bool, device=device)
for i in range(batch_size):
in_plan = False
for j in range(seq_len):
if input_ids[i, j].item() == self.plan_token_id:
in_plan = True
elif input_ids[i, j].item() == self.end_plan_token_id:
plan_mask[i, j] = True
in_plan = False
elif in_plan:
plan_mask[i, j] = True
return plan_mask if plan_mask.sum() > 0 else None
def forward(
self,
input_ids: Optional[torch.LongTensor] = None,
attention_mask: Optional[torch.Tensor] = None,
labels: Optional[torch.LongTensor] = None,
plan_mask: Optional[torch.Tensor] = None,
explanation_target: Optional[torch.Tensor] = None,
use_iterative_training: bool = False,
iterative_max_iterations: Optional[int] = None,
iterative_refine_weight: Optional[float] = None,
iterative_weight: Optional[float] = None,
current_step: Optional[int] = None,
return_dict: bool = True,
output_attentions: Optional[bool] = None,
output_hidden_states: Optional[bool] = None,
**kwargs,
) -> Union[Tuple, Dict[str, torch.Tensor]]:
"""
Unified Synergistic Forward Pass - All modules work together as ONE pipeline.
Fully HuggingFace compatible - works with or without custom modules.
Uses standard HuggingFace layers and naming conventions throughout.
"""
if attention_mask is None and input_ids is not None:
attention_mask = (input_ids != self.config.pad_token_id).long()
if iterative_max_iterations is None:
iterative_max_iterations = self.config.iterative_max_iterations
if iterative_refine_weight is None:
iterative_refine_weight = self.config.iterative_refine_weight
if iterative_weight is None:
iterative_weight = self.config.iterative_loss_weight
# Step 1: Base model forward
base_outputs = self.base_model(
input_ids=input_ids,
attention_mask=attention_mask,
labels=labels,
return_dict=True,
output_hidden_states=True,
output_attentions=output_attentions,
**kwargs,
)
logits = base_outputs.logits
lm_loss = base_outputs.loss if labels is not None else None
hidden_states = base_outputs.hidden_states[-1] if base_outputs.hidden_states else None
if hidden_states is None:
raise ValueError("Base model must support hidden_states output for integrated modules")
# Initialize all module outputs and losses
plan_loss = None
plan_vector = None
plan_similarity = None
target_vector = None
memory_info = None
# LabHead, InvariantForge, SchemaBridge outputs
lab_head_output = None
inv_forge_output = None
schema_bridge_output = None
# Step 2: PlanningHead -> plan_vector
if plan_mask is None:
plan_mask = self._extract_plan_mask_from_input(input_ids) if input_ids is not None else None
if plan_mask is not None and hidden_states is not None:
plan_vector = self.planning_head(hidden_states, plan_mask)
# Apply memory attention if enabled
if self.use_memory and self.memory_attention is not None:
memory_result = self.memory_attention(plan_vector, update_memory=True)
plan_vector = memory_result['enhanced_plan']
memory_info = memory_result
# Compute plan loss if explanation_target provided
if explanation_target is not None:
if explanation_target.dim() == 1:
explanation_target = explanation_target.unsqueeze(0)
if explanation_target.shape[1] == plan_vector.shape[1]:
target_vector = explanation_target
# Automatic iterative refinement in inference mode
if not use_iterative_training and not self.training:
refined_plan = plan_vector
confidence = self.planning_head.compute_confidence(
refined_plan, explanation_target=target_vector
)
confidence_gap = 1.0 - confidence.mean()
for _ in range(min(3, iterative_max_iterations)):
if confidence_gap < 0.05:
break
direction = target_vector - refined_plan
direction = F.normalize(direction, p=2, dim=-1)
step_size = confidence_gap * iterative_refine_weight
refined_plan = refined_plan + step_size * direction
refined_plan = F.normalize(refined_plan, p=2, dim=-1)
confidence = self.planning_head.compute_confidence(
refined_plan, explanation_target=target_vector
)
confidence_gap = 1.0 - confidence.mean()
plan_vector = refined_plan
plan_similarity = self.planning_head.compute_similarity(
plan_vector, target_vector
)
plan_loss = -plan_similarity.mean()
# Step 3-5: Unified Synergistic Pipeline
# All modules progressively refine the SAME hidden_states and logits
# Each module builds on the previous one's work for true synergy
# Initialize unified representation
unified_hidden_states = hidden_states.clone()
unified_logits = logits.clone()
# Step 3-5: Unified Synergistic Pipeline
# All modules MUST be used - no fallbacks! They progressively refine the SAME unified representation
# Step 3: SchemaBridge -> Add schema-matched features (FiLM + structural bias)
# This is the FIRST module in the synergistic pipeline - adds schema-matched features
if self.use_schema_bridge and self.schema_bridge is not None:
schema_bridge_output = self.schema_bridge(
hidden_states=unified_hidden_states,
attention_mask=attention_mask,
lm_logits=unified_logits,
plan_vector=plan_vector,
labels=labels,
lab_head=None, # Not used in pipeline - called separately
inv_forge=None, # Not used in pipeline - called separately
audit=False,
current_step=current_step,
return_dict=True,
)
# Progressively refine: blend schema features into unified representation
if schema_bridge_output.fused_hidden_states is not None:
unified_hidden_states = (
(1.0 - self.config.schema_weight) * unified_hidden_states +
self.config.schema_weight * schema_bridge_output.fused_hidden_states
)
if schema_bridge_output.fused_logits is not None:
# Blend schema bias into unified logits
unified_logits = unified_logits + 0.1 * schema_bridge_output.fused_logits
# Step 4: LabHead -> Add experiment-derived features (FiLM from outcomes)
# This builds on SchemaBridge's output - adds experiment-derived features
if self.use_lab_head and self.lab_head is not None:
lab_head_output = self.lab_head(
hidden_states=unified_hidden_states, # Uses SchemaBridge-refined hidden states
attention_mask=attention_mask,
lm_logits=unified_logits, # Uses SchemaBridge-refined logits
labels=labels,
plan_vector=plan_vector,
plan_spans=None, # Not used in this pipeline
current_step=current_step,
return_dict=True,
)
# Progressively refine: blend experiment features into unified representation
if lab_head_output.fused_hidden_states is not None:
unified_hidden_states = (
(1.0 - self.config.experiment_weight) * unified_hidden_states +
self.config.experiment_weight * lab_head_output.fused_hidden_states
)
# LabHead doesn't modify logits directly (only hidden states via FiLM)
# Logits remain from SchemaBridge refinement
# Step 5: InvariantForge -> Apply invariant constraints (final refinement)
# This is the FINAL module - applies constraints to the unified representation
if self.use_invariant_forge and self.invariant_forge is not None:
inv_forge_output = self.invariant_forge(
hidden_states=unified_hidden_states, # Uses LabHead-refined hidden states
attention_mask=attention_mask,
lm_logits=unified_logits, # Uses SchemaBridge-refined logits
labels=labels,
plan_vector=plan_vector,
plan_spans=None, # Not used in this pipeline
current_step=current_step,
return_dict=True,
)
# Progressively refine: apply invariant constraints to unified representation
if inv_forge_output.fused_hidden_states is not None:
unified_hidden_states = (
(1.0 - self.config.invariant_weight) * unified_hidden_states +
self.config.invariant_weight * inv_forge_output.fused_hidden_states
)
# InvariantForge modifies logits with constraint projection
if inv_forge_output.fused_logits is not None:
unified_logits = inv_forge_output.fused_logits # Final refined logits
# Final unified outputs (synergistic combination of all modules)
fused_hidden_states = unified_hidden_states
fused_logits = unified_logits
# Step 6: Compute all losses
# Iterative training loss
iterative_loss = None
if use_iterative_training and plan_vector is not None and target_vector is not None:
initial_confidence = self.planning_head.compute_confidence(
plan_vector,
explanation_target=target_vector,
)
confidence_gap = 1.0 - initial_confidence.mean()
if confidence_gap > 0.01:
direction = target_vector - plan_vector
direction = F.normalize(direction, p=2, dim=-1)
step_size = confidence_gap * iterative_refine_weight
refined_plan = plan_vector + step_size * direction
refined_plan = F.normalize(refined_plan, p=2, dim=-1)
iterative_loss = F.mse_loss(plan_vector, refined_plan.detach())
# Memory diversity loss
memory_loss = None
if self.use_memory and self.memory_attention is not None and memory_info is not None:
memory_bank = self.memory_attention.memory_bank
memory_cosine_sim = F.cosine_similarity(
memory_bank.unsqueeze(1),
memory_bank.unsqueeze(0),
dim=-1
)
upper_triangle = torch.triu(memory_cosine_sim, diagonal=1)
memory_loss = upper_triangle.mean()
# Extract all auxiliary losses from modules (properly computed)
# LabHead auxiliary losses
lab_head_loss = None
if self.use_lab_head and self.lab_head is not None and lab_head_output is not None:
if hasattr(lab_head_output, 'aux_losses') and lab_head_output.aux_losses:
aux_losses = lab_head_output.aux_losses
if 'total' in aux_losses and isinstance(aux_losses['total'], torch.Tensor):
lab_head_loss = aux_losses['total']
elif isinstance(aux_losses, dict) and len(aux_losses) > 0:
# Sum all auxiliary losses if no 'total' key
lab_head_loss = sum([v for v in aux_losses.values() if isinstance(v, torch.Tensor)])
# InvariantForge auxiliary losses
inv_forge_loss = None
if self.use_invariant_forge and self.invariant_forge is not None and inv_forge_output is not None:
if hasattr(inv_forge_output, 'aux_losses') and inv_forge_output.aux_losses:
aux_losses = inv_forge_output.aux_losses
if 'total' in aux_losses and isinstance(aux_losses['total'], torch.Tensor):
inv_forge_loss = aux_losses['total']
elif isinstance(aux_losses, dict) and len(aux_losses) > 0:
# Sum all auxiliary losses if no 'total' key
inv_forge_loss = sum([v for v in aux_losses.values() if isinstance(v, torch.Tensor)])
# SchemaBridge auxiliary losses
schema_bridge_loss = None
if self.use_schema_bridge and self.schema_bridge is not None and schema_bridge_output is not None:
if hasattr(schema_bridge_output, 'aux_losses') and schema_bridge_output.aux_losses:
aux_losses = schema_bridge_output.aux_losses
if 'total' in aux_losses and isinstance(aux_losses['total'], torch.Tensor):
schema_bridge_loss = aux_losses['total']
elif isinstance(aux_losses, dict) and len(aux_losses) > 0:
# Sum all auxiliary losses if no 'total' key
schema_bridge_loss = sum([v for v in aux_losses.values() if isinstance(v, torch.Tensor)])
# Recompute LM loss with final fused logits if we have labels
final_lm_loss = lm_loss
if labels is not None and fused_logits is not None and not torch.equal(logits, fused_logits):
shift_logits = fused_logits[..., :-1, :].contiguous()
shift_labels = labels[..., 1:].contiguous()
final_lm_loss = F.cross_entropy(
shift_logits.view(-1, shift_logits.size(-1)),
shift_labels.view(-1),
ignore_index=-100,
)
# Combine all losses
total_loss = torch.tensor(0.0, device=input_ids.device if input_ids is not None else next(self.parameters()).device)
if final_lm_loss is not None:
total_loss = total_loss + final_lm_loss
if plan_loss is not None:
total_loss = total_loss + self.config.plan_loss_weight * plan_loss
if iterative_loss is not None:
total_loss = total_loss + iterative_weight * iterative_loss
if memory_loss is not None:
total_loss = total_loss + self.config.memory_loss_weight * memory_loss
if lab_head_loss is not None:
total_loss = total_loss + self.config.lab_head_loss_weight * lab_head_loss
if inv_forge_loss is not None:
total_loss = total_loss + self.config.inv_forge_loss_weight * inv_forge_loss
if schema_bridge_loss is not None:
total_loss = total_loss + self.config.schema_bridge_loss_weight * schema_bridge_loss
if not return_dict:
return (
fused_logits,
final_lm_loss,
plan_loss,
iterative_loss,
memory_loss,
lab_head_loss,
inv_forge_loss,
schema_bridge_loss,
plan_vector,
)
# Build unified result dictionary (single cohesive output)
result = {
'logits': fused_logits,
'hidden_states': fused_hidden_states,
'plan_vector': plan_vector,
'plan_similarity': plan_similarity,
'loss': total_loss if total_loss.item() != 0 else final_lm_loss,
'lm_loss': final_lm_loss,
'plan_loss': plan_loss,
'iterative_loss': iterative_loss,
'memory_loss': memory_loss,
'lab_head_loss': lab_head_loss,
'inv_forge_loss': inv_forge_loss,
'schema_bridge_loss': schema_bridge_loss,
'module_contributions': {
'schema_bridge': {
'active': schema_bridge_output is not None,
'confidence': getattr(schema_bridge_output, 'confidence', torch.tensor(0.0)).mean().item() if schema_bridge_output is not None else 0.0,
'best_match': getattr(schema_bridge_output, 'best_match_idx', torch.tensor([])).tolist() if schema_bridge_output is not None else None,
} if schema_bridge_output is not None else None,
'lab_head': {
'active': lab_head_output is not None,
'chosen_experiment': getattr(lab_head_output, 'chosen_index', torch.tensor([])).tolist() if lab_head_output is not None else None,
'info_gain': getattr(lab_head_output, 'info_gain_scores', torch.tensor(0.0)).mean().item() if lab_head_output is not None else 0.0,
} if lab_head_output is not None else None,
'invariant_forge': {
'active': inv_forge_output is not None,
'confidence': getattr(inv_forge_output, 'confidence_scores', torch.tensor(0.0)).mean().item() if inv_forge_output is not None else 0.0,
'satisfaction': getattr(inv_forge_output, 'satisfaction_scores', torch.tensor(0.0)).mean().item() if inv_forge_output is not None else 0.0,
} if inv_forge_output is not None else None,
},
}
if memory_info is not None:
result['memory_similarity'] = memory_info.get('memory_similarity')
result['memory_weights'] = memory_info.get('memory_weights')
if base_outputs.attentions is not None:
result['attentions'] = base_outputs.attentions
return result
def prepare_inputs_for_generation(self, input_ids, **kwargs):
"""Prepare inputs for generation - delegate to base model."""
return self.base_model.prepare_inputs_for_generation(input_ids, **kwargs)
def get_input_embeddings(self):
"""Get input embeddings from base model."""
return self.base_model.get_input_embeddings()
def set_input_embeddings(self, value):
"""Set input embeddings in base model."""
self.base_model.set_input_embeddings(value)
def get_output_embeddings(self):
"""Get output embeddings from base model."""
return self.base_model.get_output_embeddings()
def set_output_embeddings(self, new_embeddings):
"""Set output embeddings in base model."""
self.base_model.set_output_embeddings(new_embeddings)
@property
def base_model(self):
"""Access the underlying causal LM base model."""
return self._causal_lm_base
def resize_token_embeddings(self, new_num_tokens: Optional[int] = None, pad_to_multiple_of: Optional[int] = None):
"""
Resize token embeddings in base model.
This method properly delegates to the base model's resize_token_embeddings.
All models loaded with AutoModelForCausalLM have this method.
"""
# Access the base model via the property
base_model = self.base_model
# base_model should be a PreTrainedModel (like GPT2LMHeadModel), which has resize_token_embeddings
if base_model is None:
raise AttributeError(f"Base model is None")
# Verify it's the correct type (should be GPT2LMHeadModel or similar, not PlanningHeadModule)
if 'PlanningHeadModule' in str(type(base_model)):
raise AttributeError(f"Base model is incorrectly set to PlanningHeadModule - this should not happen")
# Check if base_model has the method directly
if hasattr(base_model, 'resize_token_embeddings'):
return base_model.resize_token_embeddings(new_num_tokens, pad_to_multiple_of=pad_to_multiple_of)
else:
# This shouldn't happen for models loaded with AutoModelForCausalLM
raise AttributeError(f"Base model (type: {type(base_model)}) does not have resize_token_embeddings method")
def generate(
self,
input_ids: Optional[torch.LongTensor] = None,
attention_mask: Optional[torch.Tensor] = None,
plan_mask: Optional[torch.Tensor] = None,
explanation_target: Optional[torch.Tensor] = None,
use_iterative_planning: bool = True,
**generation_kwargs,
):
"""
Generate text with planning guidance.
Args:
input_ids: Input token ids
attention_mask: Attention mask
plan_mask: Plan mask for extracting plan vector
explanation_target: Target embedding for iterative planning
use_iterative_planning: Use iterative planning if plan_vector not provided
**generation_kwargs: Additional generation kwargs for base model
Returns:
Generated token ids
"""
if plan_mask is None and input_ids is not None:
plan_mask = self._extract_plan_mask_from_input(input_ids)
if plan_mask is not None and explanation_target is not None and use_iterative_planning:
with torch.no_grad():
outputs = self.forward(
input_ids=input_ids,
attention_mask=attention_mask,
plan_mask=plan_mask,
explanation_target=explanation_target,
use_iterative_training=False,
)
plan_vector = outputs.get('plan_vector')
if plan_vector is not None:
self._current_plan = plan_vector
return self.base_model.generate(
input_ids=input_ids,
attention_mask=attention_mask,
**generation_kwargs,
)
def _init_weights(self, module):
"""
Initialize weights for new modules.
This ensures all modules are properly initialized.
"""
if isinstance(module, (nn.Linear, nn.Embedding)):
module.weight.data.normal_(mean=0.0, std=self.config.initializer_range)
if isinstance(module, nn.Linear) and module.bias is not None:
module.bias.data.zero_()
elif isinstance(module, nn.LayerNorm):
module.bias.data.zero_()
module.weight.data.fill_(1.0)
AutoConfig.register("neuroreasoner_planninghead", NeuroReasonerPlanningHead1Config)
AutoModel.register(NeuroReasonerPlanningHead1Config, NeuroReasonerPlanningHead1)
if MODEL_FOR_CAUSAL_LM_MAPPING is not None:
MODEL_FOR_CAUSAL_LM_MAPPING.register(NeuroReasonerPlanningHead1Config, NeuroReasonerPlanningHead1)
"""
Revolutionary Planning Head AutoModel - Fully HuggingFace Compatible
This model integrates the FULL breakthrough unified synergistic architecture:
- PlanningHead with memory attention
- LabHead for experiment-derived features (HuggingFace-compatible wrapper)
- InvariantForge for invariant constraints (HuggingFace-compatible wrapper)
- SchemaBridge for schema-matched features (HuggingFace-compatible wrapper)
All custom modules are wrapped with HuggingFace-compatible interfaces and act as standard layers.
All modules use HuggingFace-compatible layers and naming conventions.
This model can be loaded directly from HuggingFace Hub using:
from transformers import AutoModel, AutoConfig
model = AutoModel.from_pretrained("your-org/planning-head-model")
config = AutoConfig.from_pretrained("your-org/planning-head-model")
No extra scripts needed - pure HuggingFace compatibility!
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Dict, Tuple, Optional, List, Union
import math
import json
import os
from transformers import (
PreTrainedModel,
PretrainedConfig,
AutoConfig,
AutoModel,
AutoModelForCausalLM,
MODEL_MAPPING,
MODEL_FOR_CAUSAL_LM_MAPPING,
GenerationMixin,
)
from transformers.modeling_outputs import CausalLMOutput, ModelOutput
from transformers.utils import logging
logger = logging.get_logger(__name__)
# Self-contained HuggingFace-compatible modules - NO EXTERNAL IMPORTS NEEDED!
# All modules are defined directly in this file with standard HuggingFace naming conventions
class LabHeadModule(nn.Module):
"""
LabHead - Self-experimental reasoning with FiLM modulation.
Mimics the real LabHead implementation with:
- Experiment proposal via Gumbel-Softmax codebook
- World kernel for outcome prediction
- Information gain scoring via InfoNCE
- FiLM fusion application
Fully HuggingFace-compatible with standard layer naming:
- q_proj, k_proj, v_proj, o_proj (attention projections)
- intermediate, output (feed-forward layers)
- layer_norm (normalization)
"""
def __init__(self, hidden_size: int, plan_dim: int = 128, config=None):
super().__init__()
self.hidden_size = hidden_size
self.plan_dim = plan_dim
self.num_experiments = 4 # K experiments
self.exp_code_dim = 64 # C dimension
self.outcome_dim = 128 # O dimension
self.num_exp_codes = 16 # Codebook size
# Context pooler (mimics real LabHead)
self.context_pooler = nn.Sequential(
nn.Linear(hidden_size, hidden_size),
nn.GELU(),
nn.LayerNorm(hidden_size),
)
# Experiment Proposal Head (EPH) - Gumbel-Softmax codebook
self.eph = nn.Sequential(
nn.Linear(hidden_size, hidden_size),
nn.GELU(),
nn.Dropout(0.1),
nn.Linear(hidden_size, self.num_experiments * self.num_exp_codes),
)
self.codebook = nn.Parameter(torch.randn(self.num_exp_codes, self.exp_code_dim) * 0.02)
# World Kernel (WK) - Outcome prediction
self.world_kernel = nn.Sequential(
nn.Linear(hidden_size + self.exp_code_dim, hidden_size * 2),
nn.GELU(),
nn.Dropout(0.1),
nn.Linear(hidden_size * 2, self.outcome_dim),
)
# Information Gain Estimator (IGE) - InfoNCE
self.target_proj = nn.Linear(hidden_size, self.outcome_dim, bias=False)
self.info_nce_temp = 0.07
# Refuter Head (RH) - FiLM fusion