NEBULA-HRM-DEMO: Hybrid Photonic + Hierarchical Reasoning Model

NEBULA-HRM is a compact research model (~30.13M params) that explores a hybrid architecture combining a hierarchical reasoning module (HRM) with additional structured processing blocks. This repository contains training scripts, checkpoints, and an example inference pipeline.

Important: internal benchmarks (ARC-AGI/Sudoku/Mazes) are small synthetic subsets intended for quick smoke tests, not official leaderboards. GLUE SST-2 validation accuracy is reported from a short training run in a controlled environment.

Highlights

• Parameters: 30.13M
• Inference speed (local): ~163 samples/sec (batch-dependent)
• Memory footprint (peak, local): ~0.32 GB
• Framework: PyTorch
• Checkpoints: nebula_hrm_final.pt, nebula_hrm_complete.pth, pytorch_model.bin

Quick Start

Option A: Download files with huggingface_hub and load the PyTorch checkpoint:

import torch
from huggingface_hub import hf_hub_download
import importlib.util
import pathlib

REPO_ID = "Agnuxo/NEBULA-HRM"

# Download model code and weights
code_path = hf_hub_download(REPO_ID, filename="NEBULA_HRM_Complete_Fixed.py")
weights_path = hf_hub_download(REPO_ID, filename="nebula_hrm_complete.pth")

# Dynamic import of model definition
spec = importlib.util.spec_from_file_location("nebula_hrm", code_path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)

# Create config and model (adjust if you maintain a separate config)
config = mod.NebulaHRMConfig()
model = mod.NebulaHRMModel(config)
model.load_state_dict(torch.load(weights_path, map_location="cpu"))
model.eval()

# Dummy inference example (classification)
import torch
input_ids = torch.randint(0, config.vocab_size, (1, config.max_sequence_length))
with torch.no_grad():
    outputs = model(input_ids=input_ids, task="classification")
print({k: v.shape if hasattr(v, 'shape') else type(v) for k, v in outputs.items()})

Option B: Use the provided inference.py script:

python inference.py --repo Agnuxo/NEBULA-HRM --task classification --device cpu

Files

• NEBULA_HRM_Complete_Fixed.py: full training/inference implementation in PyTorch
• nebula_hrm_complete.pth: PyTorch checkpoint (state_dict)
• nebula_hrm_final.pt: additional serialized artifact
• pytorch_model.bin: standard binary for Hub compatibility
• config.json, tokenizer_config.json, special_tokens_map.json: auxiliary config files
• inference.py: ready-to-run script that pulls artifacts from the Hub
• model_card.md: extended model card

Training Notes

• Environment: Windows 11, Python 3.10, CUDA 11.8, RTX 3090
• Key env vars: TOKENIZERS_PARALLELISM=false, WANDB_DISABLED=true, OMP_NUM_THREADS=1, PYTHONUTF8=1
• Debugging aids: CUDA_LAUNCH_BLOCKING=1, TORCH_SHOW_CPP_STACKTRACES=1, PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:256
• Data: GLUE SST-2 (validation accuracy ~0.51 from a brief run). Internal synthetic subsets used for quick sanity checks.

Limitations and Intended Use

• This is a research prototype. Internal benchmarks are not substitutes for official leaderboards.
• Not optimized for production latency; use as a reference for architecture and training loop design.

Citation

@misc{nebula-hrm2025,
  title={NEBULA-HRM: Hybrid Photonic + Hierarchical Reasoning Model},
  author={Francisco Angulo de Lafuente},
  year={2025},
  howpublished={Hugging Face},
  url={https://huggingface.co/Agnuxo/NEBULA-HRM}
}

Author

Francisco Angulo de Lafuente (Agnuxo)