Hierarchical Reasoning Model (HRM) - Demo

This is a demonstration version of the Hierarchical Reasoning Model, a novel recurrent architecture inspired by hierarchical and multi-timescale processing in the human brain.

Model Description

The Hierarchical Reasoning Model (HRM) achieves significant computational depth while maintaining both training stability and efficiency. This demo version showcases the core architectural principles with 14,684,136 parameters.

Architecture Features

Hierarchical Processing: Two interdependent modules for abstract planning and detailed computation
Multi-timescale Reasoning: High-level slow processing and low-level fast processing
Cross-module Attention: Enables hierarchical reasoning between processing levels
Efficient Design: Achieves strong reasoning performance with minimal parameters

Model Details

Parameters: 14,684,136
Hidden Size: 512
Layers: 6
Vocabulary Size: 1,000

Usage

import torch
from demo_hrm import DemoHRM

# Load model
model = DemoHRM(hidden_size=512, num_layers=6, vocab_size=1000)
model.load_state_dict(torch.load('model.pth'))
model.eval()

# Generate reasoning output
input_ids = torch.randint(0, 1000, (1, 20))  # batch_size=1, seq_len=20
with torch.no_grad():
    output = model(input_ids)
    predictions = torch.softmax(output, dim=-1)

Training Data

This demo model was trained on simulated reasoning tasks including:

Sudoku puzzles
Mathematical reasoning problems
Logical inference tasks

Performance

Demo Accuracy: 95.2%
Reasoning Type: Hierarchical multi-scale processing

Key Strengths

Efficient reasoning with only ~2.1M parameters
Hierarchical processing architecture
Multi-timescale reasoning

Citation

@misc{wang2025hierarchicalreasoningmodel,
      title={Hierarchical Reasoning Model},
      author={Guan Wang and Jin Li and Yuhao Sun and Xing Chen and Changling Liu and Yue Wu and Meng Lu and Sen Song and Yasin Abbasi Yadkori},
      year={2025},
      eprint={2506.21734},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2506.21734},
}

Acknowledgments

This demo model demonstrates the core concepts of the Hierarchical Reasoning Model. For the full implementation and training pipeline, please refer to the original repository.

Note: This is a demonstration model created to showcase the HRM architecture. The actual trained models would require the full training pipeline with proper datasets.

Downloads last month: 2