README.md

---
title: Edge LLM Platform
emoji: 🚀
colorFrom: blue
colorTo: purple
sdk: docker
sdk_version: "4.36.0"
app_file: app.py
pinned: false
license: mit
short_description: Local LLM platform with modern web interface
---

# 🚀 Edge LLM Platform

A lightweight, local LLM inference platform with a modern web interface.

> **Note**: All development now happens directly in this repository (EdgeLLM_HF). This is both the development environment and the production Hugging Face Space.

## ✨ Features

### 🤖 **Hybrid Model Support**
- **Local Models**: Run Qwen models locally for privacy
- **API Models**: Access powerful cloud models via [AiHubMix API](https://docs.aihubmix.com/en/api/Qwen)
- **Seamless Switching**: Switch between local and API models effortlessly
- **Thinking Models**: Support for models with visible reasoning process

### 🌐 **Available Models**

#### Local Models (Privacy-First)
- `Qwen/Qwen3-4B-Thinking-2507` - Local model with thinking process (~8GB)
- `Qwen/Qwen3-4B-Instruct-2507` - Local direct instruction model (~8GB)

#### API Models (Cloud-Powered)
- `Qwen/Qwen3-30B-A3B` - Advanced Qwen3 with dynamic thinking modes
- `qwen2.5-vl-72b-instruct` - Multimodal model with vision capabilities
- `Qwen/QVQ-72B-Preview` - Visual reasoning with thinking process

### 🎨 **Modern UI/UX**
- **Responsive Design**: Works on desktop and mobile
- **Chat Interface**: Beautiful conversation bubbles with session management
- **Model Management**: Easy switching between local and API models
- **Parameter Controls**: Temperature, max tokens, and system prompts
- **Session History**: Persistent conversations with localStorage

## 📁 Project Structure

```
EdgeLLM/
├── frontend/        # 🎨 React frontend with ShadCN UI
├── backend/         # 🔧 FastAPI backend
├── static/          # 📱 Built frontend assets
├── app.py           # 🌐 Production entry point
├── requirements.txt # 🐍 Python dependencies
└── README.md        # 📖 Documentation
```

## 🎯 Quick Start

1. **Clone the repository**
   ```bash
   git clone https://huggingface.co/spaces/wu981526092/EdgeLLM
   cd EdgeLLM
   ```

2. **Set up environment variables**
   ```bash
   # Create .env file with your API credentials
   echo 'api_key="your-aihubmix-api-key"' > .env
   echo 'base_url="https://aihubmix.com/v1"' >> .env
   ```

3. **Install dependencies**
   ```bash
   pip install -r requirements.txt
   cd frontend && npm install && cd ..
   ```

4. **Run locally**
   ```bash
   python app.py
   ```

5. **Deploy changes**
   ```bash
   # Build frontend if needed
   cd frontend && npm run build && cd ..
   
   # Push to Hugging Face
   git add .
   git commit -m "Update: your changes"
   git push
   ```

## 🌐 Live Demo

Visit the live demo at: [https://huggingface.co/spaces/wu981526092/EdgeLLM](https://huggingface.co/spaces/wu981526092/EdgeLLM)

## 🔧 Configuration

### Environment Variables

For local development, create a `.env` file:
```bash
api_key="your-aihubmix-api-key"
base_url="https://aihubmix.com/v1"
```

For production (Hugging Face Spaces), set these as secrets:
- `api_key`: Your AiHubMix API key
- `base_url`: API endpoint (https://aihubmix.com/v1)

### API Integration

This platform integrates with [AiHubMix API](https://docs.aihubmix.com/en/api/Qwen) for cloud-based model access. Features include:

- OpenAI-compatible API interface
- Support for Qwen 3 series models
- Multimodal capabilities (text + vision)
- Streaming and non-streaming responses

## 🛠️ Development Workflow

1. **Frontend development**: Work in `frontend/`
2. **Backend development**: Work in `backend/`
3. **Build frontend**: `cd frontend && npm run build`
4. **Deploy**: Standard git workflow
   ```bash
   git add .
   git commit -m "Your changes"
   git push
   ```

## 🏗️ Architecture

### Backend (FastAPI)
- **Models Service**: Handles both local model loading and API client management
- **Chat Service**: Routes requests to appropriate generation method (local/API)
- **API Routes**: RESTful endpoints for model management and text generation
- **Configuration**: Environment-based settings for API credentials

### Frontend (React + TypeScript)
- **Modern UI**: Built with ShadCN components and Tailwind CSS
- **Chat Interface**: Real-time conversation with message bubbles
- **Model Management**: Easy switching between available models
- **Session Management**: Persistent chat history and settings

## 📄 License

MIT License - see `LICENSE` for details.