README.md

metadata

title: Edge LLM Platform
emoji: 🚀
colorFrom: blue
colorTo: purple
sdk: docker
sdk_version: 4.36.0
app_file: app.py
pinned: false
license: mit
short_description: Local LLM platform with modern web interface

🚀 Edge LLM Platform

A lightweight, local LLM inference platform with a modern web interface.

Note: All development now happens directly in this repository (EdgeLLM_HF). This is both the development environment and the production Hugging Face Space.

✨ Features

🤖 Hybrid Model Support

• Local Models: Run Qwen models locally for privacy
• API Models: Access powerful cloud models via AiHubMix API
• Seamless Switching: Switch between local and API models effortlessly
• Thinking Models: Support for models with visible reasoning process

🌐 Available Models

Local Models (Privacy-First)

• Qwen/Qwen3-4B-Thinking-2507 - Local model with thinking process (~8GB)
• Qwen/Qwen3-4B-Instruct-2507 - Local direct instruction model (~8GB)

API Models (Cloud-Powered)

• Qwen/Qwen3-30B-A3B - Advanced Qwen3 with dynamic thinking modes
• qwen2.5-vl-72b-instruct - Multimodal model with vision capabilities
• Qwen/QVQ-72B-Preview - Visual reasoning with thinking process

🎨 Modern UI/UX

• Responsive Design: Works on desktop and mobile
• Chat Interface: Beautiful conversation bubbles with session management
• Model Management: Easy switching between local and API models
• Parameter Controls: Temperature, max tokens, and system prompts
• Session History: Persistent conversations with localStorage

📁 Project Structure

EdgeLLM/
├── frontend/        # 🎨 React frontend with ShadCN UI
├── backend/         # 🔧 FastAPI backend
├── static/          # 📱 Built frontend assets
├── app.py           # 🌐 Production entry point
├── requirements.txt # 🐍 Python dependencies
└── README.md        # 📖 Documentation

🎯 Quick Start

1. Clone the repository

   git clone https://huggingface.co/spaces/wu981526092/EdgeLLM
   cd EdgeLLM
   

2. Set up environment variables

   # Create .env file with your API credentials
   echo 'api_key="your-aihubmix-api-key"' > .env
   echo 'base_url="https://aihubmix.com/v1"' >> .env
   

3. Install dependencies

   pip install -r requirements.txt
   cd frontend && npm install && cd ..
   

4. Run locally

   python app.py
   

5. Deploy changes

   # Build frontend if needed
   cd frontend && npm run build && cd ..
   
   # Push to Hugging Face
   git add .
   git commit -m "Update: your changes"
   git push
   

🌐 Live Demo

Visit the live demo at: https://huggingface.co/spaces/wu981526092/EdgeLLM

🔧 Configuration

Environment Variables

For local development, create a .env file:

api_key="your-aihubmix-api-key"
base_url="https://aihubmix.com/v1"

For production (Hugging Face Spaces), set these as secrets:

• api_key: Your AiHubMix API key
• base_url: API endpoint (https://aihubmix.com/v1)

API Integration

This platform integrates with AiHubMix API for cloud-based model access. Features include:

• OpenAI-compatible API interface
• Support for Qwen 3 series models
• Multimodal capabilities (text + vision)
• Streaming and non-streaming responses

🛠️ Development Workflow

1. Frontend development: Work in frontend/
2. Backend development: Work in backend/
3. Build frontend: cd frontend && npm run build
4. Deploy: Standard git workflow

   git add .
   git commit -m "Your changes"
   git push
   

🏗️ Architecture

Backend (FastAPI)

• Models Service: Handles both local model loading and API client management
• Chat Service: Routes requests to appropriate generation method (local/API)
• API Routes: RESTful endpoints for model management and text generation
• Configuration: Environment-based settings for API credentials

Frontend (React + TypeScript)

• Modern UI: Built with ShadCN components and Tailwind CSS
• Chat Interface: Real-time conversation with message bubbles
• Model Management: Easy switching between available models
• Session Management: Persistent chat history and settings

📄 License

MIT License - see LICENSE for details.