Heygem - Open Source Alternative to Heygen 【切换中文】

Version Update Notice

The Lite version is officially released, you can download it here. If you want to experience the standard version (original version), you can click here to download

1. The Lite version can reduce two services heygem-tts / heygem-asr, reducing the installation size from 70GB to 13.5GB
2. The Lite version customizes avatars and video generation faster
3. The Lite version does not have text-to-video functionality, it can only generate videos by uploading audio

Important Notice to Developer Partners

Dear Heygem Open Source Community Members:

We sincerely thank you for your enthusiastic attention and active participation in the Heygem digital human open source project! We have noticed that some developers face challenges during local deployment. To better meet the needs of different scenarios, we are now announcing two parallel service solutions:

Project	HeyGem Open Source Local Deployment	Digital Human/Clone Voice API Service
Usage	Open Source Local Deployment	Rapid Clone API Service
Recommended	Technical Users	Business Users
Technical Threshold	Developers with deep learning framework experience/pursuing deep customization/wishing to participate in community co-construction	Quick business integration/focus on upper-level application development/need enterprise-level SLA assurance for commercial scenarios
Hardware Requirements	Need to purchase GPU server	No need to purchase GPU server
Customization	Can modify and extend the code according to your needs, fully controlling the software's functions and behavior	Cannot directly modify the source code, can only extend functions through API-provided interfaces, less flexible than open source projects
Technical Support	Community Support	Dynamic expansion support + professional technical response team
Maintenance Cost	High maintenance cost	Simple maintenance
Lip Sync Effect	Usable effect	Stunning and higher definition effect
Commercial Authorization	Supports global free commercial use (enterprises with more than 100,000 users or annual revenue exceeding 10 million USD need to sign a commercial license agreement)	Commercial use allowed
Iteration Speed	Slow updates, bug fixes depend on the community	Latest models/algorithms are prioritized, fast problem resolution

We always adhere to the open source spirit, and the launch of the API service aims to provide a more complete solution matrix for developers with different needs. No matter which method you choose, you can always obtain technical support documents through [email protected]. We look forward to working with you to promote the inclusive development of digital human technology!

Silicon-based Intelligent Developer Team

From scratch, hand-in-hand to teach you how to create your own HeyGem open source AI digital human!

Rapid Clone API | API Documentation Center

Real-time Interaction SDK | SDK Documentation Center

Local Real-time Interaction (realtime) duix.ai Open Source Address | Android Version | IOS Version

Open Source Co-Creation · Shared Glory

Since we open-sourced Heygem, global geeks have illuminated the digital avatar matrix in the code universe, with each commit reconstructing the future! But joy is better shared than enjoyed alone—now we invite all experts to join the "Open Source Co-Creation Plan," empowering everyone with AI creativity and propelling the Chinese AI fleet towards the stars!

1. Co-Creation Content Direction

   Share high-quality videos or articles on Heygem deployment tutorials, optimization guides, and practical cases (Bilibili, Douyin, Xiaohongshu, WeChat Official Accounts, Zhihu, etc.)

2. Open Source Co-Creation Special Reward Pool (Real Cash Rewards!)

   (1) Basic Rewards

   Content receiving 20-100 likes will be awarded the [Heygem.ai Master Award] and a 20 RMB cash red envelope.

   Content receiving 100+ likes will be awarded the [Heygem.ai God Award] and a 50 RMB cash red envelope.

   (2) Special Achievements:

   The monthly MVP will unlock the Open Source Hall of Fame digital badge (permanently on-chain).
   

3. Participation Method

   Send your creativity to the customer service lady, add a friend with the note "Name+999".

Outstanding Co-Creation Works Exhibition

HeyGem Digital Human One-Click Start, 8G Video Memory Available, Model Size 10G, No Need for 100G Hard Disk Space, No Need for D Drive, Based on Docker Single Image, Silicon-Based Open Source

Ai Digital Human 16 - Local Deployment! The Most Popular Open Source Digital Human HeyGem Zero-Basis Hands-On Teaching Setup Tutorial, 20% Generation Stuck Solution, Full Simplified Process with Supporting Files - T8 ComfyUI Tutorial

Heygem Open Source Witnessed History! Cyber Worker Revolution!

Digital Human Project Heygem Local Deployment Tutorial

So Tempting! From Paid to Open Source, AI Digital Humans Will Open a New Era

Open Source Free Digital Humans Are Here, Unlimited Times, Fast Cloning

AI Digital Humans Are Free! GitHub's Hot Project Can Run on Your Computer

The Most Popular Free AI Digital Human, HeyGem V1.0.3, Latest Update, One-Click Integration Package! Super Strong Lip-Sync Effect, Speed Up, Supports Long Videos, Batch Generation, 8G Video Memory Available!

HeyGem One-Click Package Windows Direct Run Without Docker Silicon-Based Open Source Digital Human

Introduction

Heygem is a fully offline video synthesis tool designed for Windows systems that can precisely clone your appearance and voice, digitalizing your image. You can create videos by driving virtual avatars through text and voice. No internet connection is required, protecting your privacy while enjoying convenient and efficient digital experiences.

• Core Features
  • Precise Appearance and Voice Cloning: Using advanced AI algorithms to capture human facial features with high precision, including facial features, contours, etc., to build realistic virtual models. It can also precisely clone voices, capturing and reproducing subtle characteristics of human voices, supporting various voice parameter settings to create highly similar cloning effects.
  • Text and Voice-Driven Virtual Avatars: Understanding text content through natural language processing technology, converting text into natural and fluent speech to drive virtual avatars. Voice input can also be used directly, allowing virtual avatars to perform corresponding actions and facial expressions based on the rhythm and intonation of the voice, making the virtual avatar's performance more natural and vivid.
  • Efficient Video Synthesis: Highly synchronizing digital human video images with sound, achieving natural and smooth lip-syncing, intelligently optimizing audio-video synchronization effects.
  • Multi-language Support: Scripts support eight languages - English, Japanese, Korean, Chinese, French, German, Arabic, and Spanish.
• Key Advantages
  • Fully Offline Operation: No internet connection required, effectively protecting user privacy, allowing users to create in a secure, independent environment, avoiding potential data leaks during network transmission.
  • User-Friendly: Clean and intuitive interface, easy to use even for beginners with no technical background, quickly mastering the software's usage to start their digital human creation journey.
  • Multiple Model Support: Supports importing multiple models and managing them through one-click startup packages, making it convenient for users to choose suitable models based on different creative needs and application scenarios.
• Technical Support
  • Voice Cloning Technology: Using advanced technologies like artificial intelligence to generate similar or identical voices based on given voice samples, covering context, intonation, speed, and other aspects of speech.
  • Automatic Speech Recognition: Technology that converts human speech vocabulary content into computer-readable input (text format), enabling computers to "understand" human speech.
  • Computer Vision Technology: Used in video synthesis for visual processing, including facial recognition and lip movement analysis, ensuring virtual avatar lip movements match voice and text content.

Dependencies

1. Nodejs 18
2. Docker Images
   • docker pull guiji2025/fun-asr
   • docker pull guiji2025/fish-speech-ziming
   • docker pull guiji2025/heygem.ai

Windows Installation

Prerequisites

1. Must have D Drive: Mainly used for storing digital human and project data

   • Free space requirement: More than 30GB

2. C Drive: Used for storing service image files

   • Free space requirement: More than 100GB

   • If less than 100GB is available, after installing Docker, you can choose a different disk folder with more than 100GB of remaining space at the location shown below.

     

3. System Requirements:

   • Currently supports Windows 10 19042.1526 or higher

4. Recommended Configuration:

   • CPU: 13th Gen Intel Core i5-13400F
   • Memory: 32GB
   • Graphics Card: RTX 4070

5. Ensure you have an NVIDIA graphics card with properly installed drivers

   NVIDIA driver download link: https://www.nvidia.cn/drivers/lookup/

   

Installing Windows Docker

1. Use the command wsl --list --verbose to check if WSL is installed. If it shows as below, it's already installed and no further installation is needed.

   

• WSL installation command: wsl --install
• May fail due to network issues, try multiple times
• During installation, you'll need to set and remember a new username and password

2. Update WSL using wsl --update.

   

3. Download Docker for Windows, choose the appropriate installation package based on your CPU architecture.

4. When you see this interface, installation is successful.

   

5. Run Docker

   

6. Accept the agreement and skip login on first run

   

   

   

Installing the Server

Installation using Docker, docker-compose as follows:

1. The docker-compose.yml file is in the /deploy directory.

2. Execute docker-compose up -d in the /deploy directory, if you want to use the lite version, execute docker-compose -f docker-compose-lite.yml up -d

3. Wait patiently (about half an hour, speed depends on network), download will consume about 70GB of traffic, make sure to use WiFi

4. When you see three services in Docker, it indicates success

   

Client

1. Directly download the officially built installation package
2. Double-click HeyGem-x.x.x-setup.exe to install

Open APIs

We have opened APIs for model training and video synthesis. After Docker starts, several ports will be exposed locally, accessible through http://127.0.0.1.

For specific code, refer to:

• src/main/service/model.js
• src/main/service/video.js
• src/main/service/voice.js

Model Training

1. Separate video into silent video + audio
2. Place audio in D:\heygem_data\voice\data

   D:\heygem_data\voice\data is agreed with the guiji2025/fish-speech-ziming service, can be modified in docker-compose

3. Call the http://127.0.0.1:18180/v1/preprocess_and_tran interface

   Parameter example:

   {
     "format": ".wav",
     "reference_audio": "xxxxxx/xxxxx.wav",
     "lang": "zh"
   }

   Response example:

   {
     "asr_format_audio_url": "xxxx/x/xxx/xxx.wav",
     "reference_audio_text": "xxxxxxxxxxxx"
   }

   Record the response results as they will be needed for subsequent audio synthesis

Audio Synthesis

Interface: http://127.0.0.1:18180/v1/invoke

// Request parameters
{
  "speaker": "{uuid}", // A unique UUID
  "text": "xxxxxxxxxx", // Text content to synthesize
  "format": "wav", // Fixed parameter
  "topP": 0.7, // Fixed parameter
  "max_new_tokens": 1024, // Fixed parameter
  "chunk_length": 100, // Fixed parameter
  "repetition_penalty": 1.2, // Fixed parameter
  "temperature": 0.7, // Fixed parameter
  "need_asr": false, // Fixed parameter
  "streaming": false, // Fixed parameter
  "is_fixed_seed": 0, // Fixed parameter
  "is_norm": 0, // Fixed parameter
  "reference_audio": "{voice.asr_format_audio_url}", // Return value from previous "Model Training" step
  "reference_text": "{voice.reference_audio_text}" // Return value from previous "Model Training" step
}

Video Synthesis

• Synthesis interface: http://127.0.0.1:8383/easy/submit

  // Request parameters
  {
    "audio_url": "{audioPath}", // Audio path
    "video_url": "{videoPath}", // Video path
    "code": "{uuid}", // Unique key
    "chaofen": 0, // Fixed value
    "watermark_switch": 0, // Fixed value
    "pn": 1 // Fixed value
  }

• Progress query: http://127.0.0.1:8383/easy/query?code=${taskCode}

  GET request, the parameter taskCode is the code from the synthesis interface input above

Self-Check Steps Before Asking Questions

1. Check if all three services are in Running status

   

2. Confirm that your machine has an NVIDIA graphics card and drivers are correctly installed.

   All computing power for this project is local. The three services won't start without an NVIDIA graphics card or proper drivers.

3. Ensure both server and client are updated to the latest version. The project is newly open-sourced, the community is very active, and updates are frequent. Your issue might have been resolved in a new version.

   • Server: Go to /deploy directory and re-execute docker-compose up -d
   • Client: pull code and re-build

4. GitHub Issues are continuously updated, issues are being resolved and closed daily. Check frequently, your issue might already be resolved.

Question Template

1. Problem Description

   Describe the reproduction steps in detail, with screenshots if possible.

2. Provide Error Logs

   • How to get client logs:

     

   • Server logs:

     Find the key location, or click on our three Docker services, and "Copy" as shown below.

     

Contact Us

  [email protected]

License

LICENSE

Acknowledgments

• ASR based on fun-asr
• TTS based on fish-speech-ziming

Star History