Vortex5 commited on
Commit
e4b6cb0
·
verified ·
1 Parent(s): 9ed1e66

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -22
README.md CHANGED
@@ -9,28 +9,25 @@ tags:
9
  - merge
10
  - roleplay
11
  ---
12
- # merge
13
 
14
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
 
16
- ## Merge Details
17
- ### Merge Method
18
 
19
- This model was merged using the [Karcher Mean](https://en.wikipedia.org/wiki/Karcher_mean) merge method.
 
20
 
21
- ### Models Merged
22
-
23
- The following models were included in the merge:
24
- * [Vortex5/Moonlit-Shadow-12B](https://huggingface.co/Vortex5/Moonlit-Shadow-12B)
25
- * ./intermediates/First
26
 
27
- ### Configuration
 
 
28
 
29
- The following YAML configuration was used to produce this model:
30
  <details>
31
- <summary><b>Merge Config</b> (click to expand)</summary>
32
 
33
- ```yaml
34
  name: First
35
  base_model: Vortex5/Vermilion-Sage-12B
36
  models:
@@ -41,19 +38,40 @@ parameters:
41
  normalize: true
42
  t: [0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.6, 0.5, 0.6, 0.6]
43
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
44
  ```yaml
45
  dtype: bfloat16
46
  merge_method: karcher
47
  modules:
48
  default:
49
  slices:
50
- - sources:
51
- - layer_range: [0, 40]
52
- model: ./intermediates/First
53
- - layer_range: [0, 40]
54
- model: Vortex5/Moonlit-Shadow-12B
55
  parameters:
56
- max_iter: 9999.0
57
- tol: 1.0e-09
58
  ```
59
- </details>
 
 
 
 
 
 
 
 
 
9
  - merge
10
  - roleplay
11
  ---
12
+ # 🩵 LunaMaid-12B
13
 
14
+ This is a multi-stage merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
15
 
16
+ ## 🧬 Merge Overview
 
17
 
18
+ **LunaMaid-12B** was produced through a **two-stage multi-model merge** using [MergeKit](https://github.com/arcee-ai/mergekit).
19
+ Each stage fuses models with complementary linguistic and stylistic traits to create a cohesive, emotionally nuanced personality.
20
 
21
+ ### 🩵 **Stage 1 — Slerp Merge (Intermediate Model `First`)**
 
 
 
 
22
 
23
+ - **Base Model:** [Vortex5/Vermilion-Sage-12B](https://huggingface.co/Vortex5/Vermilion-Sage-12B)
24
+ - **Merged With:** [yamatazen/NeonMaid-12B-v2](https://huggingface.co/yamatazen/NeonMaid-12B-v2)
25
+ - **Method:** Spherical Linear Interpolation (Slerp)
26
 
 
27
  <details>
28
+ <summary><b>Stage 1 Configuration</b></summary>
29
 
30
+ ```yaml
31
  name: First
32
  base_model: Vortex5/Vermilion-Sage-12B
33
  models:
 
38
  normalize: true
39
  t: [0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.6, 0.5, 0.6, 0.6]
40
  ```
41
+ </details>
42
+
43
+
44
+
45
+ ### 🌑 **Merge Method — Karcher Mean Merge (Final Model)**
46
+
47
+ - **Base Model:** Intermediate output from Stage 1 `./intermediates/First`
48
+ - **Merged With:** [Vortex5/Moonlit-Shadow-12B](https://huggingface.co/Vortex5/Moonlit-Shadow-12B)
49
+ - **Method:** [Karcher Mean](https://en.wikipedia.org/wiki/Karcher_mean) (Riemannian Barycenter)
50
+
51
+ <details>
52
+ <summary><b>Stage 2 Configuration</b></summary>
53
+
54
  ```yaml
55
  dtype: bfloat16
56
  merge_method: karcher
57
  modules:
58
  default:
59
  slices:
60
+ - sources:
61
+ - layer_range: [0, 40]
62
+ model: ./intermediates/First
63
+ - layer_range: [0, 40]
64
+ model: Vortex5/Moonlit-Shadow-12B
65
  parameters:
66
+ max_iter: 9999
67
+ tol: 1e-9
68
  ```
69
+ </details>
70
+
71
+ ### Models Merged
72
+
73
+ The following models were included in the merge:
74
+ * [Vortex5/Moonlit-Shadow-12B](https://huggingface.co/Vortex5/Moonlit-Shadow-12B)
75
+ * [Vortex5/Vermilion-Sage-12B](https://huggingface.co/Vortex5/Vermilion-Sage-12B)
76
+ * [yamatazen/NeonMaid-12B-v2](https://huggingface.co/yamatazen/NeonMaid-12B-v2)
77
+ * ./intermediates/First