Upload folder using huggingface_hub

README.md

@@ -1,3 +1,38 @@
+---
+language: en
+license: mit
+tags:
+  - medical
+  - sentence-transformers
+  - text-embedding
+  - sentence-similarity
+  - onnx
+  - semantic-search
+  - opensearch
+  - healthcare
+  - medical-embeddings
+datasets:
+  - abhinand/MedEmbed-corpus
+metrics:
+  - cosine-similarity
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+model-index:
+  - name: MedEmbed-large-v0.1-onnx
+    results:
+      - task:
+          type: Sentence Similarity
+          name: Semantic Retrieval
+        dataset:
+          type: abhinand/MedEmbed-corpus
+          name: MedEmbed corpus
+        metrics:
+          - type: cosine-similarity
+            value: N/A  # Replace with actual value if available
+base_model: abhinand/MedEmbed-Large-v0.1
+inference: true
+---
+
 # MedEmbed-large-v0.1 ONNX Model
 
 This repository contains an ONNX version of the MedEmbed-large-v0.1 model, which was originally a SentenceTransformer model.
@@ -6,6 +41,8 @@ This repository contains an ONNX version of the MedEmbed-large-v0.1 model, which
 
 The original MedEmbed-large-v0.1 model is a sentence embedding model specialized for medical text. This ONNX version maintains the same functionality but is optimized for deployment in production environments.
 
+This model is a derivative of [abhinand/MedEmbed-Large-v0.1](https://huggingface.co/abhinand/MedEmbed-Large-v0.1), which itself is a fine-tune of [abhinand/MedEmbed-base-v0.1](https://huggingface.co/abhinand/MedEmbed-base-v0.1).
+
 ## ONNX Conversion
 
 The model was converted to ONNX format using PyTorch's `torch.onnx.export` functionality with ONNX opset version 14.
@@ -46,4 +83,115 @@ embeddings = session.run(None, onnx_inputs)[0]
 
 ## Usage with OpenSearch
 
-This model can be used with OpenSearch's neural search capabilities. Please refer to OpenSearch documentation for details on how to load and use ONNX models for text embedding.
+This model can be integrated with OpenSearch for neural search capabilities. Here's how to set it up:
+
+### 1. Upload the model to OpenSearch
+
+```bash
+# Create a zip file containing your model files
+zip -r medembedlarge.zip MedEmbed-large-v0.1.onnx config.json tokenizer_config.json tokenizer.json vocab.txt special_tokens_map.json
+
+# Upload the model using the OpenSearch REST API
+curl -XPUT "https://your-opensearch-endpoint/_plugins/_ml/models/medembedlarge" \
+     -H "Content-Type: application/json" \
+     -d '{
+       "name": "medembedlarge",
+       "version": "1.0.0",
+       "model_format": "ONNX",
+       "model_config": {
+         "model_type": "bert",
+         "embedding_dimension": 768,
+         "framework_type": "sentence_transformers"
+       }
+     }' -u "admin:admin"
+
+# Upload the model file
+curl -XPOST "https://your-opensearch-endpoint/_plugins/_ml/models/medembedlarge/_upload" \
+     -H "Content-Type: multipart/form-data" \
+     -F "[email protected]" -u "admin:admin"
+```
+
+### 2. Deploy the model
+
+```bash
+curl -XPOST "https://your-opensearch-endpoint/_plugins/_ml/models/medembedlarge/_deploy" \
+     -H "Content-Type: application/json" -u "admin:admin"
+```
+
+### 3. Create a neural search pipeline
+
+```bash
+curl -XPUT "https://your-opensearch-endpoint/_plugins/_ml/pipelines/medembedlarge-pipeline" \
+     -H "Content-Type: application/json" \
+     -d '{
+       "description": "Neural search pipeline for medical text",
+       "processors": [
+         {
+           "text_embedding": {
+             "model_id": "medembedlarge",
+             "field_map": {
+               "text_field": "text_embedding"
+             }
+           }
+         }
+       ]
+     }' -u "admin:admin"
+```
+
+### 4. Create an index with embedding field
+
+```bash
+curl -XPUT "https://your-opensearch-endpoint/medical-documents" \
+     -H "Content-Type: application/json" \
+     -d '{
+       "settings": {
+         "index.plugins.search_pipeline.default": "medembedlarge-pipeline"
+       },
+       "mappings": {
+         "properties": {
+           "text_field": {
+             "type": "text"
+           },
+           "text_embedding": {
+             "type": "knn_vector",
+             "dimension": 768,
+             "method": {
+               "name": "hnsw",
+               "space_type": "cosinesimil",
+               "engine": "nmslib"
+             }
+           }
+         }
+       }
+     }' -u "admin:admin"
+```
+
+### 5. Index documents with the neural search pipeline
+
+```bash
+curl -XPOST "https://your-opensearch-endpoint/medical-documents/_doc" \
+     -H "Content-Type: application/json" \
+     -d '{
+       "text_field": "Patient presented with symptoms of hypertension and diabetes."
+     }' -u "admin:admin"
+```
+
+### 6. Perform a neural search query
+
+```bash
+curl -XPOST "https://your-opensearch-endpoint/medical-documents/_search" \
+     -H "Content-Type: application/json" \
+     -d '{
+       "query": {
+         "neural": {
+           "text_embedding": {
+             "query_text": "hypertension treatment options",
+             "model_id": "medembedlarge",
+             "k": 10
+           }
+         }
+       }
+     }' -u "admin:admin"
+```
+
+Note: Replace "https://your-opensearch-endpoint" with your actual OpenSearch endpoint, and adjust authentication credentials as needed for your environment.