Spaces:

---

numind

/

NuMarkdown-8B-Thinking

like 36

Sleeping

Hi
hope you doing well.
first of all thanks for the amazing model , its OCR is really good . i want to set up it locally but am getting error when try to run docker container

sudo docker run --rm -it --gpus all -p 7860:7860 numind-vlm

==========
== CUDA ==
==========

CUDA Version 12.1.1

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

=== Starting NuMarkdown-8B-Thinking Space ===
Starting vLLM server with optimized settings...
vLLM started with PID: 27
Waiting for vLLM server to start (this may take 5-10 minutes)...
Still waiting... (10/180) - checking vLLM process
❌ vLLM process died! Checking logs:
(APIServer pid=27)     async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=27)   File "/home/user/.local/lib/python3.10/site-packages/vllm/utils/__init__.py", line 1589, in inner
(APIServer pid=27)     return fn(*args, **kwargs)
(APIServer pid=27)   File "/home/user/.local/lib/python3.10/site-packages/vllm/v1/engine/async_llm.py", line 212, in from_vllm_config
(APIServer pid=27)     return cls(
(APIServer pid=27)   File "/home/user/.local/lib/python3.10/site-packages/vllm/v1/engine/async_llm.py", line 136, in __init__
(APIServer pid=27)     self.engine_core = EngineCoreClient.make_async_mp_client(
(APIServer pid=27)   File "/home/user/.local/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 102, in make_async_mp_client
(APIServer pid=27)     return AsyncMPClient(*client_args)
(APIServer pid=27)   File "/home/user/.local/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 769, in __init__
(APIServer pid=27)     super().__init__(
(APIServer pid=27)   File "/home/user/.local/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 448, in __init__
(APIServer pid=27)     with launch_core_engines(vllm_config, executor_class,
(APIServer pid=27)   File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__
(APIServer pid=27)     next(self.gen)
(APIServer pid=27)   File "/home/user/.local/lib/python3.10/site-packages/vllm/v1/engine/utils.py", line 729, in launch_core_engines
(APIServer pid=27)     wait_for_engine_startup(
(APIServer pid=27)   File "/home/user/.local/lib/python3.10/site-packages/vllm/v1/engine/utils.py", line 782, in wait_for_engine_startup
(APIServer pid=27)     raise RuntimeError("Engine core initialization failed. "
(APIServer pid=27) RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {}

thanks