IN
feat: Add Offline DeepSeek Model
intelligentnode/Intelli#119

This PR implements an offline DeepSeek model loader and inference wrapper fulfilling all requirements in issue. It provides a lightweight, memory-efficient, dependency-minimal way to load and run DeepSeek models from HuggingFace which supports all official DeepSeek R1 & Distill variants using HuggingFace + safetensors. Dynamic model config parsing and deeper memory optimizations (e.g., Triton/offloading) can be addressed in a follow-up issue if needed.

NOTES TO REVIEWERS

Tested on DeepSeek-R1-Distill-Qwen-7B that works offline and follows low-level-only policy and here’s o/p in local testing :

> python -m unittest intelli.test.integration.test_deepseek_wrapper
----------------------------------------------------------------------
Ran 1 test in 129.257s
OK
Downloading model.safetensors.index.json from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B...
Index downloaded to /home/runner/.cache/deepseek/models--deepseek-ai--DeepSeek-R1-Distill-Qwen-7B/snapshots/91[6](https://github.com/varshith257/Intelli/pull/1/checks#step:6:7)b56a44061fd5cd7d6a8fb63255[7](https://github.com/varshith257/Intelli/pull/1/checks#step:6:8)ed4f724f60/model.safetensors.index.json
Downloading config.json from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B...
Config downloaded to /home/runner/.cache/deepseek/models--deepseek-ai--DeepSeek-R1-Distill-Qwen-7B/snapshots/916b56a44061fd5cd7d6a[8](https://github.com/varshith257/Intelli/pull/1/checks#step:6:9)fb632557ed4f724f60/config.json
Downloading model.safetensors.index.json from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B...
Index downloaded to /home/runner/.cache/deepseek/models--deepseek-ai--DeepSeek-R1-Distill-Qwen-7B/snapshots/[9](https://github.com/varshith257/Intelli/pull/1/checks#step:6:10)16b56a44061fd5cd7d6a8fb632557ed4f724f60/model.safetensors.index.json
Downloading model-00002-of-000002.safetensors from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B...
Model downloaded to /home/runner/.cache/deepseek/models--deepseek-ai--DeepSeek-R1-Distill-Qwen-7B/snapshots/916b56a44061fd5cd7d6a8fb632557ed4f724f60/model-00002-of-000002.safetensors
Downloading model-00001-of-000002.safetensors from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B...
Model downloaded to /home/runner/.cache/deepseek/models--deepseek-ai--DeepSeek-R1-Distill-Qwen-7B/snapshots/916b56a44061fd5cd7d6a8fb632557ed4f724f60/model-00001-of-000002.safetensors
Model weights loaded successfully from split safetensors.
Inference successful, output shape: torch.Size([1, 16, 152064])

DETAILED SETUP AND TESTING IS DOCUMENTED IN README

/claim #82

Claim

Total prize pool $800
Total paid $0
Status Pending
Submitted May 12, 2025
Last updated May 12, 2025

Contributors

VA

Vamshi Maskuri

@varshith257

100%

Sponsors

IN

IntelliNode

@intelligentnode

$800