Commit 533ae6e
| Repository | myai |
| Branch | main |
| Author | Betty |
| Hash | 533ae6e0010dacfbd8c941d98dca96a0c84f8575 |
Description
Replace StarCoder2-3B with Qwen/Qwen2.5-Coder-1.5B:
- 1.5B params → ~3GB float16 → fits entirely in T400 4GB VRAM
- No CPU offloading → ~30s/64tok vs 3-7min before
- safetensors format → compatible with PyTorch 2.2 + transformers 4.57
- Not gated → no HF token needed for download
Removed hf_token secret dependency, restored GPU placement constraint, adjusted resource limits to 2CPU/3G RAM.
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com