Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: Can't run model Deepseek Coder 33B Instruct #3703

Open
1 of 3 tasks
imtuyethan opened this issue Sep 19, 2024 · 0 comments
Open
1 of 3 tasks

bug: Can't run model Deepseek Coder 33B Instruct #3703

imtuyethan opened this issue Sep 19, 2024 · 0 comments
Assignees
@louis-jan
Labels
category: model running os: windows Windows issues type: bug Something isn't working

Comments

@imtuyethan
Copy link
Contributor

imtuyethan commented Sep 19, 2024 •
edited
Loading

Jan version

0.5.4-641

Describe the Bug

  • I cannot run Deepseek Coder 33B Instruct (~18GB), even when turned on GPU acceleration or not.
Screen.Recording.2024-09-19.at.4.23.42.PM.mov

Here's my app logs:

Screenshot 2024-09-19 at 4 29 43 PM

Steps to Reproduce

  1. Download Deepseek Coder 33B Instruct
  2. Try to use it in threads

114 (windows-dev-tensorRT-LLM)
OS: Windows 11 Pro (Version 23H2, build 22631.4037)
CPU: AMD Ryzen Threadripper PRO 5955WX (16 cores)
RAM: 32 GB
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090
Storage: 599 GB local disk (C:)

What is your OS?

  • MacOS
  • Windows
  • Linux

Related Issue

#3662
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@louis-jan louis-jan

Labels
category: model running os: windows Windows issues type: bug Something isn't working
Projects
Jan & cortex.cpp
Status: Need Investigation
Milestone
No milestone
Development

No branches or pull requests
2 participants
@imtuyethan @louis-jan