RedditPP Mascot

RedditPP

r/LocalLLaMA • … • u/Dented_Steelbook

2019 Lambda deep learning workstation, including specs this time, what LLMs can it run and what should I expect for output?

Last post I made the error of not including all the specs for this new to me setup. Also the answers I got made sense, but they lead me to a whole new subset of questions. Sorry for wasting anyones time on the original post.

Specs are:

CPU i9-10920X 3.5GHz 12 core socket LGA2066 max memory bandwidth of 94GB/s

44GB VRAM across four 2080Ti GPUs (blower style)

256 GB of DDR4-2933 (8 sticks of 32GB)

ASUS WS X299 SAGE Mother Board

Running Ubuntu

My first question was what people thought I could run on a system with generic specs, now that I have the actual specs, I would pose the same question.

Also is there a correlation to T/S and Parameter size? I think I am asking that correctly, if I was to run a smaller model would I get faster speeds or is it my hardware that will limit me regardless?

Upgrading, if I was to reuse the case, power supply, and GPUs, would a new motherboard, DDR5 RAM and better CPU make things a lot faster or are the PCIe 3.0 GPUs going to bottleneck things right back to where I currently am?

— 0