2019 Lambda deep learning workstation, including specs this time, what LLMs can it run and what should I expect for output?
Last post I made the error of not including all the specs for this new to me setup. Also the answers I got made sense, but they lead me to a whole new subset of questions. Sorry for wasting anyones time on the original post.
Specs are:
CPU i9-10920X 3.5GHz 12 core socket LGA2066 max memory bandwidth of 94GB/s
44GB VRAM across four 2080Ti GPUs (blower style)
256 GB of DDR4-2933 (8 sticks of 32GB)
ASUS WS X299 SAGE Mother Board
Running Ubuntu
My first question was what people thought I could run on a system with generic specs, now that I have the actual specs, I would pose the same question.
Also is there a correlation to T/S and Parameter size? I think I am asking that correctly, if I was to run a smaller model would I get faster speeds or is it my hardware that will limit me regardless?
Upgrading, if I was to reuse the case, power supply, and GPUs, would a new motherboard, DDR5 RAM and better CPU make things a lot faster or are the PCIe 3.0 GPUs going to bottleneck things right back to where I currently am?