How to choose right hardware for your Deep Learning machine

Let's be honest, you can't do much deep learning without a computer. Sometimes you can use a laptop for light prototyping or google colab/kaggle for some more work, but if you do more work you will need dedicated machine or cloud service with GPU. I'm not going to touch here cloud services, but I want to dig into building your own rig.

Basics

In this section, we will briefly discuss parts you need to build a PC. Feel free to skip it if you know PC structure. Here is what's needed to build a PC:

CPU - central processing unit
Motherboard - helps parts to communicate, gives them needed power
RAM - fast memory, used for keeping every opened app. Data is held while ram is powered
Storage - used to store everything. Data is written and doesn't depend on PC being powered
PSU - power supply unit. Powers everything
GPU - graphical processing unit, main part for DL
Case and cooling - a box to store and cool parts

To build a PC, you should:

Install CPU into the socket on the motherboard
Add thermal paste to the CPU and install CPU cooler
Install RAM
Install storage
Put everything into the case
Install power supply
Install GPU

Watch a tutorial on YouTube to see how it's done. It might seem hard, but believe me, it is not. Everyone can do that with a proper video guide.

Deep Learning needs

So for Deep Learning the most important part is a GPU, everything else is needed to serve our gpu. So you would want to spend most on your GPU. But let's start from the beginning:

CPU

CPUs main 2 things are:

How fast each core is
How many cores there are

We don't need a lot of performance from our CPU. It is used for preprocessing our data and keeping GPU fed. Mid-range last gen CPU would be just fine. CPUs are reliable and can work long-long years.
Recommendations: AMD Ryzen 5, Intel Core i5

Motherboard

Motherboards can be less reliable, and you might have some problems in 5+ years. Moreover, you might use your PC a lot more than a typical user, so it's a good idea to get a mid-range motherboard. You probably don't need fancy features, just don't get the cheapest one.
Recommendations: Asus, MSI, ASRock, Gigabyte

RAM

RAM is easy too. Its speed doesn't really matter for our task, so something reliable with decent speed is going to be just fine. What is important - amount of RAM.
If you work with NLP - you would need a lot of it. Let's say 64gb is a mid-range. For Computer Vision, 32gb and more is ok if you work with 2D detection/segmentation. If you work with 3D, - 128 might not be enough sometimes.
If you are just starting your journey in Deep Learning - you can get less RAM and easily upgrade later. So don't worry too much on these numbers.
Recommendations: G-skill, Corsair

Storage

We don't need fast storage, but as the main drive you would want to get a NVMe or SATA drive. As secondary storage, I would recommend SATA, but HDD should be fine too (NVMe is the fastest, HDD is the slowest)
Recommendations: Samsung, Crucial, Sabrent

PSU

Get a nice and reliable power supply. Don't recommend to cheap out here. Calculate deeded power based on your CPU and GPU power conception. Have a 20% overhead just in case.
Recommendations: Seasonic, Corsair

Case and cooling

Case doesn't really matter too. Just get a case with a good airflow. Cooling is important, as you are going to load your hardware for long runs, and insufficient cooling in the best case will lower your hardware capability (GPU and CPU will run slower) and kill it in the worst case.
Recommendations: Fractal Design, Noctua...

GPU

Finally we got to the main part. GPU is what we use for computations while training neural nets. We are going to discus only NVIDIA GPUs, because they are far ahead for Deep Learning now.

Because of the architecture, GPUs are good for parallel computation, and that's what we need in Deep Learning. Basically, we need to do a lot of matrix/tensor multiplications, and that's exactly what Tensor Cores in Nvidia GPUs do.
They are really fast, so we need memory to be fast too, so Tensor Cores are not idle. For example, I got 60% of "GPU Time Spent Accessing Memory" on a 3090 with GPU utilization just under 100%. That's about the speed.

But you can't get much training done if your model or data doesn't fit in your GPU's VRAM. That's a third thing, and I recommend being careful with it. If you plan to study - 8gb should be enough. If you are working with Computer Vision and detection models - you probably will need 12gb and more. Sometimes it's better to get a slower card but with more VRAM, so you can train bigger models with bigger batch size. So, again, here is what's important in GPU specs:

Tensor Cores (for speed)
Bandwidth (for speed)
VRAM (for training bigger models with higher batch size)

I would recommend getting newer cards, 30 series for example (for end of 2022). Take a look at 3060 with 12gb of VRAM. It's fast enough and has enough VRAM for different tasks.

A4000 is also interesting, as it is a professional analogue of 3070 with 16gb of VRAM, lower clock speeds, lower TDP (it is quieter and uses less power) and it is more compact (it is also more expensive). If you are going to use your card 24/7, you should take a look to professional class models instead of consumer level. But in most cases, consumer level should be just fine.

3080 is pretty fast, but has only 10gb of VRAM, so I don't really recommend it. On the other hand, 3080ti has 12gb, which is a little bit better. 4080 has 16gb of VRAM, but it seems to be overpriced for its performance, and you might be better to go to 3090/4090 strait.

3090 with 24gb is great for any tasks, so 4090 is. If you can fit it into your case. And these models are also really expensive.

Conclusion

I would love to give you a chart with price to performance, but I've looked through several different benchmarks and results are too diverse. Furthermore, there are no benchmarks for some newer cards as 4080, or less exciting card as 3060 (but don't get me wrong, 3060 is good choice). So all I can recommend for now is to look at VRAM, and cost, if you will get newer cards (more $ - more performance).
If you really want to see some charts, you can have a look here or here. You can also read more about GPUs here.
And here is an example of a build.