Install the NVIDIA driver and check your card

Nothing in this series is ours here: the driver comes from NVIDIA and your distribution. Install it the way your distribution recommends, then check it properly.

Install

• Ubuntu: sudo ubuntu-drivers install, then reboot.
• Arch and derivatives: the nvidia-open package (recent cards), then reboot.
• Fedora: the akmod-nvidia package from RPM Fusion.
• Debian: the nvidia-driver package from the non-free component.

Your distribution's documentation is the reference: drivers move faster than any tutorial.

Check

nvidia-smi

You should see your card, its memory, and a CUDA version in the top right corner. If the command is missing or says it cannot talk to the driver, stop here: nothing later will work.

nvidia-smi --query-gpu=name,memory.total,memory.used --format=csv

Write down memory.total: it is the number that decides which models you can run.

A trap we paid for

A desktop session already uses some VRAM (a few hundred MB). On a 16 GB card, plan for about 15 GB usable, not 16.