Linux nVidia Laptop for AI and K8S recommendations

If you need a laptop and are going to do machine learning and AI on it, then you pretty much need to have a Linux laptop with an nVidia processor. This is a pretty narrow market, but there are two big choices: get a Linux pre-installed laptop from the big boys or go for a niche player specializing in Linux support. The reason for this is drivers. It would be really nice if Ubuntu played nice with your system.

Now in the days of containers, it wouldn’t seem like you need to do this that much, but it’s necessary if you are going to need to see graphics as X11 support is pretty rough, and its hard to get while on the road or in a remote location. So what’s a person to do? Well, first figure out the processor mess and then figure out the nVidia GPU mess and then figure out who actually ship with Ubuntu preinstalled as a marker for the drivers may work

TL;dr

If you are contemplating development locally doing machine learning and Kubernetes development (eg self-contained on an airplane or on vacation without good internet connectivity), there are four things to try:

Juno Jupiter 14″ Pro V2. This is a the only thin and light, notebook weighing just 3.1 pounds, so heavier than a 2.7 pound Macbook Air wiith an nVidia GPU, albeit a small one with 4GB of VRAM so good for small models only but with a 512GB SSD and 32GB of RAM, its $1600. if you want to all out, a 32GB and 1TB system is $1784 and a minimal one with 16GB and 512GB is $1,525 which isn’t too bad. The only drawback is that it does use a low power 11th generation processor. It has a fast 144 Hz but relatively low resolution HD display.
Juno Saturn 15″ V4. This was just released and has all the goodies and weights 4.1 pounds so somewhere between a MacBook Pro 13 and 14 (3.5 pounds and 4.5 pound respectively). It has a i7-12700H and you can get a RTX 3060 with 6GB VRAM, 32GB RAM, 512GB Disk for $1,719. This is a machine that where you can do serious development
Dell Outlet. If you have some time and want to get a potentially good deal, the tricky part is finding a 15″ that also weights 4 pounds and has a 0.30″ height, but for example right now, there is a Precision 5560 15″ notebook with 512GB SSD, 32GB RAM, RTX A2000 4GB for $2K so a bit more than a Juno, but it comes from a “safe” brand. It has a nicer high resolution display that is a nice to have.
Tensorbook. Finally if you really want a monster machine with everything that you need and don’t plan on needing a cloud connection for big model developments and are in a well funded startup that just wants to go fast, the $3.5K Tensorbook is preconfigured and ready to go with a Intel Core i7-11800H, 64GB RAM, 2TB NVMe SSD, 16GB VRAM. it has a decent 2.5K display.

Hardware Requirements: ML and Docker

OK, there is some controversy here, but Tim Dettmeers, Rukushan Pramoditha, and Quora had an old post that recommended based on the amount of VRAM as these models need to have a reasonable amount in order to run. Now of course, with these massive models, even this might not be enough but its a start of the way to think about things and shows that a laptop development machine is good for small models and testing but if you want to do real work, go to the cloud or a desktop:

State-of-the-art research running and training: 11GB
Interesting architectures and startups, prototyping: 8GB
Kaggle fun projects and for laptops: 4GB

And another rule of thumb is that the amount of RAM should equal the amount of VRAM. That’s not a problem with most laptops and that other than storing stuff, most modern CPUs and SSDs are plenty fast:

Current state-of-the-art RAM: 32GB
Tinkering on your own RAM: 16GB
Minimum: 12GB

Then the second complexity is if you want to do development with Kubernetes on this machine, here the main constraint is the additional memory requirements for doing this. And there are a sea of tools for doing this with things like Skaffold and maybe play with nix instead of brew for package management because packages are sandboxed so you don’t have dependency wars between packages and an entire ecosystem that includes a complete OS and shell. Needing all this might push you from 16GB to 32GB since you have to think about running a cluster with multiple say ROS containers.

Linux Laptop vendors

Well, there seem to be two major ones, Dell and Lenovo who are committed to shipping Linux preinstalled, and then there is a sea of smaller companies that are either reselling existing designs or actually building things themselves, so some choices are below. Now you can also get a Razer for instance and install Ubuntu on it, just be aware that you can definitely have driver problems but there’s a pretty big community of folks trying to make it work:

Dell. For a long time, their slim X13 has had Ubuntu preinstalled, but these do not have beefy GPUs. The best line for this is their Precision line of laptops. These mainly have previous generation processors, the 11th generation, and Turing (not Ampere) GPUs, but they are available.
Lenovo. This is the same way, they are slim and light have built and you can get a base system.
System76. Mayone of the original Linux-only vendors has a really deep and long line.
Juno. They are the same way with some nice units.
Kubuntu. They resell a laptop with all the fixings
Tensorbook. This is also a resold laptop that mainly has the various machine learning things.

But how do you evaluate the whole line, well here you have to understand a complicated roadmap from Intel and nVidia where a one-digit change in a five-digit number means a lot. So some recommendations are from Reviewgeek and Techradar for things that are like MacBook Air’s but at 2.5-3.5 pounds are:

Juno Jupiter 14 Pro V2. This is 3.1 pounds with i7-1165G7, HD@144 Hz, a basic nVidia RTX 3050 Ti, 16GB memory, and 512GB SSD for $1.5K, so a great entry-level machine with the ability to run small AI models although it is the 11th generation Intel and a smaller nVidia VRAM, it is small and light which is really convenient and the only ultraportable I could find with an nVidia card.
Dell XPS 13 Developer Edition. This is slim and light it has no GPU so pretty useless for AI
Lenovo X1 Carbon. The other think and light winner but like the X13 no nVidia chip and 10th gen.
System 76 Lemur. This is from a small vendor, but it is also thin and light with 12th Gen i5 and i7

Then there is the machine really for AI work (and gaming too) but is midweight and thickness like MacBook Pro 16

Juno Saturn 15 v4. Unlike the Neptune below, this is more of a mid-range box and it is available now. It has an Intel i7-12700H, an HD@144 screen, an RTX 3060 with 6GB VRAM, 16GB of memory, 1TB 3.5GB/s read SSD (with room for another one) for $1.7K at 4.1 lb so between a MacBook Pro 14 and 16 in weight.
TUXEDO Polaris 15 – Gen 3. This is the brand for Schenkery/XMG of Germany. You can get either a Ryzen 7 of Core i7 11800H or AMD Ryzen 7 5800H or the Intel is faster single-core, but the Ryzen leads on multicore by a little bit. It has an RTX 3060 6GB VRAM with 115w power consumption and a 2.5K@165Hz monitor and Thunderbolt 4 for $1,664 Euro plus shipping for the HD screen and 250 Euro more the 2.5K screen and intel processor. It is 23mm high and 2 kg (so a bit heavier than the MacBook 14)
Juno Neptune 15 v3. This is supposed to be available at the end of June 2022 from a London-based company. It has an i7-11800H so this is an 11th generation chip and it has an nVidia GeForce RTX 3070 Max-Q with 8GB VRAM, so a little slower than a Ti but no slouch. The screen is 1920×1080 but is 240 Hz which makes it a real gamer machine and it has Thunderbolt 3, or that. It’s in the regular notebook weight at 4.4lbs at about the same weight as MacBook Pro 16 which is 4.7 pounds and 0.78″ high so pretty thick. By comparison, the MacBook Pro 14 is 3.5 pounds or 1.6Kg and 0.61 inches (1.55cm) and the MacBook Air M2 is 2.7 pounds or 1.24 Kg and 0.44 or 1.13cm high. But isn’t a huge 5-pound machine (I’ve had a 7-pound one and that is a handful). It’s a pretty reasonable $2.2K for that machine.
Lambda Labs TensorBook. If you have unlimited money and do not want a hassle, then this is a great choice. It is 4.43 lb or 2.1 kg and just 0.66″ or 16.9mm high and is pretested. It is a whopping $3,500 though but you get 1TB+1TB NVME SSD, 1.4K@165 Hz screen, nVidia RTX 3080Max-Q with 16GB, Intel i7-11800H, and 64GB RAM, so if you can afford it, it the ultimate machine and has a stack of software that works. As a comparison, this sure looks like the same OEMed machine as the Juno Neptune 15 v3, that machine with the same configuration but is a bit thinner is $3.3K so the Lamdba is actually a better deal for a full machine

Then there are big and heavy notebooks which are 1 inch thick and over 5 pounds:

System 76 Oryx Pro. This is also available in late June and it has the latest goodies with a 12th generation Intel Core i7-12700H and then either an RTX 3070 Ti or 3080 Ti laptop. As you will see below, the 3070 Ti is a very decent ship. This thing is big though at 5.3 lb and nearly 0.98″ thick so it is a handful.
Kubuntu Focus M2 Gen 4. This is actually an individual build with Alder Lake 12th generation i7-12700H, a 2.5K@165 Hertz display, Thunderbolt 4, and up to RTX 3080 Ti with 16GB of memory. You can see the pricing differences from the base RTX 3060 6GB VRAM, it cost $365 to go to the RTX 3070 Ti with 8GB and a whopping another $900 to get to the 16GB RTX 3080 Ti. It is $2,270 for a nice system with a 3070 Ti, 16GB, and 512TB drive. It is heavy and big though at 5.29 lb
Laptop with Linux Clevo NH55HPQ 15.6 Graphic Designer. This is a small UK-based company so hard to get here, but it has a Core2 i7-11800H which is decent, and then RTX-3060 with 6GB VRAM. It’s a pretty reasonable $1,540 US for a basic model
Dell Outlet. One big hint is to watch the Dell Outlet, they have lots of refurbished systems for 25% off,
Dell Precision 7560 Workstation. This is a slow processor Core i5-11500H, NVIDIA RTX A2000 4GB VRAM, 16GB, 512GB for $2.3K. So a big vendor and it will work, for about $3K you can
Dell Precision 7760 Workstation. This is probably more what you wan, i5-11500 H, NVIDIA A3000 6GB VRAM which is about an RTX 3070 or so. It has a 17″ display so is huge and is $3K (so you might as well get the Tensorbook at this level.

Then there are the true giants over close to six points and over 1″ thick:

Juno Mars 15. This is the AMD-based version with an RTX 3070. It is heavy at 5.95 lb though. Wow. And 1.28″ thick.
Lenovo ThinkPad P15 Gen 2 Intel 15″. They do have pre-installed versions and this one is $2.1K for an i7-11800H with 16GB of memory (and expansion space for another 16GB), 512GB SSD, 15.6″ FHA display, RTX A3000 6GB but it is a whopping 2.87kg and 6.32lb and 24.5 to 31.4mm or 1.2″

Navigating the Intel Processors: I7-12fadK

Well, the naming of these processors is really confusing and the performance differences are small, but right now, the transition on Intel is from 11th (Rocket Lake) to 12th generations (Alder Lake) so if you can as well get the 12th generation, although the performances differences on Intel are truly minor:

The big idea is that a single Gracemont core is 40% faster than a single Skylake core at the same power. Skylake is their 14nm process for their 10th Generation.

The generations are a little easier in that the five-digit naming basically has the first two digits indicating generation, so a 12900K is the top-of-the-line unlocKed processor with 8P and 8E cores with up to 8×2 + 8 = 24 threads vs the 11900K with 8 cores in total with 16 total threads. After all that, the performance improvements are about -3 to 30% depending on the gaming benchmark.

When you combine all that, then Notebookcheck.net has a list of mobile processors to look at some top performers with some comparisons with desktop processors, those with TDP much bigger than 40W, and as an aside, you can see just how good even the M1 Pro is with its 8 cores and how the Ryzen chips are not that great in peak performance in mobile and if you can get a 12 generation, you might as well, they have the same power draw but are quite a bit better in benchmarks. The i7-1280P in particular seems to do quite well at lower power, but basically, the i9 vs i7 is a difference in class then the there is 12 vs 11th generation and finally the series which is highest at 900, then 700:

Model	TDP	Cores	Rating	Cinebench R20	Geekbench 5.3 Multi
AMD Ryzen Threadripper Pro 3975WX	280W	32	85	16K	29K
Intel Core i9-12900HK	125W	16	52	10K	18K
Intel Core i9-12900H	45W	14	39	7K	14K
Intel Core i7-1280P	28W	14	35	N/A	10K
Intel Core i7-12700H	45W	14	35	6K	12K
Apple M1 Pro 8-core	20W?	8	34.5	N/A	9.9K
Intel Core i7-12800H	45W	14	33	5.6K	12.2K
Intel Core i9-11900H	45W	8	27.2	5.0K	9.6K
AMD Ryzen 9 6900HS	35W	8	26.9	5.1K	9.2K

Mobile GPU Benchmarks: RTX 3060 or 3070 Laptop

This is even more confusing because nVidia has been changing the names of the processors from version to family for marketing reasons and they’ve split their line between RTX 30x and 20x family are gamer-oriented but the RTX A5xxx series is professional-oriented. The pricing is typically higher for the and they have different drivers. They may also have more VRAM and they have ECC, but it sure feels like marketing to me 🙂 although for many machine learning models, having lots of VRAM really matters.

The most confusing thing is what benchmark to use. 3DMark, for instance, has literally a dozen of them that are based on various video games, but 3DMark Time Spy seems like the most modern and they have 3DMark 11 Performance with its synthetic GPU benchmarks, GFXBench in contrast is multiplatform so you can compare the performance of GPU games across mobile, desktop and Mac. Cinebench is a CPU benchmark, but they do have some GPU benchmarks as well focused on OpenGL support. Basemark does automotive and has a set of benchmarks that are cross platform including browser, CPU and GPU. The performance rating is in the rough order. I’m not sure what Performance rating is, but it seems to be some sort of synthetic review score by Notebookcheck.net.

The second thing is that NVIDIA has two parallel likes, the GeForce line is targeted at gamers and has relatively less VRAM, can overclock and is generally cheaper. We use GeForce when we can because of the cost, but the RTX A series are professional so more expensive, better quality components and more VRAM. I’m sure there is a bit of enterprise pricing should be more, but generally for laptops, if you have a massive VRAM model, you should probably be running it in the cloud in an A100 for instance.

The second are the architecture transitions, the last Turing architecture are ending, they are general the GeForce RTX 20xx series vs the Ampere GeForce RTX 30xx. The professional side, the RTX Ax000 are Ampere and Turing chips are more confusing but generally named Quadro RTX laptop. So for instance, if you see Dell has the A6000 that’s 3090 Ti and so forth.

In general, unlike years past the improvement from Turing to Ampere is incremental, the flagship Titan RTX for instance is about 30% slows than the GeForce RTX 3090 Ti, so you can find some good values as a result. The top models are really expensive based on yields and demand. But for instance the 3070 Ti laptop is about the same speed as the RTX 3080 laptop, so it pays to drop down the list. Also the performance for older generations overlap, so as an example that same 3070 Ti has about the same speed as the older RTX2080 Super Mobile.

Also the highest end Laptop chips are now within range of the desktop chips, but are definitely more expensive. In general, a desktop is going to have much better performance but isn’t as portable of course and stepping even a little down gives you way lower prices with only a modest reduction in price. For example, the difference between the 3080 Ti laptop and 3080 laptop is just about 5%. Note also that it is super confusing that the version numbers are identical across laptop and desktop. But in general, laptops are pretty fast, so a 3080 laptop has about the same performance as a 3070 desktop

So to show the top GPUs and look at how it works compared with desktops:

Model	Codename	Architecture	Pixel Shaders	Perf Rating	3DMark11 PGPU	3D Mark Time Spy
NVIDIA GeForce RTX 3090 Ti	GA102	Ampere	10752	100	69.5K	21.6K
NVIDIA GeForce RTX 3090	GA102	Ampere	10496	85.9	57.7K	19.2K
NVIDIA RTX A6000		Ampere	10752	60.9	51.0K	10.5K
NVIDIA GeForce RTX 3080	GA102	Ampere	8704	79.2	53.7K	17.5K
NVIDIA Titan RTX	TU102	Turing	2304	70	49.4K	14.9K
NVIDIA GeForce RTX 3080Ti Laptop	GN20-E8	Ampere	7424	56.9	40.4K	12.1K
NVIDIA RTX A5500 Laptop	GN20-E8	Ampere	7424	~57	~40K	～12K
NVIDIA GeForce RTX 3070	GA104	Ampere	5888	60	42.0K	12.8K
NVIDIA GeForce RTX 3080 Laptop	GN20-37 GA104	Ampere	6144	52.7	38.5K	10.8K
NVIDIA RTX A5000 laptop	GA104	Ampere	6144	49.8	36.2	10.3K
NVIDIA GeForce RTX 3070 Ti Laptop	GN20-E8	Ampere	5888	51	36.8K	10.6K
NVIDIA RTX A4500 Laptop	GA104	Ampere	5888	~51	~37K	~10K
NVIDIA GeForce RTX 2080 Super Mobile	N18E-G3 TU104	Turing	3072	55.9	41K	11.3K
NVIDIA GeForce RTX 3070 Laptop	GN20-E5 GA104	Ampere	5120	47	34.1K	9.7K
NVIDIA RTX A4000 Laptop	GA104	Ampere	5120	~47	~34	~10K
NVIDIA GeForce 3080 Max-Q (low power) laptop	GN20-E7 GA104	Ampere	6144	~47	~34	~10K
Apple M1 Max 32-core GPU			32	~47	~34	~10K
NVIDIA GeForce RTX 2080 Mobile	N18E-G3 TU104	Turing	1472	47.9	34.9K	9.9K
NVIDIA GeForce GTX 1080 SLI Laptop	GP104 SLI	Pascal	2560	64.9	49.6K	12.6K
NVIDIA RTX A3000 Laptop	GA104	Ampere	4096	34.7	25.6K	7K
NVIDIA GeForce RTX 3060 Laptop	GN20-E3 GA106	Ampere	3840	37.9	26.7K	8.1K
NVIDIA GeForce RTX 2070 Super Mobile	N18E-G28	Turing	2560	40	29.1K	8.3K