Cloud AI vs Edge AI

The Cloud is where AI training happens. With access to thousands of synchronized GPUs and nearly infinite memory, the cloud can host massive models like GPT-4 or DALL-E. It is the ideal choice for Complex Tasks that aren't time-sensitive, such as processing medical records or generating high-resolution art. The downside is the 'Cloud Tax'—ongoing server costs and the hard requirement of a stable internet connection. If the Wi-Fi goes down, the intelligence disappears.

Edge AI traded raw power for Resilience and Speed. Because the model is stored locally on the device (like a smartphone or an Arduino), it works offline and reacts instantly. While you can't run a 175-billion parameter model on a watch, you can run highly optimized classifiers for heart rate monitoring, gesture recognition, or voice commands. The primary engineering challenge of Edge AI is the Constraint: you must fit your intelligence into kilobytes of RAM and milliwatts of power.

Should your model live in a data center or a pocket? In this lesson, we'll master the Cloud vs Edge trade-off—learning how to choose the right environment for every task.

Cloud AI offers unlimited compute and memory. It's the king of complex LLMs and massive dataset training. But it requires a stable, high-speed connection.

Edge AI is constrained. You have limited battery and CPU cycles. However, you have 100% availability, even in a deep forest or a moving elevator.

Checkpoint: Which scenario absolutely requires Edge AI instead of Cloud AI?

• →Processing 1 million photos overnight
• →A safety sensor on a mountain trail with no cellular coverage

Many modern systems use a 'Hybrid' approach. Simple, fast tasks happen on the device, while complex, slow tasks are offloaded to the cloud.

By mastering the Cloud/Edge divide, you've learned how to optimize for cost, performance, and user experience. You're ready to deploy smart systems.

Checkpoint: True or False: Cloud AI is always cheaper because you don't have to buy expensive edge hardware for every user.

• →True
• →False (Cloud server costs can grow infinitely with user count; Edge AI costs are fixed at hardware purchase)

Trade-offs mastered! Now, let's look at the silicon that makes Edge AI possible: Hardware for Edge AI.

Next, we'll explore the hardware landscape—from GPUs to specialized NPUs and microcontrollers.

Route Real Inference by Latency Budget. Finish the rule that routes inference to the edge when the latency budget is too tight for a round-trip to the cloud.