Skip to content
Instant download · PDF + web edition · Free sample of every book · 30-day refund promise
Latent Library Browse books
AI Engineering

Small Models and On-Device AI

Fast, Private and Affordable AI at the Edge

Pages
84
Chapters
72
Level
Advanced
Format
E-book

About this book

Not every AI task needs a giant model in a data centre. Small models running on devices can be faster, cheaper, more private and available offline. This book explains how to get there.

You will learn how to estimate memory and compute budgets, choose small model architectures, and compress models with quantisation, pruning and distillation. Chapters cover runtimes and hardware accelerators, latency and battery considerations, evaluation of compressed models, hybrid designs that combine on-device and cloud models, and updating models in the field.

Worked examples show the arithmetic behind every trade-off.

What you will learn

  • Budgeting memory, compute, latency and energy
  • Choosing and designing small models
  • Quantisation, pruning and distillation
  • Runtimes, accelerators and deployment targets
  • Evaluating compressed models fairly
  • Hybrid on-device and cloud architectures

Table of contents

  1. IntroductionIntroduction
  2. Chapter 1What Runs Where
  3. Chapter 2Why Run on the Device
  4. Chapter 3Why Not
  5. Chapter 4Deciding Where to Run
  6. Chapter 5Hybrid Designs
  7. Chapter 6Familiar Examples
  8. Chapter 7Industrial and Embedded Examples
  9. Chapter 8Tiny Machine Learning
  10. Chapter 9Small Language Models
  11. Chapter 10The Business Case
  12. Chapter 11Device Hardware
  13. Chapter 12Memory Is the Real Limit
  14. Chapter 13Battery and Heat
  15. Chapter 14Hardware Fragmentation
  16. Chapter 15Why Compress Models
  17. Chapter 16Quantisation
  18. Chapter 17Quantisation Methods
  19. Chapter 18Pruning
  20. Chapter 19Distillation
  21. Chapter 20Efficient Architectures
  22. Chapter 21Task-Specific Models
  23. Chapter 22Measuring the Accuracy Cost
  24. Chapter 23Inference Runtimes
  25. Chapter 24Model Formats and Conversion
  26. Chapter 25Hardware Delegates
  27. Chapter 26Running Language Models Locally
  28. Chapter 27Model Size Budgets
  29. Chapter 28Shipping Models
  30. Chapter 29Operating-System Models
  31. Chapter 30Updating Models
  32. Chapter 31Benchmarking
  33. Chapter 32Device Farms
  34. Chapter 33Browser-Based AI
  35. Chapter 34Edge Servers
  36. Chapter 35Designing for Speed
  37. Chapter 36Designing for Uncertainty
  38. Chapter 37Offline Experience
  39. Chapter 38Personalisation On Device
  40. Chapter 39Federated Learning
  41. Chapter 40Vision Use Cases
  42. Chapter 41Audio and Speech Use Cases
  43. Chapter 42Text Use Cases
  44. Chapter 43Sensor and Health Use Cases
  45. Chapter 44Automotive Use Cases
  46. Chapter 45Industrial Use Cases
  47. Chapter 46Assistants on Device
  48. Chapter 47Privacy Benefits
  49. Chapter 48Privacy Pitfalls
  50. Chapter 49Model Security
  51. Chapter 50Adversarial Inputs
  52. Chapter 51Update Integrity
  53. Chapter 52Testing On-Device Models
  54. Chapter 53Monitoring in the Field
  55. Chapter 54Handling Drift
  56. Chapter 55Accessibility
  57. Chapter 56Energy and Sustainability
  58. Chapter 57Licensing
  59. Chapter 58Team Skills
  60. Chapter 59Worked Example: Offline Document Scanner
  61. Chapter 60Worked Example: Local Writing Assistant
  62. Chapter 61Worked Example: Factory Defect Detection
  63. Chapter 62Case Study: Keyboard Prediction
  64. Chapter 63Case Study: Hearing Assistance
  65. Chapter 64Case Study: Retail Shelf Monitoring
  66. Chapter 65Case Study: Field Inspection App
  67. Chapter 66Common Pitfalls
  68. Chapter 67Placement Checklist
  69. Chapter 68Model Readiness Checklist
  70. Chapter 69Release Checklist
  71. Chapter 70Template: Model Budget Sheet
  72. Chapter 71A First-Project Plan
  73. Chapter 72Looking Ahead
  74. GlossaryGlossary

Who this book is for

ML and mobile engineers building AI features for phones, laptops, browsers and embedded devices.

Details

FormatPDF + web edition
Length84 pages (approx.)
Chapters72
LevelAdvanced
LanguageEnglish
TopicAI Engineering
AuthorLatent Library Editorial

Keep reading

You may also like

All books
$13.99
Details