The official recommended user plan is a mixture of local and remote large models.
Benefits of using chrome native models:
1. Handle sensitive data locally;
2. Smooth user experience;
3. Higher access rights to AI;
4. Use AI offline.
The following is from the original text:
When we use AI models on the Web to build features, we often rely on server-side solutions to build larger models. This is especially true for generative AI, where even the smallest model is about a thousand times the median page size. The same applies to other AI application scenarios, where model sizes may range from tens of megabytes to hundreds of megabytes.
Since these models are not shared across websites, each website must be downloaded when the page loads. This is an unrealistic solution for developers and users
Although server-side AI is an excellent choice for large models, device-side and hybrid approaches each have huge advantages. In order for these methods to work, we need to address model size and model delivery issues.
As a result, we are developing Web platform APIs and browser capabilities that aim to integrate AI models, including the Big Language Model (LLM), directly into the browser. These include the Gemini Nano, the most efficient version of the Gemini family LLM and is designed to run locally on most modern desktops and notebooks. With built-in AI, your website or Web application can perform AI-driven tasks without having to deploy or manage its own AI model.
Learn about the advantages of built-in AI, our implementation plans, and how to leverage this technology.
Hybrid AI: Device and Server
Although device-side AI can handle a large number of use cases, there are also some use cases that require server-side support.
For example, you may need to use a larger model or support a wider range of platforms and devices.
You can consider a hybrid approach, but this depends on the following factors:
Complexity: Using device-side AI makes it easier to support specific, easy-to-understand use cases. For complex use cases, consider using a server-side implementation.
Resilience: The server side is used by default, and the device side is used when the device is offline or the connection is unstable.
Safe fallback: Browsers with built-in AI take some time, some models may not be available, and older or less powerful devices may not be able to meet hardware requirements and run all models in the best way. Please provide server-side AI for these users.
For the Gemini model, you can use back-end integration (with Python, Go, Node.js, or REST), or implement it in a Web application through the new web version of the Google AI client SDK.
Browser architecture and API
To support built-in AI in Chrome, we built the infrastructure to get basic and expert models that execute on the device side. This infrastructure is already supporting innovative browser features (such as writing for me) and will soon support APIs for device-side AI.
You will mainly use built-in AI features through task APIs, such as translation APIs or summary APIs. The Task API is designed to reason based on the model best suited for allocation.
In Chrome, these APIs are designed to run inferences against the Gemini Nano through fine-tuning or expert models. By design, Gemini Nano runs natively on most modern devices and is best suited for language-related use cases such as summarizing, reformulating, or classifying.
If you want to learn more, you can click on the link below the video.
Thank you for watching this video. If you like it, please subscribe and like it. thank
Original: https://developer.chrome.com/docs/ai/built-in? hl=zh-cn
Oil tubing: