ArtificialInteligence @Stanford
Stanford University researchers launch Octopus v2: Enhancing super-proxy capabilities for language models on devices
Researchers at Stanford University have launched Octopus v2, an advanced on-device language model designed to address widespread latency, accuracy and privacy issues associated with current LLM applications. Unlike previous models, Octopus v2 significantly reduces latency and improves the accuracy of applications on the device. Its unique feature is that the fine-tuning method through functional tags can achieve precise function calls, surpassing GPT-4 in efficiency and speed, while significantly reducing the context length by 95%.
Octopus v2’s approach involves fine-tuning the 2 billion-parameter model of Gemma 2B derived from Google DeepMind on a custom dataset focused on Android API calls. The dataset is built from positive examples and counterexamples to improve the accuracy of function calls. The training combines a full model and low-rank adaptation (LoRA) technology to optimize performance performed on the device. The key innovation is the introduction of feature tokens during the fine-tuning period, significantly reducing latency and context length requirements. This process allows Octopus v2 to implement high-precision and efficient function calls on edge devices without requiring significant computing resources.
ArtificialInteligence @Stanford
Video: