Activefastllm Open Source

Phi-3 Medium

by Microsoft· Released May 2024· Cutoff May 2024

Phi-3 Medium is a 14B parameter language model from Microsoft, part of the Phi-3 family, designed to deliver strong performance on a wide range of tasks while being efficient enough to run on consumer hardware. It is a compact model that rivals larger models in quality, making it suitable for resource-constrained environments and local deployment.

Official Site API Docs 🤗 Hugging Face 📄 Research Paper

Input cost

Free (open source)

Output cost

Free (open source)

Context window

128K tokens

Max output

—

Modalities

text

Parameters

14B

License

MIT

Capabilities

Text GenerationCode GenerationReasoningInstruction FollowingMultilingual SupportFunction Calling

Best For

Efficient, high-quality language tasks on consumer hardware or in resource-constrained environments.

Strengths

Strong performance relative to its size, competing with larger models
Efficient inference suitable for local deployment on devices with limited compute
Long context window of 128K tokens
Open-source with permissive MIT license

Limitations

Smaller parameter count may limit performance on extremely complex tasks compared to frontier models
No native vision or multimodal capabilities
Limited to text-only input and output
May require fine-tuning for specialized domains

Use Cases

Local AI assistants on laptops or edge devices

Code generation and assistance in IDEs

Text summarization and content creation

Educational tools and tutoring

Customer support chatbots

Data extraction and analysis

Prototyping and rapid development of AI features

Improvements Over Previous Model

Introduced as a new model size in the Phi-3 family, bridging the gap between Phi-3 Mini (3.8B) and larger models
Offers a 14B parameter model with performance comparable to larger models like Llama-3-70B on certain benchmarks
Supports a 128K context window, significantly larger than Phi-3 Mini's 4K/32K variants
Released under MIT license, promoting open-source use and modification
Optimized for efficient inference on consumer hardware, including GPUs with limited VRAM

Back to all models

Improvements Over Previous Model

Introduced as a new model size in the Phi-3 family, bridging the gap between Phi-3 Mini (3.8B) and larger models

Offers a 14B parameter model with performance comparable to larger models like Llama-3-70B on certain benchmarks

Supports a 128K context window, significantly larger than Phi-3 Mini's 4K/32K variants

Released under MIT license, promoting open-source use and modification

Optimized for efficient inference on consumer hardware, including GPUs with limited VRAM