Activelegacyllm Open Source

Code Llama 34B

by Meta· Released August 2023· Cutoff July 2023

Code Llama 34B is a large language model specialized for code generation and understanding, built on top of Llama 2. It is the largest variant in the Code Llama family, offering state-of-the-art performance on coding tasks. The model supports infilling, code completion, and instruction following for multiple programming languages.

Official Site 🤗 Hugging Face 📄 Research Paper

Input cost

Free (open source)

Output cost

Free (open source)

Context window

16K tokens

Max output

—

Modalities

text

Parameters

34B

License

Llama 2 Community License

Capabilities

Code GenerationCode CompletionInfillingInstruction FollowingMulti-language Support

Best For

Advanced code generation and understanding tasks requiring high accuracy and large context.

Strengths

State-of-the-art performance on code generation benchmarks
Supports infilling (fill-in-the-middle) for code editing
Large 34B parameter count for complex reasoning
Open source with permissive license
Trained on 500B tokens of code and code-related data

Limitations

Not suitable for general natural language tasks outside code
Requires significant computational resources for inference
Context window limited to 16K tokens
May produce insecure or buggy code if not carefully prompted
No multimodal capabilities

Use Cases

Automated code generation for software development

Code completion in IDEs

Code review and bug detection

Educational tool for learning programming

Code translation between languages

Documentation generation from code

Assisting with debugging and refactoring

Improvements Over Previous Model

Introduced as a specialized code model based on Llama 2
Supports infilling (fill-in-the-middle) which predecessor Llama 2 lacked
Trained on 500B tokens of code data vs general data
Achieves state-of-the-art results on HumanEval and MBPP benchmarks
Available in three sizes (7B, 13B, 34B) for different use cases
Includes a Python-specific variant (Code Llama - Python)

Back to all models

Improvements Over Previous Model

Introduced as a specialized code model based on Llama 2

Supports infilling (fill-in-the-middle) which predecessor Llama 2 lacked

Trained on 500B tokens of code data vs general data

Achieves state-of-the-art results on HumanEval and MBPP benchmarks

Available in three sizes (7B, 13B, 34B) for different use cases

Includes a Python-specific variant (Code Llama - Python)