> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-feat-rerank-documentation.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Lepton AI
> Use Lepton AI's serverless AI endpoints for chat completions and speech-to-text through Portkey.
## Quick Start
Get started with Lepton AI in under 2 minutes:
```python Python icon="python" theme={"system"}
from portkey_ai import Portkey
# 1. Install: pip install portkey-ai
# 2. Add @lepton provider in model catalog
# 3. Use it:
portkey = Portkey(api_key="PORTKEY_API_KEY")
response = portkey.chat.completions.create(
model="@lepton/llama-3.1-8b",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
```
```js Javascript icon="square-js" theme={"system"}
import Portkey from 'portkey-ai'
// 1. Install: npm install portkey-ai
// 2. Add @lepton provider in model catalog
// 3. Use it:
const portkey = new Portkey({
apiKey: "PORTKEY_API_KEY"
})
const response = await portkey.chat.completions.create({
model: "@lepton/llama-3.1-8b",
messages: [{ role: "user", content: "Hello!" }]
})
console.log(response.choices[0].message.content)
```
```python OpenAI Py icon="python" theme={"system"}
from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL
# 1. Install: pip install openai portkey-ai
# 2. Add @lepton provider in model catalog
# 3. Use it:
client = OpenAI(
api_key="PORTKEY_API_KEY", # Portkey API key
base_url=PORTKEY_GATEWAY_URL
)
response = client.chat.completions.create(
model="@lepton/llama-3.1-8b",
messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
```
```js OpenAI JS icon="square-js" theme={"system"}
import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"
// 1. Install: npm install openai portkey-ai
// 2. Add @lepton provider in model catalog
// 3. Use it:
const client = new OpenAI({
apiKey: "PORTKEY_API_KEY", // Portkey API key
baseURL: PORTKEY_GATEWAY_URL
})
const response = await client.chat.completions.create({
model: "@lepton/llama-3.1-8b",
messages: [{ role: "user", content: "Hello!" }]
})
console.log(response.choices[0].message.content)
```
```sh cURL icon="square-terminal" theme={"system"}
# 1. Add @lepton provider in model catalog
# 2. Use it:
curl https://api.portkey.ai/v1/chat/completions \
-H "Content-Type: application/json" \
-H "x-portkey-api-key: $PORTKEY_API_KEY" \
-d '{
"model": "@lepton/llama-3.1-8b",
"messages": [{"role": "user", "content": "Hello!"}]
}'
```
## Add Provider in Model Catalog
Before making requests, add Lepton AI to your Model Catalog:
1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Lepton AI**
3. Enter your [Lepton API key](https://console.lepton.ai/)
4. Name your provider (e.g., `lepton`)
See all setup options and detailed configuration instructions
***
## Lepton AI Capabilities
### Chat Completions
Generate chat completions with Lepton's serverless models:
```python Python theme={"system"}
from portkey_ai import Portkey
portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@lepton")
response = portkey.chat.completions.create(
model="llama-3.1-8b",
messages=[{"role": "user", "content": "Write a haiku about AI"}]
)
print(response.choices[0].message.content)
```
```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';
const portkey = new Portkey({
apiKey: 'PORTKEY_API_KEY',
provider: '@lepton'
});
const response = await portkey.chat.completions.create({
model: "llama-3.1-8b",
messages: [{ role: "user", content: "Write a haiku about AI" }]
});
console.log(response.choices[0].message.content);
```
### Speech-to-Text
Transcribe audio using Lepton's Whisper models:
```python Python theme={"system"}
from portkey_ai import Portkey
portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@lepton")
with open("audio.mp3", "rb") as audio_file:
transcription = portkey.audio.transcriptions.create(
file=audio_file,
model="whisper-large-v3"
)
print(transcription.text)
```
```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';
import fs from 'fs';
const portkey = new Portkey({
apiKey: 'PORTKEY_API_KEY',
provider: '@lepton'
});
const transcription = await portkey.audio.transcriptions.create({
file: fs.createReadStream('audio.mp3'),
model: 'whisper-large-v3'
});
console.log(transcription.text);
```
### Streaming
Enable streaming for real-time responses:
```python Python theme={"system"}
from portkey_ai import Portkey
portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@lepton")
stream = portkey.chat.completions.create(
model="llama-3.1-8b",
messages=[{"role": "user", "content": "Write a story about a robot"}],
stream=True
)
for chunk in stream:
print(chunk.choices[0].delta.content, end="")
```
```javascript Node.js theme={"system"}
import Portkey from 'portkey-ai';
const portkey = new Portkey({
apiKey: 'PORTKEY_API_KEY',
provider: '@lepton'
});
const stream = await portkey.chat.completions.create({
model: "llama-3.1-8b",
messages: [{ role: "user", content: "Write a story about a robot" }],
stream: true
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
```
***
## Supported Models
Lepton AI provides serverless access to various models:
| Model | Description |
| ----------------- | ------------------ |
| llama-3.1-8b | Llama 3.1 8B model |
| llama-3-8b-sft-v1 | Fine-tuned Llama 3 |
| whisper-large-v3 | Speech-to-text |
Check [Lepton's documentation](https://www.lepton.ai/docs) for the complete model list.
***
## Next Steps
Add fallbacks, load balancing, and more
Monitor and trace your Lepton requests
Manage and version your prompts
Add custom metadata to requests
For complete SDK documentation:
Complete Portkey SDK documentation