Cohere

Integrate Cohere models with Portkey’s AI Gateway

Portkey provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including Cohere’s generation, embedding, and reranking endpoints.

With Portkey, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through Model Catalog.

Quick Start

Get Cohere working in 3 steps:

Python

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @cohere provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@cohere/command-r-plus",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)

Tip: You can also set provider="@cohere" in Portkey() and use just model="command-r-plus" in the request.

Add Provider in Model Catalog

1. Go to Model Catalog → Add Provider
2. Select Cohere
3. Choose existing credentials or create new by entering your Cohere API key
4. Name your provider (e.g., cohere-prod)

Complete Setup Guide →

See all setup options, code examples, and detailed instructions

Other Cohere Endpoints

Embeddings

Embedding endpoints are natively supported within Portkey:

Python

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@cohere"
)

embedding = portkey.embeddings.create(
    input="Name the tallest buildings in Hawaii",
    model="embed-english-v3.0"
)

print(embedding)

Re-rank

Portkey supports Cohere’s reranking models through a unified /v1/rerank endpoint.

curl https://api.portkey.ai/v1/rerank \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@cohere-prod/rerank-v3.5",
    "query": "What is the capital of the United States?",
    "documents": [
      "Carson City is the capital city of the American state of Nevada.",
      "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
      "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
      "Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."
    ],
    "top_n": 3
  }'

Cohere-Specific Parameters

Parameter	Type	Description
max_tokens_per_doc	integer	Maximum number of tokens per document for processing
priority	integer	Request priority level
=======
Use Cohere reranking with the portkey.post method and the body expected by Cohere’s reranking API:

Python

from portkey_ai import Portkey

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    provider="@cohere"
)

response = portkey.post(
    "/rerank",
    return_documents=False,
    max_chunks_per_doc=10,
    model="rerank-english-v2.0",
    query="What is the capital of the United States?",
    documents=[
        "Carson City is the capital city of the American state of Nevada.",
        "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
        "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
        "Capital punishment (the death penalty) has existed in the United States since before the United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."
    ]
)

print(response)

Managing Cohere Prompts

Manage all prompt templates to Cohere in the Prompt Library. All current Cohere models are supported, and you can easily test different prompts.

Use the portkey.prompts.completions.create interface to use the prompt in an application.

Next Steps

Add Metadata

Add metadata to your Cohere requests

Gateway Configs

Add gateway configs to your Cohere requests

Tracing

Trace your Cohere requests

Fallbacks

Setup fallback from OpenAI to Cohere

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation