> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-feat-rerank-documentation.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Together AI

> Integrate Together AI models with Portkey's AI Gateway

Portkey provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including [Together AI's hosted models](https://docs.together.ai/reference/inference).

With Portkey, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through [Model Catalog](/product/model-catalog).

## Quick Start

Get Together AI working in 3 steps:

<CodeGroup>
  ```python Python icon="python" theme={"system"}
  from portkey_ai import Portkey

  # 1. Install: pip install portkey-ai
  # 2. Add @together-ai provider in model catalog
  # 3. Use it:

  portkey = Portkey(api_key="PORTKEY_API_KEY")

  response = portkey.chat.completions.create(
      model="@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
      messages=[{"role": "user", "content": "Say this is a test"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js Javascript icon="square-js" theme={"system"}
  import Portkey from 'portkey-ai'

  // 1. Install: npm install portkey-ai
  // 2. Add @together-ai provider in model catalog
  // 3. Use it:

  const portkey = new Portkey({
      apiKey: "PORTKEY_API_KEY"
  })

  const response = await portkey.chat.completions.create({
      model: "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
      messages: [{ role: "user", content: "Say this is a test" }]
  })

  console.log(response.choices[0].message.content)
  ```

  ```python OpenAI Py icon="python" theme={"system"}
  from openai import OpenAI
  from portkey_ai import PORTKEY_GATEWAY_URL

  # 1. Install: pip install openai portkey-ai
  # 2. Add @together-ai provider in model catalog
  # 3. Use it:

  client = OpenAI(
      api_key="PORTKEY_API_KEY",  # Portkey API key
      base_url=PORTKEY_GATEWAY_URL
  )

  response = client.chat.completions.create(
      model="@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
      messages=[{"role": "user", "content": "Say this is a test"}]
  )

  print(response.choices[0].message.content)
  ```

  ```js OpenAI JS icon="square-js" theme={"system"}
  import OpenAI from "openai"
  import { PORTKEY_GATEWAY_URL } from "portkey-ai"

  // 1. Install: npm install openai portkey-ai
  // 2. Add @together-ai provider in model catalog
  // 3. Use it:

  const client = new OpenAI({
      apiKey: "PORTKEY_API_KEY",  // Portkey API key
      baseURL: PORTKEY_GATEWAY_URL
  })

  const response = await client.chat.completions.create({
      model: "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
      messages: [{ role: "user", content: "Say this is a test" }]
  })

  console.log(response.choices[0].message.content)
  ```

  ```sh cURL icon="square-terminal" theme={"system"}
  # 1. Add @together-ai provider in model catalog
  # 2. Use it:

  curl https://api.portkey.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -d '{
      "model": "@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
      "messages": [
        { "role": "user", "content": "Say this is a test" }
      ]
    }'
  ```
</CodeGroup>

<Note>
  **Tip:** You can also set `provider="@together-ai"` in `Portkey()` and use just `model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo"` in the request.
</Note>

## Add Provider in Model Catalog

1. Go to [**Model Catalog → Add Provider**](https://app.portkey.ai/model-catalog/providers)
2. Select **Together AI**
3. Choose existing credentials or create new by entering your [Together AI API key](https://api.together.ai/settings/api-keys)
4. Name your provider (e.g., `together-ai-prod`)

<Card title="Complete Setup Guide →" href="/product/model-catalog">
  See all setup options, code examples, and detailed instructions
</Card>

## Reasoning / Thinking Support

Together AI supports reasoning models that expose their internal chain of thought. Use the `reasoning_effort` parameter to control reasoning behavior, and set `strict_open_ai_compliance=False` to receive the thinking content in `content_blocks`.

<CodeGroup>
  ```python Python theme={"system"}
  from portkey_ai import Portkey

  portkey = Portkey(
      api_key="PORTKEY_API_KEY",
      strict_open_ai_compliance=False
  )

  response = portkey.chat.completions.create(
      model="@together-ai/deepseek-ai/DeepSeek-R1",
      messages=[{"role": "user", "content": "Solve step by step: What is 23 * 47?"}],
      reasoning_effort="medium"
  )

  # Access thinking content from content_blocks
  for block in response.choices[0].message.content_blocks:
      if block.get("type") == "thinking":
          print("Thinking:", block["thinking"])
      elif block.get("type") == "text":
          print("Response:", block["text"])
  ```

  ```js Javascript theme={"system"}
  import Portkey from 'portkey-ai'

  const portkey = new Portkey({
      apiKey: "PORTKEY_API_KEY",
      strictOpenAiCompliance: false
  })

  const response = await portkey.chat.completions.create({
      model: "@together-ai/deepseek-ai/DeepSeek-R1",
      messages: [{ role: "user", content: "Solve step by step: What is 23 * 47?" }],
      reasoning_effort: "medium"
  })

  // Access thinking content from content_blocks
  for (const block of response.choices[0].message.content_blocks) {
      if (block.type === "thinking") {
          console.log("Thinking:", block.thinking)
      } else if (block.type === "text") {
          console.log("Response:", block.text)
      }
  }
  ```

  ```sh cURL theme={"system"}
  curl https://api.portkey.ai/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "x-portkey-api-key: $PORTKEY_API_KEY" \
    -H "x-portkey-strict-open-ai-compliance: false" \
    -d '{
      "model": "@together-ai/deepseek-ai/DeepSeek-R1",
      "messages": [
        { "role": "user", "content": "Solve step by step: What is 23 * 47?" }
      ],
      "reasoning_effort": "medium"
    }'
  ```
</CodeGroup>

The reasoning response includes `content_blocks` with both the model's thinking process and the final answer. Streaming is also supported and returns reasoning chunks in the `content_blocks` field of the stream delta.

<Card title="Thinking Mode Documentation" icon="brain" href="/product/ai-gateway/multimodal-capabilities/thinking-mode">
  Learn more about thinking/reasoning support across providers
</Card>

## Video Generation

Together AI supports video generation through Portkey's proxy. Generate videos from text prompts and track usage in your Portkey dashboard.

<CodeGroup>
  ```python Python theme={"system"}
  from portkey_ai import Portkey

  portkey = Portkey(
      api_key="PORTKEY_API_KEY",
      provider="@together-ai"
  )

  body = {
      "prompt": "A serene sunset over the ocean with gentle waves",
      "model": "minimax/hailuo-02",
      "width": 1366,
      "height": 768
  }

  # Request to generate video
  response = portkey.post(
      url="/v2/videos",
      **body
  )

  # Request to retrieve video status
  response = portkey.get(
      path="/v2/videos/<video_id>",
  )
  ```

  ```js Javascript icon="square-js" theme={"system"}
  import Portkey from 'portkey-ai'

  const portkey = new Portkey({
      apiKey: "PORTKEY_API_KEY",
      provider: "@together-ai"
  })

  const body = {
      prompt: "A serene sunset over the ocean with gentle waves",
      model: "minimax/hailuo-02",
      width: 1366,
      height: 768
  }

  // Request to generate video
  const response = await portkey.post(
      "/v2/videos",
      body
  )

  // Request to retrieve video status (use the video id from the create response)
  // const statusResponse = await portkey.get(`/v2/videos/${videoId}`)
  ```

  ```sh cURL theme={"system"}
  curl --location 'https://api.portkey.ai/v1/v2/videos' \
    --header 'x-portkey-provider: @together-ai' \
    --header 'Content-Type: application/json' \
    --header 'x-portkey-api-key: PORTKEY_API_KEY' \
    --data '{
      "prompt": "A serene sunset over the ocean with gentle waves",
      "model": "minimax/hailuo-02",
      "width": 1366,
      "height": 768
    }'
  ```
</CodeGroup>

Video generation pricing is logged in your Portkey dashboard for cost tracking.

***

## Managing Together AI Prompts

Manage all prompt templates to Together AI in the [Prompt Library](/product/prompt-library). All current Together AI models are supported, and you can easily test different prompts.

Use the `portkey.prompts.completions.create` interface to use the prompt in an application.

## Next Steps

<CardGroup cols={2}>
  <Card title="Add Metadata" icon="tags" href="/product/observability/metadata">
    Add metadata to your Together AI requests
  </Card>

  <Card title="Gateway Configs" icon="gear" href="/product/ai-gateway/configs">
    Add gateway configs to your Together AI requests
  </Card>

  <Card title="Tracing" icon="chart-line" href="/product/observability/traces">
    Trace your Together AI requests
  </Card>

  <Card title="Fallbacks" icon="arrow-rotate-left" href="/product/ai-gateway/fallbacks">
    Setup fallback from OpenAI to Together AI
  </Card>
</CardGroup>

For complete SDK documentation:

<Card title="SDK Reference" icon="code" href="/api-reference/portkey-sdk-client">
  Complete Portkey SDK documentation
</Card>