> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-feat-rerank-documentation.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Hugging Face

Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including all the text generation models supported by [Huggingface's Inference endpoints](https://huggingface.co/docs/api-inference/index).

With Portkey, you can take advantage of features like fast AI gateway access, observability, prompt management, and more, all while ensuring the secure management of your LLM API keys through a [virtual key](/product/ai-gateway/virtual-keys) system.

<Note>
  Provider Slug. `huggingface`
</Note>

## Portkey SDK Integration with Huggingface

Portkey provides a consistent API to interact with models from various providers. To integrate Huggingface with Portkey:

### 1. Install the Portkey SDK

Add the Portkey SDK to your application to interact with Huggingface's API through Portkey's gateway.

<Tabs>
  <Tab title="NodeJS">
    ```sh theme={"system"}
    npm install --save portkey-ai
    ```
  </Tab>

  <Tab title="Python">
    ```sh theme={"system"}
    pip install portkey-ai
    ```
  </Tab>
</Tabs>

### 2. Initialize Portkey with the Virtual Key

To use Huggingface with Portkey, [get your Huggingface Access token from here](https://huggingface.co/settings/tokens), then add it to Portkey to create the virtual key.

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    import Portkey from 'portkey-ai'

    const portkey = new Portkey({
        apiKey: "PORTKEY_API_KEY", // defaults to process.env["PORTKEY_API_KEY"]
        virtualKey: "VIRTUAL_KEY", // Your Huggingface Virtual Key
        huggingfaceBaseUrl: "HUGGINGFACE_DEDICATED_URL" // Optional: Use this if you have a dedicated server hosted on Huggingface
    })
    ```
  </Tab>

  <Tab title="Python SDK">
    ```py theme={"system"}
    from portkey_ai import Portkey

    portkey = Portkey(
        api_key="PORTKEY_API_KEY",  # Replace with your Portkey API key
        virtual_key="VIRTUAL_KEY",   # Replace with your virtual key for Huggingface
        huggingface_base_url="HUGGINGFACE_DEDICATED_URL" # Optional: Use this if you have a dedicated server hosted on Huggingface
    )
    ```
  </Tab>

  <Tab title="OpenAI Python SDK">
    ```py theme={"system"}
    from openai import OpenAI
    from portkey_ai import PORTKEY_GATEWAY_URL, createHeaders

    client = OpenAI(
        api_key="HUGGINGFACE_ACCESS_TOKEN",
        base_url=PORTKEY_GATEWAY_URL,
        default_headers=createHeaders(
            api_key="PORTKEY_API_KEY",
            provider="huggingface",
            huggingface_base_url="HUGGINGFACE_DEDICATED_URL"
        )
    )
    ```
  </Tab>

  <Tab title="OpenAI Node SDK">
    ```js theme={"system"}
    import OpenAI from "openai";
    import { PORTKEY_GATEWAY_URL, createHeaders } from "portkey-ai";

    const client = new OpenAI({
      apiKey: "HUGGINGFACE_ACCESS_TOKEN",
      baseURL: PORTKEY_GATEWAY_URL,
      defaultHeaders: createHeaders({
        provider: "huggingface",
        apiKey: "PORTKEY_API_KEY",
        huggingfaceBaseUrl: "HUGGINGFACE_DEDICATED_URL"
      }),
    });
    ```
  </Tab>
</Tabs>

### **3. Invoke Chat Completions with** Huggingface

Use the Portkey instance to send requests to Huggingface. You can also override the virtual key directly in the API call if needed.

<Tabs>
  <Tab title="NodeJS SDK">
    ```js theme={"system"}
    const chatCompletion = await portkey.chat.completions.create({
        messages: [{ role: 'user', content: 'Say this is a test' }],
        model: 'meta-llama/Meta-Llama-3.1-8B-Instruct', // make sure your model is hot
    });

    console.log(chatCompletion.choices[0].message.content);
    ```
  </Tab>

  <Tab title="Python SDK">
    ```py theme={"system"}
    chat_completion = portkey.chat.completions.create(
        messages= [{ "role": 'user', "content": 'Say this is a test' }],
        model= 'meta-llama/meta-llama-3.1-8b-instruct', # make sure your model is hot
    )

    print(chat_completion.choices[0].message.content)
    ```
  </Tab>

  <Tab title="OpenAI Python SDK">
    ```py theme={"system"}
    chat_completion = client.chat.completions.create(
        messages = [{ "role": 'user', "content": 'Say this is a test' }],
        model = 'meta-llama/meta-llama-3.1-8b-instruct', # make sure your model is hot
    )

    print(chat_completion.choices[0].message.content)
    ```
  </Tab>

  <Tab title="OpenAI Node SDK">
    ```js theme={"system"}
    async function main() {
        const chatCompletion = await client.chat.completions.create({
            model: "meta-llama/meta-llama-3.1-8b-instruct", // make sure your model is hot
            messages: [{ role: "user", content: "How many points to Gryffindor?" }],
        });
        console.log(chatCompletion.choices[0].message.content);
    }

    main();
    ```
  </Tab>

  <Tab title="OpenAI Python SDK">
    ```py theme={"system"}
    chat_completion = client.chat.completions.create(
        messages = [{ "role": 'user', "content": 'Say this is a test' }],
        model = 'meta-llama/meta-llama-3.1-8b-instruct', # make sure your model is hot
    )

    print(chat_completion.choices[0].message.content)
    ```
  </Tab>
</Tabs>

## [Using Virtual Keys](https://app.portkey.ai/virtual-keys)

Virtual Keys serve as Portkey's unified authentication system for all LLM interactions, simplifying the use of multiple providers and Portkey features within your application. For self-hosted LLMs, you can configure custom authentication requirements including authorization keys, bearer tokens, or any other headers needed to access your model:

<Frame>
  <img src="https://mintcdn.com/portkey-docs-feat-rerank-documentation/igF6AIVPgbUAWApF/images/private-llm.png?fit=max&auto=format&n=igF6AIVPgbUAWApF&q=85&s=7e718322a7c65c9c20f02c11add60286" width="1174" height="914" data-path="images/private-llm.png" />
</Frame>

1. Navigate to [Virtual Keys](https://app.portkey.ai/virtual-keys) in your Portkey dashboard
2. Click **"Add Key"** and enable the **"Local/Privately hosted provider"** toggle
3. Configure your deployment:
   * Select the matching provider API specification (typically `OpenAI`)
   * Enter your model's base URL in the `Custom Host` field
   * Add required authentication headers and their values
4. Click **"Create"** to generate your virtual key

You can now use this virtual key in your requests:

<Tabs>
  <Tab title="NodeJS">
    ```js theme={"system"}
    const portkey = new Portkey({
        apiKey: "PORTKEY_API_KEY",
        virtualKey: "YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY"

    async function main() {
      const response = await client.chat.completions.create({
        messages: [{ role: "user", content: "Bob the builder.." }],
        model: "your-self-hosted-model-name",
      });

    console.log(response.choices[0].message.content);
    })
    ```
  </Tab>

  <Tab title="Python">
    ```python theme={"system"}
    portkey = Portkey(
        api_key="PORTKEY_API_KEY",
        virtual_key="YOUR_SELF_HOSTED_LLM_VIRTUAL_KEY"
    )

    response = portkey.chat.completions.create(
      model="your-self-hosted-model-name",
      messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"}
      ]

    print(response)
    )
    ```
  </Tab>
</Tabs>

For more information about managing self-hosted LLMs with Portkey, see [Bring Your Own LLM](/integrations/llms/byollm).

# Next Steps

The complete list of features supported in the SDK are available on the link below.

<Card title="SDK" href="/api-reference/portkey-sdk-client" />

You'll find more information in the relevant sections:

1. [Add metadata to your requests](/product/observability/metadata)
2. [Add gateway configs to your Huggingface requests](/product/ai-gateway/configs)[ requests](/product/ai-gateway/configs)
3. [Tracing Huggingface requests](/product/observability/traces)
4. [Setup a fallback from OpenAI to Huggingface APIs](/product/ai-gateway/fallbacks)