> ## Documentation Index
> Fetch the complete documentation index at: https://portkey-docs-feat-rerank-documentation.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Service

<Update label="1.6.0" description="2026-04-09">
  ## v1.6.0

  ***

  ### Fixes and Improvements

  * Updated dependencies to patch security vulnerabilities.
  * Fixed `Log Exports` memory leak.
</Update>

<Update label="1.5.2" description="2026-03-17">
  ## v1.5.2

  ***

  ### Fixes and Improvements

  * Relaxed validation for `completion_window` in provider batches
  * Updated dependencies to patch security vulnerabilities.
</Update>

<Update label="1.5.1" description="2026-02-11">
  ## v1.5.1

  ***

  ### Fixes and Improvements

  * Updated dependencies to patch security vulnerabilities.
</Update>

<Update label="1.5.0" description="2026-02-04">
  ## v1.5.0

  ***

  ### Pricing Improvements

  * Added support for dedicated batch pricing configurations. The exact pricing will be calculated from Gateway.
    <Info>This is a breaking change. Requires `Enterprise Gateway` version v2.1.0 or higher.</Info>

  ### Improvements

  * Security updates to dependencies.
</Update>

<Update label="1.4.4" description="2025-12-13">
  ## v1.4.4

  ***

  ### Improvements

  * Security updates to dependencies.
</Update>

<Update label="1.4.3" description="2025-12-10">
  ## v1.4.3

  ***

  ### Batches Improvements

  * Added azure responses batches pricing support.
</Update>

<Update label="1.4.2" description="2025-12-03">
  ## v1.4.2

  ***

  ### Fixes and Improvements

  * Added new batch statuses for batch jobs.
    * `file_upload_failed` - the batch job failed to upload the file to the provider after validation
    * `batch_start_failed` - the file upload succeeded but the batch job submission to the provider failed
</Update>

<Update label="1.4.1" description="2025-11-19">
  ## v1.4.1

  ***

  ### Fixes and Improvements

  * Enhanced log exports functionality to read logs based on the path format identifier released in Gateway v1.17.0
</Update>

<Update label="1.4.0" description="2025-11-17">
  ## v1.4.0

  ***

  <Note>
    **Requires a Helm repo update (>app-1.4.0)**
  </Note>

  <Note>
    **For air-gapped deployments, `Backend` version v1.5.0 is required as it adds new columns in the analytics store**
  </Note>

  ### Security Patch

  * Removed root user from container image (BREAKING CHANGE). The container image used by this chart no longer runs as root. The image now runs processes with a non-root UID and enforces a non-root container securityContext. Requires Helm repo upgrade (>app-1.4.0) to deploy the new image and chart settings.

  ### Fixes and Improvements

  * Added cost calculation logic for fine-tuning operations across multiple AI providers (OpenAI, Azure OpenAI, and Vertex AI)
  * Improved error handling to preserve provider-level file upload failures during batch processing
</Update>

<Update label="1.3.0" description="2025-10-16">
  ## v1.3.0

  ***

  ### Fixes and Improvements

  * Support KMS key support for custom batch file output.
  * Support batch operations for tracking provider batch requests from gateway.
    <Info>Works with Gateway version 1.16.0 or higher.</Info>
</Update>

<Update label="1.2.8" description="2025-10-01">
  ## v1.2.8

  ***

  ### Fixes and Improvements

  * Fixed redundant failed batch status update for batch jobs.
  * Fixed - Retry failed job status check and update status accordingly.
</Update>

<Update label="1.2.7" description="2025-08-16">
  ## v1.2.7

  ***

  ### Fixes and Improvements

  * Fixed encoding issues for file uploads to S3.
</Update>

<Update label="1.2.6" description="2025-07-22">
  ## v1.2.6

  ***

  ### Features

  * Azure Redis support for cache with auth modes including Entra and Managed Identity.
  * HTTPS Proxy support for all the external calls & HTTPs communication between PODs.
  * Added support for virtual key inclusion for custom log if passed in headers.
  * Entra and Managed Identity support for Azure Log Store.

  ### Fixes and Improvements

  * Support Batch output from gateway directly.
</Update>

<Update label="1.2.5" description="2025-06-28">
  ## v1.2.5

  ***

  ### Fixes and Improvements

  * Update dependencies for better performance for file uploads/downloads.
</Update>

<Update label="1.2.4" description="2025-06-17">
  ## v1.2.4

  ***

  ### Fixes and Improvements

  * Fixed issue with custom batches missing cost calculation for some provider models
</Update>

<Update label="1.2.3" description="2025-06-17">
  ## v1.2.3

  ***

  ### Fine-tuning and Batch Processing

  * Added support for configurable `FINETUNE_STATUS_CHECK_INTERVAL` for provider fine-tuning status check operations.
  * Added support for configurable `BATCH_STATUS_CHECK_INTERVAL` for provider batch processing status check operations.
  * Both values should be in milliseconds. Minimum value is 10000 milliseconds.
  * If not provided, will default to 10 seconds.
</Update>

<Update label="1.2.2" description="2025-05-31">
  ## v1.2.2

  ***

  ### Observability

  * Added support for below Prometheus Counters
    * `batch_count`
    * `batch_cost`
    * `batch_input_tokens`
    * `batch_total_tokens`
    * `batch_process_time`
    * `batch_success_row_count`
    * `batch_failure_row_count`
    * `batch_row_count`
  * With the below labels
    * `provider`
    * `type` (provider/custom)

  ### Fixes and Improvements

  * Fixed issue with attributing incorrect created at time stamp for batch processing
  * Including error source as `control plane` for control plane failures
</Update>

<Update label="1.2.1" description="2025-05-21">
  ## v1.2.1

  ***

  ### Data exports

  * Added support for [Data exports](/api-reference/admin-api/data-plane/logs/log-exports-beta/start-a-log-export) for hybrid deployments.

  ### Fixes and Improvements

  * Fixed issue with custom batches for small batch files
</Update>

<Update label="1.2.0" description="2025-05-17">
  ## v1.2.0

  ***

  ### Custom S3 Support

  * Added support for `s3_custom` log store option for batches and fine-tunes.

  ### Fixes and Improvements

  * Fixed issue with STS token generation for AWS.
</Update>

<Update label="1.1.12" description="2025-05-09">
  ## v1.1.12

  ***

  ### Fixes and Improvements

  * Fixed issue with cost calculation for custom batches.
</Update>

<Update label="1.1.11" description="2025-05-03">
  ## v1.1.11

  ***

  ### Fixes and Improvements

  * Fixed issue where queue remains stuck in a queued state during file validation.
</Update>

<Update label="1.1.10" description="2025-04-21">
  ## v1.1.10

  ***

  ### S3 Upload Improvements

  * Added support for passing encryption headers while uploading stream data to S3.
  * Added support for both file path and direct value from environment variables for secrets like redis connection.

  ### Stream Handling

  * Improved stream cleanup for validation processes.
</Update>

<Update label="1.1.9" description="2025-04-10">
  ## v1.1.9

  ***

  ### File Handling Fixes

  * Fixed issue with extra bytes being added to files during processing.
</Update>

<Update label="1.1.8" description="2025-04-04">
  ## v1.1.8

  ***

  ### File Upload Improvements

  * Updated socket timeout for long requests during file uploads to prevent timeouts.
</Update>

<Update label="1.1.7" description="2025-03-28">
  ## v1.1.7

  ***

  ### Fireworks Fine-tuning Support

  * Added support for `Fireworks` fine-tuning operations using Version2.

  ### Batch Processing Improvements

  * Included response tokens calculation in provider batch output.
  * Fixed file loading in memory issues for better performance.
</Update>

<Update label="1.1.6" description="2025-03-25">
  ## v1.1.6

  ***

  ### S3 SDK Updates

  * Upgraded S3 SDK to latest version for fixing issue with S3 streaming.
</Update>

<Update label="1.1.5" description="2025-03-21">
  ## v1.1.5

  ***

  ### Batch Processing Enhancements

  * Improved provider batch output handling.
  * Added support for custom batch output paths.
  * Increased maximum lines for custom batches to 500k and chunk size to 5MB for better performance.
</Update>

<Update label="1.1.4" description="2025-03-20">
  ## v1.1.4

  ***

  ### Vertex Embeddings Batches Support

  * Added support for `Vertex` batch embeddings.

  ### Batch Processing Updates

  * Included model information in log objects.
  * Implemented custom batch processing output generations.

  ### Internal POD to POD HTTPS Support

  * Added support for internal POD to POD HTTPS communication.
  * This can be enabled by mounting a volume with certificate and key.
  * `TLS_KEY_PATH` and `TLS_CERT_PATH` environment variables will be used to fetch the certificate and key from the volume.
</Update>

<Update label="1.1.3" description="2025-03-07">
  ## v1.1.3

  ***

  ### Infrastructure Updates

  * Streamlined uploaded file location for `Bedrock` operations.
</Update>

<Update label="1.1.2" description="2025-02-27">
  ## v1.1.2

  ***

  ### Vertex Integration

  * Added support for `Vertex` provider options for batches.

  ### Infrastructure Updates

  * Implemented cluster mode Redis for queues.
  * Updated fine-tune status handling.
</Update>

<Update label="1.1.1" description="2025-02-20">
  ## v1.1.1

  ***

  ### S3 Enhancements

  * Made S3 bucket optional for Bedrock batches.
  * Added S3 encryption header support for finetunes and batches.
  * Implemented SSE file upload support.

  ### Logging Improvements

  * Added filtering for log exports.
  * Implemented end limit for log export records.

  ### Performance Optimizations

  * Implemented internal memory cache for better performance.
</Update>

<Update label="1.1.0" description="2025-02-10">
  ## v1.1.0

  ***

  ### Bull Board Integration

  * Added Bull Board for visualizing job queues and their status.

  ### Batch Job Retry Support

  * Implemented retry functionality for batch jobs to handle failures gracefully.

  ### Prometheus Metrics Enhancements

  * Added Prometheus metrics for batch jobs and fine-tuning operations.
</Update>

<Update label="1.0.8" description="2025-02-04">
  ## v1.0.8

  ***

  ### Azure Fine-tuning Support

  * Added support for `Azure OpeAI` fine-tuning operations.
</Update>

<Update label="1.0.7" description="2025-01-13">
  ## v1.0.7

  ***

  ### Fine-tune v2

  * Implemented version 2 of the [fine-tuning](/product/ai-gateway/fine-tuning) functionality.
</Update>

<Update label="1.0.6" description="2024-01-08">
  ## v1.0.6

  ***

  ### Prompt Slug Filter

  * Added support for data exports filtering by PromptSlug.
</Update>

<Update label="1.0.5" description="2024-01-06">
  ## v1.0.5

  ***

  ### Batch Processing

  * Added provider and custom \[Batch] (/product/ai-gateway/batches) processing functionality.
</Update>

<Update label="1.0.4" description="2023-12-17">
  ## v1.0.4

  ***

  ### Code Quality Improvements

  * Fixed dynamic port retrieval from environment variables.
</Update>

<Update label="1.0.3" description="2023-11-28">
  ## v1.0.3

  ***

  ### Vision Fine-tuning Support

  * Added support for vision fine-tuning validation for OpenAI.
  * Implemented S3 bucket support for fine-tunes.

  ### AWS Integration Improvements

  * Fixed assumed role handling for Bedrock fine-tuning dataset URLs.
  * Improved S3 bucket path handling for Bedrock fine-tune operations.
  * Achieved parity with Enterprise Gateway for data sources.
</Update>

<Update label="1.0.2" description="2023-11-20">
  ## v1.0.2

  ***

  ### Fine-tuning Enhancements

  * Added support for OpenAI job start and Fireworks upload.
  * Improved handling of chunk type failures with JSON.
</Update>

<Update label="1.0.1" description="2023-10-25">
  ## v1.0.1

  ***

  ### Fireworks Fine-tuning Support

  * Added support for Fireworks fine-tuning operations.
</Update>

<Update label="1.0.0" description="2023-10-04">
  ## v1.0.0

  ***

  ### Initial Release

  * Base version of the Data Service with core functionality.
</Update>