Panama/user/agents/skills/infisical-api/references/pagination-and-rate-limits.md

Pagination and Rate Limits

Pagination

Infisical uses offset-based pagination for list endpoints. All responses include pagination metadata.

Pagination Parameters

Parameter	Type	Default	Max	Description
offset	integer	0	-	Number of items to skip from the beginning
limit	integer	20	100	Maximum number of items to return in this request

Pagination Response

{
  "items": [...],
  "total": 150,
  "offset": 0,
  "limit": 20
}

• total: Total count of all available items (ignoring pagination)
• offset: Requested offset
• limit: Requested limit (may be less if fewer items available)
• items: Array of results for this page

Example: Paginating Through All Results

#!/bin/bash

# Retrieve all secrets in batches of 20
offset=0
limit=20
total=-1

while [ $offset -lt $total ] || [ $total -eq -1 ]; do
  response=$(curl -s "https://us.infisical.com/api/v4/secrets?projectId=abc123&environment=dev&offset=$offset&limit=$limit" \
    -H "Authorization: Bearer TOKEN")
  
  # Extract items and total from response
  total=$(echo $response | jq '.total')
  items=$(echo $response | jq '.secrets[]')
  
  # Process items
  echo "Processing items $offset to $((offset + limit))..."
  
  offset=$((offset + limit))
done

Pagination Best Practices

1. Start with offset=0: Always begin pagination at offset 0
2. Use maximum limit: Set limit=100 for faster retrieval (unless you need fewer items)
3. Check total: Use the total value to determine if more pages exist: hasMore = (offset + limit) < total
4. Handle edge cases: Always check if limit in response is less than requested (indicates fewer items available)
5. Respect rate limits: Add delays between requests if hitting rate limits

Rate Limits (Cloud Only)

Infisical Cloud deployments have rate limits. Self-hosted deployments have no rate limits.

Rate Limit Types

Read Operations (GET, LIST)

• Free Tier: 200 reads per minute
• Pro Tier: 350 reads per minute
• Enterprise: Custom limits

Write Operations (CREATE, UPDATE, DELETE)

• Free Tier: 90 writes per minute
• Pro Tier: 200 writes per minute
• Enterprise: Custom limits

Secret Operations (All /api/v4/secrets/* endpoints)

• Free Tier: 120 secret ops per minute
• Pro Tier: 300 secret ops per minute
• Enterprise: Custom limits

Rate Limit Response Headers

When you hit a rate limit, the API returns HTTP 429 (Too Many Requests):

HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 200
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1713350400
Content-Type: application/json

{
  "statusCode": 429,
  "message": "Too many requests, please try again later."
}

• X-RateLimit-Limit: Maximum requests allowed in the window
• X-RateLimit-Remaining: Requests remaining in the current window
• X-RateLimit-Reset: Unix timestamp when the limit resets

Handling Rate Limits

Implement Exponential Backoff

async function makeRequestWithRetry(url, options, maxRetries = 3) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    const response = await fetch(url, options);
    
    if (response.status === 429) {
      const resetTime = parseInt(response.headers.get('X-RateLimit-Reset')) * 1000;
      const delayMs = Math.max(resetTime - Date.now(), 1000 * Math.pow(2, attempt - 1));
      
      console.log(`Rate limited. Waiting ${delayMs}ms before retry...`);
      await new Promise(resolve => setTimeout(resolve, delayMs));
      continue;
    }
    
    if (!response.ok) {
      throw new Error(`HTTP ${response.status}: ${response.statusText}`);
    }
    
    return response.json();
  }
  throw new Error('Max retries exceeded');
}

Monitor Rate Limit Usage

curl -s 'https://us.infisical.com/api/v4/secrets?projectId=abc123&environment=dev&limit=1' \
  -H "Authorization: Bearer TOKEN" \
  -w "\nRate Limit Remaining: %{http_header{X-RateLimit-Remaining}}\n"

Batch Operations

Group multiple operations to reduce request count:

# Instead of 100 DELETE requests, use one batch delete
curl -X DELETE 'https://us.infisical.com/api/v4/secrets/batch' \
  -H "Authorization: Bearer TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "projectId": "abc123",
    "environment": "dev",
    "secretPath": "/",
    "secretIds": ["id1", "id2", "id3", ...]
  }'

Request Queuing

Implement a request queue to spread requests over time:

import asyncio
import aiohttp
from collections import deque

class RateLimitedClient:
    def __init__(self, requests_per_minute=200):
        self.requests_per_minute = requests_per_minute
        self.min_interval = 60 / requests_per_minute
        self.last_request_time = 0
        self.queue = deque()
    
    async def request(self, session, method, url, **kwargs):
        # Wait if necessary to maintain rate limit
        elapsed = asyncio.get_event_loop().time() - self.last_request_time
        if elapsed < self.min_interval:
            await asyncio.sleep(self.min_interval - elapsed)
        
        async with session.request(method, url, **kwargs) as response:
            self.last_request_time = asyncio.get_event_loop().time()
            return await response.json()

Required Headers

All API requests must include:

Content-Type: application/json
Authorization: Bearer YOUR_ACCESS_TOKEN

Example: Complete Request with Headers

curl -X GET 'https://us.infisical.com/api/v4/secrets?projectId=abc123&environment=dev&offset=0&limit=20' \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGc..."

HTTP Status Codes

Code	Meaning	When It Occurs
200	OK	Successful GET, PATCH, DELETE
201	Created	Successful POST
400	Bad Request	Invalid parameters or request body
401	Unauthorized	Missing or invalid token
403	Forbidden	Insufficient permissions
404	Not Found	Resource doesn't exist
409	Conflict	Duplicate secret name or resource conflict
429	Too Many Requests	Rate limit exceeded (cloud only)
500	Internal Error	Server error

Performance Tips

1. Use pagination: Limit each request to 100 items maximum
2. Cache responses: Store secret values locally to reduce API calls
3. Use appropriate timeouts: Set 30-second timeouts for API calls
4. Batch operations: Combine multiple operations into single requests where possible
5. Monitor headers: Check X-RateLimit-Remaining to anticipate throttling
6. Implement exponential backoff: Automatically retry failed requests with increasing delays
7. Use webhooks: Subscribe to changes instead of polling for updates (if available)

Example: Comprehensive Pagination with Error Handling

#!/bin/bash

PROJECT_ID="abc123"
ENVIRONMENT="dev"
API_BASE="https://us.infisical.com"
TOKEN="your_access_token"
BATCH_SIZE=100

offset=0
total_processed=0

while true; do
  # Make request with error handling
  response=$(curl -s -w "\n%{http_code}" \
    "$API_BASE/api/v4/secrets?projectId=$PROJECT_ID&environment=$ENVIRONMENT&offset=$offset&limit=$BATCH_SIZE" \
    -H "Authorization: Bearer $TOKEN" \
    -H "Content-Type: application/json")
  
  # Extract body and status code
  http_code=$(echo "$response" | tail -n1)
  body=$(echo "$response" | head -n-1)
  
  # Check for errors
  if [ "$http_code" = "429" ]; then
    reset_time=$(curl -s -I "$API_BASE/api/v4/secrets?projectId=$PROJECT_ID&environment=$ENVIRONMENT" \
      -H "Authorization: Bearer $TOKEN" | grep X-RateLimit-Reset | awk '{print $2}')
    echo "Rate limited. Waiting until $reset_time..."
    sleep 60
    continue
  elif [ "$http_code" != "200" ]; then
    echo "Error: HTTP $http_code"
    echo "$body" | jq .
    exit 1
  fi
  
  # Process response
  total=$(echo "$body" | jq '.total')
  count=$(echo "$body" | jq '.secrets | length')
  
  echo "Processing items $offset-$((offset + count)) of $total..."
  
  # Do something with the secrets
  echo "$body" | jq '.secrets[] | .secretName'
  
  total_processed=$((total_processed + count))
  
  # Check if we've retrieved all items
  if [ $total_processed -ge $total ]; then
    break
  fi
  
  offset=$((offset + BATCH_SIZE))
done

echo "Processed $total_processed items total"