LLM Service

The LLM Service provides access to leading AI language models from OpenAI and Anthropic. Generate text completions in real-time for chatbots, content generation, and AI features, or use batch processing for high-volume tasks at reduced cost. All API calls use your own provider credentials for complete control and transparency.

What you can do:

  • Generate text completions with GPT-4, Claude, and other models
  • Process bulk requests with batch API (50% cost savings)
  • Configure multiple LLM providers (OpenAI, Anthropic)
  • Track usage with detailed logs and analytics
  • Map model names for easy provider switching

Interactive API Reference

For complete API documentation with interactive testing, visit our Swagger UI where you can try all endpoints with your own credentials.

View Interactive API Docs

Important: Before using the LLM API, you must configure an LLM provider (OpenAI or Anthropic) with your API credentials. You can do this through the API or in your dashboard.

Create Text Completion

Generate AI text responses in real-time for chatbots, content generation, code assistance, and more. The service automatically routes your request to the appropriate provider (OpenAI or Anthropic) based on the model you specify. Perfect for interactive features where users expect immediate responses. 

curl -X POST https://llm.stackfordevs.com/v1/completions \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-secret-key: YOUR_SECRET_KEY" \
  -H "x-tenant-id: YOUR_TENANT_ID" \
  -H "x-project-id: YOUR_PROJECT_ID" \
  -d '{
    "model": "gpt-4",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Explain quantum computing in simple terms."
      }
    ],
    "max_tokens": 500,
    "temperature": 0.7
  }'

Parameters:

  • model (required) - Model identifier
  • messages (required) - Conversation history with role and content
  • max_tokens (optional) - Maximum response length
  • temperature (optional) - Randomness (0-2, default 0.7)

Batch Processing

Process large volumes of completion requests asynchronously at up to 50% reduced cost compared to real-time API calls. Perfect for data analysis, content generation pipelines, or any non-interactive bulk processing. Batch jobs typically complete within 24 hours. 

curl -X POST https://llm.stackfordevs.com/v1/batch \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-secret-key: YOUR_SECRET_KEY" \
  -H "x-tenant-id: YOUR_TENANT_ID" \
  -H "x-project-id: YOUR_PROJECT_ID" \
  -d '{
    "requests": [
      {
        "custom_id": "request-1",
        "model": "gpt-4",
        "messages": [
          {
            "role": "user",
            "content": "Summarize the latest AI trends."
          }
        ]
      },
      {
        "custom_id": "request-2",
        "model": "gpt-4",
        "messages": [
          {
            "role": "user",
            "content": "Explain machine learning."
          }
        ]
      }
    ],
    "webhook_url": "https://yourapp.com/batch-complete"
  }'

Tip: Use custom_id to track each request. Optionally provide a webhook_url to receive a notification when the batch completes.

List Batch Jobs

View all batch jobs you've submitted, optionally filtered by status. Use this to monitor progress, check completion status, or retrieve job IDs for fetching results. 

curl -X GET "https://llm.stackfordevs.com/v1/batch?status=completed&limit=20" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-secret-key: YOUR_SECRET_KEY" \
  -H "x-tenant-id: YOUR_TENANT_ID" \
  -H "x-project-id: YOUR_PROJECT_ID"

Status Values:

  • pending - Waiting to be processed
  • processing - Currently being processed
  • completed - Finished successfully
  • failed - Processing failed

Get Batch Results

Retrieve the results of a completed batch job. Results include the AI-generated responses for each request in your batch, matched by the custom_id you provided. Download and process these results to power your application's features. 

curl -X GET https://llm.stackfordevs.com/v1/batch/batch_abc123/results \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-secret-key: YOUR_SECRET_KEY" \
  -H "x-tenant-id: YOUR_TENANT_ID" \
  -H "x-project-id: YOUR_PROJECT_ID"

Note: Returns 425 Too Early if the batch isn't completed yet. Check the job status first or wait for the webhook notification.

Cancel Batch Job

Cancel a pending or processing batch job if you no longer need the results. This can help avoid unnecessary costs for jobs that were submitted by mistake or are no longer relevant. 

curl -X POST https://llm.stackfordevs.com/v1/batch/batch_abc123/cancel \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-secret-key: YOUR_SECRET_KEY" \
  -H "x-tenant-id: YOUR_TENANT_ID" \
  -H "x-project-id: YOUR_PROJECT_ID"

Note: Returns 409 Conflict if the batch has already completed or cannot be cancelled.

Configure LLM Provider

Set up OpenAI or Anthropic as your LLM provider by providing your API credentials. The service uses your credentials to make API calls, giving you full control over billing, rate limits, and provider selection. You can configure multiple providers and the service will route requests based on the model specified. 

curl -X POST https://llm.stackfordevs.com/v1/providers \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-secret-key: YOUR_SECRET_KEY" \
  -H "x-tenant-id: YOUR_TENANT_ID" \
  -H "x-project-id: YOUR_PROJECT_ID" \
  -d '{
    "providerType": "openai",
    "apiKey": "sk-your-openai-api-key"
  }'

Supported Providers:

  • OpenAI: Requires apiKey from platform.openai.com
  • Anthropic: Requires apiKey from console.anthropic.com

Test Provider Connection

After configuring a provider, send a test request to verify your credentials are valid and the connection works. This helps catch configuration issues before making real completion requests. 

curl -X POST https://llm.stackfordevs.com/v1/providers/test \
  -H "Content-Type: application/json" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-secret-key: YOUR_SECRET_KEY" \
  -H "x-tenant-id: YOUR_TENANT_ID" \
  -H "x-project-id: YOUR_PROJECT_ID" \
  -d '{
    "providerType": "openai"
  }'

View Completion Logs

Track all completion requests made through the service. Logs include the prompt, response, model used, token counts, and timing information. Use this for debugging, auditing, and understanding usage patterns. 

curl -X GET https://llm.stackfordevs.com/v1/logs/YOUR_TENANT_ID/YOUR_PROJECT_ID \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-secret-key: YOUR_SECRET_KEY" \
  -H "x-tenant-id: YOUR_TENANT_ID" \
  -H "x-project-id: YOUR_PROJECT_ID"

Usage Statistics

Get aggregated statistics about your LLM usage—total requests, tokens consumed, costs by model, success rates, and more. Use this to monitor spending, identify optimization opportunities, and track usage trends over time. 

curl -X GET https://llm.stackfordevs.com/v1/stats/YOUR_TENANT_ID/YOUR_PROJECT_ID \
  -H "x-api-key: YOUR_API_KEY" \
  -H "x-secret-key: YOUR_SECRET_KEY" \
  -H "x-tenant-id: YOUR_TENANT_ID" \
  -H "x-project-id: YOUR_PROJECT_ID"

Error Codes

The LLM API uses standard HTTP status codes.

Status Description
200 Success - Completion generated or request completed
201 Created - Batch job created successfully
204 No Content - Resource deleted successfully
400 Bad Request - Invalid parameters or missing required fields
401 Unauthorized - Missing or invalid API keys
404 Not Found - Batch job or resource doesn't exist
409 Conflict - Batch job cannot be cancelled
425 Too Early - Batch results not yet available
500 Server Error - LLM provider error or service issue