Querying an Inference Endpoint

The Playground

The FlexAI Inference Playground is a convenient way to interact with your deployed models without needing to write any code —or cURL requests. It allows you to test your Inference Endpoint using a user-friendly interface.

It is a FlexAI-hosted instance of Chainlit 🔗, a UI tool for interacting with multimodal AI models.

You can get the Inference Playground URL by opening the Inference Endpoint’s Details drawer menu. You will find the URL under the default Details tab’s Summary section.

Querying the Inference Endpoint through code

As long as you have the correct Endpoint URL, Access key, model name, and the right request body format, you can use any HTTP client to interact with your Inference Endpoint.

cURL

Request:

curl Request

curl -X POST https://inference-e0ed242b-dbd3-423e-b525-76beac222d44-7de0c0f4.platform.flex.ai/v1/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer 4279eecf-bb20-4c58-9745-f18fd1e764bb" \
  -d '{
    "model": "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    "prompt": "What is love?",
    "max_tokens": 256
  }'

Response:

curl response (rendered as JSON by using jq)

    {
      "id": "chatcmpl-e6c2d199-b57d-4904-a1cb-89d604b146d9",
      "object": "text_completion",
      "created": 1748238665,
      "model": "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
      "choices": [
        {
          "index": 0,
          "message": {
            "role": "assistant",
            "reasoning_content": null,
            "content": "Love is a powerful and meaningful emotion that brings people close together. It is a connection between two individuals that transcends time and space, supporting and nurturing a deep and deepening relationship. Love is the foundation that holds relationships together, reflecting in the happiness, relationships, and harmony surrounding us in every aspect of life. To understand love and its complexities, one would need to delve deep into the arena of psychology, philosophy, anthropology, literature, and social sciences. It is an elusive but natural human reaction, serving as an enabling force for the bonding of our species, and it is an integral aspect of human nature.",
            "tool_calls": []
          },
          "logprobs": null,
          "finish_reason": "stop",
          "stop_reason": null
        }
      ],
      "usage": {
        "prompt_tokens": 20,
        "total_tokens": 162,
        "completion_tokens": 142,
        "prompt_tokens_details": null
      },
      "prompt_logprobs": null,
      "kv_transfer_params": null
    }

Node.js

Request:

Node.js Fetch Request

const response = await fetch('https://inference-e0ed242b-dbd3-423e-b525-76beac222d44-d41b1b44.platform.flex.ai/v1/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': 'Bearer 0ad394db-8dba-4395-84d9-c0db1b1be0a8'
  },
  body: JSON.stringify({
    model: 'TinyLlama/TinyLlama-1.1B-Chat-v1.0',
    prompt: 'What is love?',
    max_tokens: 256
  })
});

const data = await response.json();
console.log(data);

Response:

Node.js Fetch response

{
  "id": "chatcmpl-e6c2d199-b57d-4904-a1cb-89d604b146d9",
  "object": "text_completion",
  "created": 1748238665,
  "model": "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning_content": null,
        "content": "Love is a powerful and meaningful emotion that brings people close together. It is a connection between two individuals that transcends time and space, supporting and nurturing a deep and deepening relationship. Love is the foundation that holds relationships together, reflecting in the happiness, relationships, and harmony surrounding us in every aspect of life. To understand love and its complexities, one would need to delve deep into the arena of psychology, philosophy, anthropology, literature, and social sciences. It is an elusive but natural human reaction, serving as an enabling force for the bonding of our species, and it is an integral aspect of human nature.",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 20,
    "total_tokens": 162,
    "completion_tokens": 142,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null,
  "kv_transfer_params": null
}

Python

Request:

Python Request

import requests
import json

url = "https://inference-e0ed242b-dbd3-423e-b525-76beac222d44-d41b1b44.platform.flex.ai/v1/completions"
headers = {
    "Content-Type": "application/json",
    "Authorization": "Bearer 0ad394db-8dba-4395-84d9-c0db1b1be0a8"
}
data = {
    "model": "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    "prompt": "What is love?",
    "max_tokens": 256
}

response = requests.post(url, headers=headers, json=data)
formatted_json = json.dumps(response.json(), indent=2)
print(formatted_json)

Response:

Python Request response

{
  "id": "chatcmpl-e6c2d199-b57d-4904-a1cb-89d604b146d9",
  "object": "text_completion",
  "created": 1748238665,
  "model": "TinyLlama/TinyLlama-1.1B-Chat-v1.0",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "reasoning_content": null,
        "content": "Love is a powerful and meaningful emotion that brings people close together. It is a connection between two individuals that transcends time and space, supporting and nurturing a deep and deepening relationship. Love is the foundation that holds relationships together, reflecting in the happiness, relationships, and harmony surrounding us in every aspect of life. To understand love and its complexities, one would need to delve deep into the arena of psychology, philosophy, anthropology, literature, and social sciences. It is an elusive but natural human reaction, serving as an enabling force for the bonding of our species, and it is an integral aspect of human nature.",
        "tool_calls": []
      },
      "logprobs": null,
      "finish_reason": "stop",
      "stop_reason": null
    }
  ],
  "usage": {
    "prompt_tokens": 20,
    "total_tokens": 162,
    "completion_tokens": 142,
    "prompt_tokens_details": null
  },
  "prompt_logprobs": null,
  "kv_transfer_params": null
}