Skip to content

Amgix Metrics

Amgix collects metrics from all nodes and exposes them via API endpoints.

Amgix Now

Amgix Now does not collect metrics in stand-alone mode. When it joins an Amgix cluster, it collects and reports its metrics to the cluster leader. Metrics endpoints are not implemented in Amgix Now.

Definitions

GET /v1/metrics/definitions endpoint returns an array of all supported metric keys, their unit and description:

[
  {
    "key": "api_requests",
    "unit": "req",
    "description": "HTTP requests handled by the API process."
  },
  {
    "key": "api_search_ms",
    "unit": "ms",
    "description": "Sum of request durations in milliseconds for search on the API process."
  },
  ...
]

Prometheus

GET /v1/metrics/prometheus endpoint outputs metrics in Prometheus format. It may look something like this:

# HELP amgix_metrics_window_seconds Length in seconds of the rolling window used for live Amgix samples in this Prometheus exposition.
# TYPE amgix_metrics_window_seconds gauge
# HELP amgix_api_requests HTTP requests handled by the API process.
# TYPE amgix_api_requests gauge
# HELP amgix_api_requests_n Sample count (denominator) for amgix_api_requests.
# TYPE amgix_api_requests_n gauge
# HELP amgix_api_search HTTP search requests handled by the API process.
# TYPE amgix_api_search gauge
amgix_metrics_window_seconds 60
amgix_api_requests{host="api-1",leader="true",role="api"} 142
amgix_api_requests_n{host="api-1",leader="true",role="api"} 142
amgix_api_search{host="api-1",leader="true",role="api"} 38
amgix_api_search_n{host="api-1",leader="true",role="api"} 38
amgix_broker_documents_bulk{host="encoder-1",leader="false",role="index"} 12
amgix_index_queue_docs_new{host="encoder-1",leader="false",role="index"} 500

These can be consumed/scraped by any system that collects prometheus metrics and exposed in custom dashboards or used to create observability alerts, etc.

Current

GET /v1/metrics/current?window=60&keys=key1&keys=key2...

Current endpoint returns metrics live rolling window of metrics for either 30 or 60 seconds interval (defaults to 60). Optionally, you can provide a list of metrics keys to return only the ones you need. The result is JSON that may look something like this:

{
    "nodes": {
        "cbf199d4cf36": {
            "is_leader": false,
            "last_seen": 1782668324.21634,
            "meta": {
                "at_capacity": false,
                "free_ram_gb": 5.24,
                "free_vram_gb": null,
                "gpu_available": false,
                "gpu_support": false,
                "load_models": true,
                "loaded_models": [
                    {
                        "label": "BAAI/bge-small-en-v1.5",
                        "loaded_at": 1782406914.89619,
                        "model_key": [
                            "dense_model",
                            "BAAI/bge-small-en-v1.5",
                            ""
                        ]
                    }
                ],
                "model_last_used": [
                ],
                "total_ram_gb": 7.75,
                "total_vram_gb": null
            },
            "metrics": [
                {
                    "dims": [
                    ],
                    "key": "api_request_ms",
                    "last_seen": 1782668315.1483,
                    "windows": {
                        "60": {
                            "n": 2,
                            "value": 1.41173
                        }
                    }
                },
                {
                    "dims": [
                    ],
                    "key": "api_requests",
                    "last_seen": 1782668315.14829,
                    "windows": {
                        "60": {
                            "n": null,
                            "value": 2.0
                        }
                    }
                }
            ],
            "role": "now"
        },
        "e637701cab5c": {
            "is_leader": false,
        ...

Trends endpoint returns aggregated historical metric buckets for the given time range and resolution.

GET /v1/metrics/trends?since=2026-06-28T17:18:00.000Z&until=2026-06-28T17:49:00.000Z&resolution=60&keys=api_requests&keys=api_request_ms

  • since and until expect UTC timestamps of the time range.
  • resolution is either 60 (1 minute) or 300 (5 minutes).
  • keys can be specified to return metrics for specific keys only.

Values are aggregated into resolution buckets by node.

Amgix stores historical metrics for up to 7 days. It returns the following JSON shape:

[
  {
    "key": "api_requests",
    "bucket_seconds": 60,
    "buckets": [
      {
        "key": "api_requests",
        "dims": [],
        "bucket_start": 1782667080,
        "bucket_seconds": 60,
        "value": 2.0,
        "n": null,
        "hostname": "api-1",
        "source": "api"
      },
      {
        "key": "api_requests",
        "dims": [],
        "bucket_start": 1782667080,
        "bucket_seconds": 60,
        "value": 3.0,
        "n": null,
        "hostname": "api-2",
        "source": "api"
      },
      {
        "key": "api_requests",
        "dims": [],
        "bucket_start": 1782667140,
        "bucket_seconds": 60,
        "value": 5.0,
        "n": null,
        "hostname": "api-1",
        "source": "api"
      }
      ...
    ]
  },
  {
    "key": "api_request_ms",
    "bucket_seconds": 60,
    "buckets": [
      {
        "key": "api_request_ms",
        "dims": [],
        "bucket_start": 1782667080,
        "bucket_seconds": 60,
        "value": 24.37,
        "n": 3,
        "hostname": "api-2",
        "source": "api"
      }
      ...
    ]
  }
  ...
]