When querying Skyminer, the data returned can be very large. To prevent the response time from being excessively long, it is usually a good idea to limit the number of points returned by Skyminer.
Queries can become overwhelming either horizontally, that is to say that there are too many datapoints for one of the metrics returned, or vertically, when a group by returns too many groups.
There are two mechanism to limit the number of datapoints in each series returned.
You can use the limit parameter of the metric. This will stop returning points after the limit is reached, but will not throw an exception.
You can use the limit aggregator. This will stop returning points and return an error message if the limit is reached.
Safeguard is a mechanism used to facilitate the usage of horizontal and vertical limits.
In the query UI, the padlock button can be used to define 3 parameters:
Number of series: maximum number of groups returned by a group_by (vertical limit)
Limit before aggregation: maximum number of datapoints for each series, before aggregation (horizontal limit)
Limit after aggregation: maximum number of datapoints for each series, after aggregation (horizontal limit)
These limits are set by default to avoid accidental disproportionally large queries.
The different kinds of limits can be set in different ways. This section does not present new API parameters as limit mechanisms have already been introduced in the general presentation of the metric object and in the section on aggregators.
The limit aggregator can be used as any aggregator. The name of the aggregator is limit and there is only one parameter:
limit: the maximum number of datapoints for each series returned.
This aggregator throws an error when the limit is reached, and returns a response with status code 500:
{
"errors": [
"Limit Aggregator: query exceeded limit of 10000 data points, please increase limits or add filtering/downsampling"
]
}
An alternative is the limit attribute, a root attribute of the metric object. It does not throw an error when the limit is reached but stops returning datapoints.
The vertical limits are done through the group_limit attribute of the metric. It sets the maximum number of groups returned by the group by.
...
{
"tags": {... tags for filtering ...},
"name": "abc.123",
"limit": 1000,
"group_limit": 100,
"aggregators": [... a list of aggregators ...]
}
...
An error message is returned if the limit is overstepped with status code 500:
{
"errors": [
"Groups Limit: exceeded limit of 100 groups in a single metric query, please increase limits, change grouping or add filters on tags"
]
}
Alternatively you can use the safeguard object to define those limits:
{
"safeguard": {
"group_limit": 100,
"limit_before_aggregation": 10000000,
"limit_after_aggregation": 10000
},
"metrics": [
...
]
}