Skyminer system is a Big Data storage and analytics engine integrated with Kratos products, systems and solutions. It allows its users to store billions of samples with different data types over time, while maintaining efficient storage and outstanding write and read performances. Skyminer provides features to analyze data over time, organisational, or geospatial dimensions within and/or between data series.
Different time series don’t necessarily start and end at the same time. You can use the Time Align aggregator to align the start time of the data. You can also use statistical aggregation, interpolation, or the resample aggregator to get time series with samples aligned over time.
By default there is no process or filter applied to the data before displaying it on Skyminer. However you can choose to process the data in the query page.
By default there are 3 safeguards for the size of data displayed on Skyminer:
A limit of 100 series (group by)
A limit of 10 000 000 points before aggregation
A limit of 10 000 points after aggregation
These can be overridden by clicking on the padlock icon on the right of the query interface, but be aware that plotting a large number of points is rarely informative and causes performance issues.
You can export the data to a JSON or CSV file by clicking on the respective button on the right of the query page. It is also possible to send the data to Jupyter in order to process it with Python scripts and to generate a PDF report by using Jupyter and the Skyminer extensions. Other export capabilities exist, contact Kratos for more information.
Using aggregators: this can be achieved by selecting the Align Sampling option, in combination with either Align Start Time or Align End Time. The Resample aggregator also provides this capability without the need to compute statistics (using last known value, or nearest timestamp value).