Both Thanos and Cortex have enabled the cloud native ecosystem to scale Prometheus storage with the use of blocks of data persisted across many clusters into single object storage. Whilst this unlocks cheap long term retention of metrics, it presents a significant challenge of being able to efficiently read and process large volumes of data. This talk outlines the Thanos community's efforts to improve read path performance through query pushdown and query sharding and how it compares with existing Cortex approaches. Thanos deployment's are composed of stores; components that expose a consistent Prometheus compliant read API for retrieving timeseries, and queriers; components that combine raw timeseries and evaluate PromQL expressions against them. Query pushdown gives the opportunity to pre-evaluate these expressions closer to the data, while query sharding breaks down a query into distinct, disassociated datesets that can be computed concurrently thanks to Kubernetes.
Click here to view captioning/translation in the MeetingPlay platform!
