This article discusses the importance and necessity of the task queue system, and details how Kube Queue defines its role and contribution in the current Kubernetes ecosystem.
We introduce Apache Flink's adaptive batch scheduler and detail how it can automatically decide parallelism of Flink batch jobs.
Part 2 of this 2-part series will give you insight into some core design considerations and implementation details of the sort-based blocking shuffle in Flink.
Part 1 of this 2-part series will introduce the sort-based blocking shuffle, present benchmark results, and provide guidelines on how to use this new feature.
