In-person + Virtual
16 -20 May
Learn More and Register to Attend

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for KubeCon + CloudNativeCon Europe 2022 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

Please note: This schedule is automatically displayed in Central European Summer Time (UTC +2). To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date." The schedule is subject to change.
Back To Schedule
Wednesday, May 18 • 14:30 - 15:05
Spark on Kubernetes: The Elastic Story - Bowen Li & Huichao Zhao, Apple

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.

Apache Spark is a unified analytics engine for large-scale data processing. People are moving Spark and batch workload to Kubernetes due to its uprising popularity. There are many challenges to running Spark efficiently on Kubernetes, for example, supporting autoscaling-based workloads. In this talk, we discuss building a large scale Spark Service on top of Kubernetes. We will also walk through autoscaling on a multi-tenant platform with advanced features such as physical isolation, min/max capacity setting, bin-packing, scale-in and scale out controls, and more. These improvements show significant CPU and memory utilization savings for Spark on Kubernetes.

Click here to view captioning/translation in the MeetingPlay platform!

avatar for Bowen Li

Bowen Li

Engineering Manager, AIML Data Infra, Apple
Bowen Li is the manager of the Batch Processing and Interactive Analytics team at Apple AI/ML. His team builds and manages Spark batch processing service, Interactive Spark notebook service, and interactive analytics service with Trino for hundreds of data engineers and scientists... Read More →
avatar for huichao zhao

huichao zhao

Software Engineer, Apple
Software Engineer on AI/ML Data Platform team, Apple

Wednesday May 18, 2022 14:30 - 15:05 CEST
Viewable In Platform Feria Valencia