AWS Performance & Memory Optimization Engineer

Job Summary:

This role centers on diagnosing, optimizing, and scaling high-performance distributed systems running on AWS. You will lead memory-focused performance engineering across JVM-based applications, Spark workloads, EMR clusters, and containerized services. The ideal candidate brings hands-on experience with low-level memory management, tuning, and troubleshooting across multiple layers of the AWS stack.

Key Responsibilities:

A. Memory Optimization & Performance Engineering

• Analyze and optimize JVM memory usage (heap, GC tuning, off-heap strategies, JDK/JVM parameterization).
• Improve Spark job performance through executor memory tuning, shuffle optimization, and YARN resource configuration.
• Conduct EMR cluster performance audits, focusing on memory bottlenecks, cluster sizing, and workload balancing.
• Optimize PySpark pipelines for memory efficiency and distributed execution stability.
• Implement EC2-level performance tuning (instance family selection, kernel parameters, OS-level memory profiling).
• Drive memory-aware scaling strategies for EKS/Kubernetes workloads (requests/limits, node sizing, pod-level tuning).

B. AWS Platform Optimization

• Profile and optimize memory usage across AWS compute services (EC2, EMR, EKS).
• Use CloudWatch, EMR metrics, Spark UI, and custom telemetry to identify memory hotspots.
• Build automated performance dashboards and memory-usage observability pipelines.

C. PnP (Performance & Profiling) Expertise

• Apply advanced profiling tools (JFR, YourKit, async-profiler, FlameGraphs) to diagnose memory leaks and GC pressure.
• Lead root-cause analysis for memory-related production incidents.
• Establish best practices for memory-safe coding, cluster configuration, and workload orchestration.

D. Architecture & Collaboration

• Partner with data engineering, platform engineering, and SRE teams to design memory-efficient architectures.
• Provide guidance on JVM-based microservices, Spark pipelines, and containerized workloads.
• Document optimization strategies, tuning guidelines, and performance playbooks.

Required Skills & Experience:

• Strong hands-on experience with Java/JVM internals, GC algorithms, JDK tuning, and memory-profiling tools.
• Deep expertise in AWS EMR, Spark/YARN, and distributed data processing.
• Proficiency with PySpark, large-scale ETL pipelines, and memory-aware transformations.
• Solid understanding of EC2 performance characteristics, Linux memory management, and kernel-level tuning.
• Experience with EKS/Kubernetes, container memory limits, pod scheduling, and node-level optimization.
• PnP (Performance & Profiling) mastery, including flame graphs, heap dumps, GC logs, and async profiling.
• Familiarity with AWS monitoring tools (CloudWatch, EMR metrics, Spark UI, X-Ray).

Salary: Negotiable

Working Hours: Need work during 9 AM to 5 PM Pacific Standard Time of USA, which is 11PM to 7AM all thruogh the night Monday to Friday.