Senior Infrastructure – Kafka Engineer, Enterprise Data Engineering

Role Overview

Customer is seeking a Senior Infrastructure – Kafka Engineer to join the Enterprise Data Engineering team. This role is ideal for a seasoned engineer with deep experience in Apache Kafka / Confluent Kafka, messaging (MQ), SQL/NoSQL databases, and cloud infrastructure, who can lead operations and engineering for a large-scale, event-driven data platform.

You will lead Kafka platform operations and automation, integrate Kafka with core banking systems, and provide senior-level support across a broad technology stack to enable real-time data and analytics use cases across the bank.

Key Responsibilities

Platform & Infrastructure Engineering

• Administer, configure, and troubleshoot Kafka clusters (on-prem and cloud), including broker/cluster configuration, partitioning, and performance tuning.
• Design and implement scalable, highly available Kafka infrastructure, including disaster recovery and multi-environment strategies.
• Integrate Kafka with upstream/downstream systems via Kafka Connect and other connectors (e.g., MQ, MongoDB, Oracle, SQL Server, PostgreSQL, MySQL).
• Build and support real-time data pipelines using Kafka producers and streaming consumers (e.g., Spark Streaming, Kafka Streams).
• Automate infrastructure provisioning and configuration across environments using Terraform and modern DevOps practices.
• Deploy and manage Kafka components and clients in production and DR environments, ensuring resilience and recoverability.

Operations, Observability & Support

• Lead a small team of engineers/technicians in event-based monitoring, diagnosis, and remediation of infrastructure issues.
• Implement and maintain comprehensive monitoring, logging, and alerting using tools such as Splunk, Datadog, Grafana, and related observability platforms.
• Perform proactive health checks and capacity planning to identify issues before they impact service.
• Serve as a primary point of contact for daily operations, major incidents, and escalations related to Kafka and related infrastructure.
• Develop, maintain, and continuously improve runbooks and playbooks for incident response, maintenance, and common operational tasks.
• Analyze and audit support tickets to identify patterns, reduce downtime, and drive problem management and root-cause fixes.

Governance, Compliance & Collaboration

• Ensure infrastructure and platform changes comply with internal standards, regulatory requirements, and security policies.
• Collaborate with security, networking, application, and data engineering teams to design and operate secure, compliant, event-driven architectures.
• Contribute to standards, best practices, and documentation for Kafka, messaging, and integration patterns across Enterprise Data Engineering.
• Participate in and help drive agile ceremonies; influence product/technical direction for streaming and integration platforms.

Required Qualifications

• 7+ years of experience in infrastructure engineering with a strong focus on:
  • Kafka administration (on-prem and cloud) and Kafka ecosystem (brokers, topics, consumer groups, replication, failover).
  • Messaging systems (e.g., MQ) and database integration (SQL and NoSQL).
• Proven experience designing, deploying, and scaling Kafka clusters and connector infrastructure in production and DR environments.
• Hands-on experience building real-time data pipelines using Kafka producers and streaming consumers (e.g., Spark Streaming).
• Strong proficiency with at least one major cloud platform (AWS, GCP, or Azure) and event-driven architectures, including containerization and DevOps practices.
• Experience with monitoring/observability tools such as Splunk, Datadog, Grafana.
• Solid understanding of networking, operating systems (Linux/Windows), and core diagnostic tools.
• Proficiency with source control (SVN, Git) and scripting/programming (e.g., PowerShell, Bash, Python, Perl).
• Demonstrated ability to analyze complex issues, make sound decisions with limited information, and drive issues to resolution.
• Strong communication, customer service, and collaboration skills; comfortable working with cross-functional technical teams.

Desired Qualifications

• Prior experience with enterprise monitoring tools beyond those listed.
• Financial services industry experience, ideally within a regulated banking environment.

Education

• Bachelor’s degree in Computer Science, Computer Engineering, Electronics Engineering, or equivalent professional experience.

Compensation & Benefits (Client-Provided)

• Base salary: Target range $120,000 per year, plus annual discretionary bonus opportunity (final comp based on location, skills, and experience).
• Comprehensive benefits including medical, dental, vision, retirement benefits (401(k) with match and company contribution), paid parental leave, generous PTO and holidays, flexible work arrangements, education reimbursement, wellness programs, and more.

Title: Senior Infrastructure – Kafka Engineer, Enterprise Data Engineering
Locations (Hybrid – 4 days onsite / 1 day remote): Phoenix, AZ
Employment Type: Full-time (40 hours/week), Monday–Friday (6-Month Contract)
Pay Rate: Up to $120K