Databricks Automation & AI Platform Engineer

About us:

Where elite tech talent meets world-class opportunities!

At Xenon7, we work with leading enterprises and innovative startups on exciting, cutting-edge projects that leverage the latest technologies across various domains of IT including Data, Web, Infrastructure, AI, and many others. Our expertise in IT solutions development and on-demand resources allows us to partner with clients on transformative initiatives, driving innovation and business growth. Whether it's empowering global organizations or collaborating with trailblazing startups, we are committed to delivering advanced, impactful solutions that meet today’s most complex challenges.

We are building a community of top-tier experts and we’re opening the doors to an exclusive group of exceptional AI & ML Professionals ready to solve real-world problems and shape the future of intelligent systems.

Role Overview

We are seeking a senior Databricks Automation & AI Platform Engineer with deep expertise in Python, Databricks platform components, and AI/MLOps/LLMOps workflows. The engineer will support the Next‑Gen Platform Integration team by building automation frameworks, enabling scalable deployment workflows, and integrating AI capabilities across Databricks environments.

This role requires strong systems‑level Python knowledge, Databricks governance experience, and hands‑on AWS automation skills.

Key Responsibilities

🔹 Databricks Automation & Integration

• Build automation workflows for MLOps, LLMOps, and application deployment within Databricks.
• Enhance workspace onboarding automation including Unity Catalog, permissions, and environment setup.
• Develop reusable modules for workspace provisioning, catalog configuration, model deployment, and governance workflows.
• Integrate Mosaic AI components (Gateway, Model Serving, Agents) into platform automation.

🔹 Platform Engineering & Deployment

• Develop Python automation scripts for Databricks services including:

• Unity Catalog

• MLflow

• Mosaic AI

• Model Serving

• Databricks Apps

• Ensure consistency, reliability, and scalability across multi‑workspace environments.

• Implement automated rollback strategies, fail‑fast mechanisms, and environment stability checks.

🔹 AWS Integration & Automation

• Build integrations using AWS Lambda, API Gateway, and Service Principal authentication.
• Automate Databricks Job orchestration, monitoring, and deployment pipelines.
• Implement secure, scalable automation bridging AWS and Databricks.

Required Skills

🔹 Technical Skills

• Strong hands‑on Python experience including:

• GIL behavior

• Multiprocessing vs multithreading

• Memory overhead trade‑offs

• Data structure time complexities

• Deep understanding of Databricks ecosystem:

• Unity Catalog governance

• Delta architecture (Bronze/Silver/Gold)

• MLflow tracking & model lifecycle

• Mosaic AI Gateway & LLMOps workflows

• Model Serving & Databricks Apps

• Experience with Docker image immutability, container state management, and environment reproducibility.

🔹 Cloud & Automation Skills

• AWS Lambda, API Gateway, IAM, and automation pipelines.
• Service Principal‑based authentication for secure Databricks automation.
• Experience with platform engineering, deployment automation, and production stability patterns.

Ideal Candidate Profile

• 6+ years of Python development and cloud automation experience.
• Hands‑on Databricks platform engineering and governance expertise.
• Strong understanding of MLOps/LLMOps and AI deployment workflows.
• Experience working with platform, ML, and data engineering teams.
• Immediate availability preferred.