Lead Data Engineer

Company Description

Sutherland is seeking an analytical person to join us as an Associate Profession - BI Analyst to be responsible for reporting and providing insights from the data. We are a group of dynamic and driven individuals. If you are looking to build a fulfilling career and are confident you have the skills and experience to help us succeed, we want to work with you! 

Job Description

The Lead BI Engineer will be responsible for architecting, developing, and optimizing enterprise-scale business intelligence solutions. This role requires deep expertise in Power BI, Azure, Databricks, and SQL, with the ability to design scalable data pipelines, deliver advanced analytics, and drive BI adoption across the organization. The ideal candidate combines strong technical skills with leadership capabilities, guiding teams and collaborating with stakeholders to enable data-driven decision-making.

Key Responsibilities

Data Engineering & Pipeline Development

• Design and implement data ingestion pipelines from multiple sources using Azure Databricks.
• Develop scalable and reusable frameworks for ingesting and transforming datasets.
• Integrate end-to-end data pipelines ensuring data quality, consistency, and governance.
• Work with event-based/streaming technologies for real-time data ingestion and processing.
• Optimize Databricks code and perform performance tuning for large-scale workloads.

SQL Development & Optimization

• Strong expertise in SQL, T-SQL, and PL/SQL with hands-on experience in SQL Server.
• Write, analyze, and optimize complex SQL queries including stored procedures, functions, triggers, joins, updates, and constraints.
• Deep knowledge of SQL architecture: Agent Jobs, Indexes, Partitions, Views, and Procedures.
• Proven ability to design high-performance queries and implement best practices for SQL optimization.

BI & Visualization

• Lead the design and development of Power BI dashboards, reports, and KPI scorecards.
• Proficient in DAX, advanced calculations, and data modeling aligned with warehouse standards.
• Collaborate with business stakeholders to translate requirements into actionable BI solutions.
• Champion BI adoption and foster a culture of data-driven decision-making.

Cloud & Infrastructure

• Design and deploy data applications on Azure cloud solutions (Synapse, Data Lake, SQL DB, Blob Storage).
• Hands-on experience with Azure Data Factory, Azure Functions, and Azure DevOps for automation and orchestration.
• Provision and manage Databricks clusters, including scheduling and job automation.
• Familiarity with AWS/GCP integration where applicable.

Advanced Analytics & Development

• Experience with PySpark and Python for data processing and advanced analytics.
• Hands-on experience with Spark SQL, Spark Streaming, Delta Lake, and Scala/Python notebooks.
• Develop proofs of concept (POCs) and prototypes with customer teams.
• Implement transformational big data projects, including real-time streaming using Databricks.

Qualifications

Desired Skills

• Strong problem-solving, interpersonal, and communication skills.
• Self-starter with the ability to adapt to new Azure services in the analytics space.
• Provide technical leadership and mentor junior engineers.
• Collaborate with delivery leadership to ensure project delivery adheres to quality standards.

Additional Information

willing to work in UK shift