Software Engineer (Infrastructure)

Darktrace is seeking an experienced Infrastructure Engineer to manage, maintain, and optimize a dedicated NVIDIA GPU server and cloud environments for innovation projects. Responsibilities include setting up, configuring, and maintaining the servers and software stack. A successful candidate will work directly with Darktrace researchers and software engineers, ensuring optimal performance and availability for ongoing AI and HPC (high-performance computing) projects.

This is a hybrid role, with a compulsory attendance of 2 days a week in the Cambridge office.

This role focuses on maintaining and optimising the Linux operating system, file systems, and software stack (Cuda, PyTorch, Python etc) for machine learning projects as well as setting up and configuring NVIDIA HGX servers (installing and updating software, managing user access, and ensuring optimal performance) and cloud infrastructure for GPU compute projects (managing access and ensuring optimal performance). Additional responsibilities include:

• Monitoring server and application performance, identifying bottlenecks, and taking corrective actions to maintain high availability,
• Implementing and maintaining server security, including patch management, vulnerability scanning, and intrusion detection,
• Collaborating with network administrators, hardware engineers, and researchers to troubleshoot and resolve server and software-related issues,
• Working closely with the project manager to ensure efficient resource allocation, server utilisation and scaling across multiple teams,
• Collaborating with data scientists and machine learning engineers to understand their software requirements and provide guidance on best practices,
• Assisting in training team members on the capabilities and usage of the HGX servers and the software environment,
• Developing multi-use tooling to work with the HPC environments.

What experience do I need:

We welcome applications from engineers with strong problem-solving and creative thinking skills as well as excellent communication and the ability to work in a collaborative team environment. You will be an independent thinker with a startup mindset. Technology-wise, you will have experience in system administration, preferably with a focus on HPC platforms, GPU-based servers, and machine learning software environment as well as a familiarity with AI and HPC provisioning and management, both on-premises and in the cloud. You will have experience with server virtualization technologies and containerization and well versed with the linux operating system. You'll also ideally have:

• Strong knowledge of NVIDIA HGX server architectures and components,
• Strong knowledge of AWS or Azure Cloud environments,
• Experience with NVIDIA GPU technologies, such as NVLink, NVSwitch, and Tensor Core GPUs,
• Experience with machine learning frameworks and libraries, such as PyTorch and associated system optimisations,
• Experience with NAS servers,
• Experience with data version control systems.