Senior Software Engineer, Data

This role is part of the AI Insights team, which owns the services that power Affinity's industry-leading relationship intelligence platform. Our team extracts and retrieves information from billions of structured and unstructured data points to deliver insights to our customers.

As a Senior Software Engineer, Data, you will collaborate with machine learning engineers, software engineers, and product managers to shape the future of private capital's leading CRM platform. This involves designing and building scalable, efficient event streaming processing pipelines and batch processing pipelines to transform and load data into distributed data storage. You will play a key role in enhancing our frameworks to monitor data quality, safeguard data security, and implement best practices.

What you’ll be doing:

• Design scalable and reliable data pipelines to consume, integrate and analyze large volumes of complex data from different sources, supporting the evolving needs of our business.
• Help define our data roadmap. You'll collaborate with our team of machine learning engineers, software engineers, product, and business leaders to shape product development.
• Build and maintain frameworks for measuring and monitoring data quality and integrity.
• Establish and optimize CI/CD processes, test frameworks, and infrastructure-as-code tooling.
• Build and implement robust data-intensive applications over distributed systems.
• Articulate the trade-offs of different approaches to building data pipelines and storage solutions, providing clear recommendations aligned with product and business requirements.

Qualifications:

Required:

• 5+ years of experience as a Software Engineer or Data Engineer with at least 2-3 years in a Senior Engineering role, leading complex, and sometimes ambiguous, engineering projects across team boundaries.
• Proficiency in event streaming and batch processing pipelines for data-intensive applications is essential.
• Proven hands-on experience with scalable and robust distributed data systems using Kafka, AWS SQS or similar technologies. 
• Comfortable working with large datasets and high-scale data ingestion, transformation, and distributed processing tools such as Apache Spark. 
• Strong proficiency in SQL. 
• Experience with cloud platforms such as AWS, Databricks, GCP, Azure or related technologies.
• Familiar with CI/CD processes and test frameworks.
• Comfortable partnering with product and machine learning teams on large, strategic data projects.

Nice to have:

• Hands-on experience with both relational and non-relational database/data stores, including vector databases (e.g. Weaviate, Milvus), graph databases, and text search engines (e.g. OpenSearch or Vespa clusters), with a focus on indexing and query optimization. 
• Familiar with industry-standard databases and analytics technologies, including Data Lake and Lakehouse.
• Experience with Infrastructure as Code (IaC) tools, such as Terraform.
• Experience implementing data consistency measures using validation and monitoring tools.

Tech Stack: Our Data stack includes tools to build data pipelines between AWS RDS and DBX via scheduled batch jobs and streaming syncing. Spark SQL and MLlib for large-scale data processing in DBX. We also build data pipelines between RDS and other search-optimized engines, such as openSearch. In-house data quality tools and governance tools to ensure data quality, security and compliance. 

How we work:

Our culture is a key part of how we operate, as well as our hiring process:

• We iterate quickly. As such, you must be comfortable embracing ambiguity, be able to cut through it, and deliver value to our customers.
• We are candid, transparent, and speak our minds while simultaneously caring personally with each person we interact with. 
• We make data-driven decisions and make the best decision for the moment based on the information available.

If you’d want to learn more about our values click here.

What you'll enjoy at Affinity:

• We live our values: As owners, we take pride in everything we do. We embrace a growth mindset, engage in respectful candor, act as playmakers, and "taste the soup" by diving deep into experiences to create the best outcomes for our colleagues and clients.
• Health Benefits: We cover your medical, dental, and vision insurance premiums with comprehensive PPO, HDHP and HMO options (in CA), and offer flexible personal & sick days to support your well-being.
• Retirement Planning: We offer a 401(k) plan to help you plan for your future.
• Learning & Development: We provide an annual education budget and a comprehensive L&D program.
• Wellness Support: We reimburse monthly for things like home internet, meals, and wellness memberships/equipment to support your overall health and happiness.
• Team Connection: Virtual team-building activities and socials to keep our team connected, because building strong relationships is key to success.

Please note that the role compensation details below reflect the base salary only and do not include any equity or benefits. This represents the salary range that Affinity believes, in good faith, at the time of this posting, that it will pay for the posted job.

A reasonable estimate of the current range is $106,200 to $200,000 USD. Within the range, individual pay depends on various factors including geographical location and review of experience, knowledge, skills, abilities of the applicant.