Principal Data Engineer / Tech Lead
We are looking for a principal engineer and tech lead to join our Data Engineering team at HDAI. This small and nimble team owns the data ingestion pipelines from HDAI’s health system customers and plays a critical part of our operation. High quality ingestion and interpretation of clinical data is at the core of producing AI models which can be trusted by clinicians. This role comes at a key turning point in our product development: We are transitioning from a legacy application to a Python-based, AWS-native, event driven architecture (you should be familiar with Lambdas, ECS, SQS, and S3, or similar technologies). Strong software development skills are a must, your code must meet a high bar of quality assurance while achieving deadlines effectively. At HDAI, we nurture a culture of continually improving, and one of the best assets of our engineers is a love for learning. We hope that you too share this goal. A successful candidate will demonstrate strong attention to detail, clear communication, and effective follow-through. We are excited to have you as a part of this team!
Responsibilities:
1. Participate in building our new ETL framework built on AWS Lambdas, ECS, SQS, and S3.
2. Maintain and optimize existing data pipelines and ETL code to support data ingestion from HDAI’s customers.
3. Work with clinicians to understand the data and design new ingestion frameworks for new customers.
4. Maintain software using agile best practices, document and fix bugs, update release notes, update documentation, and address backlog items on a regular basis.
5. Lead project planning, execution, and delivery, ensuring projects are completed on time and within scope, with risk minimization strategies.
6. Contribute to a culture of continuous learning, growth, and innovation within the Data Engineering team.
7. Contribute to a collaborative work environment with peers across many teams.
Qualifications:
1. Bachelor’s Degree in Computer Science or equivalent.
2. 5+ years of working experience in data engineering, including ETL and dataset management, with at least 2 years in a tech lead or leadership role.
3. Familiarity with developing for cloud platforms (AWS, Azure, GCP) and Docker. Experience with AWS Lambdas, ECS, SQS, S3, as well as Terraform are a plus.
4. Proficient in implementing event-driven architectures composed of stateless services.
5. Extensive software development experience in Python or similar languages.
6. Applicants must possess a strong ability to diagnose and resolve complex software issues through effective debugging techniques.
7. Familiarity with DevOps practices and tools for CI/CD pipelines.
8. Proven track record of successfully leading the implementation of complex projects.
9. Excellent interpersonal skills with demonstrated ability to effectively communicate with internal and external teams; ability to develop trust, cooperation, and mutual respect.
10. You exemplify strong accountability and ensure the quality of your work and your team.
Job Type: Full-time
Pay: $175,432.00 - $185,576.00 per year
Benefits:
- 401(k)
- 401(k) matching
- Dental insurance
- Employee discount
- Health insurance
- Life insurance
- Paid time off
- Referral program
- Tuition reimbursement
- Vision insurance
Compensation package:
- Bonus opportunities
- Stock options
Experience level:
Schedule:
Ability to Relocate:
- Dedham, MA 02026: Relocate before starting work (Required)
Work Location: Hybrid remote in Dedham, MA 02026