Job Information
Delta Dental Insurance Company Senior Data Engineer in Alpharetta, Georgia
DUTIES: Design, develop, and maintain scalable data pipelines and ETL processes to collect, transform, and store large volumes of data. Collaborate with cross-functional teams to understand data requirements, identify opportunities for data integration, and propose efficient data engineering solutions. Implement data governance and quality control measures to ensure data accuracy, consistency, and integrity throughout the data lifecycle. Optimize and tune database systems and data processing frameworks for performance and scalability. Lead the evaluation, selection, and implementation of data engineering technologies, tools, and frameworks. Mentor and provide guidance to junior data engineers, assisting in their professional development. Collaborate with data scientists and analysts to support their data needs, providing efficient access to relevant datasets. Develop and maintain data models, schemas, and documentation to facilitate data understanding and accessibility. Work closely with infrastructure and DevOps teams to ensure data systems are secure, reliable, and properly integrated with existing infrastructure. Perform data profiling, analysis, and data mining tasks to identify patterns, trends, and insights in large datasets. Collaborate with stakeholders to identify business requirements and translate them into technical data engineering solutions. Implement data warehousing and data mart solutions to support reporting and analytics needs. Stay up to date with emerging data engineering technologies and best practices, and proactively recommend improvements to existing systems. Troubleshoot and resolve data-related issues, ensuring minimal disruption to data pipelines and systems. Collaborate with data privacy and compliance teams to ensure adherence to data privacy regulations and industry standards. Conduct performance monitoring and capacity planning to anticipate and address future data engineering needs. Develop and maintain data governance policies, procedures, and documentation. Participate in data architecture discussions and provide input on data storage, data flow, and data integration strategies. Collaborate with stakeholders to define and implement data retention and archival strategies. Continuously monitor and optimize data engineering processes, striving for efficiency, reliability, and scalability. Telecommuting permissible from anywhere in the U.S.
Requirements
:
Bachelors degree or foreign degree equivalent in Computer Science, Computer Engineering, or related field and six (6) years of progressive, post-baccalaureate experience in the job offered or related role.
Skills:
Experience and/or education must include:
1.
Snowflake for data modeling, query optimization, and data integration;
2.
Snowflake or Matillion for Azure;
3.
Building custom integration processes with Python;
4.
Ability to write Unix shell scripts and invoke them from Matillion pipelines to integrate them into an overall data pipeline; and
5.
Azure Cloud (Cloud Computing Platform) and Azure services including Azure Data Lake Storage, Azure VMs, and Azure SQL Database.
OTHER: Job Site:
1130 Sanctuary Pkwy #600, Alpharetta, GA 30009; Telecommuting permissible from anywhere in the U.S.; 40 hours/week; $171,267/year - $196,500/year. If offered employment, the applicant must have legal right to work in U.S. EOE.
CONTACT: Please reference Job # 2047578 and send resume to DDTA@delta.org.
