Easter Seals Jobs

Job Description

A fortune 100 organization is seeking a Data Engineer that will sit hybrid onsite in the greater Cincinnati Ohio area for a contract. This person will be working to ingest data from various sources using flat files or APIs and handle operational support for ETL jobs that are already in place. It is crucial they have a strong understanding of Azure Data Factory and Data Bricks to be successful in this role.

Primary responsibilities:

Responsible to design, and integrate fault tolerance and enhancements into data pipelines to improve quality and performance

Perform root cause analysis and solve problems using analytical and technical skills to optimize data delivery and reduce costs

Backup and Disaster Recovery: Develop and implement backup and disaster recovery plans for Databricks environments. Ensure that data and configurations are regularly backed up and that there are clear procedures in place for restoring services in the event of a failure.

ETL Process Management: Implement ETL (Extract, Transform, Load) processes to integrate data from various sources into Databricks, ensuring data quality and integrity.

Data Integration: Integrate Databricks with other data storage solutions and data lakes, ensuring seamless data flow and accessibility.

Performance Optimization: Optimize data processing and query performance within Databricks to ensure efficient data retrieval and processing.

Monitoring and Troubleshooting: Monitor data pipelines and workflows for performance issues or errors, and troubleshoot any problems that arise to maintain smooth operations.

Data Pipeline Development: Design, develop, and maintain robust data pipelines using Databricks to process and transform large volumes of data.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .

To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .

Skills and Requirements

4+ years of experience with data bricks and azure data factory (ADF)

Strong working knowledge of ETL moving the data from source to source

4+ years of experience with python/pyspark/spark what have they done, what libraries have they used, etc. (what is work space, what is unity how have you used them) Hive

hadoop null

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.