What's your preference?
Job Description
- Req#: 16406
- Develop and maintain data processing pipelines using Python, Spark, and Hadoop ecosystems.
- Build ETL workflows on AWS EMR clusters for processing large datasets.
- Assist in optimizing Spark applications for performance and reliability.
- Support data visualization and exploration using Hue and other analytics platforms.
- Work with data scientists and analysts on machine learning pipeline implementation.
- Monitor distributed computing environments and assist with troubleshooting.
- 3+ years of software development experience.
- Proficiency in programming languages like Python & SQL.
- Knowledge of database modeling and familiarity with ETL tools.
- Experience with big data technologies and cloud platforms (AWS).
- Working knowledge of Apache Spark (PySpark, Spark SQL) and Hadoop ecosystem.
- Experience with AWS EMR cluster operations.
- Familiarity with data exploration and workflow management tools.
- Understanding of distributed systems concepts.
- Exposure to real-time data streaming (Kafka, Kinesis).
- Basic understanding of data governance & security.
- Familiarity with machine learning concepts.
- AWS certifications or willingness to obtain.
- Experience with AWS services (S3, Glue, Redshift, Athena).
- Basic knowledge of containerization (Docker, Kubernetes).
- Understanding of CI/CD pipelines.
Overview
We are seeking a Data Engineer to join our data engineering team, focusing on data processing and analytics solutions using big modern data technologies.
Responsibilities
Qualifications
About the company
Cotiviti is a solutions and analytics company leveraging unparalleled clinical and financial datasets to deliver insight into the healthcare system’s performance.
Notice
Talentify is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or protected veteran status.
Talentify provides reasonable accommodations to qualified applicants with disabilities, including disabled veterans. Request assistance at accessibility@talentify.io or 407-000-0000.
Federal law requires every new hire to complete Form I-9 and present proof of identity and U.S. work eligibility.
An Automated Employment Decision Tool (AEDT) will score your job-related skills and responses. Bias-audit & data-use details: www.talentify.io/bias-audit-report. NYC applicants may request an alternative process or accommodation at aedt@talentify.io or 407-000-0000.