• Job Description

  Req#: REQ-21761
  Data Engineer Consultant
  We are looking for a talented data and visualization engineer that is passionate about data in all its forms. In this role the engineer will be responsible for being part of a talented team building a modern data pipeline for ingesting, processing and distributing large datasets using cloud based and open-source technologies. The ideal candidate has broad experience in building data pipelines that scale across many data sources and consumers along with presenting the data in visualization tools. The engineer should have previously worked in agile teams building cloud-based analytics platforms and delivering solutions in an iterative team focused manner.
  Responsibilities
  • Create and maintain data pipelines that ingest, transform and stage data for use in analytics processes.
  • Build integrations with external systems using common integration patterns and tools.
  • Integrate with various source systems across the organization to pull data into a data lake for processing.
  • Understand the data model across multiple business domains and appropriately map into cross functional reports and dashboards.
  • Develop and manage data processing services on Linux servers.
  • Migrate and refactor legacy process to modern cloud infrastructure
  Skills and Requirements
  • Very strong hands-on Python development experience using pandas, SQLAlchemy, Django.
  • Professional experience in core AWS services and architecture supporting data processing systems.
  • Strong knowledge of open-source libraries for data processing like Airflow.
  • Can clearly explain typical data processing architectures and their benefits and drawbacks.
  • Familiar with containers and microservice architectures.
  • Comfortable working in either an agile team or independently depending on project needs
  • Self-starter that is able to quickly come up to speed in multiple development environments.
  • Ability to understand and explain complex business processes and data, ideally within financial companies.
  • Some experience in Linux to manage processing jobs.
  • Solid understanding of relational and columnar databases.
  • Experience with Rest API development.
  • Ability to understand and modify complex queries across multiple data sources.

• About the company

  Millennium is a global alternative investment management firm, founded in 1989, which manages $47.1 billion in assets. We seek to pursue a diverse array of investment strategies across industry sectors, asset classes, and geographies.

Notice