Manager of Data Ingestion and Transformation

About The Position

The Data Ingestion and Transformation Leader will oversee data engineer scrum teams responsible for Infutor’s ingestion of external and customer’s data sources, transformation and cleansing of the data into a consolidated data model, and creation of output data to create customer data products and to feed customer applications.  This role will report to the Chief Technology Officer, and be based in the corporate headquarters in Oakbrook Terrace, IL.

Responsibilities

  • Responsible for the success and management of the US and Costa Rica scrum teams that run legacy processes to ingest, transform and deliver data products to Infutor file install customers.
  • Partner with the CTO and lead the evaluation and choice of new innovative Big Data technology, Data Lake approaches and commercial or open source ETL software to create a new future data ingestion platform running in AWS.
  • This new ETL/ELT/Ingestion platform will be built by moving industry leading data matching and cleansing logic from the legacy applications into a Cloud Microservices architecture (Lambda/Docker), with all rules stored in a central data repository (not in the code) that can then be leveraged internally by the new platform, and eventually externally by our customers as data cleansing API’s.
  • The new platform will feed a consolidated data model (InfutorGraphDB) in AWS (Redshift) as a source of truth for consumer and household identities along with other consumer attributes and intelligence.
  • Transformation routines will need to be orchestrated in the new platform to ingest customer and external data sources from the Data Lake, as well as streaming data, and perform Master Data Management (MDM), cleansing and execution of Data Intelligence Microservices to feed the consolidated InfutorGraph DB.
  • Transformation routines will also need to be created to create customer install files to be delivered to customers via SFTP and to feed legacy applications to replace the current Q Files process. The goal of the new platform is to eventually replace all current legacy data ingestion processes.
  • Ensure the new platform is secure and engineered to meet high quality standards that not only meet business and continuity needs, but are designed to adapt and scale in the long-term.
  • Recruit, coach and mentor a world-class, data engineering team based in Costa Rica to build this new platform. Provide clear direction, encourage professional development, and motivate the team to achieve success. Reinforce the cultural principles of Infutor and set the leadership tone for the team
  • During the transition to the new platform, oversee continuous improvement of existing legacy data ingestion environments and processes.

Ideal Business Experience
  • Bachelor's degree in discipline appropriate to assignment or an equivalent combination of education and experience (related certifications are a plus).
  • Strong and innovative leader who can mentor, manage and develop multiple geographically distributed engineering/data scrum teams.
  • Excellent project management skills with the ability to balance multiple priorities simultaneously. Demonstrates creativity and innovation in problem solving and applying appropriate methodologies relevant to the complexity of the challenge.
  • Foundational experience in a well-managed, process oriented, financially and metric driven company. An ability to apply best-in-class methodologies to a legacy environment is crucial.
  • Experience collaborating with internal and external resources to develop strategies that meet department goals within budget and established timelines.
  • Minimum of 5 years of experience in team management and leadership roles.
Ideal Technical Experience
  • Proven hands-on experience, leadership and expertise with leading commercial or open-source ETL software such as Informatica, Talend, Matillion, Boomi, Jitterbit or similar.
  • Strong track record in MDM (Master Data Management), Data Architecture, Data matching and cleansing.
  • Experience in Big Data processing technologies such as Hadoop, Apache Spark, EMR.
  • Minimum of 4 years of experience in using ETL software to ingest, transform and load data.
  • Minimum of 2 years of experience in Master Data Management (MDM) tools and patterns.
  • Strong understanding of relational database technologies, concepts and SQL.
  • Experience in No-SQL databases / JSON is a plus.
Qualified candidates are encouraged to email their cover letter and resume to careers@infutor.com.

Contact Infutor today to find out how accurate, reliable, and agile data solutions can provide key consumer insights and give you the power to make more informed business decisions