Your Role:

• Lead a mid-size team of Data Engineers.

• Create and maintain optimal data pipeline architecture.

• Assemble large, complex data sets that meet functional / non-functional business requirements.

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using ETL processes and modern cloud technologies.

• Take ownership or clarification of requirements and solutions proposition before implementation.

• Lead the building of scaled machine learning production systems by designing pipelines and engineering infrastructure.

• Facilitate the development and deployment of offline ML models into production through the use of scalable tools and services to handle machine learning training and inference processes.

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.

• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.

• Keep up to date with latest open-source tools for data engineering.

• Mentor junior team members in ML Production best practices.

• Design, develop, implement, and debug large and complex data platforms.

• Analyze and improve performance of existing platforms.

• Implement new technologies, policies and practices that can help increase resiliency, automation and improve platform health.

• Identifying technology gaps and help business build and deliver viable solutions.

• Drive the evolution of Data & Services products/platforms with an impact-focused on data science and engineering.

• Participate in the development of data and analytic infrastructure for product development.

• Continuously innovate and determine new approaches, tools, techniques & technologies to solve business problems and generate business insights & recommendations.

• Partner with roles across the organization including consultants, engineering, and sales to determine the highest priority problems to solve.

• Evaluate trade-offs between many possible analytics solutions to a problem, taking into account usability, technical feasibility, timelines, and differing stakeholder opinions to make a decision.

• Break large solutions into smaller, releasable milestones to collect data and feedback from product managers, clients, and other stakeholders.

• Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.

• Work with small, cross-functional teams to define the vision, establish team culture and processes.

• Consistently focus on key drivers of organization value and prioritize operational activities accordingly.

• Maintain awareness of relevant technical and product trends through self-learning/study, training classes, and job shadowing.

Ideal Candidate Qualifications:

Experience leveraging open-source tools, predictive analytics, machine learning, Advanced Statistics, and other data techniques to perform basic analyses.

• Demonstrated basic knowledge of statistical analytical techniques, coding, and data engineering.

• Experience developing and configuring dashboards is a plus.

• Demonstrated judgement when escalating issues to the project team.

• High proficiency in Python/Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), SQL.

• Curiosity, creativity, and excitement for technology and innovation.

• Demonstrated quantitative and problem-solving abilities.

• Ability to multi-task and strong attention to detail.

• Motivation, flexibility, self-direction, and desire to thrive on small project teams.

• Expert proficiency in using Python/Scala, Spark(tuning jobs), SQL, Hadoop platforms to build Big Data products & platforms.

• Experience with data pipeline and workflow management tools: NIFI, Airflow.

• Comfortable in developing shell scripts for automation.

• Proficient in standard software development, such as version control, testing, and deployment.

• Experience with visualization tools like tableau, looker.

• At least 5 year leading collaborative work in complex engineering projects in an Agile setting e.g. Scrum.

• Extensive data warehousing/data lake development experience with strong data modeling and data integration experience.

• Good SQL and higher-level programming languages with solid knowledge of data mining, machine learning algorithms and tools.

• Strong hands-on experience in Analytics & Computer Science.

• Demonstrated basic knowledge of statistical analytical techniques, coding, and data engineering.

• Experience in building and deploying production-level data-driven applications and data processing workflows/pipelines and/or implementing machine learning systems at scale in Java, Scala, or Python and deliver analytics involving all phases like data ingestion, feature engineering, modeling, tuning, evaluating, monitoring, and presenting.

• Outstanding communication and organizational skills.

• Strong English written and verbal communication skills.

• At least 10 years of relevant hands-on experience as a Data Engineer in an individual contributor capacity.

• Able to lead the implementation of machine learning production systems.

• Demonstrated ability, through hands-on experience, to develop production machine learning pipelines.

• At least a bachelor’s degree in computer architecture, Computer Science, Electrical Engineering or equivalent experience. Postgraduate degree is an advantage.

The following skills will be considered as a plus

• Hands-on experience with cloud computing and big data frameworks e.g. GCP, AWS, Azure, Flink, Elasticsearch, and Beam

• Knowledge in MLOps frameworks such as TensorFlow Extended, Kubeflow, or MLFlow

• Financial Institution or a Payments experience a plus

• Experience in managing/working in Agile teams

• Experience developing and configuring dashboards