Senior Data Engineer

Popis práce

Our is a London based technology company on its journey to become the Smart Data Analytics Platform of choice for Financial Services professionals globally. The Platform gives influential decision makers a competitive edge by ingesting a large number of data sources, structuring and credentialling unstructured data, running advanced analytics and machine learning models to derive decision insights - in a World’s First Fully Integrated Solution.

 

DETAILED ROLE DESCRIPTION:

Senior Data engineer helps to architect, design, implement, and optimise our data ingestion, transformation and spreading pipelines and processes. He or she develops data models, processing pipelines, back-end services supporting the data science teams, automates processes, build integrations, builds analytics.

DESIRED SKILLS:

  • A university degree from a reputed University with a record of academic excellence in Mathematics, Computer Science, Engineering, Physics or similar.
  • 8+ years of relevant experience in Data Engineering, warehousing, ETL, automation, big data and cloud technologies, etc.
  • Ability to write clean, scalable, maintainable code in Python. Has good understanding of software engineering concepts and patterns. Proficiency in other languages like Scala, Java, C#, C++ are an advantage.
  • Proven record of building and maintaining data pipelines deployed in at least one of the big 3 cloud ML stacks (AWS, Azure, GCP).
  • Hands on experience with
    1. Open-source ETL, and data pipeline and orchestration such tools as Apache Airflow and Nifi.
    2. Large Scale/ Big Data technology, such as Hadoop, Spark, Hive, Impala, PrestoDb, Kafka.
    3. Workflow orchestration tools like Apache Airflow.
    4. Containerisation with Docker, deployment on Kubernetes.
    5. NoSql and graph databases.
    6. Unix server administration and shell scripting.
  • Experience in building scalable data pipelines for highly unstructured data.
  • Experience in building DWH and data lakes architectures.
  • Experience in working in cross-functional team with software engineers, data scientists and machine learning engineers.
  • Experience in working with or leading an off-shore team.
  • Proven record of building data science environments deploying ML solutions in at least one of the big 3 cloud ML stacks (Azure/Aws/GCP) and on Kubernetes clusters.
  • Excellent written and verbal command of English.
  • Strong problem-solving, analytical, and quantitative skills.
  • A professional attitude and service orientation with the ability to work with our international teams based out of India and European Union.
show
#6fcfe1
Woman thinking and looking to the right

Vyjádřete svůj názor v našem Průzkumu mezd 2024

a předběžně se zaregistrujte a získejte ZDARMA digitální kopii Průvodce mzdami 2024