itgeek.vnbeta
The best job search engine for the IT community
Big Data - Hadoop Engineer (up to 4000Usd)
Ha Noi
Tech stack
Thông tin công việc
Mô tả
Salary up to 4000USD with JOINING BONUS 30,000,000VND
As a Hadoop big data engineer, you will develop, operate and drive scalable and resilient data platform based on Hadoop ecosystem to address the business requirements:
• Ensure industry best practices around data pipelines, metadata management, data quality, data governance and data privacy
• Design and implement business-specific large-scale data processing pipelines
• Work with complex data structures, manipulate, cleanse data, and perform transformations to make insights from data.
• Responsible to Ingest data from files, streams, and databases. Process the data with PySpark, Kafka, Hive, Hive LLAP…
• Develop efficient software code for multiple use cases leveraging Spark and Big Data Technologies for various use cases built on the platform
• Provide high operational excellence guaranteeing high availability and platform stability.
Yêu cầu
• Experience in Hadoop ecosystem including HDFS, MapReduce, YARN, HBase, Zookeeper, Pig, Hive…
• Experience with Hadoop distributions such as Cloudera, HortonWorks, GCP…
• Experience with Apache Spark using PySpark
• Experience with Apache Kafka
• Experience with Apache Beam
• Experience with SRE, Patching & Automation: Kubernetes or Docker & Containerization
• Good in programming language Python, Java
• Good to have cloud Skills and Experience, SQL/NoSQL database on Cloud, especially GCP
• Experience in building large-scale data processing (batch-processing, stream processing)
Candidate Data Privacy Notice: www.hcltech.com/candidate-privacy-notice
Thông tin khác
100% salary in probation period
Premium Health Insurance service for Employee and Family
18 leaves per year
Nơi làm việc
- Ha Noi