Short Description
InterGlobe Technologies (IGT) is looking to Data Engineer (SSE) â?? Big Data technologies.Should have good understanding about Lambda architecture and Big Data analytics framework.Job Description
- Having over all 7+ years of IT development experience in Database, Java or any programming languages.
- Minimum 5+ years of working experience in the Apache Hadoop framework, HDFS, Map Reduce, Hive, Flume, Sqoop, Oozie, Spark, Spark Streaming and Scala.
- Experience in Big Data and batch/real-time ingestion and analytical solutions leveraging transformational technologies.
- Experience in Java, spring, Elasticsearch and Cassandra.
- Experience in data mining, data modelling, data processing, machine learning, natural language processing or information retrieval.
- Understand the source systems data, gather & able to transfer the data into HDFS as RAW and Decomposed layer (including writing Oozie workflow, coordinators, Sqoop and Flume)
- Capability to design and develop independently.
- Experience and knowledge of a wide variety of testing methods and tools covering functional, performance, regression, system integration, smoke testing of Business Intelligence, ETL, BigData ( Ingestion, Modeling, Serving layer) and Business Analytical landscape.
- Good understanding about Lambda architecture and Big Data analytics framework.
- Having exposure on the Cloudera Manager, Cloudera Navigator, Hue, Shell Scripting, Apache Pig and Python
- Good understanding in XSD, XML and JSON schemas
- Integrating Hadoop with Oracle and NoSQL databases
- Use technical tools or native programming languages, to find, identify and diagnose solution defects based on data models or data extracts. Write advanced SQL queries to verify the integrity of databases including data transformations
- Degree in Software Engineering, Computer Information Systems, Commerce, Business Administration, Operations Research or a related field
- Transparency and Collaboration
- Self-driven and Self-learning
- Problem solving and decision making
- End-End solution
- Appetite to develop in multiple platforms
Data Engineer