This job was posted by https://www.azjobconnection.gov : For more information, please see: https://www.azjobconnection.gov/jobs/6443031

Development of a big data platform in Hadoop using Spark, Oozie, and related; Implement and support big data tools, frameworks such as HDFS, HIVE and streaming technologies such as Kafka and Spark; Set standards for warehouse and schema design; Write HIVE Queries to process and manage data in external tables; Automating task of loading the data into HDFS by developing workflow jobs in Oozie; Develop/build Maven scripts to integrate them with Jenkins and automate the compilation, deployment and testing of web applications through XL Release; feed machine learning models in big data environment; Developing, optimising hadoop/Spark jobs and automation of hadoop/spark jobs using shell scrip, event engine tool; Developing scala, python and java based rest API for batch processing, Real time processing of streaming requests through API in big data platform; Responsible for unit testing, debugging, verifying splunk logs and deploying the production ready code and various micro services for US and International market projects. Migrating big-data projects to new architectural platform and configuration driven applications from old platform using technologies Java, Hadoop Distribution, Map-Reduce, HIVE, Shell Scripting, SQL, REST Services; Developing and maintaining the big-data pipelines (batch and real-time) from end-to-end flow. Required Masters inInfo.Sys/CS/Inf.Tech/related plus 6 months experience asProgrammer/System Analyst/Soft.Dev/Soft.Engg/related. Travel may be required.