SUMMARY:

• Around 7 years of programming and software development experience with skills in data analysis, design and development, testing and deployment of software systems from development stage to production stage.
• Excellent knowledge on Hadoop and ecosystems such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigm.
• Experience in working with BI team and transform big data requirements into Hadoop centric technologies.
• Experience in Installing, Configuring and Administrating Hadoop cluster of major Hadoop distributions.
• Experience in deploying Hadoop cluster using Cloudera Manager and Hortonworks Ambari.
• Experience in performance tuning a Hadoop cluster by gathering and analyzing an existing infrastructure.
• Extensively worked with Unix and Linux file systems and their administration.
• Working experience on designing and implementing complete end - to-end Hadoop Infrastructure including Pig, Hive, Sqoop, Flume and zookeeper.
• Experience in ingesting streaming data into hadoop using Spark, Storm, Kafka Framework.
• Experience in working with flume to load the log data from multiple sources directly into HDFS.
• Experience in configuring the Zookeeper to coordinate the servers in clusters and to maintain the data consistency.
• Experience in Designing/developing ETL jobs across multiple platforms and tools including Hadoop.
• Experienced with various Services of AWS: EC2, Elastic Beanstalk, ELB, Route53, S3, CloudFront, CloudWatch, and CloudFormation.
• Experience in upgrading the existing Hadoop cluster to latest releases.
• Expert in working with Hive data warehouse tool - creating tables, data distribution by implementing Partitioning and Bucketing, writing and optimizing the HiveQL queries.
• Strong experience in writing Pig Latin scripts to process data in HDFS, and writing Pig UDFs (Eval, Filter, Load and Store) in Java to process complicated data. Stored pig relations into Hive table using HCatalog libraries.
• Experience in supporting analysts by administering and configuring Hive.
• Experience in providing support to data analyst in running Pig and Hive queries.
• ExpertImporting and exporting data into HDFS and Hive using Sqoop.
• Experience in writing scripts to dump the Shared data from MySql, Oracle, Teradata to HDFS.
• Good Knowledge on Splunk, Talend, Microsoft Azure, Kerberos, Knox, Ranger, Phoenix.
• Proficient programming abilities in Portal and Client-Server based Application using Java, J2ee, JSP, Servlet, Hibernate, Spring, Jdbc, Teradata, My-SQL Server.
• Experience in working on Restful and SOAP web services.
• Excellent exposure on Software Development Life Cycle & Software Testing Life Cycle.
• Adept in end-to-end development of software products from requirement analysis to system study, designing, testing, de-bugging, documentation and implementation.
• Experience working on Test Driven Development, Domain driven patterns and Agile Methodologies.

SKILL:

Big Data Ecosystems: Hadoop, Map-Reduce, EMR, YARN, HDFS, Zoo keeper, Hive, Pig, Impala, Spark, Storm, Sqoop, Flume,Oozie,Tez,Splunk, Kafka, Talend, Microsoft Azure, Kerberos, Knox, Ranger, Phoenix.

Cluster Administration Tools: Cloudera Manager, Cloudera Navigator, Hortonworks Ambari.

Programming Languages: Java, SQL, Python, Scala, Ruby, UNIX Shell Scripting.

Web Services: RESTful, SOAP.

Web Technologies: JQuery,JMS, JavaScript, Angular JS,Ext JS, CSS, HTML, XML

Web Servers: Apache Tomcat, BEA WebLogic, IBM WebSphere

Virtualization: AWS (EC2, Elastic Beanstalk, ELB, Route53, S3, CloudFront, CloudWatch, and CloudFormation.

Frameworks: Spring, Hibernate.

NoSQL Databases: HBase, MongoDB,Cassandra,MarkLogic, DynamoDB.

Databases: Teradata, MySQL, Oracle, DB2.

Version Control & Tools: SVN, Git, Git Hub, Rally, R, TableauOperating Systems:: Unix,Linux, Windows (XP/7/8/10)

EXPERIENCE:

Confidential, Miami, FL

Java/Hadoop consultant

Responsibilities:

• Created Complex ETL pipelines using java and Hadoop.
• Involved in loading data from different file systems to HDFS.
• Extracted the data from various RDBMS servers like Teradata, Oracle into HDFS using Sqoop. Developed custom MapReduce codes, generated JAR files for user defined functions and integrated it with Hive to extend the accessibility of statistical procedures within the entire analysis team.
• Managed 200+ Nodes CDH cluster with 14 petabytes of data using CM and Linux Cent OS.
• Installed and configured Cloudera Manager for easy management of existing Hadoop cluster.
• Deployed cluster and implemented ETL pipelines on AWS.
• Worked on Data Serialization formats for converting Complex objects into sequence bits by using AVRO, PARQUET, JSON, CSV formats.
• Maintenance of Splunk Environment with multiple Indexers.
• Used EMR, Kinesis and DynamoDB.
• Used Kafka to implement real time data ingestion pipelines.
• Data ingestion is done using Flume with source as Kafka Source & sink as HDFS.
• For one of the use case, used Spark Streaming with Kafka&HDFS/HBase to build a continuous ETL pipeline. This is used for real time analytics performed on the data.
• Used storm for an automatic mechanism for repeating attempts to download and manipulate the data when there is a hiccup.
• Developed Spark scripts by using Scala shell commands as per the requirement.
• Used Spark API over Hadoop YARN to perform analytics on data in Hive.
• Implemented Spark for fast interactive data analysis.
• Implemented Partitioning, Dynamic partitioning and Bucketing in Hive using internal and external table for more efficient data.
• Implemented Kerberos security authentication protocol for existing cluster.
• Designed and developed complex application flows using spring web flow and Hibernate.
• Generated Business Logic using servlets, Session beans and deployed them on Web logic server.
• Requirement Analysis, Design, Design Reviews, Coding, Code Reviews.
• Used GIT and Rally for integration.

Environment: Hadoop (HDFS/MapReduce), Pig, Hive,Spark, Scala,Storm,Sqoop, Kafka, HBase, Teradata, Oracle,RedHatLinux, Java, Spring, Servlets,Hibernate,Jsf, Html, Css, JavaScript.