SUMMARY:

• Around 10+ years of IT experience in full System Development Life Cycle (Analysis, Design, Development, Testing, Deployment & Support) using WATERFALL & AGILE methodologies. Expert in BigData/Hadoop with strong skills in providing solutions to business problems using Big
• Data Analytics. Extensive experience in implementing, consulting & managing Hadoop Clusters & eco system components like HDFS, MapReduce, Pig, Hive, Sqoop, Flume, Oozie, Zookeeper & HBase.
• Strong architectural experience in building large scale distributed data processing and In - depth knowledge of Hadoop Architecture MR1 & MR2 (YARN)
• Experience in writing analytical applications with MapReduce and processing implementation.
• Experience in writing Pig scripts and custom UDF's for cleansing the data
• Experience in creating Hive Internal and External tables and optimizing the hive query
• Experience in partitioning, bucketing and joining Hive tables for Hive query optimization
• Experience in working with Flume to load the log data from multiple sources directly into HDFS
• Experience in importing and exporting data from HDFS to RDBMS and vice-versa using Sqoop,
• Experience in writing Oozie workflow engine to run multiple jobs
• Experience in configuring the Zookeeper to coordinate the servers in clusters and to maintain the data consistency
• Experience working with Real-time data Streaming & Processing using Kafka & Storm
• Experienced in Amazon AWS cloud services (EC2, EBS and S3).
• Experience in using SequenceFile, RCFile, ORC, Parquet file formats and AVRO data serialization
• Experience in Jobs management & performance tuning using Fair scheduler & Capacity scheduler
• Good Exposure to technologies like Solr, Falcon, Phoenix, Chukwa, Whirr & Tez
• Worked on real-time in-memory processing engines such as Spark, Impala and integration with BI Tools such as Tableau.
• Working in Onsite/Offshore delivery model when development centers span in multiple countries
• Configure and monitor Hadoop clusters with different Hadoop distributions (Cloudera, Hortonworks, Apache)
• Experience in monitoring and managing the Hadoop cluster using Cloudera Manager
• Experienced in administrative tasks such as installing, configuring, commission & de-commission nodes, troubleshooting, backups and recovery of Hadoop and its ecosystem components
• Experience in installing, configuring and deploying on premise Hadoop development cluster on AWS EC2.
• Extensive experience in implementing, consultation, managing and Benchmarking Hadoop clusters
• Good Knowledge on R and DataScience and configuration management tool CHEF
• Experience in all facets of Software Development Life Cycle (SDLC), including requirements gathering, designing, coding, testing, and deployment
• Committed to excellence, self-motivator, team-player, and a far-sighted developer with strong problem-solving skills and with zeal to learn new technologies.
• Strengths include good team player, excellent communication interpersonal and analytical skills and ability to work effectively in a fast-paced, high volume, deadline-driven environment
• Adept at gathering and documenting requirements, assessing business objectives, and managing all stages of software development and implementation.
• Excellent interpersonal skills and communication skills.

TECHNICAL SKILLS:

Big Data Technologies: HDFS, MapReduce, YARN, Cloudera manager and Navigator

NoSQL: HBase, MongoDB, Cassandra

RDBMS: MySQL, Oracle, DB2

Data Ingestion Tools: Flume, Sqoop

Realtime & Adhoc Queries: Impala, Elastic Search, Hive, Hue

Monitoring Tools: Ganglia, Nagios, Splunk

Visualization Tools: Tableau, Pentaho, Kibana

Realtime Streaming & Processing: Storm, Spark

Languages: C, C++, Java, Scala, Python

Data Mining Tools: R, SPSS

Development Tools: Eclipse, RSA, RAD

Build and Log Tools: Build tools (ANT, MAVEN), Logging tool(Log4J), Version Control (CVS, SVN, Clear Case, GitHub)

Methodologies & Standards: Waterfall Model and Agile

Operating Systems: Unix, Linux (Redhat, CentOS, Ubuntu), Mac & Windows

PROFESSIONAL EXPERIENCE:

Senior Professional Application Architect

Confidential

Responsibilities:

• Designing technical architecture and developed various Big Data workflows using MapReduce, Hive, YARN, Kafka, Storm & Spark
• Deployed on premise cluster and tuned the cluster for optimal performance for job execution needs and processes large data sets.
• The logs that are stored on HDFS are analyzed and the cleaned data is imported into Hive warehouse which enabled end business analysts to write Hive queries.
• Configured various big data workflows to run on the top of Hadoop using oozie and these workflows comprises of heterogeneous jobs like Hive, Pig, MapReduce & Storm
• Experience in working with search engine ElasticSearch in getting real time data analytics integrating with Kibana dashboard.
• Developed suit of Unit Test Cases for Mapper, Reducer and Driver classes using MR Testing library.
• Benchmarked Kafka message broker for High throughput
• Performed performance tuning & troubleshooting of MapReduce jobs by analyzing and reviewinglog files
• Implemented Kafka Custom Producer/Consumer for publishing messages to topic and subscription from the topics and written the topology.
• Written Spouts to read data from Kafka message broker and passing to processing logic
• Written Bolts to filter, aggregate, join interacting with data stores and emit tuples for the subsequent bolts to process
• Written Storm topology which defines the flow of data between the edges
• Performed CRUD operation on Cassandra and executed CQL queries(Cassandra Query Language)
• Experience in data migration from RDBMS & processed events from Storm Bolts to Cassandra
• Worked with various HDFS file formats like Avro, Sequence File and various compression formats like Snappy, bzip2 & lz2.
• Used Maven extensively for building jar files of Storm topology and deployed to Cluster.
• Used Git as version control to checkout and check-in of files.
• Analyzing the requirements, designing and developing solutions.
• Managing Project team in achieving the project goals including resource allocation, resolving technical issues and mentoring the resources.
• Providing technical solutions/assistance to all development projects
• Advising various teams on the impact of new enhancements/products
• Bug fixing and 24-7production support running processes.
• Participated in SCRUM Daily stand-up, sprint planning, Backlog grooming & Retrospective meetings.

Environment: MapReduce, Pig, Hive, FLUME, JDK 1.6, Linux, Kafka, Storm, Spark, Elastic-Search, Cassandra, YARN, Hue, HiveServer2, Impala, HDFS, Oozie, Splunk, Git, Kibana, Linux Scripting

Confidential, IL

Responsibilities:

• Design and Development Application Architecture and setup Hadoop Environment.
• Set up Splunk Servers and Forwarders on the cluster nodes
• The configuration for additional data nodes was managed using Chef
• Written Linux Scripts & Cron Jobs for Monitoring Services & the health Cluster health.
• Developed Map Reduce programs to cleanse and parse data in HDFS obtained from various data sources and to perform joins on the Map side
• Written M/R jobs to process trip summary & scheduled to execute hourly, daily, weekly, monthly & quarterly.
• Responsible for loading machine data into Hadoop cluster coming from different sources using Flume
• Written workflow on Oozie to schedule M/R Jobs
• Configured Flume and written Custom Sinks and Sources
• Used Flume to collect, aggregate, and store the log data from different web servers.
• Ingested data into HBase and retrieve using Java API's
• Used SPARK SQL from extracting data from different data sources and placing the processed data into NoSQL(MongoDB)
• Used SPARK for analyzing the machine emitted & sensor data to help extracting data sets for meaningful information such as location, driving speed, acceleration, braking speed, driving pattern and so on.
• Created SPARK SQL(metadata) tables to store the processed results in a tabular format
• Used Git as version control to checkout and check-in of files.
• Reviewed high level design & code & mentoring team members.
• Participated in SCRUM Daily stand-up, sprint planning, Backlog grooming & Retrospective meetings.

Environment: Hadoop, MapReduce, OpenStack, Flume-NG, Free IPA, HBase 0.98.2, MongoDB, Spark, Kerberos, PostgreSQL, RabbitMQ Server, Map/Reduce, HDFS, ZooKeeper, Oozie, Splunk, GitHub, Chef

Senior BigData/Hadoop Developer

Confidential

Responsibilities:

• Design and Development of Hadoop Stack
• Analyzed the functional specification
• The configuration of data nodes on the cluster was managed using CHEF.
• Responsible to manage data coming from different sources and involved in HDFS maintenance and loading of structured and semi-structured data.
• Load data to External tables by using Hive Scripts
• Performed aggregate Joins, transformation using Hive queries
• Implemented Partitions, Dynamic Partitions, Buckets in Hive
• Optimized HIVE SQL queries and thus improved the job performance
• Developed Sqoop scripts to import and export the data from relational sources and handled incremental loading on the customer and transaction data by date
• Loaded data into the cluster from dynamically generated files using Flume and from relational database management systems using Sqoop
• Used Oozie to automate/schedule business workflows which invoke Sqoop, MapReduce and Pig jobs as per the requirements
• Performed Hadoop cluster environment administration that includes adding & removing cluster nodes, cluster capacity planning, performance tuning, cluster monitoring, and trouble shooting
• Written Unit Test Cases for Hive Scripts

Environment: Hadoop, HDFS, MapReduce, Pig, Hive, Flume, Oozie, ZooKeeper, CHEF

Senior Software Engineer

Confidential

Responsibilities:

• Understanding the functional requirements of the client for designing the technical specifications, to develop the system and subsequently documenting the requirement
• Responsible for developing class diagrams, sequence diagrams
• Designed and implemented a separate middle ware Java component on Fusion
• Designing and coding of Business Classes using Spring features
• Prepared Unit test cases for the developed UI.
• Ensuring appropriate process standards are met and maintained

Environment: Java1.6, Oracle Fusion Middleware, Eclipse, Websphere, Spring F/w

Senior System Engineer

Confidential

Responsibilities:

• Understanding the functional requirements of the client for designing the technical specifications, to develop the system and subsequently documenting the requirement.
• Prepared LLD - Class Diagrams, Sequence Diagrams, Activity Diagram using Enterprise Architect UML Tool
• Development, checking Coding Standards
• Worked on Hibernate, Spring IOC, DAO, JSON Parsing
• Written POJO classes. .hbm files for Hibernate ORM
• Prepared Unit test cases for the developed UI.
• Responsible for problem tracking, diagnosis, replications, troubleshooting, and resolution of client problems.
• Ensuring appropriate process standards are met and maintained.
• Involved in preparing Adhoc Reports.

Environment: Java, ACG Proprietary F/w using DOJO, Hibernate, Spring, DB2, RSA, Rational ClearCase, RPM, RQM, Mantis

IT Consultant

Confidential

Responsibilities:

• Understanding the functional requirements of the client for designing the technical specifications, to develop the system and subsequently documenting the requirement.
• Prepared LLD - Class Diagrams, Sequence Diagrams, Activity Diagram using Enterprise Architect UML Tool
• Developing UI on JSF with RichFaces
• Writing TestNG test Cases
• Prepared Unit test cases for the developed UI.
• Involved in writing code in Core Java, JSP and Struts Framework.
• Setting up Environment for servers, Monitoring the calls
• Responsible for problem tracking, diagnosis, replications, troubleshooting, and resolution of client problems.
• Ensuring appropriate process standards are met and maintained.
• Involved in preparing Adhoc Reports.

Environment: Windows, Unix, Java, Struts, Hibernate, Tomcat, Lenya, Remedy Tool, WinSCP, Putty, VPN, Eclipse

Confidential

Responsibilities:

• Studying CR, understanding functionality and coding.
• Development for HRMS 6.x Application which supports various clients like Omnicom, Mervyns, Pepsi, Cintas, etc.,
• Debugging, enhancement of framework Java objects
• Unit Testing
• Replicate the Issues and fixing.
• Version Maintenance in CM Synergy Continuous

Environment: Windows, Unix, Java, Struts, JSP, Servlet, Tomcat, DB2, Eclipse, Lotus Notes 6.5, CM Synergy, Junit