We provide IT Staff Augmentation Services!

Sr. Big Data/hadoop Developer Resume

Nyc, NY

SUMMARY:

  • Over 8 years of combined experience in IT industry, played major role in implementing, developing and maintenance of various Web Based applications using Java, J2EE Technologies and Big Data Ecosystem.
  • Around 3+years of work experience as Hadoop Developer, who expertise in providing end to end solutions for real time bigdata problems.
  • 3+ years of hands on experience in Hadoop Ecosystem components such as Hive, Pig, Sqoop, Flume, Impala, Oozie, Zookeeper, HBase.
  • Hands on experience on working in multiple domains such as Healthcare, Finance & Banking etc.
  • Strong knowledge of Hadoop Architecture and Daemons such as HDFS , Job Tracker , Task Tracker , Name None , Data Node and Map Reduce concepts.
  • Hands on experience in writing Map Reduce programs using Java to handle different data sets using Map and Reduce tasks .
  • Developed multiple Map Reduce jobs to perform data cleaning and preprocessing.
  • Involved in designing the data model in Hive for migrating the ETL process into Hadoop and wrote Pig Scripts to load data into Hadoop environment
  • Designed HIVE queries & Pig scripts to perform data analysis, data transfer and table design.
  • Expertise in writing Hive UDF, Generic UDF's to in corporate complex business logic into Hive Queries .
  • Experienced in optimizing Hive queries by tuning configuration parameters.
  • Implemented SQOOP for large dataset transfer between Hadoop and RDBMS.
  • Extensively used Apache Flume to collect the logs and error messages across the cluster.
  • Experience in working on NO - SQL databases like HBase, Cassandra, MongoDB
  • Experienced in performing real time analytics on HDFS using HBase.
  • Experienced in transporting, and processing real time event streaming using Kafka and Storm.
  • Experienced in Real time data ingestion into HBASE and HIVE using Storm.
  • Experience in implementing applications on Spark frameworks.
  • Hands on Experience with ETL Tool using Talend, Informatica.
  • Experience in working with Cloudera (CDH3 & CDH4&CDH5 ) and Horton Works Hadoop Distributions.
  • Hands on experience on AWS infrastructure services Amazon Simple Storage Service ( Amazon S3 ) and Amazon Elastic Compute Cloud ( Amazon EC2 ).
  • Good Knowledge on Apache Solr , Elasticsearch .
  • Worked with Oozie and Zookeeper to manage the flow of jobs and coordination in the cluster
  • Experience in performance tuning, monitoring the Hadoop cluster by gathering and analyzing the existing infrastructure using Cloudera manager.
  • Experience with configuration of Hadoop Ecosystem components: Map Reduce, Hive, HBase, Pig, Sqoop, Oozie, Zookeeper, Flume, Storm, Spark, Yarn, Tez.
  • Hands on experience on implementing Core Java concepts such as collections framework, garbage collection, Exception handling.
  • Experienced writing Test cases and implement unit test cases using testing frame works like Junit, Easy mock and mockito.
  • Adequate knowledge and working experience with Agile and waterfall methodologies and performed role of Scrum master
  • Good understanding of all aspects of Testing such as Unit, Regression, Agile, White-box, Black-box.
  • Experience in developing applications using Struts, Enterprise Java Beans (EJB), Servlets, JPA, JMS, SOAP, JSF, Hibernate, spring frameworks.
  • Extensive experience in implementing/ consume SOAP and Rest Based Web Services.
  • Good knowledge of Web/Application Servers like JBoss, Apache Tomcat, IBM Web Sphere and Oracle Web Logic.
  • Working with onsite and offshore team members, mentoring junior team members and ability to work in team.
  • Able to work on own initiative, highly proactive, self-motivated commitment towards work and resourceful.
  • Strong debugging and critical thinking ability with fabulous understanding of frameworks advancement in methodologies and strategies.

TECHNICAL SKILLS:

Bigdata Technologies: Hadoop, MapReduce, HDFS, Hive, Pig, Zookeeper, Sqoop, Oozie, Flume, IMPALA, HBASE, Kafka, Storm

Big Data Frameworks: HDFS, YARN, Spark

Hadoop Distributions: Cloudera(CDH3,CDH4,CDH5),Hortonworks, Amazon EMR

Programming Languages: Java, C, C++,shell scripting, Scala

Databases: RDBMS, MySQL, Oracle, Microsoft SQL Server, Teradata, DB2, PL/SQL, CASSANDRA, MongoDB

IDE and Tools: Eclipse, NetBeans, Tableau

Operating System: Windows XP/vista/7, Linux/Unix

Frameworks: Spring, Hibernate, JSF, EJB, JMS

Scripting Languages: JSP & Servlets, JavaScript, XML, HTML, Python

Application Servers: Apache Tomcat, Web Sphere, Web logic, JBoss

Methodologies: Agile, SDLC, Waterfall

Web Services: Restful, SOAP

ETL Tools: Talend, Informatica

Others: Solr, elasticsearch

PROFESSIONAL EXPERIENCE:

Confidential, NYC, NY

Sr. Big Data/Hadoop Developer

Responsibilities:

  • Experience with Hortonworks distribution of Hadoop.
  • Experience in Importing and exporting data into HDFS and Hive using Sqoop.
  • Hands on experience in defining, partitioning, bucketing, compressing Hive tables to meet business requirement.
  • Experience in performance tuning of Hive queries.
  • Implemented Ad-hoc query using Hive to perform analytics on structured data.
  • Worked extensively with HIVE DDLs and Hive Query language (HQLs) and implemented business logic using Hive UDF's to perform ad-hoc queries on structured data.
  • Implemented Optimized joins to perform analysis on different data sets using Map Reduce programs.
  • Written Hive queries for data analysis to meet the business requirements.
  • Hands on experience in writing Map Reduce programs to meet business needs.
  • Hands on experience in Sequence files, RC files, Combiners, Counters, Dynamic Partitions, Bucketing for best practice and performance improvement.
  • Implemented secondary sorting to sort reducer output globally in map reduce.
  • Implemented data pipeline by chaining multiple mappers by using Chained Mapper.
  • Hands on experience in writing Linux/Unix Shell scripting.
  • Experienced in transporting, and processing real time event streaming using Kafka and Storm.
  • Experienced in Real time data ingestion into HBASE and HIVE using Storm.
  • Experience in implementing applications on Spark frameworks.
  • Experienced in defining CRON job flows.
  • Have solid understanding of REST architecture and its applications to well performing web sites for global usage.
  • Experience in managing and reviewing Hadoop log files by developing Pig program for loading and filtering the streaming data into HDFS using Flume.
  • Implemented test scripts to support test driven development and continuous integration.
  • Responsible for managing data coming from disparate data sources.
  • Used Pig as ETL tool to do transformations, event joins, filtering and some pre-aggregations before storing the data onto HDFS.
  • Hands on experience in writing, executing pig scripts.
  • Hands on experience in writing Pig UDFs.
  • Configured Oozie work flows to automate data flow, preprocess and cleaning tasks using Hadoop Actions.
  • Hands on Experience with ETL Tool using Informatica.
  • Knowledge on rendering and delivering reports in desired formats by using reporting tools such as Tableau.

Environment: Hadoop, Hive, Map Reduce, HDFS, Pig, Sqoop, Java (JDK 1.6), Java 6, Eclipse, Oracle 10g, PL/SQL, SQL*PLUS, Linux, flat files, elasticsearch, Amazon EMR

Confidential, New York, NYC

Big data Hadoop Developer/Hadoop Administrator

Responsibilities:.

  • Experience with Cloudera distribution of Hadoop.
  • Worked on Hadoop cluster which ranged from 5-8 nodes during pre-production stage and it was sometimes extended up to 26 nodes during production
  • Experience in Importing and exporting data into HDFS and Hive using Sqoop.
  • Developed Pig program for loading and filtering the streaming data into HDFS using Flume.
  • Experienced in handling data from different data sets, join them and pre process using Pig join operations.
  • Moving Bulk amount data into HBase using Map Reduce Integration.
  • Developed Map-Reduce programs to clean and aggregate the data
  • Developed HBase data model on top of HDFS data to perform real time analytics using Java API.
  • Developed different kind of custom filters and handled pre-defined filters on HBase data using API.
  • Used Sqoop to import the data from RDBMS to Hadoop Distributed File System (HDFS) and later analyzed the imported data using Hadoop Components
  • Responsible for building scalable distributed data solutions using Hadoop.
  • Implemented counters on HBase data to count total records on different tables.
  • Experienced in handling Avro data files by passing schema into HDFS using Avro tools and Map Reduce.
  • Worked on custom Pig Loaders and Storage classes to work with a variety of data formats such as JSON, Compressed CSV, etc.
  • Implemented secondary sorting to sort reducer output globally in map reduce.
  • Implemented data pipeline by chaining multiple mappers by using Chained Mapper.
  • Worked with join patterns and implemented Map side joins and Reduce side joins using Map Reduce.
  • Hands on experience in Sequence files, RC files, Combiners, Counters, Dynamic Partitions, Bucketing for best practice and performance improvement.
  • Created Hive Dynamic partitions to load time series data.
  • Experienced in handling different types of joins in Hive like Map joins, bucker map joins, sorted bucket map joins.
  • Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts.
  • Created tables, partitions, buckets and perform analytics using Hive ad-hoc queries.
  • Provided batch processing solution to certain unstructured and large volume of data by using Hadoop MapReduce framework
  • Experience with configuration of Hadoop Ecosystem components: Map Reduce, Hive, HBase, Pig, Sqoop, Oozie, Zookeeper, Flume, Storm, Spark, Yarn, Tez.
  • Automated common maintenance and installation mechanisms.
  • Monitored jobs and other hardware/internal aspects of the cluster.
  • Assembled newly bought hardware into racks with switches, assign IP addresses properly, firewalling, enable/disable ports, VPN etc
  • Experience in Commissioning/de- commissioning of nodes, and providing backup and restore etc
  • Experience in enabling users to view job progress via web.
  • Worked with Cloudera on issues and applied upgrades and security patches
  • Experience in applying Security ( Kerberos ) and linking with Active Directory and/or LDAP .
  • Experience with CDH distribution and Cloudera Manager to manage and monitor Hadoop clusters.
  • Involved in story-driven agile development methodology and actively participated in daily scrum meetings.

Environment: Apache Hadoop, HDFS, Hive, Pig, Map Reduce, Cloudera CDH4,Solr,Oracle, MySQL, HBase, Flume, Sqoop, Oozie, MRunit, Java, Web services, Eclipse, Shell Scripting, LDAP, Cloudera Manager

Confidential, Kansas City, MO

Java/J2EE Developer

Responsibilities:

  • Actively participated in software development lifecycle (scope, design, implement, deploy, test), including design and code reviews.
  • Developed user interface using HTML, CSS, JSPs and AJAX
  • Client side validation using JavaScript and JQuery
  • Performed client side validation with JavaScript and applied server side validation as well to the web pages.
  • Developed the application leveraging the Model View Layer (MVC) architecture, Build tools Maven, ANT.
  • Used JIRA for BUG Tracking of Web application.
  • Used SVN version controlling to maintain project versions.
  • Developed Maven and Ant scripts to assist in the Continuous Integration
  • Managed the team’s build tasks with Jenkins CI and Subversion with Jenkins build script
  • Stateless Session beans were used to carry out client requests.
  • Designed and developed JMS service using message driven beans
  • Written web services SOAP, WSDL and web services clients using Axis and XML, XSD implemented SOA architecture.
  • Written Spring Core and Spring MVC files to associate DAO with Business Layer.
  • Worked with HTML, DHTML, CSS, and JAVASCRIPT in UI pages.
  • Wrote Web Services using SOAP for sending and getting data from the external interface.
  • Extensively worked with JUnit framework to write JUnit test cases to perform unit testing of the application
  • Implemented JDBC modules in java beans to access the database.
  • Designed the architecture, tables for the back-end Oracle database.
  • Application hosted under Web Logic and developed utilizing Eclipse IDE.
  • Used XSL/XSLT for transforming and displaying reports. Developed Schemas for XML.
  • Involved in writing the ANT scripts to build and deploy the application.
  • Developed a web-based reporting for monitoring system with HTML and Tiles using Struts framework.
  • Middleware Services layer is implemented using EJB (Enterprise Java Bean - stateless) in Web Sphere environment
  • Worked on Session Beans for user authentication and preventing denial of service attacks.
  • Application was tested using JUnit.
  • Designed the Application using Model View Controller Architecture.
  • Developed different modules using J2EE (Servlets, JSP, JDBC)
  • Integrated the Application with Database using Hibernate.
  • Communicated between different applications using JMS.
  • Created web services using RESTful , to allow application and services to connect to them over the Internet
  • Interacted with backend Oracle and wrote stored procedures, functions and cursors for deleting and inserting using PL/SQL.
  • Deployed the Application in Web Logic App Server.
  • Developed test cases and performed unit testing using JUnit framework.
  • Worked on production issues.
  • Interacted with the Users and Documented the Application

Environment: Java, Web Logic App Server, Struts, JDBC, JSP, JMS, Restful, HTML, XML/XSL/XSLT, J2EE, PL/SQL, SQL, Oracle 9i, JUnit, Windows.NET.

Confidential, Southlake, TX

Java Developer

Responsibilities:

  • Involved in various SDLC Life cycle phases like development, deployment, testing, documentation, implementation & maintenance of application software.
  • Extensively used Core Java, Servlets, JSP and XML.
  • Used Struts 1.2 in presentation tier.
  • Involved in writing JSP and JSF components. Used JSTL Tag library (Core, Logic, Nested, and Bean and Html taglib’s) to create standard dynamic web pages.
  • Application was based on MVC architecture with JSP serving as presentation layer, Servlets as controller and Hibernate in business layer to access to Oracle Database.
  • Developed the DAO layer for the application using Spring Hibernate Template support.
  • Used XML and JSON for transferring/retrieving data between different Applications.
  • Used the Spring Dependency Injection and developed the presentation layer using Struts MVC Framework.
  • Used JQuery for creating JavaScript behaviors.
  • Business-logic controller framework development utilizing spring framework and dependency injection, catering to both standard server-side and AJAX client requests.
  • Collaborated with technical architects to ensure that the design meets the requirements.
  • Implemented the JBoss server logging configuration which is represented by the logging subsystem.
  • Implemented EJB Session Beans and MDB.
  • Responsible for developing JUnit test cases using Easy Mock and DB units for unit and integration units.
  • Used Maven script for building and deploying the application.
  • Assisted in development and improvement of application maintenance plans, processes, procedures, standards and priorities.
  • Generated the Hibernate XML and Java Mappings for the schemas
  • Used DB2 Database to store the system data
  • Used Rational Application Developer (RAD) as Integrated Development Environment (IDE)
  • Used unit testing for all the components using JUnit
  • Used Apache log 4j Logging framework for logging of trace and Auditing.
  • Used Asynchronous JavaScript and XML (AJAX) for better and faster interactive Front-End.
  • Involved in developing UML Diagrams like Use Case, Class, Sequence diagrams.

Environment: Java, J2EE, HTML, Spring 3, Hibernate 3, Web Services, JBoss, Java SDK, SOA, JavaScript, JMS, Struts 2.0, JSON, Servlets 2.2, JSF, JSP, CSS3, HTML, DHTML, JQuery, AJAX, JDBC, Oracle, Clear Case, TOAD, XML, Scrum, Unix/Linux.

Confidential

Java Developer

Responsibilities:

  • Involved in various SDLC Life cycle phases like development, deployment, testing, documentation, implementation & maintenance of application software.
  • Developing Identity Management Application with Single Sign On using SAML
  • Working with Shibboleth Identity Provider and Service Provider
  • Using IIS and Apache for web Server.
  • Developed analysis level documentation such as Use Case Model, Activity, Sequence and Class Diagrams.
  • Developed the application using Struts MVC for the web layer.
  • Developed UI layer logics of the application using JS P, JavaScript, HTML/DHTML, and CSS.
  • Developed business components of the applications using EJB.
  • Involved in developing complex Hibernate mapping files, mapping different kinds of associations between tables.
  • Developed queries using PL/SQL to retrieve data from the database.
  • Developed Test plans, cases and executed them TEST and Stage environments.
  • Developed GUI and Business Logic using JSP and Servlets.
  • Involved in requirements gathering and converting them into specifications.
  • Designed JSP pages using different t Tag libraries.
  • Involved in bug fixing, enhancements and support.
  • Created Stored Procedures, Triggers for the application.
  • Developed unit test cases using JUnit for testing functionalities and performed integration testing of the application.
  • Implemented client side validations using JavaScript functions.
  • Support to UAT, production environments and resolving issues with other deployment and testing groups.
  • Implemented REST web services for Deployment and Publishing purposes.
  • Wrote SQL queries that store and retrieve information, Worked on Microsoft SQL server and Postgre SQL database.
  • Developing UI using different Frontend Technologies like HTML, CSS, JavaScript

Environment: Java, J2EE, Spring, JSP, Hibernate, Java Script, CSS, JDBC, IntelliJ, LDAP, REST, Active Directory, SAML, Web Services, Microsoft SQL Server, HTML.

Hire Now