We provide IT Staff Augmentation Services!

Big Data /hadoop Consultant Resume

2.00/5 (Submit Your Rating)

Alpharetta, Ga

SUMMARY:

  • Over 6 years of experience in various IT sectors such as banking, health - care, and financial services, which includes hands-on experience in Big Data technologies.
  • 3 years of experience as a Hadoop Developer in all phases of Hadoop and HDFS development.
  • Excellent programming skills include Java/J2EE, Spring, Spring Boot, Web Services (Jersey REST Service), Struts, Hibernate, JSP, Servlets, JSP Custom Tag Libraries, JDBC, JMS, HTML, XML, JavaScript and AJAX.
  • Hands on experience with HDFS, MapReduce and Hadoop Ecosystem (Pig, Hive, Spark, Oozie, Hbase, Zookeeper, Flume and Sqoop).
  • Have experience with Spark processing Framework such as Spark and Spark Sql.
  • Experience in NoSQL databases like HBase and knowledge on NoSQL databases like Cassandra and Mongodb.
  • Procedural knowledge in cleansing and analyzing data using HiveQL, Pig Latin, and custom MapReduce programs in Java
  • Good experience working with Distributions such Hortonworks and Cloudera.
  • Hands-on experience with message broker such as Apache Kafka.
  • Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems (RDBMS), Teradata and vice versa.
  • Experience in creating workflows using Oozie to schedule jobs.
  • Strong experience in Hadoop Administration and Linux.
  • Experience in developing middleware applications using Spring REST API.
  • Worked extensively with Dimensional modeling, Data migration, Data cleansing, Data profiling, and ETL Processes features for data warehouses.
  • Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
  • Have exposure on tools like PLATFORA and DATAMEER.
  • Extensive experience with SQL, PL/SQL and database concepts.
  • Expertise in debugging and optimizing Oracle and java performance tuning with strong knowledge in Oracle 11g and SQL.
  • Experience in performance tuning the Hadoop cluster by gathering and analyzing the existing infrastructure.
  • Experience with creating unit test cases using Junit.
  • Strong in Object Oriented Programming and Distributed Computing.
  • Experience with a variety of application servers like BEA WebLogic, WebSphere and Tomcat.
  • Good Inter personnel skills and ability to work as part of a team. Exceptional ability to learn, master new technologies and to deliver outputs in short deadlines.

TECHNICAL SKILLS:

Languages:: JAVA, C, PL/SQL and SQL

J2EE Technologies: J2EE, J2EE Design Patterns, UML, JMS and JDBC

Frameworks: Struts, Hibernate, Spring and Spring REST API Services

Web/App Server: WebSphere 8.5, BEA Web logic 10.3 and Tomcat 6

Hadoop/Big Data: Hadoop, HDFS, Map Reduce, Hive, Pig, Sqoop, Spark, HBase, Flume, Oozie, yarn and Zookeeper

RDBMS: Oracle 11g (SQL, PL/SQL), MS-SQL and Sybase

Messaging Systems: JMS and IBM MQ Series

Web Languages: XML, XSL, HTML5, JavaScript, AJAX and JQuery

IDE: RAD, Eclipse, TOAD, MYSQL Workbench.

Version Tools: SVN, Git and CVS

Operating System: Windows and Linux

Others: Unix Shell Scripts, MS-Visio, WinSCP, PUTTY and ANT

PROFESSIONAL EXPERIENCE:

Confidential, Alpharetta, GA

Big Data /Hadoop Consultant

Responsibilities:

  • Involved in analyze functional requirements and design big data implementations.
  • Involved in design and setup environment for different Big Data projects.
  • Responsible to ingest data into HDFS using shell scripts for different projects.
  • Written Map Reduce jobs to ingest data into Hive or Hbase tables.
  • Written Map Reduce programs to analyze the large data sets.
  • Dumped the data using Sqoop into HDFS/Hive for analyzing.
  • Developed map reduce to parse JSON data and store into HBase and Hive.
  • Developed Pig Latin Scripts to extract the data and load into HDFS/Hive/HBase.
  • Developed custom UDFS and implemented pig scripts.
  • Developed Hive Query Language (HQL) scripts to load map reduce output into Hive data warehouse.
  • Developed Oozie workflows to Sqoop Teradata into Hive Tables.
  • Written Pig scripts to analyze and generate summary data for ad-hoc reports.
  • Written shell scripts to move external log data into HDFS for processing.
  • Written Oozi workflow to automate the workflow and map reduce jobs.
  • Developed HQL scripts to extract data from hive and Sqoop out to RDBMS.
  • Developed tableau reports for diagnostics and smart apps.
  • Importing and exporting data from oracle to HDFS/Hive using Sqoop.
  • Developed Hive Scripts to create Hive Database and tables for the different projects.
  • Created HBase column family tables and developed MR jobs to load data into HBase.
  • Created IBM SPSS data modeler streams for predictive analytics.
  • Configured IBM SPSS modeler to connect Hive HCatlog for modeler streams.
  • Scheduling / Monitoring Oozi coordinator and workflow controls task.
  • Support other teams to setup project environments.
  • Involved in migrating applications and support applications in production environment.

Environment: Java, JDK1.6, Horton works, Hadoop, HDFS, Hive, Tez, Pig, Sqoop, Hue, HBase, Solr/Lucene, Kafka, Storm, Oozi, Zookeeper, yarn Map Reduce, HCatalog, Avro, Parquet, Tableau, JSP, Oracle, Teradata, SQL, Log4J, RAD, Web sphere, Eclipse, AJAX, JavaScript, JQuery, CSS3, SVN, WinScp, Putty, FTP, Linux, Cronjob, Shell Script and SQL Developer.

Confidential, NJ

Java / Hadoop Consultant

Responsibilities:

  • Involved in developing data ingestion process for different vendor’s data formats like FW, DFPP, digital data and Omniture.
  • Developed java code to cure data and integrated with data ingestion process.
  • Developed shell scripts to download files and ingest into HDFS.
  • Involved in creating Hive external tables.
  • Involved in writing aggregate Hive SQL's and provides the results to Business Team.
  • Responsible to ingest data into HDFS using shell scripts for different vendors.
  • Imported the data using Sqoop into HDFS/Hive.
  • Developed Pig Latin scripts to extract the data and load into HDFS.
  • Developed Map Reduce to process different data sets and generate desired output files for external hive tables.
  • Developed Oozie workflow to automate the workflow.
  • Developed HAWQ SQL’s to load data from HDFS into HAWQ tables.
  • Generate aggregate SQL’s on HAWQ tables.
  • Involved in migrating applications and support applications in different environments.

Environment: Java, Oracle, Horton works, Hadoop, HDFS, Hive, Pig, Sqoop, HBase, Oozie, HCatalog, HAWQ, Oracle, SQL, Log4J, Eclipse, SVN, WinScp, Putty, FTP, Linux, Cronjob and Shell Script.

Confidential, New York City, New York

Hadoop Developer

Responsibilities:

  • Responsible for loading customer's data and event logs into HBase using Java API.
  • Created HBase tables to store variable data formats of input data coming from different portfolios
  • Involved in adding huge volumes of data in rows and columns to store data in HBase.
  • Responsible for architecting Hadoop clusters with CDH4 on CentOS, managing with Cloudera Manager.
  • Involved in initiating and successfully completing Proof of Concept on FLUME for Pre-Processing, Increased
  • Reliability and Ease of Scalability over traditional MSMQ.
  • Use Flume to collect the log data from different resources and transfer the data type to hive tables using different SerDe's to store in JSON, XML and Sequence file formats.
  • Used Hive to find correlations between customer's browser logs in different sites and analyzed them to build risk profile for such sites.
  • End-to-end performance tuning of Hadoop clusters and Hadoop Map/Reduce routines against very large data sets.
  • Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts.
  • Created User accounts and given the users the access to the Hadoop Cluster
  • Implemented the secure authentication for the Hadoop Cluster using Kerberos Authentication protocol.
  • Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
  • Developed the Pig UDF'S to pre-process the data for analysis.
  • Familiarity with a NoSQL database such as MongoDB Solr.
  • Experience in working with various kinds of data sources such as MongoDB Solr and Oracle.
  • Successfully loaded files to Hive and HDFS from MongoDB Solr.
  • Experience in managing development time, bug tracking, project releases, development speed, release forecast, scheduling and many more. Using a custom framework of Nodes and MongoDb to take care of the back-end calls with a lightning fast speed.
  • Monitored Hadoop cluster job performance and performed capacity planning and managed nodes on
  • Hadoop cluster.
  • Responsible for using Cloudera Manager, an end to end tool to manage Hadoop operations.

Environment: Hadoop, Big Data, HDFS, Pig, Hive, MapReduce, Sqoop, Cloudera manager, LINUX, CDH4, FLUME, HBase, Pig, Hive

Confidential

Java/ J2EE Developer

Responsibilities:

  • Developed the application using Struts Framework that leverages classical Model View Layer (MVC) Architecture UML diagrams like use cases, class diagrams, interaction diagrams (sequence and collaboration) and activity diagrams were used
  • Gathered business requirements and wrote functional specifications and detailed design documents
  • Extensively used Core Java, Servlets, JSP and XML
  • Designed the logical and physical data model, generated DDL scripts, and wrote DML scripts for Oracle 9i database
  • Implemented Enterprise Logging service using JMS and apache CXF.
  • Developed Unit Test Cases, and used JUNIT for unit testing of the application
  • Implemented Framework Component to consume ELS service.
  • Implemented JMS producer and Consumer using Mule ESB.
  • Wrote SQL queries, stored procedures, and triggers to perform back-end database operations
  • Designed Low Level design documents for ELS Service.
  • Developed SQL stored procedures and prepared statements for updating and accessing data from database.
  • Development carried out under Eclipse Integrated Development Environment (IDE).
  • Used JBoss for deploying various components of application.
  • Involved in Unit testing, Integration testing and User Acceptance testing.
  • Utilizes Java and SQL day to day to debug and fix issues with client processes.

Environment: Java, spring core, JBoss, JUNIT, JMS, JDK, SVN, Maven, Servlets, JSP and XML

Confidential

Java Developer

Responsibilities:

  • Used AGILE methodology for developing the application.
  • As part of the lifecycle development prepared class model, sequence model and flow diagrams by analyzing Use cases using Rational Tools.
  • Extensive use of SOA Framework for Controller components and view components.
  • Involved in writing the exception and validation classes using Struts validation rules.
  • Involved in writing the validation rules classes for general server-side validations for implementing validation rules as part observer J2EE design pattern.
  • Used OR mapping tool Hibernate for the interaction with database. Involved in writing Hibernate queries and Hibernate specific configuration and mapping files.
  • Developed EJB tier using Session Facade, Singleton and DAO design patterns, which contains business logic, and database access functions.
  • Involved in developing JSP pages and custom tag for presentation layer in Spring framework.
  • Developed tabs with rollover effect using JavaScript plus other effects.
  • Developed web services using SOAP and WSDL with Apache Axis 2.
  • Developed, implemented, and maintained an asynchronous, AJAX based rich client for improved
  • Developed SQL stored procedures and prepared statements for updating and accessing data from database.
  • Development carried out under Eclipse Integrated Development Environment (IDE).
  • Used JBoss for deploying various components of application.
  • Used JUNIT for testing and check API performance. Involved in fixing bugs and minor enhancements for the front-end modules. Responsible for troubleshooting issues, monitoring and guiding team members to deploy and support the product.
  • Worked with the Android SDK, and implemented Android Bluetooth and Location Connectivity components.
  • Worked with business and System Analyst to complete the development in time.
  • Implemented the presentation layer with HTML, CSS and JavaScript.
  • Developed web components using JSP, Servlets and JDBC
  • Wrote complex SQL queries and stored procedures.
  • Implemented Persistent layer using Hibernate API
  • Implemented Transaction and session handling using Hibernate Utils.
  • Implemented Search queries using Hibernate Criteria interface.
  • Maintained Jasper server on client server and resolved issues.
  • Actively involved in system testing.
  • Fine tuning SQL queries for maximum efficiency to improve the performance
  • Designed Tables and indexes by following normalizations.
  • Involved in Unit testing, Integration testing and User Acceptance testing.
  • Utilizes Java and SQL day to day to debug and fix issues with client processes.

Environment: Java, Servlets, JSP, Hibernate, Junit Testing, Oracle DB, SQL, Jasper Reports, iReport, Maven, Jenkins.

We'd love your feedback!