Hadoop Developer Resume
Atlanta, GA
PROFESSIONAL SUMMARY:
- Over 7 years of professional IT experience which includes more than 2 years of experience developing Hadoop/Big Data applications.
- Hands on experience with major components in Hadoop Ecosystem including Hive, HBase, Pig, Sqoop, YARN, Apache Spark, Flume, Oozie, Zookeeper, MongoDB and Map Reduce frameworks.
- Worked with Oozie work flow engine to schedule time based jobs to perform multiple actions.
- Analyzed large amounts of data sets writing Pig scripts and Hive queries.
- Experience in Importing and Exporting the Data using SQOOP from HDFS to Relational Database (Teradata) systems/Mainframe and vice - versa.
- Experience in building, maintaining multiple Hadoop clusters of different sizes and configuration and setting up the rack topology for large clusters.
- Worked in Informatica for the Extraction, Transformation and Loading from various sources to the enterprise data warehouse Developed and tested extraction, transformation and load (ETL) processes.
- Worked on Hadoop Architecture and various components such as HDFS, Job Tracker, Name node, Data Node and Map Reduce concepts.
- Experienced with performing real time analytics on NoSQL data bases like HBase and Cassandra.
- Supported Map Reduce Programs running on the cluster and wrote custom Map Reduce Scripts for Data Processing in Java.
- Experience in analyzing data using HiveQL, Pig Latin and custom Map Reduce programs in Java.
- Experienced on loading large sets of structured, semi structured and unstructured data and also performed importing and exporting data into HDFS and Hive using Sqoop.
- Strong understanding of Data warehouse concepts, ETL, data modeling experience using Normalization, Business Process Analysis, Reengineering, Dimensional Data modeling, physical & logical data modeling.
- Good knowledge on distributed, parallel, Cloud computing, No SQL concepts.
- Proficient in developing web based applications and client server distributed architecture applications in Java/J2EE technologies using Object Oriented Methodology.
- Expertise in design and development of various web and enterprise applications using various technologies like JSP, Servlets, Struts, Hibernate, Spring, JDBC, XML, AJAX, SOAP and Web Services.
- Leading, guiding and developing, Debugging, performance optimization during implementation of project.
- Highly adept at promptly and thoroughly mastering new technologies with a keen awareness of new industry developments and the evolution of next generation programming solutions
- Creative problem solver with the ability to rapidly analyze challenges, applying strategic thinking to tactical concerns with strong problem solving skills and a result oriented attitude. Excellent team Player with good Technical, Analytical and interpersonal skills
TECHNICAL SKILLS:
Big Data Technologies: HDFS, Hive, Map Reduce, Pig, Sqoop, Oozie, Zookeeper, YARN, Spark.
Scripting Languages: Shell, Python, Scala.
Programming Languages: Java, C++,C,SQL,PL/SQL.
Front End Technologies: HTML, XHTML, CSS, XML, JavaScript, AJAX, Servlets, JSP.
Java Frameworks: MVC, Apache Struts2.0, Spring and Hibernate.
Web Services: SOAP(JAX-WS), WSDL, SOA, Restful(JAX-RS), JMS.
Application Servers: Apache Tomcat, WebLogic Server, WebSphere, JBoss.
Databases: Oracle 11g, MySQL, MS SQL Server.
NoSQL Databases: HBase, MongoDB, Cassandra.
IDE: Eclipse, NetBeans.
Operating Systems: Linux, UNIX, Mac, Windows 7/8/10
PROFESSIONAL EXPERIENCE:
Confidential, Atlanta, GA
Hadoop Developer
Responsibilities:
- Used Sqoop to import customer information data from MySQL database into HDFS for data processing.
- Developed a workflow using Oozie to automate the tasks of loading the data into HDFS from analyzing the data.
- Developed Map Reduce jobs to calculate the total usage of data by commercial routers in different locations.
- Developed Map reduce programs for data sorting in HDFS.
- Optimized Hive queries to extract the customer information from HDFS or Hbase.
- Developed Pig Latin scripts to aggregate the log files of the business clients.
- Loaded and transformed large sets of structured, semi structured data using Pig Scripts.
- Involved in loading data from UNIX file system to HDFS.
- Wrote Map Reduce jobs to generate reports for the number of activities created on a particular day, during a dumped from the multiple sources and the output was written back to HDFS
- Designed workflows by scheduling Hive processes for Log file data, which is streamed into HDFS using Flume.
- Managed and reviewed Hadoop log files to identify issues when job fails.
- Developed Hive UDFs for preprocessing the data for analysis.
- Involved with a team who worked on NoSQL databases like MongoDB for POC (proof of concept) in storing documents using GridFs.
- Reviewed the HDFS usage and system design for future scalability and fault-tolerance.
- Involved in writing shell scripts in scheduling and automation of tasks.
- Worked on Hive for further analysis and for generating transforming files from different analytical formats to text files.
- Involved in ETL, Data Integration and Migration. Imported data using Sqoop to load data from Oracle to HDFS on regular basis.
- Very good understanding of Partitions, bucketing concepts in Hive and designed both Managed and External tables in Hive for optimized performance.
- Wrote Map Reduce jobs to generate reports for the number of activities created on a particular day, during a dumped from the multiple sources and the output was written back to HDFS
- Managed and reviewed Hadoop log files to identify issues when job fails.
- Used Hue for UI based PIG script execution, Oozie scheduling and creating tables in Hive.
Environment: Hadoop, HDFS, Map Reduce, Pig, Hive, Sqoop, Oozie, Java, Linux Shell Scripting and Big Data.
Confidential, Huntsville, AL
Java/J2EE Developer
Responsibilities:
- Involved in designing, developing, bug fixing, documenting and unit testing the application.
- Applied design patterns including MVC Pattern, Factor y Design Pattern and Singleton Design Pattern.
- Developed Business logic with the help of Spring and Data Access was implemented by using Hibernate.
- Used spring framework for Dependency Injection and integrated with Hibernate.
- Used Hibernate to interact with DB2 databases for storage, modification and retrieval of data.
- Developed stored procedures, triggers and functions to process the data using PL/SQL and mapped it to Hibernate Configuration File.
- Configured routes in Apache Camel using XML for routing messages between end points.
- Developed RESTful Web services using Jersey and returned the response in JSON format.
- Developed integration tests using REST-assured framework to validate RESTful web services.
- Product development was in an Agile (SCRUM) methodology, where progress and task assignment is discussed in everyday meetings.
- Worked with QA team for testing and resolving defects.
- Wrote unit tests using JUnit and used Apache's Log4j for debugging.
- Used Maven for building and managing dependencies of the application.
Environment: Eclipse, J2EE, Spring, Hibernate, Oracle 11g, MySQL, Tomcat, JBoss, HTML5, JUnit, CSS, Maven.
Confidential
Software Engineer
Responsibilities:
- Responsible for designing and leading project for paperless office automation project.
- Extensively used Java, Adobe API, JavaScript, SOAP, VBA with Advance Business Application Programming Language to deploy across the company offices.
- Worked with non-technical professionals to better understand the business knowledge, supporting live software systems and deploying project to all company offices across the country.
- Managed legal claims portal for electronic exchange of all information relating to a claim via web services, SOAP, JavaScript, Java application to application secure, trusted, reliable electronic communications
- Involved in liaising with partners in order to develop project that will connect offices remotely giving access to data securely using Java, SOAP, Proclaim Legal Case Management System
- Recognized system deficiencies and implements effective solutions by providing support
- Handled confidential information in line with the firm’s data security protocols
- Assisted the technical team in their design and development tasks
Environment: Java, Adobe API, VBScript, Proclaim Legal Case Management System, SOAP Web Services, HTML, Java Script, JDBC, Eclipse, JSP, HTML, Eclipse, XML, JUnit, Windows, UNIX.
Confidential
Java/J2EEDeveloper
Responsibilities:
- Involved in Analysis, Designing, Development and Testing phases of the application.
- Involved in creation and maintenance of the backend services using Spring, Hibernate, SQL Server and Oracle.
- Developed Web pages using JSPs with Tag libraries, HTML, and Java Script.
- Writing J2EE code using Spring, hibernate to upload input CSV files for credit risk data.
- Implemented Dependency Injection (IOC) feature of spring framework to inject dependency into objects and AOP is used for Logging.
- Designed and developed persistence layer build on ORM framework and developed it using Hibernate
- Implemented various Design patterns like Business Delegate, Data Transfer Objects DTO, Service locator, Session Facade and Data Access Objects DAO patterns.
- Involved in writing SQL, Stored procedure and PL/SQL for back end. Used Views and Functions at the Oracle Database end.
- Developed various documents within the application using XML by using Eclipse as IDE tool.
- Developed SOAP requests to interact with billing schedule system.
- Used Web Services (SOAP & WSDL) to exchange data between Server part and client.
- Integrating and deploying the application on WebLogic application server using ANT.
- Developed user interfaces for presenting the expense reports, transaction details using JSP, XML, HTML and Java Script.
- Used Log4J for logging the application exceptions and debugging statements.
- Proficient in doing Object Oriented Design using UML-Rational Rose.
Environment: JDK, JSP, Tiles, HTML, Java Script, WebLogic, Eclipse, Spring JDBC/ORM/DI, JSF, JPA/Hibernate, Spring, PL/SQL, Windows, CVS, Log4J, Ant.
