Hadoop Developer Resume
Dallas, TX
SUMMARY
- 8+ years of overall IT experience in a variety of industries, which includes hands on experience of 3+ years in Big Data technologies and extensive experience of 4+ years in Java.
- Good understanding of ClassicHadoop and Yarn architecture along with variousHadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Secondary Name Node, Resource Manager, Node Manager, Application Master and Containers.
- Hands on experience in working with Ecosystems like Hive, Pig, Sqoop, Map Reduce, Flume, Oozie., Zookeeper, HDFS, Impala, HBase, Spark.
- Strong knowledge of Pig and Hive’s analytical functions, extending Hive and Pig core functionality by writing custom UDFs.
- Experience in importing and exporting terra bytes of data using Sqoop from HDFS to Relational Database Systems and vice - versa.
- Knowledge of job workflow scheduling and monitoring tools like oozie and Zookeeper, of NoSQL databases such as HBase, Cassandra.
- Experience in deployment, configuration and support of Platform as a Service provider solutions like "PivotalCloudFoundry"
- Involvement in all phases of SDLC from project proposal, planning, analysis, development, testing, deployment and support.
- Experience in technologies Java, EJB, Servlets, JDBC, JSP, Struts, Spring, JavaScript on WebLogic, Apache Tomcat Web/Application Servers.
- Good Experience with databases, writing complex queries and stored procedures using SQL and PL/SQL.
- Experience in developing and implementing web applications using Java, JSP, CSS, HTML, HTML5, XHTML and Java script, JSON, XML, JDBC.
- Experience in using sequence file, RC file and Avro file formats.
- Sound Relational Database Concepts and extensively worked with ORACLE, MySQL, SQL Server.
- Highly skilled in integrating Kafka with Spark streaming for high speed data processing.
- Very good experience with both MapReduce 1 (Job Tracker) and MapReduce 2 (YARN) setups.
- Detailed knowledge and experience of Design, Development and Testing Software solutions using Java and J2EE technologies.
- Research-oriented, motivated, proactive, self-starter with strong technical, analytical and interpersonal skills.
TECHNICAL SKILLS
Hadoop Components: HDFS, Hue, MapReduce, PIG, Hive, HCatalog, Hbase, Sqoop, Impala, Zookeeper, Flume, Kafka, Yarn, Cloudera Manager, Kerberos.
Spark Components: Apache Spark,Data Frames, SparkSQL, Spark, YARN, Pair RDDs
Web Technologies / Other components: J2EE, XML, Log4j, HTML, XML, CSS, JavaScript
Server Side Scripting: UNIX Shell Scripting.
Databases: Oracle 10g, Microsoft SQL Server, MySQL, DB2, Teradata
Programming Languages: Java, C, C++, Scala, Impala.
Web Servers: Apache Tomcat, BEA WebLogic.
IDE: Eclipse, Dreamweaver
OS/Platforms: Windows 2005/2008, Linux (All major distributions), Unix.
NoSQL Databases: Hbase, MongoDB.
Methodologies: Agile (Scrum), Waterfall, UML, Design Patterns, SDLC.
Currently Exploring: Apache Flink, Drill, Tachyon.
PROFESSIONAL EXPERIENCE
Confidential, Dallas, TX
Hadoop Developer
Responsibilities:
- Responsible for building scalable distributed data solutions usingHadoop.
- Job duties include design and development of various modules inHadoopBig Data platform and processing data using MapReduce, Hive, SQOOP, Pig and Oozie.
- Developed job processing scripts using Oozie workflow
- Implemented POC to migrate map reduce jobs into Spark RDD transformations using Scala.
- Spark streaming collects the data from Kafka in near real time and performs necessary transformations and aggregations on the fly to build the common learner data model and persists the data in Cassandra
- Used Pig for analysis of large data sets and brought data back to Hbase by using Pig.
- Involved inHadoopcluster task like commissioning & decommissioning Nodes without any effect to running jobs and data.
- Wrote Map Reduce jobs to discover trends in data usage by users.
- Uses Talend Open Studio to load files intoHadoopHIVE tables and performed ELT aggregations in HadoopHIVE.
- Worked on upgrade and configuration of PivotalCloudFoundryfrom version 1.7 to version 1.8.
- Designing and creating ETL jobs through Talend to load huge volumes of data into Cassandra, HadoopEcosystem and relational databases.
- Used Sqoop to import data from SQL server to Cassandra.
- Job management using Fair scheduler.
- Worked extensively with Sqoop for importing metadata from Oracle.
- Responsible for working with different teams in buildingHadoopInfrastructure.
- Involved in creating Hive tables, and loading and analyzing data using hive queries.
- Designed, developed and did maintenance of data integration programs in aHadoopand RDBMS environment with both traditional and non-traditional source systems as we as RDBMS and NoSQL data stores for data access and analysis. Experienced in runningHadoopstreaming jobs to process terabytes of xml format data.
- Load and transform large sets of structured, semi structured and unstructured data.
- Responsible to manage data coming from different sources.
- Assisted in exporting analyzed data to relational databases using Sqoop.
- Wrote Hive Queries and UDF's.
- Developed Hive queries to process the data and generate the data cubes for visualizing.
Environment: MapReduce, Spark, HDFS, Pig, HBase, Oozie, Kafka, Zookeeper, Sqoop, Cassandra, Linux, XML, Hadoop, Toad, Maven, NoSQL, MySQL, Hive, Java, Eclipse, Oracle 10g,ETL
Confidential, Bloomington, IL
Hadoop Developer
Responsibilities:
- Worked on analyzingHadoopstack and different big data analytic tools including Pig and Hive, HBase database and Sqoop.
- Designed high level ETL architecture for overall data transfer from the OLTP to OLAP.
- Created various Documents such as Source-To-Target Data mapping Document, Unit Test Cases and Data Migration Document.
- Installed and configured Pig for ETL jobs Designed high level ETL architecture for overall da ta transfer from the OLTP to OLAP.
- Wrote MapReduce jobs to perform operations like copying data on HDFS and defining job flows on EC2 server, load and transform large sets of structured, semi-structured and unstructured data.
- Developed PL/SQL procedures for processing business logic in the database.
- Imported data using Sqoop from Teradata using Teradata connector.
- Creating Hive tables and working on them for data analysis in order to meet the business requirements.
- Worked on a POC on Spark and Scala parallel processing.
- Real streaming the data using Spark with Kafka.
- Implemented data ingestion and handling clusters in real time processing using Kafka.
- Experience with Core Distributed computing and Data Mining Library using Apache Spark.
- Used Hive to process data and Batch data filtering .Used Spark for any other value centric data filtering.
- Designed and implemented Spark test bench application to evaluate quality of recommendations made by the engine. Deployed application using PivotalCloudFoundry(PCF) CLI.
- Monitored and identified performance bottlenecks in ETL code. Worked on data utilizing aHadoop, Zookeeper, and Accumulo stack, aiding in the development of specialized indexes for performant queries on big data implementations.
- Used Zookeeper for various types of centralized configurations, SVN for version control, Maven for project management, Jira for internal bug/defect management, MapReduce.
- Installed the Operating System on Solaris and Linux servers and Blades over the network.
- Got good experience with NoSQL database.
- Worked on MongoDB database concepts such as locking, transactions, indexes, Sharding, replication, schema design.
- Configuring high availability using geographical MongoDB replica sets across multiple data centers.
- Generated Java APIs for retrieval and analysis on No-SQL database such as HBase and Cassandra.
- Designed and implemented MapReduce-based large-scale parallel relation-learning system, Installed and benchmarkedHadoop/HBase clusters for internal use.
- Written HBASE Client program in Java and web services.
Environment: Informatica Power Center 9.5,Hadoop, Spark, HDFS, Hive, Pig, HBase, Oozie, Sqoop, Kafka, Strom, Zookeeper, MongoD, MapReduce, Cassandra, Linux, XML, Toad, Maven, NoSQL, MySQL Workbench, Java 6, Eclipse, Oracle 10g, PL/SQL, SQL*PLUS, PivotalCloudFoundry
Confidential, Houston, TX
Java/J2EE Developer
Responsibilities:
- Created the UI tool - using Java, XML, DHTML, and JavaScript.
- Wrote stored procedures using PL/SQL for data retrieval from different tables.
- Worked extensively on bug fixes on the server side and made cosmetic changes on the UI side.
- Part of performance tuning team and implemented caching mechanism and other changes.
- Recreated the system architecture diagram and created numerous new class and sequence diagrams.
- Designed and developed UI using HTML, JSP and Struts where users have all the items listed for auctions.
- Developed Authentication and Authorization modules where authorized persons can only access the inventory related operations.
- Developed Controller Servlets, Action and Form objects for process of interacting with Oracle database and retrieving dynamic data.
- Responsible for coding SQL Statements and Stored procedures for back end communication using JDBC.
- Developed the Login screen so that only authorized and authenticated administrators can only access the application.
- Developed various activities like transaction history, search products that enable users to understand the system efficiently.
- Involved in preparing the Documentation of the project to understand the system efficiently.
Environment: JDK1.2, JavaScript, HTML, DHTML, XML, Struts, JSP, Servlet, JNDI, J2EE, Tomcat, Rational Rose, Oracle.
Confidential
Java Developer
Responsibilities:
- Involved in Presentation Tier Development using JSF Framework and ICE Faces tag Libraries.
- Involved in business requirement gathering and technical specifications.
- Implemented J2EE standards, MVC2 architecture using JSF Framework.
- Implementing Servlets, JSP and Ajax to design the user interface.
- Extensive experience in building GUI (Graphical User Interface) using JSF and ICE Faces.
- Developed Rich Enterprise Applications using ICE Faces and Portlets technologies.
- Experience using ICE Faces Tag Libraries to develop user interface components.
- Used JSF, JSP, Java Script, HTML, and CSS for manipulating, validating, customizing, error messages to the User Interface.
- Used EJB (Session beans) to implement the business logic, JMS for communication for sending updates to various other applications and MDB for routing priority requests.
- All the Business logic in all the modules is written in core Java.
- Wrote Web Services using SOAP for sending and getting data from the external interface.
- Developed a web-based reporting for monitoring system with HTML and Tiles using Struts framework.
- Middleware Services layer is implemented using EJB (Enterprise Java Bean - stateless) in WebSphere environment.
- Used Design patterns such as Business delegate, Service locator, Model View Controller, Session façade, DAO.
- Funds Transfers are sent to another application using JMS technology asynchronously.
- Involved in implementing the JMS (Java messaging service) for asynchronous communication.
- Involved in writing JMS Publishers to post messages.
- Involved in writing MDB (Message Driven Beans) as subscribers.
- Created Stored procedures using PL-SQL for data modification (Using DML insert, update, delete) in Oracle.
Environment: J2EE, EJB, JSF, ICE Faces, EJB, Web Services, XML, XSD, Agile, Microsoft Visio, Clear Case, Oracle 9.i/10.g, Weblogic8.1/10.3,RAD, Log4j,Servlets, JSP, Unix.
Confidential
Java Developer
Responsibilities:
- Individually worked on all the stages of a Software Development Life Cycle (SDLC).
- Used JavaScript code, HTML and CSS style declarations to enrich websites.
- Implemented the application using Spring MVC Framework which is based on MVC design pattern.
- Developed application service components and configured beans using (applicationContext.xml) Spring IOC
- Designed User Interface and the business logic for customer registration and maintenance.
- Integrating Web services and working with data in different servers.
- Involved in designing and Development of SOA services using Web Services.
- Understanding the requirements from business users and end users.
- Working with XML/XSLT files.
- Experience creating UML class and sequence diagram.
- Experience in Creating Tables, Views, Triggers, Indexes, Constraints and functions in SQL
- Server2005.
- Worked in content management for versioning and notifications.
Environment: Java, J2EE, JSP, Spring, Struts, Hibernate, Eclipse, SOA, WebLogic, Oracle, HTML, CSS, Web Services, JUnit, SVN, Windows, UNIX.
