Hadoop/ Spark Developer Resume
Charlotte, NC
PROFESSIONAL SUMMARY:
- Around 8 years of experience in IT developing industry, 4 years in designing and developing various web applications using Java, Servlets, HTML, CSS and JavaScript and 3+ years of comprehensive experience in Big Data processing using Hadoop and its ecosystem ( MapReduce, Pig, Hive, Sqoop, Flume, Spark, Kafka and HBase ).
- Solid understanding of the Hadoop Distributed File System.
- Excellent Experience in Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce programming paradigm.
- Strong Work experience in Enterprise Financial Transactions and Money movement activities. Dealt with huge transaction volumes while interfacing the front end application written in Java, JSP, Struts, Webworks, Spring, JSF, Hibernate, Web service and EJB with Web sphere Application server and Jboss.
- Hands on experience in installing configuring and using Hadoop ecosystem components like Hadoop MapReduce HDFS HBase Hive Sqoop Pig Zookeeper Spark and Scala
- Good Exposure on Apache Hadoop Map Reduce programming PIG Scripting and Distribute Application and HDFS
- Experience in importing and exporting data using Sqoop from HDFS to Relational Database Systems and vice - versa
- Cluster coordination services through Zookeeper.
- Experience in managing Hadoop clusters using Cloudera Manager tool.
- Worked on managing and automating Infrastructure Development and operations involving AMAZON WEB SERVICES(AWS).
- Install and configure chef server /workstation and nodes via CLI tools to AWS nodes.
- Created users and groups using IAM and assigned individual policies to each group.
- Configured Security group for EC2 window and Linux instances.
- Experience in working with Github private repositories and docker repositories.
- Experience with Docker to create, manage, deploy and run containerized applications
- Sound knowledge in various databases like MySQL & NoSQL (cassandra)
- Experience in working with various build tools like Maven.
- Working Experience in frameworks like Spring and Hibernate .
- Good Knowledge in different webservices like Soap,Restful .
- Strong working experience using Agile methodologies including Scrum.
- Some Knowledge in some of unix/linux commands.
- Excellent ability to understand complex scenarios and business problems and transfer the knowledge to other team members in the most comprehensive manner.
- Good Analytical thinking, Inter-personal, thinking and communication skills.
TECHNICAL SKILLS:
BigData: Hadoop, Hbase, Hive, Sqoop,Oozie, Spark, Kafka, Flume, Mapreduce.Zookeper
Operating Systems: AIX, UNIX, Mac, Linux, Windows 2000 / NT / XP / Vista.
Programming Languages: Java, Scala, Go, C, C++, VB, Objective C.
Databases/technologies: DB2,MySQL, Oracle 9i, MongoDB, Cassandra.
IDE/Development Tools: Eclipse 5.x, JBuilder 7.x and RAD 7.0 NetBeans, Intellij
Web Technologies: J2EE, Soap & REST Web Services, JSP, Servlets, EJB, JavaScriptStruts, Spring, Web works, Direct Web remoting, HTML, XML,JMS, JSF, Ajax
Frameworks: Struts2.0, Spring Framework and Hibernate
AWS Compute Services: EC2, ECS, Lambda
AWS Storage Services: S3
AWS Database: RDS
Version Control Tools: Git
Web/Application Servers: IBM Web sphere Application server, Jboss, Apache TomcatNginx
Automation/Build Tools: Docker, Vagrant, Maven, Developement Strategies: Agile, Lean Agile, Pair Programming, Water-Fall and Test DrivenDevelopment
PROFESSIONAL EXPERIENCE:
Confidential, Charlotte, NC
Hadoop/ Spark Developer
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop Ecosystem.
- Responsible for troubleshooting issues in the execution of MapReduce jobs by inspecting and reviewing log files.
- Participate in the design and development of software using agile development practices.
- Develop Scala and SQL code to extract data from various databases, Apply innovative ideas around the Data Science and Advanced Analytics practices Creatively and present models to business customers and executives, utilizing a variety of formats and visualization methodologies.
- Implement POC for using Apache Impala for data processing on the top of Hive.
- Implemented POC to migrate map reduce jobs into Spark RDD transformations.
- Converts GoLang scripts into spark jobs which takes necessary fields from impala and populate them into HBase.
- Developing and building different spark projects using SBT and Maven.
- Understanding of data storage and retrieval techniques, ETL and databases, to include graph stores, relational databases, tuple stores, NOSQL like HBASE. Hadoop, MySQL. Spark MLLIB libraries for designing recommendation Engines Analysis predicted by Statistical analysis using Spark
- Experience in using Sqoop to import and export the data from Oracle DB into HDFS and HIVE .
- Developed workflow in Oozie to manage and schedule jobs on Hadoop cluster for generating reports on nightly, weekly and monthly basis and clean up jobs.
- Commissioning and decommissioning the nodes, manage cluster through performance tuning and enhancement.
- Worked in Agile development environment in sprint cycles of two weeks by dividing and organizing tasks. Participated in daily scrum and other design related meetings.
- Perform live tests and maintain expert knowledge in areas of expertise.
Environment: Hadoop, CDH 5.5, Map Reduce, Hive, Pig, Sqoop, Flume, HBase, Java, Scala, Go, Spark, Oozie, Linux, UNIX
Confidential, Santa Barbara, CA
Hadoop Developer
Responsibilities:
- Responsible for building scalable distributed data solutions using Hadoop Ecosystem.
- Involved in architecture of Customer Matching Module
- Involved in the implementation of Multi-tenancy
- Developed HIVE UDFs to perform data cleansing and transforming for ETL activities.
- Developed HIVE UDFs and UDAFs for Data analysis and Hive table loads.
- Wrote Hive Generic UDF’s to implement Customer Matching Algorithm which is core of ETL process
- Wrote HIVE UDF’s to get the Data from HBase and Put the Data on HBase
- Developed data pipeline using Flume and Sqoop to ingest data into HDFS for analysis
- Developed Sqoop scripts to import the data from SQL Server to HDFS
- Optimized hive queries to make them run in several of hours from multiple days
- Worked extensively on tuning Hive Jobs
- Successfully loaded files to Hive and HDFS from Cassandra. Processed the source data to structured data and store in NoSQL database Cassandra. Created alter, insert and delete queries involving lists, sets and maps in DataStax Cassandra. Worked in a language agnostic environment with exposure to multiple web platforms such as AWS, databases like Cassandra.
- Worked on Apache Drill set up, connectivity to tableau
- Wrote UDF’s for Apache Drill
- Worked on Tableau connectivity to Apache Drill
- Configured Hive Storage plugin, Hbase Storage Plugin and DFS plugin to create Views on Drill
- Implemented POC on Talend Open Studio for Big data
Environment: Hadoop Framework, MapReduce, Hive, Sqoop, HBase, Cassandra, Flume, Oozie, Java(JDK1.6), SQL Server, Talend for Big Data, Apache Drill
Confidential, Westborough, MA
Senior Java/ J2EE Developer
Responsibilities:
- Used Struts2 framework to handle application requests using REST and SOAP web services. Implemented the data persistence using Hibernate 4.
- Developed Object Relational Mapping (ORM) using Hibernate 4 and JPA.
- Developed stored procedures, triggers and functions to process the data using PL/SQL and mapped it to hibernate configuration file.
- Used HQL for querying Oracle database using ORM classes for CRUD (Create, Read, Update and Delete) to manage the shopping cart.
- Involved in the development of project back-end logic layer by using most of the core java features such as Collection Framework, Interfaces, and Exception Handling programming.
- Developed necessary DAO (Data Access Objects) for products and catalogs management.
- Implemented various design patterns in the framework such as Singleton, Factory and Session Facade.
- Designed and developed custom tags, action classes and action form beans.
- Developed cross-browser/platform HTML5, CSS3, JavaScript to match design specs for complex page layouts while adhering to code standards.
- Used JSP, Struts libraries to create web interfaces.
- Consumed REST web services by using jQuery and AJAX.
- Provided client side validation through JavaScript and AJAX for asynchronous communication.
- Configured JMS with MDB components for notifications and used Java Mail API for sending emails.
- Prepared Class Diagrams, Use Case Scenarios, and Sequence Diagrams using UML.
- Used Maven to build the project and deployed the application on Tomcat 7 Server.
- Attended daily Scrum meetings and got defects from QA team
- Involved in Unit Testing and Bug Fixing using JUnit and Eclipse.
- Used Log4j framework to log/track application and CVS for Version Control.
Environment: Java 1.8, J2EE 6 (JSP, JMS), Struts2, Hibernate 4.2, JavaScript, HTML5, CSS3, jQuery, AJAX, Oracle 11g, XML, JAXP 2.0, Tomcat 7, Eclipse Luna 3.7, CVS, Maven 3, JUnit, Log4j
Confidential
Java Developer
Responsibilities:
- Involved in Unified Modeling Language (UML) in the design of project, including several diagrams in three types, which are Structure diagrams, Behavior diagrams and Interaction diagrams using Rational Rose.
- Participated in the daily SCRUM meetings to produce quality deliverables within time.
- Involved in the development of presentation layer in JSP. Client Side validations were done using JavaScript.
- Used AJAX to display form data and for channeling equities data, excess information, market value information etc. onto user screens.
- Involved in multi-tiered J2EE design and coding utilizing Spring MVC framework.
- Used various Java, J2EE design patterns like Factory, Singleton, DAO, DTO, etc.
- Involved in developing of various Java classes, interfaces, modules for calculating margin account information, excess information based on various requirements imposed on each entity.
- Used Spring IOC to dynamically load Java Beans for the web-application.
- Used Spring AOP to handle user authentication, transaction management etc.
- Used Spring JDBC template for the development of the DAO layer.
- Worked on project deployment descriptor files such as web.xml and context definition files for servlet mappings, Java bean class definitions, transactions and database connection configuration.
- Used Log4J components for logging. Perform daily monitoring of log files and resolve issues and used Tibco EMS to persist them in the database.
- Involved in unit testing the application using JUnit and performed the code review.
- Used VSS for version control and Maven scripts were used for building the Java artifacts.
Environment:
- Java, Servlets, Swings, C++,Java/J2ee 1.5, JMS, Struts, Multi-threading, Spring MVC
- Spring Core and DAO, XML, Tibco Java API, Flex 3.0, Action Script, Windows XP, Linux,
- Eclipse 3.3.2, Web logic, Oracle, Toad, VSS, Web services, Ant, BuildForge.BEA Web logic 7.0, and Windows 2000 server, Unix.
