Hadoop Developer Resume
Miami, FL
SUMMARY:
- Over 7 years of experience in Application analysis, Design, Development, Maintenance and Supporting web, Client - server based applications in Java/J2EE technologies which includes 5years of experience with Big Data and Hadoop related components like HDFS, Map Reduce, Pig, Hive, YARN, Sqoop, Flume, Spark, Strom, Oozie, Zookeeper, Kafka.
- Experience working in environments using Agile (SCRUM) and Waterfall methodologies.
- Experience in multiple Hadoop distributions like MapR, Cloudera and Hortonworks.
- Experience on working structured, unstructured data with various file formats such as XML files, JSON files, and sequence files using MapReduce programs.
- Work experience with cloud configurations like Azure.
- Good Knowledge in Amazon AWS concepts like EMR and EC2 web services which provides fast and efficient processing of Big Data.
- Experience with Business Intelligence tools like Tableau and MicroStrategyfor exception reports.
- Expertise in Data ingestion using Sqoop, Apache Kafka, Spark Streaming and Flume.
- Implemented business logic using Pig scripts. Wrote custom Pig UDF’s to analyze data.
- Performed different ETL operations using Pig for joining operations and transformations on data to join, clean, aggregate and analyze data.
- Experience with Oozie Workflow Engine to automate and parallelize Hadoop, MapReduce and Pig jobs.
- Experience with Apache Airflow a workflow Manager to automate and parallelize the daily jobs.
- Extensive experience with wiring SQL queries using HiveQL to perform analytics on structured data.
- Reporting and statistical analysis with Splunk.
- Worked with SQL, Oracle PL/SQL, Stored Procedures, Table Partitions, Triggers, SQL queries, PL/SQL Packages, and loading data into Data Warehouse/Data Marts.
- Experience in performing data validation using HIVE dynamic partitioning and bucketing.
- Experience in working with Windows, UNIX/Linux platform with different technologies such as Big Data, SQL, XML, JSON, HTML, Core Java, Shell Scripting.
- Experienced in importing and exporting data between RDBMS and Teradata into HDFS using Sqoop.
- Good knowledge analyzing data using Python development and scripting for Hadoop Streaming.
- Experience in implementing algorithms for analyzing using Spark.
- Experience in implementing Spark using Scala and Spark SQL for faster processing of data.
- Experience in creating tables on top of data on AWSS3 obtained from different data sources and providing them to analytics team building reports using Tableau.
- Extensive Hands on experience with Accessing and perform CRUD operations against HBase data using Java API and implementing time series data management.
- Expert knowledge over J2EE Design Patterns like MVC Architecture, Singleton, Factory Pattern, Front Controller, Session Facade, Business Delegate and Data Access Object for building J2EE Applications.
- Experienced in J2EE, Spring, Hibernate, and SOAP/Rest web services, JMS, JNDI, EJB, and JAX-WS.
- Expertise with Application servers and web servers like WebLogic, IBM WebSphere, Apache Tomcat, JBOSS and VMware .
- Proven expertise in implementing IOC/Dependency Injection features in various aspects of Spring Framework .
- Experienced in developing the unit test cases using Junit, Mockito.
- Knowledge on Build tool Jenkins.
- Experience in using Maven and ANT for build automation.
- Experience in using version control and configuration management tools like SVN, CVS, Git, GitHub.
- Expertise in database modeling, administration and development using SQL and PL/SQL in Oracle, MySQL, DB2and SQL Server.
TECHNICAL SKILLS:
Hadoop/Big Data: HDFS, Map Reduce, Hive, Pig, YARN, Sqoop, Flume, Oozie, Strom, Scala, Kafka, Spark
Workflow Manager Tools: Apache Oozie, Apache Airflow.
Methodologies: Agile, Waterfall
Language: Java, C#,C, SQL and Python
Java EE Technologies: JSP, Servlets, JNDI, JDBC, JPA, JMS, JSF
Java EE Frameworks: Jakarta Struts, Spring, Hibernate.
Apache: Tomcat, JBoss, IBM WebSphere and WebLogic.
Web Technologies: Angular.JS, Node.js EXPRESS, jQuery UI, Ajax, HTML/HTML5, CSS/CSS3, RESTful Service, JavaScript, jQuery, Bootstrap, JSON
XML Technologies: XML,DOM
Database: Oracle 10g/11g, PL/SQL, MongoDB, MySQL, MS SQL Server 2012, HBase.
Build Tool: Ant, Maven
RESTful, SOAP, JAX: WS
Testing: Junit, Mockito
IDE Tools: Eclipse, NetBeans, JBoss Developer Studio, IBM Rational Rose, IBM RADVersion Control SVN, CVS, Git
Operating Systems: Windows 7/8/10, Vista, UNIX, Linux, Ubuntu, Mac OS X
Other Tools: Visual Paradigm, LOG4J, Jenkins, AWS, Azure, OpenStack
PROFESSIONAL EXPERIENCE:
Confidential, Miami, FL
Hadoop Developer
Roles and Responsibilities:
- Work experience with different Hadoop distributions namely Cloudera and MapR.
- Work on different Hadoop Eco-System components like HIVE, IMPALA, OOZIE, SQOOP , SPARK etc., to improve the performance of the existing process.
- Develop python code to Ingest, standardize and perform certain validations on different file formats.
- Implement Spark using Scala and Spark SQL for faster processing of big data.
- Work on running Hadoop streaming jobs to process terabytes of XML format data.
- Develop Scripts to integrate Spark-Streaming and Spark-Batch Processing.
- Work on Apache HIVE LLAP on Azure HDInsight .
- Work experience on Storage account and the Datalake on Azure.
- Work experience on with different file formats like Avro, Parquet etc.
- Work on importing and exporting data between Teradata and HDFS using SQOOP .
- Work on bringing the data from various sources into HDFS and building reports for business to improve the business strategies using Tableau .
- Work experience on building an API to trigger the Hadoop jobs from Informatica.
- Create, build and deploy scripts using Maven and SBT .
- Work on SQL queries when required mainly using IMPALA and SPARK-SQL .
- Work on Apache Airflow a Workflow Manager to automate and parallelize the daily jobs written in python.
- Work on Flower to monitor the workers related to the Apache Airflow.
- Work on supervisor to monitor the heartbeat of the Scheduler, Webserver and Workers of Airflow Architecture.
- Work on OOZIE Workflow Engine to automate and parallelize Hadoop, Map Reduce and Spark Jobs.
- Use JIRA to keep track of bugs and features.
- Work with Continuous Integration (CI) using Jenkins for timely builds and running Tests.
- Develop a script using Jenkins with the integration of the GIT repository for the build, testing, code review and the deployment of the build Jar file, shell-scripts and OOZIE workflows to the destination HDFS paths.
- Experience with on-call support if any of the Hadoop services are down or with any job failure.
- Experience with data science team in analyzing the LTO promotions data.
- Experience working with the data science in building a DAG for the R-code.
Environment: Hadoop, HDFS, Hive, Flume, HBase, Sqoop, PIG, Java JDK 1.6, Eclipse, MySQL, JSON, Apache Kafka, Spark, Ubuntu, Zookeeper, AZure.
Confidential, Overland Park, KS
Hadoop Developer
Roles and Responsibilities:
- Involved in Analysis, Design, and Development and testing process based on the new business requirements.
- Develop Scala Source Code to process heavy RAW JSON data
- Use Apache Spark to execute Scala Source Code for JSON Data Processing.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Experienced in defining job flows. Experienced in managing and reviewing Hadoop log files.
- Extracted files from MongoDB through Sqoop and placed in HDFS and processed.
- Experienced in running Hadoop streaming jobs to process terabytes of xml format data.
- Load and transform large sets of structured, semi structured and unstructured data.
- Responsible to manage data coming from different sources.
- Experience in writing monitoring/start up shell scripts for Unix and Linux.
- Supported Map Reduce Programs those are running on the cluster.
- Use Spark to process live Streaming data using Apache Flume.
- Use Spark to process live Streaming data using Apache Kafka.
- Develop Scripts to Integrate Spark- Streaming and Spark-Batch Processing.
- Used Scala collection framework to store and process the complex information.
- Developed UNIX Shell scripts to automate repetitive database processes.
- Writing entities in Scala and Java along with named queries to interact with database.
- Involved in loading data from UNIX/Linux file system to HDFS.
- Involved in creating Hive tables loading with data and writing hive queries which will run internally in map reduce way.
- Involved in designing of MapReduce jobs with Greenplum Hadoop system (HDFS).
- Worked on analyzing Hadoop cluster and different big data analytic tools including Pig HBase database and Sqoop
- Use Impala to determine statistical information about Operational Data.
- Participated in development/implementation of Cloudera Hadoop environment (CDH).
- Hands on Experience with Talend Data Integration ETL Tool.
- Generate reports and predictions using Tableau.
- Installed and configured Hive and also written Hive UDFs.
- Create Oozie workflows to automate scripts for collecting input and initiate Spark jobs.
- Used Spark SQL for faster processing of the data.
- Involved in development of the web pages using AngularJS, JSP and Java Script.
- Work on Git repositories, version tagging and Pull Requests.
- Involved in daily SCRUM meetings to discuss the development/progress ofSprints and was active in making scrum meetings more productive.
Environment: Hadoop, HDFS, Hive, Flume, HBase, Sqoop, PIG, Java JDK 1.6, Eclipse, MySQL, JSON, Apache Kafka, Spark, Ubuntu, Zookeeper, Amazon EC2 SOLR, AWS.
Confidential, Millville, NJ
Java/J2EE Developer/ Hadoop Developer
Role and Responsibilities:
- Responsible in gathering requirements from users and designing Use cases, Technical Design and Implementation.
- Extensively worked on Spring and Hibernate Frameworks.
- Installed and configured Hadoop MapReduceHDFS,developed multiple MapReduce jobs in java for data cleaning and preprocessing.
- Experience in installing configuring and using Hadoop ecosystem components.
- Importing and exporting data into HDFS and Hive using Sqoop.
- Worked on Front Controller, Dependency Injection, MVC, Data Access Objects and other J2EE core patterns.
- Developed the entire front end screens using AJAX, JSP, JSP Tag Libraries, CSS, HTML and JavaScript.
- Used JavaScript and jQuery for front end validations and functionalities.
- Participated in development/implementation of HortonWorks Hadoop environment.
- Created the Node.js EXPRESS Server combined with Socket.io to build MVC framework from front-end side AngularJS to back-end MongoDB, in order to provide broadcast service as well as chatting service.
- Contributed significantly in applying the MVC Design pattern using Spring.
- Implemented action Form classes for data transfer and server side data validation.
- Performed Unit Testing JUnit, System Testing and Integration Testing.
- Developed web services using SOAP and WSDL.
- Application deployment is done in WebSphere, JBoss servers.
- Used Eclipse as an IDE for developing application.
- Involved in the complete software development life cycle.
- Involved in unit testing and user documentation and used Log4j for creating the logs.
- Involved in Maintenance and Bug Fixing.
Environment: Hadoop, HDFS, Hive, Flume, HBase, Sqoop, PIG, Java JDK 1.6, Eclipse, MySQL, JSON, Spring IOC, Hibernate, AJAX, HTML, JSP, JSTL, Java Script, jQuery, Junit, SOAP, WSDL, WebSphere, LOG4j, OpenStack.
Confidential
Java/J2EE Developer
Role and Responsibilities:
- Created Use case Sequence diagrams functional specifications and User Interface diagrams using IBM Rational Rose.
- Involved in complete requirement analysis design coding and testing phases of the project.
- Used Hibernate as ORM to map Java classes to data base tables.
- Involved in writing JDBC code for doing CRUD operations.
- Involved in the design of Data-warehouse using Star-Schema methodology and converted data from various sources to oracle tables.
- Involved in developing PL/SQL queries, stored procedures, and functions.
- Implemented the Business logic by efficiently utilizing the OOPS features of core Java and also Performed Unit Testing to using JUNIT.
- Used JUnit for Unit testing and Maven for build
- Generated XML Schemas and used XML Beans to parse XML files.
- Created Stored Procedures Functions. Used JDBC to process database calls for DB2 and SQL Server databases.
- Developed the code which will create XML files and Flat files with the data retrieved from Databases and XML files.
- Created Data sources and Helper classes which will be utilized by all the interfaces to access the data and manipulate the data.
- Developed web application Spring Framework,JSP and HTML.
- Developed the interfaces using EclipseandJBoss, involved in integrated testing Bug fixing and in Production Support
Environment: Java 1.5, Servlets, JSP, Java Mail API, JavaScript,HTML,Spring, XML, MySQL,JBoss, IBM Rational Rose.