We provide IT Staff Augmentation Services!

Hadoop Developer Resume

SUMMARY:

  • 9 years of experience in the Information Technology Industry with strong exposure to Software Project Management, Design, Development,Implementation, Maintenance/Support and Integration of Software Applications.
  • 5 plus years of work experience as Hadoop Developer with good knowledge of Hadoop Framework, Hadoop Distributed File System and Parallel Processing Implementation.
  • Experience in Hadoop Ecosystems HDFS, Map Reduce, Hive, Pig, HBase, Sqoop, AWS.
  • Excellent understanding / knowledge of Hadoop architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and Map Reduce programming paradigm.
  • Good Exposure on Apache Hadoop Map Reduce programming, Hive, PIG scripting andHDFS.
  • Experience in managing and reviewing Hadoop log files.
  • Experience in importing and exporting data from RDBMS to HDFS, Hive tables, HBase by using Soop.
  • Experienced in Big Data Hadoop Ecosystems experience in Ingestion, Storage, Querying, Processing and Analysis of Big Data.
  • Strong experience in writing Map Reduce programs for Data Analysis. Hands on experience in writing custom partioners for Map Reduce.
  • Performed data analysis using Hive and Pig.
  • Excellent understanding and knowledge of NOSQL databases like Mongo DB, HBase, and Cassandra.
  • Experience with distributed systems, large - scale non-relational data stores, RDBMS, NoSQL map-reduce systems, data modeling, database performance, and multi-terabyte data warehouses.
  • Hands on experience in application development using Java, RDBMS, and Linux shell scripting.
  • Experience working with JAVA J2EE, JDBC, ODBC, JSP, Java Eclipse, Java Beans, EJB, Servlets.
  • Hands on experience in SQL programs, Packages, Stored procedures, Triggers, Dynamic SQL, SQL Loader, SQL Plus, Shell Scripting, performance tuning and Query Optimization.
  • Support development, testing, and operations teams during new system deployments.
  • Evaluate and propose new tools and technologies to meet the needs of the organization.
  • An excellent team player and self-starter with good communication skills and proven abilities to finish tasks before target deadlines.

TECHNICAL SKILLS:

Hadoop ECO Systems;: Hadoop MapReduce, HDFS, HBase, Hive, Pig, Sqoop,ZooKeeper, NO SQL Mongo DB, Cassandra

Databases: MS SQL Server, MY SQL, Oracle 9i/10g, MS accessLanguages: Java, C/C++, SQL, Teradata SQL, PL/SQL, Servlets, JavaBeans, JDBC

Operating Systems: Windows, Server Windows XP/Vista, UNIX,LINUX

SQL Server Tools: SQL Server Management Studio, Enterprise Manager, QueryAnalyser, Profiler, Export/Import (DTS).

PROFESSIONAL EXPERIENCE:

Confidential, VA

Hadoop Developer

Responsibilities:

  • Used Hive to do transformations, event joins, filter both traffic and some pre-aggregations before storing the data onto HDFS.
  • Develop simple to complex MapReduce Jobs using Hive to cleanse and load mainframe data
  • Handle importing of data from various data sources, perform transformations using Hive, MapReduce, load data into HDFS and extract the data from MySQL into HDFS using Sqoop.
  • Involved in developing Hive queries and UDFs for the needed functionality that is not out of the box available from Apache Hive.
  • Effectively used Sqoop to transfer data between databases and HDFS.
  • Worked on streaming the data into HDFS from Web Servers using Flume.
  • Manage and review Hadoop log files.
  • Worked on Agile Scrum Methodologies.
  • Implemented Complex MapReduce programs to perform joins on Map side using Distributed Cache.
  • Experience in administering the Linux Systems to deploy Hadoop Cluster and monitoring the cluster using Ganglia.
  • Load and Transform large sets of Structured, Semi Structured and Unstructured Data.
  • Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Java MapReduce, Hive and Sqoop as well as system specific jobs.
  • Experience in upgrading clusters (major and minor) upgrades as part of migration.
  • Experience in developing Web Services with Python Programming language.

Environment: Hadoop, MapReduce, HDFS, Hive, Pig, Flume, Oozie, Java, SQL, Sqoop, Eclipse, DB2

Confidential

Hadoop Developer

Responsibilities:

  • Completely involved in Requirement Analysis and Documentation on Requirement Specification.
  • Involved in project from Analysis to Production Implementation, with emphasis on identifying the source and source data validation, developing logic and transformations as per the requirement and creating mappings and loading the data into target tables.
  • Analyze large amounts of data sets to determine optimal way to aggregate and report on it.
  • Develop simple to complex MapReduce Jobs using Hive and Pig.
  • Optimized MapReduce jobs to use HDFS efficiently by using various compression mechanisms.
  • Handle importing of data from various data sources, perform transformations using Hive, MapReduce, load data into HDFS and extract the data from MySQL into HDFS using Sqoop.
  • Export the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Extensively used Pig for data cleansing.
  • Create partitioned tables in Hive.
  • Manage and review Hadoop log files.
  • Involved in creating Hive tables, loading with data and writing Hive queries which will run internally in MapReduce way.
  • Install and configure Pig and also write Pig Latin scripts.
  • Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
  • Involved in HDFS Maintenance and Loading of Structured and Unstructured Data and also in Debugging and Resolving the problem.

Environment: Hadoop, MapReduce, HDFS, Hive, Pig, Java, SQL, Sqoop, Java (jdk 1.6), Eclipse

Confidential, NJ

Hadoop Developer

Responsibilities:

  • Involved in review of functional and non-functional requirements.
  • Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in Java for Data cleaning and Pre-processing.
  • Experienced in defining job flows and managing and reviewing of Hadoop log files.
  • Involved in loading data from LINUX file system to HDFS.
  • Worked on installing Cluster, Commissioning and Decommissioning of Data node, Name node recovery, capacity planning, and slots configuration.
  • Extracted files from CouchDB through Sqoop and placed in HDFS and processed.
  • Involved in creating Hive tables, loading with data and writing Hive queries which will run internally in Map Reduce way.
  • Implemented best income logic using Pig scripts and UDFs.
  • Implemented test scripts to support test driven development and continuous integration.
  • Worked on tuning the performance Pig queries.
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
  • Responsible to manage data coming from different sources.
  • Involved in loading data from UNIX file system to HDFS.
  • Loaded and transformed large sets of structured, semi structured and unstructured data.
  • Clustered coordination services through Zookeeper.
  • Experienced in managing and reviewing Hadoop log files.
  • Job management using Fair Scheduler.
  • Developed a Custom File System plug in for Hadoop so it can access files on Data Platform.
  • Responsible for cluster maintenance, added and removed cluster nodes, cluster monitoring and troubleshooting, managed and reviewed data backups, managed and reviewed Hadoop log files.
  • Installed Oozie workflow engine to run multiple Hive and Pig jobs.
  • Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
  • Supported in setting up QA environment and updated configurations for implementing scripts with Pig and Sqoop.

Environment: Hadoop, HDFS, Map Reduce, Pig, Sqoop, Zookeeper, HBase, Shell Scripting, Ubuntu, Linux Red Hat.

Confidential, OH

Java Developer

Responsibilities:

  • Involved in gathering business requirements, analyzing the project and creating use Cases.
  • Coordination with the Design team, Business analysts and end users of the system.
  • Designed and developed front-end using JSP, Java Script, HTML.
  • Worked with Solr for indexing the data and used JSP for the Web application.
  • Developed Servlets and JSP based on MVC pattern using Struts Action framework.
  • Involved in writing Hibernate queries and Hibernate specific configuration and mapping files.
  • Used Junit framework for writing Test Classes and for starting up the application server in various modes.
  • Wrote Junit test classes for the services and prepared documentation.

Environment: Java, Hibernate, Jakarta Struts 1.2, JSP, Servlet, JavaScript, MS SQL Server 7.0, Eclipse, Windows XP, Unix,

Confidential

Associate Software Engineer

Responsibilities:

  • Responsible in developing regression test cases and test plans.
  • Used tools like FireBug and FirePath to understand the HTML code for websites.
  • Responsible in creating Absolute x-paths to locate elements on the page.
  • Performed end to end and Ad-hoc testing on multi OS platforms.
  • Executed smoke tests on E-commerce website, reader application and reader devices whenever new build is released.
  • Performed unit, functional, performance, Stress, Regression testing and cross browser testing.
  • Performed Business acceptance testing to make sure the application meets the business requirements and made sure to include all the test cases in the test plan.
  • Worked closely with Developers to triage the defects.
  • Developed additional functionality in the Software as per business requirements.
  • Followed Sun Standard Coding and Documentation Standards.
  • Participated in project planning with Business Analysts and team members to Analyse the Business requirements into working software.
  • Developed Software application modules using disciplined Software Development.

Environment: Java, J2EE, JSP, EJB, ANT, STRUTS1.2, Log4J, Weblogic 7.0, JDBC, MyEclipse, Windows • XP, Oracle.

Hire Now