We provide IT Staff Augmentation Services!

Senior Hadoop Developer Resume

2.00 Rating

Austin, TX

SUMMARY:

  • 7+ years of professional IT experience which includes 4+ years of proven experience inHadoop Developer using Cloudera (CDH) and Hortonworks (HDP) Distributions and 3 years of Java developer and SQL Server Database Administration.
  • Extensive experience and knowledge of processing BigData using Hadoop ecosystem components HDFS, Map Reduce, Hive, Pig, Sqoop, Oozie, Flume, Spark.
  • Very Good experience on Unix, Linux, QSHELL scripting.
  • Well versed in installing, upgrading Apache, Cloud era (CDH4) distributions for Hadoop.
  • Experience in Hadoop Distributions like Cloudera, HortonWorks, BigInsights, MapR Windows Azure, and Impala. Hands - on experience with Hadoop applications (such as administration, configuration management, monitoring, debugging, and performance tuning).
  • Strong knowledge on Hadoop HDFS architecture and MapReduce framework.
  • Responsible for writing MapReduce programs using Java.
  • Experience in troubleshooting errors in Pig, Hive and MapReduce.
  • Performed data analysis using Hive and Pig.
  • Experience in importing and exporting data between HDFS and Relational Database Management systems using Sqoop.
  • Good experience working with Hortonworks Distribution and Cloudera Distribution.
  • Implemented standards and processes for Hadoop based application design and implementation.
  • Strong understanding of NoSQL databases like HBase, MongoDB & Cassandra.
  • Developed MapReduce jobs to automate transfer of data from HBase.
  • Loaded streaming log data from various webservers into HDFS using Flume.
  • Support development, testing, and operations teams during new system deployments.
  • Evaluate and propose new tools and technologies to meet the needs of the organization.
  • Always worked closely with system users and management personnel and gained reputation of loyalty.
  • Major strengths are familiarity with multiple software systems, ability to learn quickly new technologies, adapt to new environments, self-motivated, focused, adaptive and quick learner with excellent interpersonal, technical and communication skills.

TECHNICAL SKILLS:

METHODS: HDFS, MapReduce, SqoopHive, PIG, HBASE, Flume, Oozie. Hbase,Cassandra, mongoDB. Java, C++, C, SQL, PIG LatinMATLAB, PL/SQL, MVCMaven, HTML, XML, TCP/IPHTTP, HTTPS, LINUXUNIX, UBUNTU, MacOSWINDOWS 98/00/NT/XP, Oracle8i/9i, MY SQLMS SQL server

CHRONOLOGICAL SUMMARY OF EXPERIENCE:

Senior Hadoop Developer

Confidential, Austin, TX

Responsibilities:

  • Installing and configuring fully distributed Hadoop Cluster.
  • Installing Hadoop Eco-system Components (Pig, Hive and Hbase).
  • Involved in Hadoop Cluster environment administration that includes cluster capacity planning, performance tuning, cluster Monitoring and Troubleshooting.
  • Creating and configuring Hadoop cluster in Cloudera.
  • Coordinating and managing relations with vendors, IT developers and end users.
  • Managing the work streams, process and coordinate the team members and their activities to ensure that the technology solutions are in line with the overall vision and goals.
  • Analyzed the web log data using the HiveQL.
  • Worked on analyzingHadoopcluster and different big data analytic tools including Pig, Hbase NoSQL database and Sqoop.
  • Integrated Cassandra Querying Language called CQL for Apache Cassandra.
  • Installed and configured Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in java for data cleaning and preprocessing.
  • Developed workflows using custom MapReduce, Pig, Hive, Sqoop.
  • Importing and exporting data into HDFS and Hive using Sqoop.
  • Written the Apache PIG scripts to process the HDFS data and send the data to Hbase.
  • Configuration of various database connectivity (Oracle11g, SQL Server 2005)
  • Experienced in running Hadoop streaming jobs to process terabytes of xml format data
  • Load and transform large sets of structured, semi structured and unstructured data.
  • Responsible to manage data coming from different sources.
  • Created Mapreduce jobs using Hive/Pig Queries.
  • Supported Map Reduce Programs those are running on the cluster
  • Experience in providing security for Hadoop Cluster with Kerberos
  • Cluster coordination services through Zoo Keeper.
  • Installed and configured Hive and also written Hive UDFs.
  • Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way
  • Automated all the jobs, for pulling data from FTP server to load data into Hive tables,Using Oozie workflows.
  • Maintained, audited and built new clusters for testing purposes using the cloudera manager.

Environment: Cassandra, MapReduce, HDFS, Hive, Flume, Cloudera Manager, Sqoop MySQL, UNIX Shell Scripting, Zookeeper.

Senior Hadoop Developer

Confidential, San Francisco, CA

Responsibilities:

  • Developed Simple to complex MapReduce Jobs.
  • Maintain the data in Accumulo using Java APIs and using SQRRL shell
  • Create and publish REST Clients for the middleware to interact with the Accumulo NoSQL DB
  • Create ETL jobs using the Talend Bigdata Studio.
  • Handled importing of data from various data sources, performed transformations using Hive, MapReduce, loaded data into HDFS and Extracted the data from Postgres into HDFS using Sqoop.
  • Analyzed the data by performing Hive queries to study customer behavior.
  • Used UDF's to implement business logic in Hadoop.
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
  • Automated all the jobs, for pulling data from FTP server to load data into Hive tables,Using Oozie workflows.
  • Installed and configured Hive, Pig, Sqoop and Oozie on the Hadoop cluster.
  • Installed Oozie workflow engine to run multiple MapReduce, Hive and Pig jobs.
  • Exported the analyzed data to the relational databases using Sqoop for visualization and to generate reports for the BI team.
  • Monitor System health and logs and respond accordingly to any warning or failure conditions.
  • Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts.
  • Manage the team.
  • Interact and co-ordinate with external teams to resolve technical issues
  • Co-ordinate with clients to ensure delivery on time

Environment: Hadoop, Map Reduce, HDFS, Hive, Postgres 9.6, Accumulo (SQRRL), Oozie, Java (jdk1.7), UNIX, SVN and Zookeeper.

Hadoop Developer

Confidential, Woburn, MA

Responsibilities:

  • Capturing data from existing databases that provide SQL interfaces using Sqoop.
  • Implemented Hadoop stack and different bigdata analytic tools, migration from different databases to Hadoop.
  • Processed information from Hadoop HDFS. This information will comprise of various useful insights that can be used in the decision making process. All these insights will be presented to the users in the form of Charts.
  • Working on different Big Data technologies, good knowledge of Hadoop, Map-Reduce, Hive.
  • Developed various POCs over Hadoop, Big data.
  • Worked on deployments and automation task.
  • Installed and configured Hadoop cluster in pseudo and fully distributed mode environments.
  • Involved in developing the data loading and extraction processes for big data analysis.
  • Worked on professional services engagements to help customers design, build clusters, applications, troubleshoot network, disk and operating system related issues.
  • Administer linux servers, other UNIX variants, and managed hadoop clusters.
  • Installed and configured local Hadoop Cluster with 3 nodes and set up 4 nodes cluster on EC2 cloud.
  • Written MapReduce code to process and parsing the data from various sources and storing parsed data into HBase and Hive using HBase-Hive Integration.
  • Work with HBase and Hive scripts to extract, transform and load the data into HBase and Hive.
  • Continuous monitoring and managing of the Hadoop cluster.
  • Analyzed the data by performing Hive queries and running Pig scripts to know user behavior.
  • Installed Oozie workflow engine to run multiple Hive and Pig jobs.
  • Developing scripts andbatch jobto scheduleabundle(group of coordinators), which consists ofvarious Hadoop programsusingOozie.
  • Exported theanalyzed datato the relational databases using Sqoop for visualization and to generate reports.

Environment: Hadoop, HDFS, Map Reduce, Hive, Flume, Sqoop, Cloudera CDH4, HBase, Oozie, Pig, AWS EC2 cloud.

Hadoop Developer

Confidential, Framingham, MA

Responsibilities:

  • Worked on analyzing, writing Hadoop Mapreduce jobs using Java API, Pig and Hive.
  • Responsible for building scalable distributed data solutions using Hadoop.
  • Involved in loading data from edge node to HDFS using shell scripting.
  • Worked on installing cluster, commissioning & decommissioning of datanode, namenode high availability, capacity planning, and slots configuration.
  • Created HBase tables to store variable data formats of PII data coming from different portfolios.
  • Implemented a script to transmit sysprin information from Oracle to Hbase using Sqoop.
  • Implemented test scripts to support test driven development and continuous integration.
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
  • Load and transform large sets of structured, semi structured and unstructured data
  • Experience in managing and reviewing Hadoop log files.
  • Responsible for cluster maintenance, adding and removing cluster nodes, cluster monitoring and troubleshooting, manage and review data backups, manage and review Hadoop log files.
  • Installed Oozie workflow engine to run multiple Hive and pig jobs.
  • Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
  • Supported in setting up QA environment and updating configurations for implementing scripts with Pig and Sqoop.

Environment: Hadoop, HDFS, Pig, Sqoop, HBase, Shell Scripting, Ubuntu, Linux Red Hat.

Java Developer

Confidential

Responsibilities:

  • Involved in Requirement Analysis, Development and Documentation.
  • Participation in developing form-beans and action mappings required for struts implementation and validation framework using struts.
  • Development of front-end screens with JSP Using Eclipse.
  • XML and XSDs are used to define data formats.
  • Implemented J2EE design patterns value object singleton, DAO for the presentation tier, business tier and Integration Tier layers of the project.
  • Involved in Bug fixing and functionality enhancements.
  • Designed and developed excellent Logging Mechanism for each order process using Log4J.
  • Involved in writing Oracle SQL Queries.
  • Involved in Check-in and Checkout process using CVS.
  • Developed additional functionality in the software as per business requirements.
  • Involved in requirement analysis and complete development of client side code.

Environment: Java, J2EE, JSP, EJB, ANT, Weblogic 7.0, JDBC, MyEclipse, Windows XP, CVS, Oracle.

Java Developer

Confidential

Responsibilities:

  • End to end responsibility of Server management and deployment of packages
  • Involved in requirements gathering and analysis for VCare application
  • Involved in the project from its initiation and driven it through all the phases of SDLC like design, development and Testing (UT, ST, SIT and UAT)
  • Designing the workflow
  • Involved in the Performance Tuning of Tomcat Application Server for VCare application.
  • Implementation of Billing and Report Generation for VCare Application
  • Train the associates in PIM2R/ Confidential framework

Environment: J2EE (Servlets, JSPs), Brew, Windows Server 2003, Oracle 10G, MYSQL5.0, Java, PL/SQL, Java Script, Eclipse Ganymede 5.2, Oracle SQL Developer.

Java Developer

Confidential

Responsibilities:

  • Created UML class diagrams that depict the code's design and its compliance with the functional requirements.
  • Used J2EE design patterns for the middle tier development.
  • Developed EJB's in Web Logic for handling business process, database access and asynchronous messaging.
  • Used Java Mail notification mechanism to send confirmation email to customers about scheduled payments.
  • Wrote stored procedures and Triggers using PL/SQL.
  • Involved in building and parsing XML documents using JAX parser.
  • Experience in implementing Web Services and XML/HTTP technologies.
  • Created Unix shell and Perl utilities for testing, data parsing and manipulation.

Environment: Java, JDBC, Servlets, JSP, Struts, Eclipse, Oracle 9i, CVS, JavaScript, J2EE, JDK6, Web Services, SQL, Log4j, XML, XSD, HTML, JUnit, CSS.

We'd love your feedback!