We provide IT Staff Augmentation Services!

Big Data/hadoop Developer Resume

0/5 (Submit Your Rating)

Phoenix, AZ

SUMMARY

  • 7+years of extensive Professional IT experience, including 4+ years of Hadoop /Big data experience, 3 years of Java Experience, capable of processing large sets of structured, semi - structured and unstructured data and supporting systems application architecture.
  • Expert in importing and exporting of data using Sqoop from HDFS to relational database system and vice-versa.
  • Expert level of scripting using Pig scripts and Hive queries for processing and analyzing large volume of data.
  • Experience with Oozie Workflow Engine in running workflow designing, job scheduling with actions that run Hadoop Map/Reduce and Pig jobs.
  • Good Experience in developing and implementing big data solutions and data mining applications on Hadoopusing Hive, PIG, Hbase, Hue, Oozie workflows and designing and implementing Java, Python Map Reduce programs.
  • Good knowledge onHadoopcluster administration, Monitoring and managingHadoopcluster using cloudera manager.
  • Knowledge in installing, configuring, and usingHadoopecosystem components likeHadoopMap Reduce, HDFS, HBase, Oozie, Hive, Sqoop, Pig, Flume, Apache Spark, Zookeeper and Kafka.
  • Experience in managing and reviewingHadooplog files.
  • Hands on experience in converting Hive/SQL queries into Spark transformations using Scala.
  • Good Experience with flume tool for data ingestion from various data producers (webservers) intoHadoop.
  • Good knowledge in NOSQL databases HBASE, MongoDB.
  • Sound Relational Database Concepts and extensively worked with ORACLE, MySQL, SQL Server.
  • Good Experience with databases, writing complex queries and stored procedures using SQL and PL/SQL.
  • Experience in using sequence file, RC file and Avro file formats.
  • Good understanding of ClassicHadoopand Yarn architecture along with variousHadoopDemons such as Job Tracker, Task Tracker, Name Node, Data Node, Secondary Name Node, Resource Manager, Node Manager, Application Master and Containers.
  • Very good experience with both MapReduce 1 (Job Tracker) and MapReduce 2 (YARN) setups.
  • Expert in Java MapReduce Jobs, User Defined functions for Pig and Hive.
  • Knowledge in handling messaging services using Apache Kafka.
  • Familiarity on real time streaming data with Sparkfor fast large scale in memory Map Reduce.
  • Experience in Business Intelligence tools such as Tableau, Datameer for visually analyzing the data.
  • Experience in building, maintaining multipleHadoopclusters of different sizes and configuration and setting up the rack topology for large clusters.
  • Developedmachine learning algorithms using Mahout for clustering and data mining.
  • Involvement in all phases of SDLC from project proposal, planning, analysis, development, testing, deployment and support.
  • Experience in developing and implementing web applications using Java, JSP, CSS, HTML, HTML5, XHTML and Java script, JSON, XML, JDBC.
  • Experience in working in 24X7 Support and used to meet deadlines, adaptable to ever changing priorities.
  • Proven ability to work with senior technical managers and staff to provide expert-level support for the installation, maintenance, upgrading, and administration of full-featured database management systems.
  • Excellent interpersonal and communication skills, creative, research-minded, technically competent, result-oriented with problem solving as well and ability to work well with people and to maintain a good relation with the organization.

TECHNICAL SKILLS

Technology Tools: Big data andHadoop, Apache/Cloudera HDFS 1.X/2.X, MapReduce, YARN, Sqoop, Flume, Spark, Scala, Hive, Pig, Oozie, Zookeeper, Kafka

Operating Systems: MS Windows, Linux, Ubuntu, CentOS

Programming Languages: C, C++, Java, SQL, PL/SQL, Python, JavaScript,Unix Shell Scripting

Database: Oracle, MySQL, Microsoft Sql Server

NoSQL Databases: Hbase, MongoDB, Cassandra.

IDE Tools: Eclipse, Net Beans

Other Skills: Tableau, Datameer, HTML5, Java Script, JSON, CSS XML, Apache Tomcat

PROFESSIONAL EXPERIENCE

Confidential, Phoenix, AZ

Big Data/Hadoop Developer

Responsibilities:

  • Installed and Configured ApacheHadoopclusters for application development andHadooptools like Hive, Pig, Oozie, Zookeeper, Hbase, Flume and Sqoop.
  • Implemented multiple Map Reduce Jobs in java for data cleaning and pre - processing.
  • Worked in a team with 40 node cluster and increase cluster by adding Nodes, the configuration for additional data nodes was done by Commissioning process inHadoop.
  • Responsible for Cluster maintenance, adding and removing cluster nodes, Cluster Monitoring and Troubleshooting, manage and review data backups and log files.
  • Responsible to manage data coming from different sources.
  • Managed and scheduled Jobs on aHadoopcluster.
  • Implemented a script to transmit information from Oracle to Hbase using Sqoop.
  • Involved in defining job flows, managing and reviewing log files.
  • Installed Oozie workflow engine to run multiple Map Reduce, HiveQL and Pig jobs.
  • Participated in requirement gathering form the Experts and Business Partners and converting the requirements into technical specifications.
  • Created Hive tables to store the processed results in a tabular format.
  • Was done various compressions and file formats like snappy, Gzip, Avro, Sequence, text.
  • Wrote complex Hive queries and UDFs in Java and Python.
  • Created and exposed Hive views through Impalafor the business Users
  • Involved in forecast based on the present results and insights derived from data analysis.
  • Involved in collecting the data and identifying data patterns to build trained model using Machine Learning.
  • Prepare Developer(Unit) Test cases and execute developertesting.
  • Implemented test scripts to support test driven development and continuous integration.
  • Developed and implemented some machine learning algorithmsusing Mahout for data mining for the data stored in HDFS
  • Created and maintained Technical documentation for launchingHadoopClusters and for executing Hive queries and Pig Scripts.
  • Worked on visualization tool tableau for visually analyzing the data.

Environment: Hadoop, HDFS, Pig, Hive, Map Reduce, Impala, Sqoop, Flume, Oozie, Big Data, java, Python, Mahout, Junit testing, Oracle, MySQL, Tableau, LINUX, Windows.

Confidential, Phoenix, AZ

Java/Hadoop Developer

Responsibilities:

  • Worked on data migration from existing data sources toHadoopfile system.
  • Interact with the Business Analysts to get the requirements and formulate them into system use cases.
  • Understand customer business use cases and be able to translate them to analytical data applications and models to implement a solution.
  • Created flow charts, sequence diagrams, schemas, data model of underlying system, pseudocode and class diagrams using Microsoft Vision.
  • Ability to create Hive Avro Tables to read and write the data with the appropriate compressions.
  • Created custom Database Encryption & Decryption UDF that could be plugged in while Ingesting data to External Hive Tables for maintaining security at table or column level.
  • Worked on No - SqlHbase database for processing very large semi structured and structured tables by defining column families, region servers etc.
  • ImplementedSparkSQL for faster testing and processing of data.
  • Knowledge on handling Hive queries usingSparkSQL that integrate withSparkenvironment.
  • Worked on migrating MapReduce programs intoSparktransformations usingSpark.
  • ConfiguredSparkStreaming to receive real time data and store the stream data to HDFS.
  • Developed map-reduce programs for different patterns of data onHadoopcluster.
  • Developed java map reduce programs using core concepts like OOPS, Multithreading, Collections and IO. Compiled and built the application using MAVEN and used SVN as version control system.
  • Created data ingestion plans for loading the data from external sources using Sqoop.
  • Used Python for pattern matching in build logs to format errors and warnings
  • Created flow charts, sequence diagrams, schemas, data model of underlying system, pseudo code and class diagrams using Microsoft Visio.

Environment: Hadoop, HDFS, Pig, Hive, Map Reduce, Sqoop, Oozie, Spark, Big Data, Hbase, java, Python,Unix Shell Scripting, Oracle 11g/10g, MySQL, MS Visio, LINUX, Windows.

Confidential, Chicago, IL

Java/Hadoop Developer

Responsibilities:

  • Analyzed large data sets by running Hive queries and Pig scripts.
  • Involved in creating Hive tables, and loading and analyzing data using hive queries.
  • Developed Simple to complex Map Reduce Jobs using Hive and Pig
  • Involved in runningHadoopjobs for processing millions of records of text data.
  • Load and transform large sets of structured, semi structured and unstructured data.
  • Responsible to manage data coming from different sources.
  • Implemented multiple Map Reduce Jobs in java for data cleaning and pre - processing.
  • Implemented Partitioning, Dynamic Partitions, Buckets in HIVE.
  • Involved in loading data from Unix file system to HDFS.
  • Monitor System health and logs and respond accordingly to any warning or failure conditions.
  • Implemented the workflows using Apache Oozie framework to automate tasks.
  • Worked with application teams to install operating system,Hadoopupdates, patches, version upgrades as required.
  • Developed unit test cases forHadoopMapReduce jobs with JUnit.
  • Developed multiple Map Reduce jobs in java for data cleaning and preprocessing.
  • Involved in maintaining variousUnix Shell scripts.
  • Assisted in exporting analyzed data to relational databases using Sqoop.
  • Supported Map Reduce Programs those are running on the cluster.
  • Created and maintained Technical documentation for launchingHadoopClusters and for executing Hive queries and Pig Scripts.

Environment: Hadoop, HDFS, Pig, Hive, MongoDB, Map Reduce, Sqoop, Oozie, Unix Shell Scripting, Big Data, java, Oracle 11g/10g, MySQL, LINUX, Windows.

Confidential 

Java Developer

Responsibilities:

  • Managed connectivity using JDBC for querying/inserting & data management including triggers and stored procedures.
  • Developed UI using HTML, JavaScript and JSP developed Business Logic and Interfacing components using Business Objects, XML.
  • Communicated with client to Analyze and Review of Business/Technical Requirements
  • Designed and developed Co - Branding framework to change UI for different clients using same application.
  • Used Struts, Spring (MVC) Framework to develop the application.
  • Used Factory, DAO, Singleton and DTO and Value Object, Business Delegate design patterns.
  • Designed and developed database layer using ORM technologies like Hibernate.
  • Developed Web services client to consume with Vender Web services
  • Generated Excel reports Using POI framework
  • Implementing the business logic and creating dynamic web pages with JSP and using JavaScript to in corporate client side and server side validations and functionality.
  • Preparation and Review of Unit Test Plans, Scripts and Results
  • Used respective designed patterns to implement the reusable component of the application.
  • Implemented various GUI screens using JSP, AJAX Frame work and JQuery.
  • Worked with web methods for deploying and administering services.
  • Used Hibernate to interact with database.
  • Worked with requirement analysis team to gather software requirements for application development.
  • Provided support and maintenance after deploying the web application.
  • Resolved issues reported by the client.

Environment: Linux, Windows, Core Java, JSP, JavaScript Hibernate, Servlets, Spring MVC Framework, Hibernate, Oracle, JMS, Ajax, JQuery, XML, Log4j, Apache Tomcat and Eclipse.

We'd love your feedback!