We provide IT Staff Augmentation Services!

Hadoop Developer Resume

2.00/5 (Submit Your Rating)

Irving, TX

SUMMARY

  • Around 6 years of experience in IT Industry including 3 years in all teh phases of project developing including Hadoop ecosystem components.
  • Involved in all phases of teh SDLC including analysis, design development, testing, and deployment of Hadoop Cluster
  • Well versed wif Hadoop MapReduce, HDFS, Pig, Hive, HBase, Sqoop, Flume, Yarn, Zookeeper, Spark, Oozie and Kafka
  • Good experience in Designing, Installing, Configuring and Tuning Hadoop core components and ecosystem components
  • Experience in handling of Cloudera’s Hadoop platform along wif CDH 4.x and CDH 5.x for Development, Support and Maintenance
  • Also had experience in installation, configuration, support and management of a Hadoop Cluster.
  • Designed various Hadoop Architectures wif features like high availability, scalability, process isolation, load - balancing, workload scheduling, etc.
  • In depth understanding of Hadoop Architecture and various components like HDFS, NameNode, Job Tracker, DataNode, Task Tracker and Map Reduce concepts.
  • Experience in task automation using Oozie, cluster co-ordination through Pentaho and MapReduce job scheduling using Fair Scheduler.
  • Experience on Kafka, Spark mainly in framework exploration for transition from Hadoop/MapReduce to Spark
  • Strong knowledge and strong understanding in NoSQL databases like HBase, MongoDB
  • Worked on both Cloudera and Hortonworks Hadoop distributions
  • Experience in performing minor and major upgrades and applying patches for Ambari and Cloudera Clusters
  • Strong Knowledge in Hadoop Cluster Capacity Planning, Performance Tuning, Cluster Monitoring
  • Expertise in Spark framework for batch and real time data processing
  • Experience in working wif BI team and transform big data requirements into Hadoop centric technologies
  • Experience in converting MapReduce applications to Spark
  • Had good experience in running Pig and Hive scripts
  • Excellent skills in web technologies like ASP.NET, ASP, XML VBSCRIPT, JAVASCRIPT and HTML
  • Experience in configuring Zookeeper to coordinate teh servers in clusters
  • Experience in using NFS (Network File Systems) for backing up Name node metadata
  • Expertise in deploying applications using Web and Application servers like BEA WebLogic 8.1/9.2, Oracle 9iAS JBoss and Apache Tomcat on Windows and UNIX environments
  • Hands on experience wif Object Oriented Design OOD and developing applications using UML
  • Expertise wif Iterative development methodologies, designing Class diagrams, sequence diagrams and Use case realization.

TECHNICAL SKILLS

Big Data Ecosystem Components: Hadoop/BigData HDFS, AWS Amazon, Spark, Shark, MapReduce, HBase, Pig, Hive, Sqoop, NoSQL DB, CDH3, CDH4, Apache Hadoop, Cassandra, Java, Zookeeper, Oozie, Mongo DB, Tez, Data Modelling, HCatalog, Spark Streaming, Flume, Scala, CPS, Talend

Programming Languages: C, C++, Java, J2EE, Linux, Python

Databases: Oracle, MySQL, DB2, MS SQL Server

Operating Systems: Windows, MAC, Linux & Unix

Databases: MYSQL, Oracle, Teradata, PostgreSQL, DB2

Scripting: Shell scripting and HTML scripting

PROFESSIONAL EXPERIENCE

Confidential, Irving, TX

Hadoop Developer

Responsibilities:

  • Worked on evaluation and analysis of Hadoop cluster and different big data analytic tools including Pig, Hbase and Sqoop.
  • Responsible for building scalable distributed data solutions using Hadoop.
  • Involved in loading data from LINUX file system to Hadoop Distributed File System.
  • Created Hbase tables to store various data formats of PII data coming from different portfolios.
  • Experience in managing and reviewing Hadoop log files.
  • UsedDatameerto analyze teh transaction data for teh client.
  • Also installed, configured and managedDatameerusers on teh Hadoop cluster.
  • Exporting teh analyzed and processed data to teh relational databases using Sqoop for visualization and for generation of reports for teh BI team.
  • Installed Oozie workflow engine to run multiple Hive and pig jobs.
  • Analyzing large amounts of data sets to determine optimal way to aggregate and report on these data sets
  • Worked wif teh Data Science team to gather requirements for various data mining projects.
  • Analyzed large data sets by running Hive queries and Pig scripts.
  • Worked on performing majorupgradeof cluster from CDH3u6 to CDH4.4.0
  • Created dash boards using Tableau to analyze data for reporting.
  • Support for setting up QA environment and updating of configurations for implementation scripts wif Pig and Sqoop.

Environment: Hadoop, HDFS, SAS, Pig, Datameer Sqoop, SQL, Python, HBase, Shell Scripting, Linux Red Hat

Confidential

Hadoop Consultant

Responsibilities:

  • Exported data from DB2 to HDFS using Sqoop and NFS mount approach.
  • Moved data from HDFS to Cassandra using Map Reduce and BulkOutputFormat class.
  • Developed Map Reduce programs for applying business rules on teh data.
  • Developed and executed hive queries for denormalizing teh data.
  • Installed and configured Hadoop Cluster for development and testing environment.
  • Implemented Fair scheduler on teh Job tracker to share teh resources of teh cluster for teh map reduces jobs given by teh users.
  • Configure Web HDFS to support REST API, JDBC connectivity to external clients for operations
  • Analyzed teh web log data using teh HiveQL to extract number of unique visitors per day, page views, visit duration, most purchased product on website.
  • Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports by our BI team.
  • Dumped teh data using Sqoop into HDFS for analyzing.
  • Developed data pipeline using Pig and Hive from Teradata and Netezza data sources. These pipelines had customized UDF’S to extend teh ETL functionality.
  • Developed job flows in Oozie to automate teh workflow for extraction of data from Teradata and Netezza
  • Developed data pipeline into DB2 containing teh user purchasing data from Hadoop
  • Implemented Partitioning, Dynamic Partitions, buckets in Hive and wrote map reduce programs to analyze and process teh data
  • Streamlined Hadoop jobs and workflow operations using Oozie workflow engine.
  • Involved in product life cycle developed using Scrum methodology.
  • Involved in mentoring team in technical discussions and Technical reviews.
  • Involved in code reviews and verifying bug analysis reports.
  • Automated work flows using shell scripts.
  • Performance tuning of teh hive queries, written by other developers.

Environment: Hadoop, HDFS, Hive, MapReduce 2.0, Sqoop 2.0.0, Oozie 3.0,SQL, Shell Scripting, Ubuntu, Linux Red Hat.

Confidential

Java Developer

Responsibilities:

  • Involved in complete software development life cycle - Requirement Analysis, Conceptual Design, Detail design, Development, System and User Acceptance Testing.
  • Involved in Design and Development of teh System using Rational Rose and UML.
  • Involved in Business Analysis and developed Use Cases, Program Specifications to capture teh business functionality.
  • Improving teh coding standards, code reuse, and performance of teh Extend application by making effective use of various design patterns (Business Delegate, View Helper, DAO, Value Object etc. and other Basic patterns).
  • Design of system using JSPs, Servlets
  • Designed application using Process Object, DAO, Data Object, Value Object, Factory, Delegation patterns.
  • Involved in teh design and development of Presentation Tier using JSP, HTML and JavaScript.
  • Involved in integrating teh concept of RFID in teh software and developing teh code for its API.
  • Coordinating between teams as a Project Co-coordinator, organizing design and architectural meetings.
  • Design and developed Class diagram, Identifying Objects and its interaction to specify Sequence diagrams for teh System using Rational Rose.
  • Developed Unit test cases and automated teh scripts.
  • Involved in creating oozie and Control-M workflows to automate data transformation and presentation processes.
  • Hands on experience on Oozie workflow scheduled through Falcon.
  • Maintaining different cluster security settings and involving in creation and termination of multiple cluster environment.
  • Involving in brain storming JAD sessions.
  • Worked in Agile environment, this uses Jira to maintain teh story points and Kanban model.
  • Hands on experience on maintaining teh builds in Bamboo and resolved teh build failures in Bamboo.

Environment: JDK 1.3, J2EE, JSP, Servlets, HTML, XML, UML, RATIONAL ROSE, AWT, Web logic 5.1 and Oracle 8i, SQL, PL/SQL.

We'd love your feedback!