We provide IT Staff Augmentation Services!

Big Data/hadoop Developer Resume

3.00/5 (Submit Your Rating)

Seattle, WA

SUMMARY

  • Extensive IT experience in Big Data domain wif tools like Hadoop, Hive and other open source tools/technologies in Banking, Healthcare and Energy.
  • Substantial experience writing MapReduce jobs in Java, PIG, Flume, Zookeeper and Hive and Storm.
  • Experience in development ofBigDataprojects using Hadoop, Hive, HDP, PIG, Flume, Storm and MapReduce open source tools/technologies.
  • Extensive Knowledge on automation tools such as Puppet and Chef.
  • Hands on experience in installing, configuring and using ecosystem components like Hadoop MapReduce, HDFS, Hbase, AVRO, Zoo Keeper, Oozie, Hive, HDP, Cassandra, Sqoop, PIG, Flume.
  • Extensive experience in SQL and NoSQL development.
  • In - depth understanding of Data Structure and Algorithms.
  • Experience in web-based languages such as HTML, CSS, PHP, XML and other web methodologies including Web Services and SOAP.
  • Extensive experience in all teh phases of teh software development lifecycle (SDLC).
  • Experience in deploying applications in heterogeneous Application Servers TOMCAT, Web Logic and Oracle Application. Server.
  • Extensive knowledge of NoSQL databases such as HBase.
  • Worked on Multi Clustered environment and setting up Cloudera Hadoop echo System.
  • Background wif traditional databases such as Oracle, Teradata, Netezza, SQL Server, ETL tools / processes and data warehousing architectures.

TECHNICAL SKILLS

  • Hadoop
  • HIVE
  • HDP
  • PIG
  • Sqoop
  • Flume
  • MapReduce
  • Splunk
  • HDFS
  • Zookeeper
  • Storm
  • Shell
  • Python
  • AVRO
  • AIX 5.1
  • Red Hat Linux. Cent OS
  • Lucene and Solr
  • Apache Contributer
  • Puppet and Chef
  • JIRA
  • SDLC
  • MongoDB
  • Cloudera
  • HortonWork
  • Datastage
  • Talend Open Studio
  • Tableau
  • Qlickview
  • Giraph
  • IBM DB2
  • Teradata
  • MySQL
  • NoSQL
  • AWS ( Confidential Web Services)
  • EMR
  • Data Pipeline and Redshift
  • ETL Tool (Informatica)
  • Data Warehouse/Business Intelligence (BI)

PROFESSIONAL EXPERIENCE

Confidential, Seattle, WA

Big Data/Hadoop Developer

Responsibilities:

  • Worked on Distributed/Cloud Computing (MapReduce/Hadoop, PIG, HBase, AVRO, Zookeeper, etc.), Confidential Web Services (S3, EC2, EMR, etc.), Oracle SQL Performance Tuning and ETL, Java 2 Enterprise, Web Development, Mobile Application Developement (Objective-C, Java Native Mobile Apps, Mobile Web Apps), Agile Software Development, Team Building & Leadership, Engineering Management, Internet of Things (Amateur Sensor Networks, Embedded Systems and Electrical Engineering).
  • Working as a lead on Big Data Integration and Analytics based on Hadoop, SOLR and web Methods technologies.
  • Working in implementing Hadoop wif teh AWS EC2 system using a few instances in gathering and analyzing data log files.
  • Working on data using Sqoop from HDFS to Relational Database Systems and vice-versa.
  • Working on Confidential on loading files to HIVE and HDFS from MongoDB.
  • Founded and developed environmental search engine engine using PHP5, JAVA, Lucene/SOLR, Apache and MYSQL.
  • Led teh evaluation of Big Data software like Splunk, Hadoop for augmenting teh warehouse, identified use cases and led Big Data Analytics solution development for Customer Insights and Customer Engagement teams.
  • Designed techniques and wrote TEMPeffective and successful programs in JAVA, Linux shell scripting to push teh large data including teh Text and Byte type of data to successfully migrate to NO SQL Stores using various Data Parser techniques in addition to Map Reduce jobs.
  • Involved in scheduling Oozie workflow engine to run multiple Hive and PIG jobs.
  • Worked on TOADfor Data Analysis, ETL/Informatica for data mapping and teh data transformation between teh source and teh target database.
  • Working on Hive/Hbase vs RDBMS, imported data to Hive, HDP created tables, partitions, indexes, views, queries and reports for BI data analysis.
  • Developing data pipeline using Flume, Sqoop, PIG and Java map reduce to ingest customer behavioral data and financial histories into HDFS for analysis.
  • Currently working on XML parsing using PIG, Hive, HDP and Redshift.
  • Working on architected solutions dat process massive amounts of data on corporate and AWS cloud based servers.
  • UsesSplunkto detect any malicious activity against webservers
  • Tuned teh Hadoop Clusters and Monitored for teh memory management and for teh Map Reduce jobs, to enable healthy operation of Map Reduce jobs to push teh data from SQL to No SQL store.
  • Working in importing streaming logs and aggregating teh data to HDFS through Flume.
  • Delivered Working Widget Software using EXTJS4, HTML5, RESTFUL Web services, JSON Store, Linux, Hadoop, ZOOKEEPER, NO SQL databases, JAVA, SPRING Security, JBOSS Application Server for Big Data analytics.
  • Developed a custom AVRO Framework capable of solving small files problem in Hadoop and also extended PIG and Hive tools to work wif it.
  • Working on use cases, data requirements and business value for implementing a Big Data Analytics platform.
  • Working on configuring and Maintaining Hadoop environment on AWS.
  • Developed application component interacting wif MongoDB.
  • Working on Modifying Chef Recipes used to configure teh Hadoop stack.
  • Evaluate Puppet framework and tools to automate teh cloud deployment and operations.
  • Working on Installing and configuring Hive, HDP, PIG, Sqoop, Flume, Storm and Oozie on teh Hadoop cluster.
  • Working in analyzing data using Hive, PIG, Storm and custom MapReduce programs in Java.
  • Developed Use cases and Technical prototyping for implementing PIG, HDP, HIVE and HBASE.

Confidential, Milwaukee, WI

Big Data/Hadoop Developer

Responsibilities:

  • Installed and configured Hadoop through Confidential Web Services in cloud.
  • Designed, planned and delivered proof of concept and business function/division based implementation of Big Data roadmap and strategy project (Apache Hadoop stack wif Tableau) in UnitedHealthcare using Hadoop.
  • Developed MapReduce jobs in Java for data cleaning and preprocessing.
  • Importing and exporting data into HDFS and Hive using Sqoop.
  • Used Bash Shell Scripting, Sqoop, AVRO, Hive, HDP, Redshift, PIG and Java Map/Reduce daily to develop ETL, batch processing, and data storage functionality.
  • Responsible for developing data pipeline using flume, Sqoop and PIG to extract teh data from weblogs and store in HDFS.
  • Worked on NoSQL databases including Hbase and MongoDB.
  • Used data stores included Accumulo/Hadoop and graph database.
  • Exploited Hadoop MySQL - Connector to store Map Reduce results in RDBMS.
  • Used Dell Crowbar as a wrapper to Chef to deploy Hadoop.
  • Collected data from different databases (me.e. Teradata, Oracle, MySQL) to Hadoop
  • Used Oozie and Zookeeper for workflow scheduling and monitoring.
  • Worked on Designing and Developing ETL Workflows using Java for processing data in HDFS/Hbase using Oozie.
  • Created Hbase tables to store various data formats of PII data coming from different portfolios
  • Conduct vulnerability analyses; reviewing, analyzing and correlating threat data from available sources such asSplunk.
  • Working on extracting files from MongoDB through Sqoop and placed in HDFS and processed.
  • Worked on Hadoop installation & configuration of multiple nodes on AWS EC2 system.
  • Experienced in running Hadoop streaming jobs to process terabytes of XML format data.
  • Cluster coordination services through Zoo Keeper.
  • Involved in loading data from UNIX file system to HDFS
  • Installed and configured Hive and also written Hive UDFs.
  • Worked on setting up PIG, Hive, Redshift and Hbase on multiple nodes and developed using PIG, Hive, Hbase, MapReduce and Storm.
  • Design and implement data processing using AWS Data Pipeline.
  • Drove holistic tech transformation to Big Data platform for UnitedHealthcare, create strategy, define blueprint, design roadmap, build end-to-end stack, evaluate leading technology options, benchmark selected products, migrate products, reconstruct information architecture, introduce metadata management, leverage machine learning, productionize consolidated data store: Hadoop, MR, Hive, HDP and MapReduce.
  • Developed Simple to complex MapReduce Jobs using Hive and PIG.
  • Worked on automate monitoring and optimizing large volume data transfer processes between Hadoop clusters and AWS.
  • Strong knowledge on Data Warehousing experience using Informatica Power Center.
  • Configure and manageSplunkForwarders,SplunkIndexers andSplunkSearch Heads.
  • Automated all teh jobs for pulling data from FTP server to load data into Hive tables using Oozie workflows.

Confidential

JAVA Developer

Responsibilities:

  • Worked wif Big Data platform Cloudera Hadoop (CDH) and wif different modules wifin Hadoop including HIVE and PIG to generate queries to teh data.
  • Created several AWS instances in teh Confidential cloud for some interim solutions
  • Worked on building scalable, fault tolerant and distributed data solutions wif cloud computing stack such as Hadoop, Hbase & Accumulo.
  • Worked on entire data pipeline for automating using Flume and for teh jobs scheduled periodically using Oozie.
  • Expertise in writing shell scripts and Oozie workflows
  • Worked on developing and extending serialization frameworks like AVRO.
  • Development and maintenance ofSplunkdashboards based on teh requirements.
  • Worked on configure AWS EMR (Elastic MapReduce).
  • Written teh Apache PIG scripts to process teh HDFS data.
  • Reviewing teh existingHadoopenvironment and make recommendations of new features dat may be available and performance tuning wif teh other tools like Hive, PIG, MapReduce, Storm and Flume.
  • Worked on Monitoring, Replication and Sharding Techniques in MongoDB.
  • Performed Manual Testing, reported defects in JIRA and was responsible to keep track of them.
  • Used BI solution as a custom application build using OBIEE as front end and ODI and PL/SQL used for ETL.
  • Worked in Installation, Configuration and Management of Hadoop Cluster spanning multiple racks using automated tools like puppet and chef.
  • Worked on providing support for AWS Data Pipeline.
  • Worked on automating teh jobs using Oozie in teh project.
  • Used sequence and AVRO file formats and snappy compressions while storing data in HDFS.
  • Developed MapReduce application using Hadoop, Redshift, MapReduce programming and Hbase.
  • Monitoring Hadoop scripts which take teh input from HDFS and load teh data into Hive.

We'd love your feedback!