We provide IT Staff Augmentation Services!

Big Data Analyst Resume

Seatac, WA

SUMMARY:

  • Overall 12+ years of experience in all the phases of development life cycle SDLC.
  • More than 4+ years of experience in Hadoop ecosystem tools build on eleven years of IT experience.
  • Hands on Experience and Knowledge in Hadoop (HDFS/HIVE/HBASE/SPARK SQL/SPARK STREAMING/ Scala/MapReduce) concepts and ability to write Scala/Spark RDD functions and proven understanding with listed Hadoop eco system
  • Hands on experience on Data pipelined and SPARK STREAMING data Analytics.
  • Familiarity with data loading/communicating tools like Flume, KAFKA and Sqoop, also workflow/coordinator like Oozie and Zookeeper.
  • Strong Knowledge of Pig and Hive's analytical functions.
  • Implemented Proofs of Concept on Hadoop stack and different big data analytic tools, migration from different databases (Oracle, MYSQL) to Hadoop.
  • Loaded the dataset into Hive for ETL Operation.
  • Good knowledge on Hadoop Cluster architecture and monitoring the cluster.
  • Hands on experience in IDE tools like Eclipse, Visual Studio.
  • Experience in database design using Stored Procedure, Functions, Triggers and strong experience in writing complex queries for MYSQL, SQL Server.
  • Strong understanding of Agile, Scrum and Waterfall Methodologies.
  • Quick learner, Excellent Communication skills and team player, ability to meet tight deadlines and work under pressure, highly committed to responsibilities and result - oriented.
  • Expert of automated test tools - Selenium Web driver, Soap UI, Web Services, and Backend Testing using Oracle.
  • Participate in all walk-through meetings, project status & release calls.
  • Executed various projects in Airlines, Banking, Healthcare and Telecoms Domains.
  • Solid understanding of RDD operations in Apache Spark i.e., Transformations & Actions, Persistence (Caching), Accumulators, Broadcast Variables, Optimising Broadcasts.
  • In depth understanding of Apache spark job execution Components like DAG, lineage graph, Dag Scheduler, Task scheduler, Stages and task.
  • Very good experience in Functional, Integration, System, Database, Load, Regression testing, Performance testing and UAT Testing.
  • Executed SQL queries on Oracle/SQL server in order to verify successful transactions of data from the application.
  • Hands on experience for Defect tracking & defect management from the discovery of defects, submitting the defects with elaboration so that developers are able to recreate & provide the fix faster, retesting of the fixed defects & tracking the defect status in defect tracking tools till it is closed.

TECHNICAL SKILLS:

Operating Systems: WINDOWS 95/98/2000/XP, UNIX, Linux

Languages & Tools: C/C++, SQL, SCALA, JAVA, PYTHON, COBOL JCL, DB2, XML, QTP 10.0, ALM, Version-One, JIRA, SELENIUM, SOAPUI

Hadoop Ecosystem: Map Reduce, Sqoop, Hive, Pig, Hbase 1.1.2, HDFS, Zookeeper, OozieSCALA 2.11.8, Spark 1.6.2, Kafka, AWS, FLUME, HUE, IMPALA

Distribution Platform: (Cloudera, Hartonworks), AWS, JAVA

RDBMS/NoSQL: SQL server, DB2, HBase, Cassandra, MangoDB

Methodologies: Waterfall, Software Development Life Cycle (SDLC),Agile, Scrum, SDET

EXPERIENCE:

Confidential, SeaTac, WA

Big Data Analyst

Responsibilities:

  • Replaced default metadata storage system for Hive with oracle system.
  • Involved in DATAPIPELINE SPARK Streaming with KAFKA and FLUME
  • Executed queries using Hive and developed Map-Reduce jobs to analyze data.
  • Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
  • Testing load test by adding more throughputs and comparing the velocity of the application.
  • Developed Hive queries for the analysts.
  • Analytical and problem solving skills, applied to Big Data domain
  • Involved to implement Scala code in Spark/RDD
  • Implemented and exposed SparkSQL/HiveSQL
  • Utilized Apache Hadoop Distribution environment by Hortonworks.
  • Involved in loading data from LINUX and UNIX file system to HDFS.
  • Supported in setting up QA environment and updating configurations for implementing scripts with HIVE.

Environment: Apache Hadoop, Mapreducer 2.7.3, Pig 0.16.0, Hive 1.2.1, AWS, SQOOP 1.4.6, FLUME 1.5.2, Kafka 0.10.0, HBASE 1.1.2, Spark 1.6.2, Scala, OOZIE

Confidential, Denver, CO

Hadoop Developer

Responsibilities:

  • Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hbase NoSQL database and Sqoop.
  • Importing and exporting data in HDFS and Hive using Sqoop.
  • Extracted files from MongoDB through Sqoop and placed in HDFS and processed.
  • Experience with NoSQL databases.
  • Written Hive HDFS to extract data from staging tables.
  • Involved in creating Hive tables, loading with data and writing hive queries which will run internally in map reduce way.
  • Familiarized with job scheduling using Fair Scheduler so that CPU time is well distributed amongst all the jobs.
  • Involved in the regular Hadoop Cluster maintenance such as patching security holes and updating system packages.
  • Managed Hadoop log files.
  • Analyzed the web log data using the HiveQL.

Environment: Hadoop, Hive, Pig, Map Reduce, Java 8, SQOOP, Oozie, SCALA, SPARK, KAFKA 0.10, CASSANDRA 3.X, ALM

Confidential, Orlando, FL

SDET

Responsibilities:

  • Automated and customized the scripts for the entire application using Selenium.
  • Reviewed the Business Requirement specification Document and the technical specification of the application.
  • Reviewing System Requirements to define testing scope for various levels of testing (Functional, System, Regression, and UAT).
  • Experience testing Transaction processing system for End Users during UAT and ensuring that payments made and received were calculated based on all the Business rules.
  • Developed Test Objectives and Test Plans.
  • Created manual and automated Test Cases and Test Scripts.
  • Created Standard, Image, Table, Page, Text/Text Area Checkpoints.
  • Parameterized the test(s) and modified steps affected by parameterization.
  • Worked with developers to communicate regarding issues in the application.
  • Coordinated the testing effort towards End-to-End testing to verify the proper functioning.
  • Created several compiled modules.
  • Performed Regression Testing.
  • Wrote SQL queries to test the application for data integrity.
  • Tested User Interface inconsistency and application functionality.

Environment: Selenium Web Driver 2.40, Java, TestNG, Eclipse, ANT, Soap UI, Windows, SQLServer, FireBug, FirePath, JIRA, ALM, JAVA

Confidential

Mainframe Developer

Responsibilities:

  • Understanding the requirements of Existing Application Systems
  • Knowledge acquisition of the existing business processes and application design
  • Analysis, Design, Coding and Testing of new business requirements
  • Modification of the applications as per business requirements
  • Interacting with the client on regular basis to solve the issues
  • Ensuring that the delivered products/services conform to the customer requirements and agreed quality standards
  • Reviewed the Business Requirement specification Document and the technical specification of the application.
  • Worked with testers to communicate regarding issues in the application.
  • Wrote SQL queries to test the application for data integrity in QMF.

Environment: COBOL, JCL, DB2, VSAM, File-Aid, Endevor, SPUFI

Confidential

Mainframe Developer

Responsibilities:

  • Understanding the requirements of Existing Application Systems
  • Knowledge acquisition of the existing business processes and application design
  • Analysis, Design, Coding and Testing of new business requirements
  • Modification of the applications as per business requirements
  • Interacting with the client on regular basis to solve the issues
  • Ensuring that the delivered products/services confirm to the customer requirements and agreed quality standards
  • Reviewed the Business Requirement specification Document and the technical specification of the application.
  • Worked with testers to communicate regarding issues in the application.
  • Wrote SQL queries to test the application for data integrity in QMF.

Environment: COBOL, JCL, DB2, VSAM, IMS-DB, File-Aid, Endevor, SPUFI

Confidential

Software Engineer

Responsibilities:

  • Understanding the requirements of Existing Application Systems
  • Knowledge acquisition of the existing business processes and application design
  • Analysis, Design, Coding and Testing of new business requirements
  • Modification of the applications as per business requirements
  • Interacting with the client on regular basis to solve the issues
  • Ensuring that the delivered products/services conform to the customer requirements and agreed quality standards

Environment: COBOL, JCL, DB2, VSAM, IMS-DB/Dc, File-Aid, SPUFI

Hire Now