We provide IT Staff Augmentation Services!

Big Data Developer Resume

Columbus, OH

SUMMARY:

  • Overall 9 years of IT experience with around 4 years of experience on Big Data and Data Warehouse/ Analytical applications supporting Insurance and Retail Marketing industry.
  • Expertise in HDFS, Apache Spark, Hive, Spark SQL, Sqoop, Hbase, Oozie, Flume and various other ecosystem components.
  • Working experience on designing and implementing complete end - to-end Hadoop Infrastructure including Spark SQL, HIVE, Sqoop, Oozie, Flume and zookeeper.
  • Experience in working with flume to load the log data from multiple sources directly into HDFS.
  • Strong knowledge on creating and monitoring Hadoop clusters on VM.
  • Manager on Linux, Ubuntu OS etc.
  • Experience in fine tuning spark jobs for better scalability and performance.
  • Performed Importing and exporting data into HDFS and Hive using Sqoop.
  • Good exposure on usage of NoSQL databases column-oriented Hbase.
  • Knowledge and working experience on DB2 and SQL databases.
  • Extensively worked on COBOL, JCL, VSAM, DB2, CICS, Easytrieve and Endeavor.
  • Experience in analysis, development, production support and testing of various applications using Python.

TECHNICAL SKILLS:

Big Data & Data Warehouse Technologies: Apache Pig, Apache Sqoop, Apache Hive, Apache Oozie, Apache HiveQL

Programming Languages: Java, Apache HiveQL, SPARK SQL, Scala, COBOL and IBM Assembler

Databases: DB2, MySQL

Operating System: UNIX, Linux, Windows XP and Server 2008.

PROFESSIONAL EXPERIENCE:

Confidential - Columbus, OH

Big Data Developer

Roles and Responsibilities:

  • Importing data from relational data stores to Hadoop using Sqoop
  • Creating various Spark jobs for performing ETL transformations on the transactional and application specific data sources.
  • Wrote and executed queries in SPARK SQL using Spark-shell.
  • Big data analysis using Pig and User defined functions (UDF).
  • Performed joins, group by and other operations in Spark by using Scala and SPARK SQL.
  • Processed the output from Hive and SPARK SQL and formatted it before sending to the Hadoop output file.
  • Used HIVE definition to map the output file to tables.
  • Wrote map reduce/Hbase jobs
  • Reviewed the HDFS usage and system design for future scalability and fault-tolerance;
  • Worked with HBASE NOSQL database.
  • Experienced in analysing database data and compare it with other open-source NoSQL databases to find which one of them better suites the current requirements.

Environment: APACHE SPARK, Hbase, HDFS, Hive and SPARK SQL.

Confidential - Bethpage, NY

Big Data Developer

Roles and Responsibilities:

  • Importing data from relational data stores to Hadoop using Sqoop
  • Creating various Spark jobs for performing ETL transformations on the transactional and application specific data sources.
  • Wrote and executed queries in SPARK SQL using Spark-shell.
  • Big data analysis using Pig and User defined functions (UDF).
  • Performed joins, group by and other operations in Spark by using Scala and SPARK SQL.
  • Processed the output from Hive and SPARK SQL and formatted it before sending to the Hadoop output file.
  • Used HIVE definition to map the output file to tables.
  • Wrote map reduce/Hbase jobs
  • Reviewed the HDFS usage and system design for future scalability and fault-tolerance;
  • Worked with HBASE NOSQL database.
  • Experienced in analysing database data and compare it with other open-source NoSQL databases to find which one of them better suites the current requirements.

Environment: APACHE SPARK, Hbase, HDFS, Hive and SPARK SQL.

Confidential - Bloomfield, CT

Big Data Developer

Roles and Responsibilities:

  • Importing data from relational data stores to Hadoop using Sqoop
  • Creating various Mapreduce jobs for performing ETL transformations on the transactional and application specific data sources.
  • Wrote and executed PIG scripts using Grunt shell.
  • Big data analysis using Pig and User defined functions (UDF).
  • Performed joins, group by and other operations in MapReduce by using Java and PIG.
  • Processed the output from PIG, Hive and formatted it before sending to the Hadoop output file.
  • Used HIVE definition to map the output file to tables.
  • Wrote map reduce/Hbase jobs
  • Reviewed the HDFS usage and system design for future scalability and fault-tolerance;
  • Worked with HBASE NOSQL database.
  • Experienced in analyzing Cassandra database and compare it with other open-source NoSQL databases to find which one of them better suites the current requirements.

Environment: Apache Hadoop2, MapReduce, Hive, Hbase, HDFS, MapReduce, Hive, PIG, Java.

Confidential - Orange county, CA

DB2, SQL Developer

Roles and Responsibilities:

  • Analysis of Business Requirements.
  • Creating Design Strategy documents.
  • Development of application using Rdz and AQT tool
  • Writing the procedures to report generation.
  • Also involved in SQL tables creation.
  • Involved in the creation of test cases for the testing phase.
  • Unit testing of the newly created components.
  • End to end integration testing of online scenarios.
  • Involved in defect fixing activity as a part of system testing.

Environment: JIRA, Quality Center, Advanced Query Tool (AQT), IBM DB2 Cop, DB2, Microsoft SQL Server.

Confidential

Mainframe Developer

Roles and Responsibilities:

  • Developed MapReduce programs to parse the raw data, populate staging tables and store the refined data in partitioned tables in the HDFS.
  • Supported Phase-2 analysis of Rockblocks business requirements and file types to establish framework and data flows as part of the functional specifications
  • Implemented Phase-1 of Cards Cloudera Hadoop initiative to archive Default Management data including structured non-performing operations data in SQL Server and Oracle databases. Developed high-level and low-level design documents to map to the functional data flows.
  • Enabled speedy reviews and first mover advantages by using Oozie to automate data loading into the Hadoop Distributed File System and PIG to pre-process the data.
  • Primarily developed scripts and programs using shell scripting, Sqoop and Informatica to extract, move/copy/SFTP and load production data from Non-Performing Servicing System DRI (SQL Server), Online Collections System OCS (Oracle), Content Management System CMS (Oracle) to Hadoop HDFS
  • Profiled large source data, including structured and semi-structured/web activity data, worked with data originators to fill in gaps in the data collected, and/or to adjust source-system data structures to facilitate analysis and integration
  • Followed agile methodology from analysis to implementation driven by data type and business functionality
  • Analysed the data by performing Hive queries (HiveQL) and running Pig scripts (Pig Latin) to study customer behaviour
  • Executed multiple performance improvement strategies ensuring proper usage of indexes, and partitioning and bucketing data
  • Monitored workload, job performance and capacity planning using Cloudera Manager.

Environment: JAVA, Shell Scripting, Tomcat web server, Jsp, Cloudera CDH 4, HDFS, Hadoop(Yarn), Flume, Eclipse, Map Reduce, Hive, Pig Latin, Java, SQL, Sqoop, Zookeeper and NOSQL database.

Confidential

Application Developer

Roles and Responsibilities:

  • Impact analysis of the requirements.
  • Design strategy documents preparation.
  • Impact analysis of the design strategy.
  • Prepare the Low level design documentation.
  • Work delegation to the team members.
  • Construction and unit testing.
  • Developing database applications using DB2.
  • Development of application business logic using MasterCraft, TSO/ISPF, SQL.
  • Preparing the Q++ codes
  • Generating C++ codes using MasterCraft
  • Creation of test cases for the Unit and Integration testing.
  • Performing Unit and Integration testing.
  • Responsible for coordinating with testers during system testing.
  • Defect fixing activity as a part of system testing.

Environment: COBOL, JCL, DB2, CICS, Easytrieve, File-Aid, Endevor, CA7, QMF, Online and Batch Debugger, Mastercraft Tool, C++.

Confidential

Mainframe Developer

Roles and Responsibilities:

  • Impact analysis of the requirements.
  • Project Planning and Monitoring
  • Played the role of team member did initial study and implementation of the package
  • Ensure the overall quality of the project
  • Unit Testing and integration testing
  • Implementation of the project.
  • SIT/CIT cycle support.
  • Production monitoring.

Environment: COBOL, JCL, DB2, CICS, Easytrieve, File-Aid, Changeman, CA7, QMF, Online and Batch Debugger, VSAM.

Hire Now