We provide IT Staff Augmentation Services!

Sr. Bigdata Developer Resume

Dallas, TX

PROFESSIONAL SUMMARY:

  • Having 6 years of total experience in IT industry.
  • Responsible for complete SDLC management using different methodologies like Agile, Incremental, Waterfall, etc
  • Underwent extensive training on Big Data spanning over 45 days. Covered modules like Big Data Technology Landscape and worked on Business use cases (PoC). Technologies like Pig, Hive, Spark, Oozie, MapReduce and Flume were used.
  • Worked hands on with ETL process.
  • Extensive Experience in Setting Hadoop Cluster.
  • Experience in working with different data sources like Flat files, XML files and Databases.
  • Hands on experience in migrating data from RDBMS to Hadoop ecosystem.
  • Worked on writing various Linux scripts to stream data from multiple data sources like Oracle and Teradata.
  • Extracted the data from Teradata into HDFS using Sqoop.
  • Hands on experience in working with Ecosystems like Hive, Pig, Sqoop, Map Reduce, Ooozie.
  • Trained and hands on work experience of Datastax distribution of Cassandra.
  • Load and transform large sets of structured, semi structured and unstructured data.
  • Responsible to manage data coming from different sources.
  • Been a part of Project Delivery team for a reputed client and interacted directly with customer.
  • Work experience in supporting Kafka, Storm, Hbase, Sqoop and other Hadoop components in administrative as well as development capacities for Big Data applications
  • Good knowledge in Hadoop architecture and ecosystem components
  • Data scrubbing and processing with Oozie.
  • Good interpersonal skills, fast learner, team player, self - directed and self-motivated.

TECHNICAL SKILLS:

Big Data: Spark, Flume, Hadoop, Pig, Kafka, Oozie, Hive, Sqoop, MapReduce, IBM BigInsight, Cloudera Distribution.

NoSQL: Cassandra, Dynamo DB, HBase.

IDE: Eclipse, Visual studio 2008, Visual studio 2010.

Programming Language: Java, Scala.

Database: SQL Server 2005, SQL Server 2008.

Operating System: Windows XP, Windows 7, Windows 8, CentOS 6.5.

Web Tools: HTML, CSS.

Other Tools: Shell Scripting, Maven, AWS EMR.

Development Platform: GitHub.

PROFESSIONAL EXPERIENCE:

Confidential, Dallas, TX

Sr. BigData Developer

Responsibilities:

  • Business and scope analysis for developing and managing data lake.
  • Overall requirement analysis
  • Low level design and High Level Design preparation
  • Designed and Developed the code for modeling jobs as per client requirement in agile mode, and standardized it using coding standards and code versioning techniques
  • Writing spark code to read CSV file from S3 bucket.
  • Spark code to read metadata from DynamoDB.
  • Perform data analysis on the Data frames and calculating the matrices.
  • Perform validation for each column depending upon the predefined rules.
  • Updating the DynamoDB table for the report.

Technologies Used: Amazon Web Services EMR, S3 Bucket, Amazon Lambda, Spark 2.0, DynamoDB, Scala, SFTP, Linux Shell Scripting.

Confidential, Bethesda, MD

Sr. BigData Developer

Responsibilities:

  • Overall analysis of requirements
  • Design of the entire system
  • Writing spark code to read XML data as RDD and transform into DataFrame.
  • Oozie workflow creation for Spark Action.
  • Filtering and Auditing XML Data frames and logging in HDFS as flat files.
  • Storing DataFrames as Hive tables.
  • Creating Hive Structure.

Technologies Used: Hortonworks Distribution of hadoop IBM BigInsight 4.1, Spark, Kafka, Hive, Oozie, Git., Scala, SFTP, Linux Shell Scripting.

Confidential

Sr. BigData Developer

Responsibilities:

  • Requirement Gathering And Analysis.
  • Low Level Design And Data Structuring.
  • Setting up of Apache Cassandra cluster.
  • Cassandra Data Modelling.
  • MapR educe Programs for Data Transformation And Validation
  • Data transformation by writing Pig Scripts.
  • Pig Scripts for Data Validation.
  • Oozie workflow creation.
  • Writing MapReduce Program for Cassandra Connectivity.
  • Pig connectivity.
  • Cassandra Performance Tuning.
  • Cassandra Security Enabling (SSL and data encryption on rest).
  • Cassandra OpsCentre security and administration.
  • Took initiatives to learn about the new technology which were required to be implemented for the development, and provided knowledge transfer to the fellow team members to stay in trend.

Technologies Used: Hortonworks Distribution of hadoop IBM BigInsight 4.1, Datastax Distribution of Cassandra, MapReduce,Pig, Oozie, Git., Scala, SFTP, Linux Shell Scripting, CentOS 6.0.

Confidential

BigData Developer

Responsibilities:

  • Business and scope analysis for developing and managing data lake.
  • Overall requirement analysis
  • Setting up of HDP 6 node cluster.
  • Transferring data using Scoop.
  • Writing Pig Scripts to transform the data.
  • Refining data using Pig.
  • Creating Hive Structure.

Technologies Used: Hortonworks Hadoop, Tableau, Sqoop, Hive, Pig Linux Shell Scripting.

Confidential

BigData Developer

Responsibilities:

  • Developing the framework by writing Map Reduce program in java and also transforming the framework into Spark by writing the code in Scala for faster real time processing.
  • Responsible for processing the data with Pig by writing Pig Scripts.
  • Storing the data into Hive tables.
  • Automate the jobs by using Oozie scheduler.
  • Analyze the data to produce useful insights.
  • Verified the functionality of the application as per the requirements.

Technologies Used: Spark, Scala, Hive, Impala, Qlik Sense, Oozie, Linux Shell Scripting.

Confidential

BigData Developer

Responsibilities:

  • Developing the engine.
  • Architect design.
  • Writing MapReduce Code for Data Analysis.
  • Modelling And Creating Hive Tables for Structured Data Storage.
  • Creating various D3JS charts for Data Visualization.
  • Writing Scoop Scripts for data transfer from RDBMS to HDFS.
  • Benchmarking tools (Pig, Hive, R).

Technologies Used: Pig, Sqoop, Hive, R, RHadoop, D3JS.

Confidential

Developer

Responsibilities:

  • Worked in a team to carry out the change requests and bug fixes.
  • Responsibilities include functionality and content of the site.
  • Capture the issue that is reported that by the business team
  • Find the root cause of the issue
  • Find a solution/fix
  • Implement the fix and test it in the local environment
  • Pass it on to the testing team for further tests.
  • Once approved merge the fix in the production environment.

Technologies Used: Java, JavaScript, Web services.

Confidential

Developer

Responsibilities:

  • Requirement Gathering And Analysis.
  • Low Level Design And Data Structuring
  • Code development and Designing.
  • Getting client feedback and applying the same.
  • Implement the fix and test it in the local environment
  • Created online portal and was successfully deployed on the customer LAN network.

Technologies Used: Java, JavaScript, Web services, MySql, Eclipse, PHP.

Hire Now