We provide IT Staff Augmentation Services!

Bigdata Developer / Production Support Resume

4.00 Rating

Halifax, NS

SUMMARY

  • 6+ years of extensive experience and knowledge in IT that includes 2+ experience in Big data ecosystem related technologies.
  • Experience in Development, Implementation and testing in various application architectures, including HealthCare, Insurance, Banking, and Telecom.
  • Good Knowledge and understanding of Hadoop Architecture and various components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node and MapReduce programming paradigm.
  • Experience installing, configuring and testing Hadoop Ecosystem components like Hadoop MapReduce, HDFS, Pig, Hive, HBase, Spark, Storm, Sqoop, Kafka, Flume, Oozie, Zookeeper and Ambari.
  • Good Exposure PIG Scripts and Hive queries.
  • Good knowledge on Hadoop cluster architecture and monitoring teh cluster.
  • In - depth understanding of Object oriented programming, Data Structures and Algorithms.
  • Experience in managing and reviewing Hadoop log files, have good understanding of NoSQL databases MongoDB and Cassandra.
  • Experience in importing and exporting data using Sqoop from Relational Database Systems to HDFS and vice-versa.
  • Collecting and aggregating large amount of Log data using Apache Flume and storing data in HDFS for further analysis.
  • Job/workflow scheduling and monitoring tools like Oozie and Zookeeper.
  • Experience in Object Oriented Analysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and Core Java design patterns.
  • Experience using different file formats like Parquet, ProtoBuff (Protocol Buffer), and Apache Avro, Sequence file, JSON, XML and Flat file.
  • Configured and Maintained different topologies in Storm cluster and deployed them on regular basis.
  • Experience on Hadoop clusters using major Hadoop distributions like Cloudera (CDH), Hortonworks (HDP) and MapR.
  • Expertise in developing and coding web applications using Core Java, J2EE technologies - JSP, Servlets and JDBC.
  • Hands on experience in creating various database objects like tables, functions, stored procedures and triggers using SQL, PL/SQL.
  • Experience in application development using Java, RDBMS, and Linux shell scripting.
  • Experience in all phase of SDLC like Requirement Analysis, Implementation and Maintenance, extensive experience with Agile and SCRUM.
  • Experience working on Kafka cluster. Also have experience in working on Spark and Spark streaming.
  • Experience in creating event processing data pipelines using Kafka and Spark Streaming.
  • Experience in using version controls like Git, tortoise SVN and build tools like Maven.

TECHNICAL SKILLS

Hadoop/ Big Data: HDFS, MapReduce, Sqoop, Kafka, Zookeeper, Ambari, Oozie

Hadoop/ Data Access: Pig, Hive, Solr, Impala, Spark, HBase, Storm

Programming Languages: Java, Python, Shell Scripting, Scala, SQL, C, C++

Web Servers: Tomcat, and Apache

Databases: Oracle 11g, SQL Server, MySQL, NoSQL (Cassandra)

IDE: Sublime Text, IntelliJ, WebStorm, Eclipse, Notepad++

Web Technologies: HTML, XML, JSON, JavaScript

Operating Systems: Windows, Linux, and Mac

PROFESSIONAL EXPERIENCE

Confidential

Bigdata Developer / Production Support

Responsibilities:

  • Involved in cluster planning, deployment and monitoring Hadoop cluster with Hortonworks on Redhat Linux, managing with Ambari.
  • Used Hive to find correlations between customer’s browser logs in various features of site, analyzed them to build risk profile for such sites.
  • Analyzed large data sets by running Hive queries and Pig scripts.
  • Involved in creating Hive tables, loading and analyzing data using hive queries.
  • Developed simple to complex MapReduce Jobs using Hive and Pig.
  • Involved in running Hadoop jobs for processing millions of records of text data.
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
  • Responsible for performing extensive data summarization using Hive.
  • Worked on RC files, Sequence files, Map side joins, bucketing, partitioning for hive performance enhancement and storage improvement.
  • Extracted records from Oracle DB through Sqoop and placed in HDFS and processed into Impala DB.
  • Experienced in running Hadoop streaming jobs to process terabytes of text, xml data.
  • Involved in submitting and tracking Map Reduce jobs using Job Tracker.
  • Knowledge on handling Hive queries using Spark SQL that integrate Spark environment.
  • Implemented business logic by writing Pig UDFs in Java and used various UDFs from Piggybanks and other sources.
  • Responsible in exporting analyzed data to relational databases using Sqoop.
  • Implemented Daily Oozie coordination jobs that automate parallel tasks of loading teh data into HDFS and pre-processing with Pig using Oozie co-coordinator jobs.
  • Responsible fortuning Hive and Pig scriptsto improve performance.
  • Implemented unit tests with MRUnit and PIGUnit.
  • Documented teh technical details of Hadoop cluster management and daily batch pipeline, which includes several jobs of MapReduce, Pig, Hive, Sqoop, Oozie and other scripts.

Environment: Hadoop, HDFS, Pig, Hive, MapReduce, Sqoop, Spark, Oozie, Linux, Grunt, Git/Stash, Jenkins, Java, Oracle SQL DB.

Confidential

Etl/Hadoop Developer

Responsibilities:

  • Loaded teh customer profiles data, customer information, billing information onto HDFS using Sqoop and Flume.
  • Worked on analyzing Hadoop cluster and different big data analytic tools including Pig, Hive, Hbase and Sqoop.
  • Involved in loading data from LINUX file system to HDFS.
  • Exported teh analyzed data to teh relational databases using Sqoop for visualization and to generate reports for teh BI team.
  • Responsible for importing data into Hive and HDFS using Sqoop.
  • Implemented test scripts to support test driven development and continuous integration.
  • Installed and configured Hadoop, MapReduce, HDFS (Hadoop Distributed File System).
  • Installed and configured Hadoop MapReduce, HDFS, Developed multiple MapReduce jobs in java for data cleansing and preprocessing.
  • Created Pig Latin scripts to sort, group, join and filter teh enterprise wise to get transformed data sets.
  • Involved in creating Hive tables, loading with data and writing hive queries that will run internally in MapReduce way.
  • Worked on tuning teh performance Pig scripts.
  • Mentored analyst and test team for writing Hive Queries.
  • Installed Oozie workflow engine to run multiple Mapreduce jobs.
  • Loaded bulk amounts of data to Hbase using MapReduce.
  • Worked with application teams to install operating system, Hadoop updates, patches, version upgrades as required.
  • Created pull request on Git repository for manager approval before merging teh code to teh master branch.
  • Worked on aggressive planning and deadlines for completion of teh project.

Environment: Hadoop, HDFS, MapReduce, Hive, Pig, Sqoop, Storm, Linux, Java, Oozie, HBase.

Confidential, Halifax, NS

Consultant

Responsibilities:

  • Participated in all phases including Requirement Analysis; Client Interaction; Design, Coding, Testing and Documentation.
  • Involved in teh continuous enhancements and fixing of production problems.
  • Design and development of UNIX Shell Scripts to handle pre-and post-session processes.
  • Developed and scheduled various pre-and post-sessions commands and workflows for all mappings to load data from source files to target tables.
  • Analyzed existing system and developed business documentation on changes required.
  • Used Informatica Designer to Extract & Transform teh data from various source systems by incorporating various business rules. Also used different transformations, sessions and command tasks.
  • Created mappings using different transformations like Aggregator, Expression, Stored Procedure, Filter, Joiner, Lookup, Router, SQL transform and Update Strategy.
  • Worked with dynamic parameter file, mapping variable, Mapping parameters and variable functions like Setvariable, Countvariable, Setminvariable and Setmaxvariable.
  • Tuned performance of Informatica session for large data files by increasing block size, data cache size, sequence buffer length and target based commit interval.
  • Deployed reusable transformation objects such as mapplets to avoid duplication of metadata, reducing teh development time.
  • Involved in business analysis and technical design sessions with business and technical members to develop requirements document, and ETL specifications.
  • Provided support for testing team, UAT and production issues.

Environment: Informatica Power Center 9.0, Informatica Power Connect, Cognos Flat files, XML Files, Oracle 10g/9i, MS SQL Server 2000, Shell Programming, SQL * Loader, Toad, Excel and Unix scripting, Sun Solaris, Windows XP.

Confidential

Application Developer

Responsibilities:

  • Designed web application user interface, interaction scenarios and navigation based on analysts, interpretations of requirement and use cases.
  • Closely worked with business system analyst to understand teh requirements to ensure that right set of UI modules been built.
  • Wrote application level code to perform client-side validation using jQuery and JavaScript.
  • Involved in creating HTML 5 templates.
  • Designed teh front-end applications, user interactive (UI) web pages using web technologies like HTML5, XHTML, and CSS3.
  • Defined new validations through AngularJS for teh form field validation implemented through HTML5.
  • Using advanced level of jQuery, AJAX, JavaScript, CSS and pure CSS.
  • Maintain teh code base by frequent updates to revision control using SVN.
  • Worked on CSS Background, CSS Positioning, CSS Text, CSS Border, Pseudo classes, Pseudo elements etc.
  • Structured teh JavaScript code using jQuery.
  • Performed unit testing and requested for code review.
  • Identifying and fixing teh browser/performance related issues in teh application.
  • Production deployment and validation support.
  • Fixed teh post production issues in teh application.

Environment: HTML5, CSS3, JavaScript, AJAX, jQuery, XML, SQL, Python, Bootstrap, Tortoise SVN, Eclipse, EPIC Hyperspace.

Confidential

Programmer Analyst

Responsibilities:

  • Developed artifacts including high-level object and data models, interaction diagrams and codes teh solution.
  • Implemented java code for all layers of teh application using Struts2 and Hibernate Framework.
  • Used Struts2 for developing business logic and presentation layer.
  • Developed JSP's for user interface using Struts2 Tag Libraries and Core Tag Libraries.
  • Used Java Script and its libraries like jQuery with JSP for User interface.
  • Used JSON with Ajax for Data Interchange.
  • Wrote JUnit for Unit Testing.
  • Good programming experience in SQL Plus and PL/SQL. Wrote Stored Procedures, Functions, Cursors, Indexes, Sequences and Views.
  • Good experience with build tool Ant for deployment to teh server and creating different views for releases.
  • Involved in Integration testing, System Testing, Bug fixing and Maintenance.
  • Hands of work with SVN version controlling system.
  • Logger implementation for debugs, info and error messages.
  • Responsible for Interfacing with teh business and give them status updates about teh progress of teh enhancements.
  • Responsible for prioritizing defects with teh business users and ensuring that they are fixed in teh promised time.

Environment: Java, Struts, Hibernate, JSP, Tiles, HTML, Java script, JUnit, XML, XSL/XSLT, ANT, SVN, Oracle, Tomcat, Eclipse, Weblogic, Toad, Unix, Windows XP.

We'd love your feedback!