SUMMARY:

• Around 5+ years of implementation experience in Hadoop Technologies.
• Worked in multiple Hadoop distributions like HORTONWORKS, AWS, Cloudera and MapR.
• Experience in end to end implementation of project like Data Lake.
• Experience with different file formats like ORC, Parquet, AVRO, JSON.
• Expert in data ingestion tools like Sqoop, Flume, Talend, Kafka, Spark Streaming.
• Experience in data cleansing scripts like Spark, MapReduce and Pig.
• Intensive experience in Hive, Impala and Tez.
• Exposure on NoSQL Db’s like HBase, Cassandra and Mongo DB.
• Implemented centralized search using Solr or Cloudera search.
• Ingested data from different sources like oracle, Teradata, Sql server.
• Experience in developing pipelines in spark using Scala and python.
• Developing streaming pipelines using Kafka and Storm.
• Experience in working with Data science teams and support teams.
• Orchestrated multiple Hadoop application jobs using Oozie.
• Experience in implementing optimization techniques in Hive and Spark.
• Experience in scheduling TWS jobs for processing millions of records using ITG.
• Experience in developing customized UDF’s in java to extend Hive and Pig Latin functionality
• Hands on experience in Sequence files, Combiners, Counters, Dynamic Partitions, Bucketing for best practice and performance improvement.

TECHNICAL SUMMARY

Java Space:

• Experience in developing spring Boot applications for transformations.
• Experience in connecting different end points using JDBC.
• Worked in implementing RESTFUL and SOAP projects.
• Experience in writing JUnit test cases and build tools like Maven.

Others:

• Experience in developing SQL and PL/SQL scripts.
• Exposure in cloud tools like AWS, Azure.
• Exposure on BI Tools like Tableau and Qlikview.
• Intensive skill in scripting using Bash, Python and Shell.
• Exposure in Devops technology like Chef, Puppet, Ansible, Jenkins
• Experience in software methodologies like Agile, Waterfall model.
• Experience in test management tools HP ALM
• Experience in database query tools like Aqua Data Studio, Advance Query Tool, TOAD and SQL Developer

PROFESSIONAL EXPERIENCE:

Confidential, Matawan, NJ

Java/J2EE Developer

Responsibilities:

• Involved in design and development phases of Software Development Life Cycle (SDLC)
• Involved in designing UML Use case diagrams, Class diagrams, and Sequence diagrams using Rational Rose
• Implemented Model View Controller (MVC) architecture using Jakarta Struts 1.3 frameworks at presentation tier
• Developed a Dojo based front end including forms and controls and programmed event handling
• Implemented SOA architecture with web services using JAX - RS (REST) and JAX-WS (SOAP)
• Developed various Enterprise Java Bean components to fulfill the business functionality.
• Validated all forms using Struts validation framework and implemented Tiles framework in the presentation layer
• Extensively used Hibernate in data access layer to access and update information in the database.
• Used Spring 2.0 Framework for Dependency injection and integrated it with the Struts Framework and Hibernate.
• Used JDBC to connect to backend databases, Oracle and SQL Server 2005.
• Deployed application on windows using IBM Web Sphere Application Server.
• Implemented SOA architecture with web services using Web Services like JAX-WS.
• Implemented Persistence layer using Hibernate to interact with Oracle 10g and SQL Server 2005 databases.
• Used ANT scripts to build the application and deployed on Web Sphere Application Server

Technologies used: Core Java, J2EE, Web Logic 9.2, Oracle 10g, SQL Server, JSP, STRUTS, JDK, JSF, JAX-RS (REST), JAX-WS (SOAP), JMS, Hibernate, JavaScript, HTML, CSS, IBM RAD 7.0,AJAX, JSTL, ANT1.7 build tool, Junit, Spring, Log4j, Web Services

Confidential, Cypress, CA

Sr.Big Data Analyst

Responsibilities:

• Developed MapReduce programs to parse the raw data, and create intermediate data which would be further used to be loaded into Hive portioned data.
• Involved in creating Hive ORC tables, loading the data into it and writing Hive queries to analyze the data.
• Involved in data ingestion into HDFS using Sqoop for full load and Flume for incremental load on variety of sources like web server, RDBMS and Data API’s.
• Performed multiple MapReduce jobs in PIG and Hive for data cleaning and pre - processing
• Used different file formats like Text files, Sequence Files, Avro, Optimized Row Columnar (ORC)
• Ingest real-time and near-real time (NRT) streaming data into HDFS using Flume
• Experience in custom aggregate functions using Spark SQL and performed interactive querying.
• Expertise in creating TWS Jobs and Jobstreams and automate them as per schedule
• Worked on Golden Gate replication tool to get data from various data sources into HDFS
• Worked on HBase for support enterprise production and loading data into HBASE using SQOOP.
• Collecting and aggregating large amounts of log data using Apache Flume and staging data in HDFS for further analysis.
• Exported the data from Avro files and indexed the documents in ORC file format.
• Responsible for created Technical Specification documents for the generated extracts
• Involved in performance tuning using Partitioning, bucketing of Hive tables
• Created UDFs to calculate the pending payment for the given customer data based on last day of every month and used in Hive Scripts.
• Involved in writing shell scripts to run the jobs in parallel and increase the performance
• Involved in running TWS jobs for processing millions of records using ITG.

Technologies used: Core Java, J2EE, Hadoop, HDFS, Flume, Hive, MapReduce, Sqoop, LINUX, Mapr, Big Data, Golden Gate, UNIX Shell Scripting, TWS,HP ALM 12,ITG

Confidential, Akron, OH

Sr.Big Data Analyst

Responsibilities:

• Understanding business needs, analyzing functional specifications and map those to develop and designing MapReduce programs and algorithms
• Designed and implemented MapReduce-based large-scale parallel relation-learning system.
• Customized Flume interceptors to encrypt and mask customer sensitive data as per requirement
• Recommendations using Item Based Collaborative Filtering in Apache Spark.
• Worked with NoSQL databases like Hbase in creating Hbase tables to load large sets of semi structured data coming from various sources.
• Performed importing data from various sources to the Cassandra cluster using Java APIs or Sqoop.
• Developed iterative algorithms using Spark Streaming in Scala for near real-time dashboards.
• Installed and configured Hadoop and Hadoop stack on a 40 node cluster.
• Involved in customizing the partitioner in MapReduce in order to root Key value pairs from Mapper to Reducers in XML format according to requirement.
• Configured Flume for efficiently collecting, aggregating and moving large amounts of log data.
• Involved in creating Hive tables, loading the data using it and in writing Hive queries to analyze the data.
• Involved in scheduling Oozie workflow engine to run multiple Hive and pig jobs
• Designed and built the Reporting Application, which uses the Spark SQL to fetch and generate reports on HBase table data.
• Extracted the needed data from the server into HDFS and Bulk Loaded the cleaned data into HBase.
• Used different file formats like Text files, Sequence Files, Avro, Record Columnar CRC, ORC
• Worked with cloud services like Amazon web services (AWS)
• Involved in ETL, Data Integration and Migration
• Responsible for creating Hive UDF’s that helped spot market trends.
• Optimizing Hadoop MapReduce code, Hive/Pig scripts for better scalability, reliability and performance
• Worked on creating indexes and working with Indexes using SOLR on Hadoop Distributed Platform
• Experience in storing the analyzed results back into the Cassandra cluster.
• Developed custom aggregate functions using Spark SQL and performed interactive querying

Technologies used: HDFS, MapReduce, Cloudera, Hbase, Hive, Pig, Solr, Sqoop, Spark, Cassandra, Scala, Flume, Oozie, Zookeeper, Maven, Linux, UNIX Shell Scripting

Confidential, Fremont, CA

Sr.Big Data Analyst

Responsibilities:

• Developed MapReduce programs to parse and filter the raw data store the refined data in partitioned tables in the Greenplum.
• Created Hive queries that helped market analysts spot emerging trends by comparing fresh data with Greenplum reference tables and historical metrics
• Developed a data pipeline using Kafka and Storm to store data into HDFS.
• Managing and scheduling Jobs on a Hadoop cluster using Oozie and cron jobs.
• Involved in running MapReduce jobs for processing millions of records.
• Responsible for creating Hive tables, loading the structured data resulted from MapReduce jobs into the tables and writing hive queries to further analyze the logs to identify issues and behavioral patterns.
• Experienced in migrating Hive QL into Impala to minimize query response time.
• Created UDFs to calculate the pending payment for the given Residential or Small Business customer, and used in Pig and Hive Scripts.
• Wrote test cases in MRunit for unit testing of Mapreduce Programs
• Used Elastic Search & MongoDB for storing and querying the offers and non-offers data.
• Deployed and built the application using Maven.
• Maintain Hadoop, Hadoop ecosystems, third party software, and database(s) with updates/upgrades, performance tuning and monitoring using Ambari
• Experience in managing and reviewing Hadoop log files
• Extensively worked on User Interface for few modules using JSPs, JavaScript and Ajax
• Used Python scripting for large scale text processing utilities
• Experienced in moving data from Hive tables into Cassandra for real time analytics on Hive tables
• Responsible for data modeling in MongoDB in order to load data which is coming as structured as well as unstructured data
• Implemented CRUD operations involving lists, sets and maps in DataStax Cassandra.
• Obtained good experience with NOSQL database Cassandra.
• Used Cassandra CQL with Java API's to retrieve data from Cassandra tables.
• Participated in development/implementation of Cloudera Hadoop environment.

Technologies used: HDFS, MapReduce, Hive, Pig, Cloudera, Impala, Oozie, Greenplum, MongoDB, Cassandra, Kafka, Storm, Maven, Python, CloudManager, NagiOS, Ambari, JDK, J2EE, Ajax, Struts, JSP, Servlets, Elastic Search, WebSphere, JavaScript, MRunit

Confidential, Pasadena, CA

Jr.Big Data Developer

Responsibilities:

• Worked on analyzing Hadoop cluster using different big data analytic tools including Pig, Hive, and MapReduce.
• Involved in loading data from LINUX file system to HDFS.
• Importing and exporting data into HDFS and Hive using Sqoop.
• Experience working on processing unstructured data using Pig and Hive.
• Experienced in running Hadoop streaming jobs to process terabytes of xml format data.
• Involved in scheduling Oozie workflow engine to run multiple Hive and pig jobs.
• Developed Pig Latin scripts to extract data from the web server output files to load into HDFS.
• Extensively used Pig for data cleansing.
• Implemented SQL, PL/SQL Stored Procedures.
• Worked on debugging, performance tuning of Hive & Pig Jobs.
• Implemented test scripts to support test driven development and continuous integration.
• Worked on tuning the performance Pig queries.
• Created and maintained Technical documentation for launching HADOOP Clusters and for executing Hive queries and Pig Scripts.
• Actively involved in code review and bug fixing for improving the performance.

Technologies used: Hadoop, HDFS, Pig, Hive, MapReduce, Sqoop, LINUX, Cloudera, Big Data, Java APIs, Java collection, SQL.

Confidential

Java Developer

Responsibilities:

• Extensively involved in the design and development of JSP screens to suit specific modules.
• Converted the application’s console printing of process information to proper logging technology using log4j.
• Developed the business components (in core Java) used in the JSP screens.
• Involved in the implementation of logical and physical database design by creating suitable tables, views and triggers.
• Developed related procedures and functions used by JDBC calls in the above components.
• Extensively involved in performance tuning of Oracle queries.
• Created components to extract application messages stored in xml files.
• Executed UNIX shell scripts for command line administrative access to oracle database and for scheduling backup jobs.
• Created war files and deployed in web server.
• Performed source and version control using VSS.
• Involved in maintenance support.

Technologies used: JDK, HTML, JavaScript, XML, JSP, Servlets, JDBC, Oracle 9i, Eclipse, Toad, UNIX Shell Scripting, MS Visual SourceSafe, Windows 2000.