PROFESSIONAL SUMMARY:

• Overall 8+ years of proactive IT experience in Analysis, Design, Development, Implementation,andTesting of software applications which includes an accomplished almost 4+ Years of experience in BigData, Development and Design of Java based enterprise applications.
• Leveraged strong skills in developing applications involving BigDatatechnologies like Hadoop, MapReduce, Yarn, Flume, Hive, Pig, Sqoop, HBase, PIvotal, Cloudera, MapR, Avro, Spark and Scala.
• Extensively worked on major components of HadoopEcosystem like
• HDFS, HBase, Hive, Sqoop, PIG,andMapreduce.
• Developed various scripts, numerous batch jobs to schedule various Hadoop programs.
• Experience in analyzing data using HiveQL, and custom MapReduce programs in Java.
• Hands on experience in importing and exporting data from different databases like Oracle, Mysql, into HDFS and Hive using Sqoop.
• Implemented Flume for collecting, aggregating and moving a largenumber of server logs and streaming data to HDFS.
• Hands on experience in spark, scala,andMarklogic.
• Extensively used MapReduceDesignPatterns to solve complex MapReduce programs.
• Developed Hive queries for data analysis to meet the business requirements.
• Experience in extending Hive and Pigcore functionality by writing custom UDFs like UDAFs and UDTFs.
• Experienced implementing Securitymechanism for HiveData.
• Extensively used ETL methodology for performing DataProfiling, DataMigration, Extraction, Transformation and loading using Talend/SSIS and designed data conversions from a wide variety of source systems.
• Extensively created mappings in Talend using tMap, tJoin, tReplicate, tParallelize, tJava, tJavarow, tDie, tAggregateRow, tWarn, tLogCatcher, tMysqlScd, tFilter, tGlobalmap etc.
• Experience with HiveQueriesPerformanceTuning.
• Experienced with improving datacleansing process using PigLatinoperations, transformations and joinoperations.
• Extensive knowledge ofNoSQL database like HBase.
• Experienced with performing CRUD operations using HBaseJavaClientAPI and RestAPI.
• Experience in designing both time driven and data driven automated workflows using Bedrock and Talend.
• Good knowledge in creating PL/SQLStoredProcedures, Creatingindexes, Packages, Functions, Triggers, Cursors with Oracle (9i, 10g, 11g), and MySQL server.
• Expert in designing and writing on - demand UNIXshellscripts.
• Extensively worked with ObjectOrientedAnalysis, Design (OOAD) and development of software using UML Methodology, good knowledge of J2EE design patterns and CoreJavaDesignPatterns.
• Excellent Java development skills using J2EEFrameworks like Struts, EJBs and WebServices.
• Proficient in development methodologies such as Scrum, Agile,andWaterfall.
• Passion to excel in any assignment and have good debugging and problem solving skills.
• Ability to work under high pressure and close deadlines.
• Excellent adaptability, ability to learn, good analytical and programmatic skills.

TECHNICAL SKILLS:

Hadoop/Big Data Technologies: HDFS, MapReduce, HBase, Hive, Pig, Impala, SQOOP, Flume, OOZIE, Spark, SparkQL, and Zookeeper, AWS, Cloudera, Hortonworks, Kafka, Avro, BigQuery.

Languages: Core Java, XML, HTML and HiveQL.

J2EE Technologies: Servlets, JSP, JMS, JSTL, AJAX, DOJO, JSON and Blaze DS.

Frameworks: Spring 2, Struts 2 and Hibernate 3.

Application & Web Services: WebSphere 6.0, JBoss 4.X and Tomcat 5.

Scripting Languages: Java Script, Angular JS, Pig Latin, Python 2.7and Scala.

Database (SQL/No SQL): Oracle 9i, SQL Server 2005, MySQL, HBase and Mongo DB 2.2

IDE: Eclipse and Edit plus.

PM Tools: MS MPP, Risk Management, ESA.

Other Tools: SVN, Apache Ant, Junit and Star UML, TOAD, Pl/SQL Developer, Perforce, JIRA, Bugzilla, Visual Source, QC, Agile Methodology.

EAI Tools: TIBCO 5.6.

Bug Tracking/ Ticketing: Mercury Quality Center and Service Now.

Operating System: Windows 98/2000, Linux /Unix and Mac.

PROFESSIONAL EXPERIENCE:

Confidential

Hadoop/Big Data Developer

Responsibilities:

• Translation of functional and technical requirements into detailed architecture and design.
• Developed automated scripts for all jobs starting from pulling the data from Mainframes to HDFS system.
• Developed and Designed ETL Applications and Automated using Oozie workflows and Shellscripts with error handling and mailing System.
• Implemented nine nodes CDH4Hadoop cluster on UbuntuLINUX.
• Implemented MapReduce programs by joining data sets from different sources using joins.
• Optimized mapreduces programs by configuring mapreduce configurationally parameters and implemented optimized joins.
• Implemented map reduce solutions like Top-K, summarizations, datapartitions using mapreduce design patterns.
• Implemented map reduce programs to handle different file formats like XML, Avro, sequencefiles and implemented compression techniques.
• Developed hivequeries according to business requirement.
• Designed/created HiveInternaltables, partitions to store structured data.
• Used HiveOptimized file formats like ORC formats and Parquet formats.
• Implemented HiveSerializes, desterilizes to handle Avrofiles, used Xpath expressions to handleXML files.
• Importing & exporting data from RDBMS through Sqoop.
• Designed Cassandradatamodeling to analyze near real time analysis.
• Configured Cassandraclustered, v-nodes, replicationstrategies, and configured data model using data stax community.
• Ensured NFS is configured for NameNode.
• Designed/Implemented time series data analysis using Cassandra file system.
• Implemented CRUD operation on top of Cassandra data using CQL and RestAPI.
• Implemented data import/export data from structured data using Sqoop import/export options.
• Implemented Sqoop saved jobs, incremental imports to import data.
• Used CompressionTechniques (snappy) with file formats to leverage the storage in HDFS.
• Used cloud era manager to perform cluster monitoring, debug mapreducejobs, handlejob submission on the cluster.
• Successfully migrated Legacy application to BigData application using Hive/Pig in Productionlevel.

Environment: Hadoop, HDFS, Map Reduce, Hive, Sqoop, OOzie, Cloudera, PIG, Java (JDK 1.6), Eclipse, MySQL and Ubuntu, Zookeeper.

Confidential, Boston, MA.

Hadoop/Big Data Developer

Responsibilities:

• Involved in all phases of development activities from requirements collection to production support.
• Detailed understanding of the current system and find out the different sources of data for EMR.
• Involved in a Cluster setup.
• Performed Batch processing of logs from various data sources using MapReduce
• Automated job cloudera submission ViaJenkinsscripts and Chef.
• Predictive analytics (which can monitor inventory levels and ensure product availability)
• Analysis of customers' purchasing behaviors in JMS.
• Response to value-added services based on clients' profiles and purchasing habits
• Worked on gathering and refining requirements, interviewing business users to understand and document data requirements including elements, entities,and relationships, in addition to visualization and report specifications.
• Defined UDFs using PIG and Hive in order to capture customer behavior.
• Design and implement map reduce jobs to support distributed processing using java, Hive, SparkSQL and ApachePig, Oozie.
• Integrated ApacheKafka for dataingestion.
• Create Scala, Hive external tables on the mapreduce output before partitioning, bucketing is applied on top of it.
• Providing Shell scripting pivotal graphs in order to show the trends
• Maintenance of data importing scripts using HBase, Hive and MapReduce jobs
• Developed and maintain several batch jobs to run automatically depending on business requirements
• Import and export data between the environments like MySQL, HDFSand Unit testing and Deploying for internal usage monitoring performance of solution.

Environment: Apache Hadoop, Cloudera, RHEL, Hive, HBase, PIG, HDFS, Java Map-Reduce, Core Java, Python, Maven, GIT, Jenkins, UNIX, MYSQL, Eclipse, Oozie, Sqoop, Flume and Cloudera Distribution, Oracle, Teradata,and MySql.

Confidential

Hadoop Developer

Responsibilities:

• Detailed Understanding on the existing build system, Tools related for information of various products and releases and test results information
• Designed and implemented mapreduce jobs to support distributed processing using java, Hive and ApachePig.
• Developed UDF's to provide custom hive and pig capabilities using SOAP/RESTful services.
• Built a mechanism for talend, automatically moving the existing proprietary binary format data files to HDFS using a service called Ingestion service.
• Implemented a prototype to integrate PDF documents into a web application using Github.
• Comprehensive knowledge and experience in process improvement, normalization/de-normalization, dataextraction, datacleansing, Scrumdatamanipulation
• Performed Scala, Data transformations in Scala, HIVE and used partitions, buckets for performance improvements.
• Written custom Input format and record reader classes for reading and processing the binary format in MapReduce.
• Written Custom writable classes for Hadoop serialization and De serialization of Time series tuples.
• Implemented Custom File loader for Pig so that we can query directly on the large Data files such as build logs
• Used Python for pattern matching in build logs to format errors and warnings
• Developed PigLatinscripts&Shellscript for validating the different query modes in Historian.
• Created Hive external tables on the Mapreduce output before partitioning bucketing is applied on top of it.
• Improved the Performance by Scala, tuning of HIVE and map reduce using talend, ActiveMQ,andJBoss.
• Developed Daily Testengine using Python for continuous tests.
• Used Shellscripting for Jenkins job automation with Talend.
• Building a custom calculation engine which can be programmed according to user needs.
• Ingestion of data into Hadoop using Shellscripting for Scrum, ElasticSqoop and apply data transformations and using Pig and HIVE.
• Handled the performance improvement changes to Pre-Ingestionservice which is responsible for generating the BigDataFormatbinaryfiles from an older version of Historian.
• Worked with support teams and resolved operational & performance issues
• Research, Scrum, evaluate and utilize new technologies/tools/frameworks around Hadoop eco system
• Prepared graphs from test results posted to MIA.

Environment: Apache Hadoop, Hive, Scala, PIG, HDFS, Cloudera, Java Map-Reduce, Core Java, Python, Maven, GIT, Jenkins, UNIX, MYSQL, Eclipse, Oozie, Sqoop, Flume, Oracle, My SQL,and CDH4.X.

Confidential, New Jersey.

Hadoop Developer

Responsibilities:

• Worked on analyzing Hadoopcluster using different BigDataanalytic tools including Hive, Pig and MapReduce.
• Installed and configured the Hadoopcluster using the Cloudera'sCDH distribution and monitored the cluster performance using the ClouderaManager.
• Monitored workload, job performance and capacity planning using ClouderaManager.
• Implemented schedulers on the Job tracker to share resources of the cluster for the MapReduce jobs given by cluster.
• Involved in Design, developHiveDatamodel, loading with data and writing JavaUDF for Hive.
• Handled importing and exporting data into HDFS by developing solutions, analyzed the data using MapReduce, Hive and produce summary results from Hadoop to downstream systems.
• Used Sqoop to import and export the data from HadoopDistributedFileSystem (HDFS) to RDBMS.
• Created Hive tables and loaded data from HDFS to hive tables as per the requirement.
• Established custom MapReduces programs to analyze data and used HQL queries for datacleansing.
• Created components like HiveUDFs for missing functionality in Hive to analyze and process large volumes of data extracted from No-SQL database Cassandra.
• Collecting and aggregating substantial amounts of log data using ApacheFlume and staging data in HDFS for further analysis.
• Developed workflow in Oozie to automate the tasks of loading the data into HDFS and pre-processing with Pig.
• Worked on optimization of MapReduce algorithm using combiners and partitions to deliver the best results and worked on Application performance optimization for an HDFScluster.
• Comprehensive knowledge and experience in process improvement, normalization/de-normalization, dataextraction, datacleansing, Scrumdatamanipulation.

Environment: Cloudera Distribution (CDH), HDFS, Pig, Hive, Map Reduce, Sqoop, Hbase, Impala, Java, SQL, Cassandra.

Confidential

Java Developer

Responsibilities:

• Involved in the analysis, design, and development, testing phases of SoftwareDevelopmentLifeCycle (SDLC).
• Analysis, design,and development of Application based on J2EE using Struts and Tiles, Spring 2.0 and Hibernate 3.0.
• Involved in interacting with the Business Analyst and Architect during the SprintPlanningSessions.
• Used XML Web Services for transferring data between different applications.
• Used Apache CXF web service stack for developing web services and SOAPUI and XML-SPY for testing web services.
• Used JAXB for binding XML to Java.
• Used SAX and DOM parsers to parse XML data.
• Hibernate was used for Object-Relationalmapping with Oracledatabase.
• Worked with SpringIOC for injecting the beans and reduced the coupling between the classes.
• Implemented SpringIOC (InversionofControl)/DI (DependencyInjection) for wiring the object dependencies across the application.
• Implemented spring transaction management for implementing transactions for the application.
• Implemented design patterns for Service Locator.
• Performed unit testing using JUnit3, EasyMock Testing Framework for performing Unit testing.
• Worked on PL/SQL stored procedures using PL/SQL Developer.
• Involved in Fixing the production Defects for the application.
• Used ANT as build-tool for building J2EE applications.

Environment: MapReduce, HDFS, HBase, HDP Horton, Sqoop, Data Processing Layer, HUE, AZURE, Erwin, MS Visio, Tableau, SQL, MongoDB, Oozie,UNIX, MySQL, RDBMS, Ambari, Solr Cloud, PL/SQL, TOAD, Java.

Confidential

Data Analyst

Responsibilities:

• Involved in design and implementation of server-side programming.
• Involved in gathering requirements, analyzed them and prepared high-level documents.
• Participated in all client meetings to understand the requirements.
• Actively involved in designing and data modelling using RationalRoseTool (UML)
• Involved in the design of the SPACE database.
• Designed and development of UserInterfaces, Menus using HTML, JSP, JSPCustomTag, JavaScript.
• Implemented User Interface using spring tiles framework.
• Tuxedo server, which provides case details, is fetched by the help of web services technology (i.e. binding, finding a service and use of XML message format etc.)
• Involved in integration system with BT's systems like GTC, CSS or through e Linkhub and IBMMQ series.
• Developed, Deployed and tested JSP's, Servlets in Weblogic.
• Used Eclipse as IDE tool and integrated WebLogic with Eclipse to deploy and develop the applications and JDBC to connect the database.

Environment: Struts Framework, Java 1.3, XML, Data Modelling, JDBC, SQL, Pl/SQL, JMS, Web Services, SOAP, Solaris 9, ANT tool, Toad, Eclipse.