- Around 4 years of extensive and diverse experience in Big Data a nalytics using Hadoop Map - Reduce/YARN, HDFS, Hive, HBase, Flume, Sqoop, Zookeeper, Pig, and Oozie.
- Experience in writing test cases and testing map-reduce application.
- Designed and implemented image and video processing with Hadoop cluster.
- 3+ years Extensive experience in developing Enterprise applications using Java and various J2EE technologies including JSP, Servlets, RMI, Enterprise Java Beans3.0 (Entity Beans, Session Beans, Message Driven Beans), JSP Custom Tags Libraries, JavaBeans, JDBC, Networking, Threads, JNDI and JMS.
- Experience in developing Web Applications with various Open Source frameworks: Spring, Jakarta Struts Framework 1.x, Hibernate 2.0/3.0 ORM, Struts Validation and Tiles Frameworks .
- Experience working with different databases, such as Oracle, SQL Server, MySQL and writing stored procedures, functions, joins, and triggers for different Data Models.
- Expertise in implementing Service Oriented Architectures (SOA) with XML based Web Services (SOAP/UDDI/WSDL/REST).
- Well versed with J2EE Design Patterns like Factory, Front-Controller, Session Façade, Service-Locator, Data Access Object (DAO, Singleton, MVC (Struts), Business-Delegate.
- Programming experience in UNIX Shell Script .
- Experience in developing applications using IBM WSAD, RAD6, NetBeans and Eclipse IDE and deploying those applications on Web sphere Application Server, Web Logic and Apache Tomcat with J2EE.
- Experience in Unit testing, Integration testing and generating Test cases for web/Java core/mobile applications using Junit, Selenium and Selendroid (android applications).
- Experience with Behavior driven development and test driven development methodologies using JBehave and Mockito.
- Good working knowledge of Ant project build/test/deployment, Log4j for error logging and Debugging, and XMLSpy for XML validation.
Big Data Technologies: Hadoop, HDFS, Hive, MapReduce, Pig, Sqoop, Flume, Zookeeper, Oozie, Hadoop distribution of Cloudera CHD4/CDH5, and HBase
ProgrammingLanguages: C, C++,Objective-C and Java (5, 6, 7, 8)
Databases/RDBMS: Mongodb, MySQL, PL-SQL, MS-SQL Server 2005/2008, Oracle 9i/10g/11g
ETL Tools: InformaticsPower center 9.6.1
OperatingSystems: Linux, Windows XP/7/8
Software LifeCycles: SDLC, Waterfall and Agile models
Office Tools: MS-Office, MS-Project and Risk Analysis tools, Visio
Utilities/Tools: Eclipse, Tomcat, NetBeans, JUnit, SQL, SVN, Log4j, Tiles, SOAP UI, ANT, Maven, QTP Automation and MR-Unit
Cloud Platforms: Amazon EC2
Confidential, Fremont, CA
- Developed on-the-fly decryption support for Hive, Pig and custom MapReduce use cases using hadoop java API.
- Developed batch decrypter module using MapReduce programming.
- Developed automatic hadoop output file compression feature based on input data compression format for masking/encryption/batch-decryption modules.
- Developed MapReduce layers to support Sequence & Avro formatted input data for Discovery module.
- Developed MapReduce layer to support HBase table input data for Discovery module.
- Fixed bugs and made enhancements both in Java code base( MapReduce, HDFS file system API etc) and in Tomcat deployment procedures to support CDH4 and Hortonworks clusters.
- Developed MapReduce layer for structured masking of sequence input files. The module can simultaneously read sequence files with different key/value types, perform masking on value objects using java reflection and write out the modified output sequence files in the original format.
- Developed MapReduce layer for structured masking of Avro input files. The module can perform masking on appropriate fields of the generic Avro records and write out the modified output Avro files with the original schema.
Salesforce Developer/ Administrator
Role and Responsibilities:
- Worked with end users and business analysts for requirements gathering, analysis and provided design inputs.
- Interacted with various business user groups for gathering the requirements for Salesforce implementation and documented the Business and Software Requirements.
- Worked closely with sales team and business analysts and performed detailed analysis of business and technical requirements and designed the solution by customizing various standard objects of SalesForce.com (SFDC).
- Implemented the requirements on Salesforce.com platform and Force.com IDE Plug-in using Eclipse.
- Implemented web based case management automation - Web to Case (on Case Object) to track and solve customer's issues.
- Created Custom objects to build new application functionality in the Salesforce.com.
- Responsible for understanding the data migrating requirements and analyze data to be loaded from legacy systems to Salesforce.com.
- Created new user accounts and configure Salesforce.com to fit security needs at the user and organization levels.
- Developed Apex Classes & Triggers to implement the business logic as per the requirements.
- Involved in data mapping and migration of data from legacy systems to Salesforce.com Objects and fields.
- Worked on various salesforce.com standard objects like Accounts, Contacts, Cases, Opportunities, Products, Opportunity Line Items, Leads, Campaigns, Reports and dashboards.
- Designed, and deployed the Custom objects, Custom tabs, Entity-Relationship data model, validation rules, Workflow Rules, Page layouts, Components, Visual Force Pages to suit to the needs of the application.
- Customized page layouts for Opportunity, Contacts, and Accounts depending upon user roles, and groups.
- Defined lookup and master-detail relationships on the objects and created junction objects to establish connectivity among objects.
- Performed Unit, Integration and Regression Testing.
Environment: Saleforce.com platform, Apex Language, Visual Force Pages, HTML, Java Script, Workflow & Approvals, Reports, Custom Objects, Email Services, Eclipse IDE, Windows XP, COBOL, CICS, JCL, VSAM.
Confidential, Houston, TX
- Developing parser and loader map reduce application to retrieve data from HDFS and store to HBase and Hive.
- Importing the data from the MySQL into the HDFS using Sqoop .
- Importing the unstructured data into the HDFS using Flume .
- Used Oozie to orchestrate the map reduce jobs that extract the data on a timely manner.
- Written Map Reduce java programs to analyze the log data for large-scale data sets.
- Involved in using HBase Java API on Java application.
- Automated all the jobs for extracting the data from different Data Sources like MySQL to pushing the result set data to Hadoop Distributed File System.
- Customize parser loader application of Data migration to HBase.
- Developed Pig Latin scripts to extract the data from the output files to load into HDFS.
- Developed custom UDFS and implemented Pig scripts .
- Implemented MapReduce jobs using Java API and PIG Latin as well HIVEQL
- Participated in the setup and deployment of Hadoop cluster
- Hands on design and development of an application using Hive (UDF).
- Responsible for writing Hive Queries for analyzing data in Hive warehouse using Hive Query Language (HQL).
- Provide support data analysts in running Pig and Hive queries.
- Involved in HiveQL.
- Involved in Pig Latin.
- Importing and exporting Data from MySQL/Oracle to HiveQL Using SQOOP.
- Importing and exporting Data from MySQL/Oracle to HDFS.
- Configured HA cluster for both Manual failover and Automatic failover.
- Designed and built many applications to deal with vast amounts of data flowing through multiple Hadoop clusters, using Pig Latin and Java-based map-reduce.
- Specifying the cluster size, allocating Resource pool, Distribution of Hadoop by writing the specification texts in JSON File format.
- Responsible for defining the data flow within Hadoop eco system and direct the team in implement them.
- Exported the result set from Hive to MySQL using Shell scripts.
- Developed HIVE queries for the analysts.
Environment: Apache Hadoop, Hive, Hue Tool, Zookeeper, Map Reduce, Sqoop, crunch API, Pig 0.10 and 0.11, HCatalog, Unix, Java, JSP, Eclipse, Maven, SQL, HTML, XML, Oracle, SQL Server, MYSQL
Confidential, Philadelphia, PA
- Work closely with various levels of individuals to coordinate and prioritize multiple projects throughout SDLC.
- Worked in the BI team in the area of Big Data Hadoop cluster implementation and data integration in developing large-scale system software.
- Configured Hadoop cluster with NameNode and slaves and formatted HDFS.
- Worked extensively in creating Map Reduce jobs to power data for search and aggregation.
- Designed a data warehouse using Hive.
- Worked extensively with Sqoop for importing and exporting the data from HDFS to Relational Database systems/mainframe and vice-versa.
- Extensively used Pig for data cleansing.
- Created partitioned tables in Hive.
- Implementing/settingAutosys Jobs.
- Each node used Ubuntu and Hadoop 0.20.
- Effective data storage of 40-node cluster => 16 TB.
- Implemented test scripts to support test driven development and continuous integration.
- Worked on tuning the performance Pig queries.
Environment: Hadoop, MapReduce, HDFS, Pig, Hive, Java (jdk1.7), Flat files, Oracle 11g/10g, PL/SQL, SQL*PLUS, Windows NT, Sqoop.
Confidential, Champaign, IL
BIG DATA ANALYST / Java developer
- Installed and configured Apache Hadoop to test the maintenance of log files in Hadoop cluster.
- Installed and configured Hive, Pig, Sqoop, and Oozie on the Hadoop cluster.
- Installed Oozie Workflow engine to run multiple Hive and Pig Jobs.
- Developed multiple MapReduce jobs in Java for data cleansing and preprocessing.
- Developed Simple to complex Map/Reduce Jobs using Hive and Pig.
- Involved in loading data from UNIX file system to HDFS.
- Evaluated business requirements and prepared detailed specifications that follow project guidelines required to develop written programs.
- Analyzed large amounts of data sets to determine optimal way to aggregate and report on it.
- Provided quick response to ad hoc internal and external client requests for data and experienced in creating ad hoc reports.
- Responsible for building scalable distributed data solutions using Hadoop.
- Migration of ETL processes from Oracle to Hive to test the easy data manipulation.
- Performed optimization on Pig scripts and Hive queries increase efficiency and add new features to existing code.
- Stored and retrieved data from data-warehouses using Amazon Redshift.
- Developed PIG Latin scripts for the analysis of semi structured data.
- Used Hive and created Hive tables and involved in data loading and writing Hive UDFs.
- Used Sqoop to import data into HDFS and Hive from other data systems.
- Installed Oozie workflow engine to run multiple Hive.
- Generated aggregations and groups and visualizations using Tableau.
- Continuous monitoring and managing the Hadoop cluster using Cloudera Manager .
- Conducted some unit testing for the development team within the sandbox environment.
- Developed Hive queries to process the data for visualizing and reporting.
Environment: Apache Hadoop, Cloudera Manager, CDH2, CDH3 CentOS, Java, MapReduce, Apache Hama, Eclipse Indigo, Pig, Hive, Sqoop, Oozie and SQL, Struts, Junit.
- Project overview and component Assembly inspect product output and QA whole procedure according to the staging documents guideline.
- Software loading, imaging, Ghost walk, set IP/Mac address by using UNIX, Lynx, Window NT Servers, Window XP. used UNIX shells:· The Bourne Shell (sh),· The C Shell (csh or tsch) · The Bourne Again Shell (bash),· The Korn Shell (ksh)
- Programming for scanner, printer, adjusted jumper for customer display, printer, scanner, set right time zone, BIOS, booth Sequence, PXE set up, IP address, Password, Security code and printer setup.
- JMB Modem Configuration & Installation, Antenna Installation, Caster Check, Smart Tag Removal, MSR Check, Media Manager Setup, Media Loading, Software Configuration, Image Personalization, Graphics panel added to monitor, Testing soft Stop, ID Check, Rental Testing, Three Way final Check,
- Upgrading Memory, network and system peripheral devices. Perform 2 & 4 point Calibration for Kiosk Peripherals and other test screens.
- Application and Peripherals Testing for Cash register, Scanner, Customer display and peripherals devises to fulfill the staging standard requirements,