Hadoop Lead/architect Resume
PROFESSIONAL SUMMARY:
- Over 10+ years of IT experience including Hadoop System administration, working with Confidential Solutions, continuing as Hadoop Lead/Architectin .
- Working on Cloudera Hadoop (CDH 5), where playing the role of both Hadoop Admin and Developer. Have expertise in Hive, Impala, Hue, Pig, MapReduce, HBASE, Oozie, Sqoop, Sentryand cluster administration through Cloudera Manager, Cloudera Navigator. Basic knowledge of AWS, Apache Zeepline, DMX - h ETL tool and Spark.
- Great experience on Cluster Administration (Setup, Upgrade, Patch installation, Security Management) with Cloudera
- A Cloudera certified Hadoop developer (CCDH) and Zend PHP 5 Certified Engineer (ZCE)
- Hands on experience in server side scripting likeJava, PHP, DBMS packages like MySQL, Oracle,BigData technologies like Hadoop, Web technologies like AJAX, JSON, Javascript, JQuery, RSS, XHTML, CSSand operating systems like Linux, Windows.
- Around 10+years of total industry experiencewith 2.5+ years in BigData Hadoop, 4-6 years in web application development (CMS, E-commerce, Social networking and digital marketing) and 2-3 years in CRM development and consulting.
- Proficiency in managing widely-divergent, parallel projects from inception, requirement specs, planning, designing, configuration management & documentation to roll out and post production support within agreed cost / timelines.
- Expertise in developing & effectuating project management plans and risk mitigation plans for ongoing engagements. Solution architecting the applications
- Good exposure to SDLC process and MVC framework design pattern with agile methodology of working
- A Team player, who can manage middle to big size team with modular design pattern and can deliver a quality product in time
- A fast learner and adaptable to all kinds of new technologies. Hard working and ability to take responsibilities with comprehensive problem solving ability.
- Designing and Architecting Big Data solutions based on Hadoop
- Managing D&B BigData platform and leading team for deliveries around data availability, data quality and accessibility through analytic and reporting tools
- Supporting multiple projects and enhancing them with different features
- Great experience/expertise in Cluster management, while setting up new clusters for D&B initiatives and performing upgrades on existing clusters when needed, with regular operations related to Sentry security management, data node addition/removal etc.
- Using Hadoop ETL components like Sqoop for data extraction from RDBMS systems like Oracle, MySQL and load into HDFS, Hive
- Using Pig and MapReduce programs for data transformation of both structure and semi structured (JSON, XML) data
- Creating and Monitoring Oozie workflows for scheduled jobs and performing metadata management through MySQL/Hive
- Writing Hive and Impala queries for data validation and analytics
- Gathering new data source upload requirement, suggesting format and standardizations to be followed and load them to Hadoop environment
- Using Solr for search and Cloudera Navigator for Audit related activities
- Doing POC activities on new technology components like Spark, Drools, DMX-h ETL tool, Apache Zeepline etc. for system enhancement
- Supporting analytics & reporting tools to connect and extract data from Hadoop
PERSONAL SKILLS:
- Comprehensive problem solving abilities
- Adaptive to new technologies and willingness to learn
- Team Player
- Leadership quality
- Ability of taking the ownership/responsibility
- A Smart worker
TECHNICAL PROFICIENCY:
BigData Technologies: HDFS, Hive, Impala, Pig, Hue, Oozie, MapReduce, Sqoop, Sentry, HBase, Cloudera Manager, Cloudera Navigator etc, Basic knowledge on AWS, Apache Zeepline, DMX-h ETL tool, Apache Ambari and Spark.
Programming Languages: Java, PHP, JS
Databases Packages: MySQL, Oracle
Frameworks/Web Technologies: Netbiscuits mobile platform, Codeigniter, Zend, Smarty, JSON, AJAX, JQuery, RSS, XHTML, CSS
CRMTools/Products: Sugar CRM, VtigerCRM, Suite CRM
CMS Tools/Products: Autonomy/InterwovenTeamsite, Wordpress
IDE/Tools: Eclipse, Aptana Studio, Zend Studio, Dreamweaver, Tortoise SVN, Git Stash
Domain Expertise: BigData DW, CRM, CMS, Digital Marketing
Operating Systems: Unix, Linux, Windows
PROJECT ANNEXURE:
Confidential
Hadoop Lead/ArchitectResponsibilities:
- Setting up and managing the CDH 5 Hadoop Cluster
- Building MapReduce, Oozie based automated jobs to load multiple data sources into HDFS, after validation and cleansing
- Setting up Oozie workflow to load incremental data on monthly, quarterly basis
- Creating Hive schema and tables for users, to give easy access to HDFS structured data
- Using Pig scripts to transform semi structured data into structured format
- Using Sqoop tool to extract data from RDBMS and load into Hive
- Managing ACL through Hadoop Sentry component
- Maintaining file meta data information into MySQL for online reporting
- Archive setup for both raw and processed HDFS data
- Enhancing the system by doing research on new Hadoop components like Flume, Spark etc.
Technology: CDH 5.5, HDFS, Hue, Hive, Pig, Oozie, MapReduce, Sqoop, HBase
Confidential
Hadoop Lead/Architect
Responsibilities:
- Setting up process and flow to receive files through secured process and get it available for the Hadoop cluster
- Using Pig scripts to transform some XML, JSON format data
- Building MapReduce, Oozie based automated jobs to parse, transform data and load them in staging tables
- Perform data validation and load into production schema and manually fix the rejected records
- Restricting the access of Hive schema through Hadoop Sentry
- Fulfillment process where we extract required data elements and push to external FTP
- Setting up Oozie workflow to load incremental data on daily basis to day partitions of Hive table
- Building QA automation tools, to validate data in HBase
Technology: CDH 5.2, HDFS, Hue, Impala, Hive, Pig, Oozie, MapReduce, Sqoop, HBase
Confidential
Hadoop Lead
Responsibilities:
- Amazon API integration
- Integration with Google CI and merge it with Amazon response and CI response
- Building web service which receives input for brands and SKU and provide retailer list as part of response
- Building store locator feature with Google map integration
- Support for XML/HTML response format
- Scalability and performance measurement to support all P& Confidential brands and locales
- Localization support
Technology: Java, MS SQL Server
Confidential
Project Manager
Responsibilities:
- Integration with Teamsite CMS with both open deploy and data deploy support
- RWD support to enable desktop, tablet and mobile device view
- Product selector tool
- BazaarVoice integration for ratings & review feature
- Google Channel Intelligence integration for buy it now solution
- Social network API integration to show latest feed from Facebook & Twitter
- GSA integration for search feature and GA integration for event tracking
- Registration, Login features
- Localization support
- SSIS package for scheduler job
Technology: .NET, MVC Entity Framework, SQL Server, Interwoven Teamsite CMS, CSS 3, JQuery
Confidential
Project Lead
Responsibilities:
- Integration with Teamsite CMS with both open deploy support
- Netbiscuits BML integration for view pages
- Product category, sub-category feature with product selector tool
- BazaarVoice integration for ratings & review feature
- Google Channel Intelligence integration for buy it now solution
- Shopping Cart integration
- GSA integration for search feature and GA integration for event tracking
- Localization support
- SSIS package for scheduler job
Technology: .NET, SQL Server, Netbiscuits BML, Teamsite CMS
Confidential
Project Lead
Responsibilities:
- Integration with Vkontakte (social network in Russia) API
- Integration with Facebook API
- Integration with email engines of Russia like Yandex, Rambler, Gmail etc
- Registration, Login, Forgot Password, User profile modules for user
- Invite Friend, Voting, Sharing features for videos created by users
- Contest module which deals with a flash tool
- Admin module which has moderation and report generation feature
- Localization support with release of 4 locales
Technology: PHP 5, MySql, AJAX, JSON, JQuery, Smarty, Facebook (FBML, FQL) &Vkontakte API, Unix
Confidential
Technical Lead
Responsibilities:
- Integration with Teamsite CMS with open deploy workflow support
- Product landing with multi select facet filter mechanism
- Product recommender tool
- BazaarVoice integration for ratings & review feature
- ExpoTV integration for product video reviews
- Stain Brain web service creation
- Google Channel Intelligence integration for buy it now solution
- Lucene search API integration for search feature and GA integration for event tracking
- Localization support
- Package for scheduler job
Technology: J2EE, Spring, Oracle, Teamsite CMS, JQuery
Confidential
Module Lead
Responsibilities:
- Product landing with multi select facet filter mechanism
- Article listing and detail page
- Product and category detail pages
- Bunny section
- BazaarVoice integration for ratings & review feature
- Lucene search API integration
- Localization support
Technology: J2EE, Spring, Hibernet, Oracle,TeamsiteCMS,JQuery
Confidential
Senior Developer
Responsibilities:
- Added module as Claims without module builder.
- Added module as Supplies and Shipping Address without module builder.
- Integrated Zucker Report.
- Customized Project module and Task Grid view
- Configured various workflows on time basis and on several events.
- Integrated Google map to Contacts Address.
- Integrated Just CRM Time-Invoice module and customized to support Tasks & Contacts module
Technology: PHP, MySQL, SugarCRM Prof 5.20e, Smarty,AJAX, Unix
Confidential
Senior Developer
Responsibilities:
- Multi-tenant CRM system
- Layout Manager for custom fields.
- Send IM Notifications to admin for different operations.
- Manage product template backup to be used for other sections
- Tabbed menu style configuration.
- Telemarketing dashboard.
- Ticket Dashboard
- Mobile Interface
- Magento-CRM Integration through Vtigerwebservices
Technology: PHP, MySQL, AJAX, Vtiger 5.1, Smarty, Unix