Hadoop Developer Resume
SUMMARY:
- Extensive experience as Production Support Lead.
- Extensive experience in System Analysis, Database Design, Development and Administration of SQL Server 2000/2005/2008 in production, development and testing environments and also good experience in development using Big Data/Hadoop ecosystem tools.
- Good experience in processing Unstructured, Semi - structured and Structured data.
- Hands on Experience with objects creation such as tables, views, indexes, stored procedures, triggers, user defined functions, and data dictionaries using T-SQL scripts.
- Proficient in SQL Server installation, configuration, query optimization, backup/ recovery and database consistency checks.
- Working experience in managing, monitoring and tuning SQL Server performance using SQL Server Profiler/Monitor and Windows Performance Monitor.
- Experience in scripting, analyzing and debugging both new and existing Complex Stored procedures.
- Thorough understanding of the HDFS, Map Reduce framework and extensive experience in developing Map Reduce Jobs
- Experienced in building highly scalable Big-data solutions using Hadoopandmultiple distributions i.e., Cloudera, Horton works and NoSQL platforms
- Well - versed in scheduling Jobs, Alerts and SQL Mail Agent using SQL Server Agent Services.
- Hands-on experience on major components in Hadoop Ecosystem including Hive, HBase, HBase-Hive Integration, PIG, Sqoop, Flume & knowledge of Mapper/Reduce/HDFS Framework.
- Involved in migration from MS SQL Server , DTS to SSIS, running and scheduling SSIS packages, Look-up for cleaning bad data, validating data using SSIS.
- Outstanding in data migration from heterogeneous databases like Oracle, MS access and flat files to SQL Server using SSIS .
- SAS Certified Programmer with experience in Pharmaceutical industry involving data management and statistical analysis for clinical trials.
- Extensive experience in BASE SAS, SAS MACROS, SAS/STAT, SAS/ACCESS, SAS PC (V9.0), SAS/GRAPH.
- Flexible, enthusiastic and project oriented team player with excellent written, verbal communication and leadership skills to develop creative solutions for challenging client needs.
- Experience is providing 24X7 support and on call rotation schedule
TECHNICAL SKILLS:
IDE: Eclipse, net beans
Hadoop Ecosystem: HDFS, Map Reduce, Hive, Pig, Hbase, Sqoop, Oozie, ElasticSearch.
Database Tools: Enterprise Manager, Query Analyzer, SQL Profiler, Upgrade Wizard, Replication, Database Engine Tuning Advisor, Business Intelligence Development Studio (BIDS), Lite Speed.
Databases: MS SQL Server 2000/2005/2008/ 2012 , MySQL, Oracle.
ETL Tools: SQL Server Integration Services (SSIS), Data Transformation Services (DTS), BCP
SAS Tools: SAS/BASE, SAS/MACROS, SAS/STAT, SAS/GRAPH, SAS/SQL, SAS/ACCESS, SAS/ODS, SAS/REPORTS.
Languages: C, C++, T-SQL, PL/SQL
Web Technologies: Web Services, XML, HTML
Operating System: Windows NT/2000/2003/XP, Vista, BizTalk Server
Applications: MS-Office suite, MS Visual Studio 2005
Protocols: HTTP, TCP/IP, FTP
PROFESSIONAL EXPERIENCE:
Confidential
Hadoop Developer
Responsibilities:
- Good understanding and related experience with Hive, Pig and Map/Reduce.
- Deep understanding of schedulers, workload management, availability, scalability and distributed data platforms.
- Installed and configured Hadoop MapReduce, HDFS, developed multiple MapReduce jobs in Java for data cleaning and pre-processing.
- Involved in loading data from UNIX file system to HDFS using HDFS Api and Flume.
- Wrote MapReduce jobs to discover trends in data usage by users.
- Involved in managing and reviewing Hadoop log files.
- Involved in running Hadoop streaming jobs to process terabytes of text data.
- Load and transform large sets of structured, semi structured and unstructured data.
- Wrote pig UDF’s.
- Automated all the jobs starting from pulling the Data from different Data Sources and pushing the result dataset to Hadoop Distributed File System and running MR jobs and PIG.
- Wrote unit test cases using MR Unit
Confidential
Software Analyst (PRODUCTION SUPPORT LEAD)
Responsibilities:
- Provide functional expertise in production environment.
- Involved in job scheduling and alerts.
- Provided a range of services that ensures data integrity, delivery, and security.
- Involved in managing and scheduling daily jobs related to data transfers, backup and recovery using SQL Server Agent and job schedulers.
- Created and maintained database objects, complex Stored Procedures, Triggers, and Tables, Views and SQL Joins and other statements for various applications.
- Wrote new stored procedures and modified existing ones and tuned them such that they reduced the manual work by sending automated alerts.
- Monitored and Managed SQL Server Agent jobs and Alerts to ensure successful completion of tasks on timely manner.
- Created and maintained database objects, complex Stored Procedures, Triggers, and Tables, Views and SQL Joins and other statements for various applications.
- Worked with team in Tuning SQL Statements using Indexes and designed and implemented the Stored Procedures and Triggers for automating tasks.
- Work with Team Members to ensure the project meets business needs.
- Analysis of current processes/systems to endure jobs run well in production.
- Documented various processes and applications for users.
- Responsible for transferring the data using SSIS packages.
- Create SPs & UDF to processing data & as well as fetch data for Reporting.
- Involved in Release and Change Management responsibilities involving track of weekly changes, which are coming into SIT and later getting deployed in Production.
Confidential
SQL Developer
Responsibilities:
- Responsible for managing scope, planning, tracking and change control aspects of the project.
- Installation and configuration of SQL Server 2008 in production and testing environments.
- Involved in database design in client server by analyzing business requirements.
- Communicate activities/progress to client team, business analysts and QA team through weekly feature meetings.
- Responsible for leading, guiding, mentoring ORMB offshore and onshore implementation team.
- Supported Implementation Team in understanding the Business Requirement. Work on change requests and thoroughly testing the changed functionality with impact analysis.
- Implemented customized incremental release packages.
- Documented Use Cases, Functional and Technical Design Documents.
- Generated T-SQL scripts to create and maintain database objects.
- Created and maintained database objects, complex Stored Procedures, Triggers, and Tables, Views and SQL Joins and other statements for various applications.
- Wrote new stored procedures and modified existing ones and tuned them such that they performed well.
- Effectively used temporary tables for stored procedures considering the performance issues with the front-end application.
Confidential
SAS/SQL Programmer
Responsibilities:
- Performed data analysis, statistical analysis, generated reports, listings and graphs using SAS tools- SAS/Base, SAS/Macros, SAS/Graph, SAS/SQL, SAS/Connect and SAS/Access.
- Responsible for locating appropriate databases for projects and performing relevant analysis.
- Processed data collection to ensure proper quality of data.
- Maintained the daily error log for cleaning the data.
- Developed SAS macros for data cleaning, reporting and to support routing processing.
- Analyzed and interpreted numerical data from various sources organized and summarized data for analysis.
- Presented numerical information in various formats. Created SAS Customized reports using the Data null technique.
- Generated graphs using SAS/GRAPH and the SAS Graphics Editor. Used Proc Report to generate reports.
- Prolifically generated SQL queries for the reports.
- Successfully identified problems with the data, produced derived data sets, tables, listings and figures, which analyzed the data to facilitate correction.
Confidential
Application Developer
Responsibilities:
- Streamlined records management within enrollment management program provided by 9 campuses. Automate student enrollment process. Decrease time for required creation of 4,000 records on spreadsheet.
- Designed and built SQL Server database that supported 1,000,000+ active and alumni student records. Combined database with excel to automate spreadsheet development.
- Reduced hours for creating spreadsheet to 2 hours, with zero errors, decrease in man-hours, improvement in system scalability, and production of additional reports.
- Architected Access database, connected system to server, and integrated network with system’s GUI that facilitated teamwork among agents.
- Created manuals on database functionality and provided training and support to staff and student workers.
- Created online surveys for students.
- Worked on university websites, also trained staff on updating the website/ personal profiles on the website.