Hadoop Developer Resume
Pittsburgh, PA
EXPERIENCE SUMMARY
- Over 12 years of experience in Windows and UNIX based applications using JAVA, C++, C, SQL, VB, Perl and Shell Scripts.
- Around 10 years of experience as anOracle PL/SQLDeveloper in Analysis, Design and Implementation of Business Applications along with 4+ years of experience in JAVA application development and maintenance.
- Implementation and working experience of several Big Data technology stack such as Hadoop 1.2.1, MapReduce, HDFS, Hive, Pig, Sqoop, Oozie and good knowledge of YARN.
- In depth understanding of MapReduce concepts and its critical role in data analysis of huge and complex datasets.
- Experienced in performance tuning JAVA MapReduce code and HIVE queries.
- Extended Hive and Pig core functionality by writing custom JAVA User Defined Functions (UDF) and User Defined Aggregating Functions (UDAF); implemented DAG workflows with error handling and mailing through Oozie.
- Proficiency with mentoring and on - boarding new engineers on Hadoop and taught courses on Hadoop and its eco system, setup multi node cluster on Amazon Cloud AWS, installation and configuration of VMs.
- Good knowledge on NoSQL databases like MongoDB and Cassandra.
- Good knowledge of Data warehousing concepts and ETL processes
- Expertise in Client-Server application development using Oracle11g/10g/9i, PL/SQL, SQL *PLUS, TOADandSQL*LOADER.
- Experience inOracle supplied packages (UTL FILE to read and write from database, DBMS SQL to write dynamic SQL), Dynamic SQL, Materialized Views, RecordsandPL/SQL Tables, Indexes and Triggers.
- Strong UNIX Shell Scripting skills.
- Experienced in large development projects involving object oriented technologies and worked with leading firms in Banking Financial Systems, Medical Systems and Insurance.
- Sound knowledge in Fixed Income Securities, Trading, Trade Lifecycle, Market/Reference Data.
TECHNICAL SKILLS
- Hadoop, MapReduce, Hive, Pig, Sqoop, Oozie, MongoDB
- C, C++, VB 6.0, JAVA, Python, SQL, C#, VB.Net
- Oracle 9i/ 10g/ 11g, Sybase 12.5.1, MS-Access
- MFC 6.0, COM, Windows SDK
- XML, HTML, Perl, Shell Scripts
- Windows XP, Windows 2000, Windows 7, Linux, HP-UX, Solaris
PROFESSIONAL EXPERIENCE
Confidential, Pittsburgh, PA
Hadoop Developer
Responsibilities:
- Involved in development of Metadata Management Framework POC to document and accelerate the sourcing and distribution of data utilizing the Big Data technologies such as Hadoop, Hive, Pig Scripts, Sqoop and Oozie.
- Coded Pig Latin, Hive and Map Reduce scripts for data feeds to determine Data Quality checks, metrics and perform Data Profiling and analysis tasks and run them through the Analytic warehouse system.
- Developed data pipelines to import/export structured & unstructured datasets using Sqoop, to move data in and out of the Hadoop ecosystem.
- Developed MapReduce programs using JAVA API to perform data scrubbing for unstructured data.
- Performance tuned existing JAVA MapReduce jobs and Improved performance of HIVE queries by modifying the file formats and introducing partitions/bucketing.
- Created JAVA UDFs for data specific processing and extended HIVE and PIG functionalities.
- Scheduling jobs through Oozie.
Oracle/ C++ Developer
Confidential
Responsibilities:
- Interacting with Business Analyst’s to gather the requirements, analysis and solution implementation and providing application support and enhancement.
- Improvised the C++ Trade Blotter screen to reuse the analytics and calculate the yields automatically while importing the mergers.
- Enhanced the Trade Edit business logic to calculate and use the daily liquidity when present, so that forward settlement trades and 17A7 trades use non-daily liquidity first when a trade is edited.
- Enhanced many UI screens (Portfolio view & summary, Interest Rate, Issuer maintenance, Security Maintenance et al) using Neuron Data, C and SQL scripts, to include the new UI components requested by the trader and investment analyst.
- Designed and implemented a new Ratings screen to capture the Fund Ratings to get away with the hard coded ratings using the C++ framework of the client.
- Migrated Legacy apps to Windows 7 environment from Windows XP.
- Create ad-hoc SQL queries, reports, complex packages, procedures, functions, triggers and other DB objects as needed.
- Implemented the PostTrade workflow packages to automate the Settlement process through Omgeo Connect.
- Improved performance of large data transactions by using bulk data processing like FORALL et al and tuned SQL reports and PL/SQL procedures.
- Improvised the performance of front end by 35%, by using oracle Table Functions.
- Perform data loads from flat files to DB using SQL*Loader and External tables.
- Contributed to development of ETL utility using UNIX Shell scripts, PL/SQL and SQL Loader.
- Re-engineered the Analytics engine.
Sr. Software Engineer
Confidential - Syosset, NY
Environment: C++, JAVA, MFC, Sybase 12.5.1, Shell Scripting, Solaris, Autosys, PVCS
Responsibilities:
- Enhanced the FinCalc Module, to incorporate Trip Segmentation dimension and Specialty Panel.
- Involved in full SDLC of the project.
- Knowledge acquisition from client and transition to offshore development team.
- Setup a 6 member offshore team and lead them from on-site.
Associate Developer
Confidential
Environment: C++, JAVA, PL/SQL, Oracle 9i, Perl, Shell Scripting, Solaris.
Responsibilities:
- Lead an off-shore team of 5 members; coordinate with the onsite coordinator and extensive client interaction to get things done.
- Implementation of Simple Excel Funding File component written in Perl script that takes care of easy movement of data between JPMC, TopCorp and Plan Sponsor.
- Maintenance of the Bulk Funding Process, which handles Funding transactions through SQL Scripts and Perl Scripts.
- Implemented the batch modules involving Shell, SQL & Perl scripts, for File Delivery Manager and File Upload medium.
- Developed SQL scripts to generate reports for the Free Agent Automation module.
- Involved in development of Perl & UNIX scripts for Check Posting module which transmits check posting files from legacy systems to Routing Interface Manager.
- Involved in design and development of Escheatment reports module for HSA, using PL/SQL scripts.
- Implemented the HSA business layers using core JAVA algorithms.
- Worked on code migration from C/C++ to JAVA
- Enhanced EE ER statement, using C++ to track contributions via bulk file as employee (EE), employer(ER) or employer matching contributions (EM).
Sr. Systems Engineer
Confidential
Environment: VC++, COM, MFC, XML, DOM, Syngo, .Net, ClearCase, Windows XP.
Responsibilities:
- Implemented the prototype TDC ActiveX Control, Vertical Tab Control.
- Optimized the Circle ROI algorithm in C++.
Software Programmer
Confidential
Environment: VC++, COM, ATL, MFC, JAVA, XML, DOM, SAX, MS-Access, Oracle 9i.
Responsibilities:
- Design and development of COM Interfaces for the whole Sync System.
- Implemented Record Matching Algorithm, Conflict Handler, Deletion Handler, Multi-threaded Data retrieval and Pagination for fast retrieval of data.
- Involved in Report Generation tool which parses the XML data feed and uses HTML to generate report.
Jr. Programmer
Confidential
Environment: VB, VC++, Oracle 8i, MS-Access, Windows 2000/ 98/ NT.
Responsibilities:
- Support maintenance and enhancement of the product.
- Customizing the product for different clients and deploying it.
- Involved in full SDLC Lifecycle.
