- Around 5 years of extensive experience in complete Software Development Life Cycle (SDLC) including gathering System Requirements, Design, Coding, Development, Testing, Production Support, Maintenance and Enhancement.
- Hands on experience in installing, configuring and using Apache Hadoop ecosystems such as HDFS, MapReduce, HIVE, PIG, SQOOP, FLUME and OOZIE.
- Hands on experience on Hortonworks and Cloudera Hadoop environments.
- Strong understanding of Hadoop daemons and MapReduce concepts.
- Strong understanding of Scala
- Developed scripts in importing - exporting data into HDFS format.
- Developed scripts in analyzing big data using Hadoop environment.
- Developed scripts in handling Hadoop Ecosystem Projects such as Hive, Pig and Sqoop.
- Developed in devUDFs for Hive using Java.
- Strong experience in working with Elastic MapReduce and setting up environments on Amazon AWS EC2 instances
- Strong understanding of NoSQL databases like HBase, MongoDB.
- Experience in writing Microsoft SQL queries, Stored Procedures, Functions and Packages for various validation purposes using, MySQL, SQLServer, Crystal reports, TSQL, TOAD.
- Experience in Data modeling for Data Warehouse/Data Mart development, Data Analysis for Online Transaction Processing (OLTP) and Data Warehousing (OLAP)/Business Intelligence (BI) applications and tuning SQL Server Integration Services (SSIS) packages and SSRS reports.
- Comprehensive knowledge and experience in process improvement, Experience in extracting, transforming and loading (ETL ) data from spreadsheets, database normalization/de-normalization, data extraction, data cleansing, data manipulation.
- Experience in BI Development, SSIS packages from MS-Access, Excel, Oracle.
- Designing and Deployment of Reports for the End-User requests using Web Interface & SSRS.
- Experience in controlling the User privileges, Monitoring the Security and Audit issues as per Standards.
- Expertise in generating reports using SQL Server reporting Services, Crystal Reports, and MS Excel Power View.
- Excellent T-SQL Developer skills including Stored Procedures, Indexed Views, User Defined Functions, Triggers, and Distributed Transactions.
- Implemented new solutions with emerging technologies for resolving issues such as inefficiency, performance bottleneck in distributed applications.
- Proficient in Unit Testing and Integrated Testing.
- Mentored team member to learn and use emerging technologies and domain knowledge.
- Provided solutions to automate development and testing for web services-based application.
- Strong analytical thinking and problem-solving skills backed by good communication skill.
Hadoop/Big Data: HDFS, MapReduce, Hive, Pig, Sqoop, Spark, Hbase, Flume, Oozie, and ZooKeeper, AWS (EMR)
Programming Languages: PHP,JAVA, C, C++, C#, XML, SQL, PL/SQL, Unix shell scripts.
Application/Web Servers: Apache Tomcat, IBM Web Sphere, BEA Web Logic.
Tools & Utilities: SQL Server Reporting Services (SSRS), ETL and SQL Server Integration Services (SSIS), Business Intelligence Development Studio (BIDS), Eclipse, Edit plus, Text pad, WinSCP.
RDBMS: MS SQL Server 2012, DB2, BI Tools SSIS, SSRS, Oracle10g,11g,12c., MySQL 4.0, T-SQL, MS Access.
Source Control: Subversion (SVN)
Operating Systems: Windows 9x/2000/XP, Linux, Unix.
Big Data Developer
- Involved in extracting customer’s Big data from various data sources into Hadoop HDFS. This included data from databases and also log data from servers.
- Used Sqoop to efficiently transfer data between databases and HDFS and used Flume to stream the log data from servers.
- Developed Map Reduce programs to cleanse the data in HDFS obtained from heterogeneous data sources to make it suitable for ingestion into Hive schema for analysis.
- Implemented Partitioning, Bucketing in Hive for better organization of the data.
- Developed User Defined Functions in Pig and Hive
- Used Oozie workflow engine to manage interdependent Hadoop jobs and to automate several types of Hadoop jobs such as Java Map-reduce, Hive and Sqoop as well as system specific jobs.
- Installed and configured Hive, Sqoop, Pig, Cloudera Manager and Oozie on the Hadoop Cluster.
- Involved in implementing High Availability and automatic failover infrastructure to overcome single point of failure for Name node utilizing Zookeeper services.
- Involved in upgrading the cluster from CDH 4.x to CDH5.x
- Worked on Commissioning and decommissioning the Datanodes in Hadoop Cluster.
- Experienced in access controls, security using Kerberos and Sentry.
- Involved in Migrating the data from Test Cluster to Virtual Cluster.
- Worked with BI teams in generating the reports on Tableau.
Environment: Sqoop, Pig, Hive, Oozie, Flume, Tableau, HDFS, Map Reduce
SQL and Web Developer
Platform: Amanda 6, SQL Server 2012
- Actively involved in software development life cycle starting from requirements gathering and performing Object Oriented Analysis.
- Involved in Requirement analysis for implementing Amanda for Public Works division in Confidential .
- Developed and optimized database structures, stored procedures, Dynamic Management views, and user-defined functions.
- Involved in writing Amanda batch jobs, SQL storedprocedures, database functions, database triggers.
- Created Common Codes folders for use across the team.
- Created custom reports using Crystal Reports and Mail Merge Documents to generate the license from Amanda application
- Written complex batch jobs and procedures for Importing and Exporting data.
- Involved in SQL Server Configuration, Implementation and Trouble-shooting for Business work.
- Involved in daily code drops and archives on the production databases during maintenance windows.
- Created custom reports using SAPCrystal Reportsreports according to business requirement.
- Developed user interface using JSP, HTML5, jQueryand performed Client-side validations using Java Script, and CSS.
- Developed different Scripts for the Data analysts for Verification of reports.
- Performed detailed documentation of Report design, development, testing and deployment activities.
- Implementing all Amanda application changes into production
- Maintaining the production system for day to day issues and resolve all the issues opened in Forge ticketing system
- Attending daily meeting with IDPH ( Confidential ) to walk thru fixed issues and prioritize existing issues.
- Watching daily/weekly interfaces runs smoothly between NREMT (National Registry of Emergency Medical Technicians) And IDPH, also other interfaces from Department of workforce development.
Environment: s: Business Intelligence Development Studio (BIDS), SQL Server Reporting Services (SSRS), ETL, SQL Server Integration Services (SSIS), Microsoft SQL Server 2012, Tomcat in a Linux Environment, Crystal Reports, JSP, Java Script, jQuery, HTML, CSS, Agile Methodology.
Confidential, Dallas, TX
- Involving in creating Technical Specification Document (TSD) for the project by conducting meeting with the users to meet their business requirements.
- Extracted data from sources like SQL server, DB2 and fixed width and delimited flat files and Loading into the Oracle target database.
- Responsible for developing ETL processes/programs with Informatica Power Center to extract data from the client's operational databases and Flat Files, Transform and filter the data, and Load it into Target database and Flat Files
- Used various transforms in SSIS to load data from flat files, Siebel CRM and FTP to the SQL databases.
- Using Informatica Power Center Work Flow manager to create sessions with the task developer and workflows with workflow manager to run the logic embedded in the mappings.
- Creating mappings with Informatica PowerCenter 9.6.1 using Transformations (Expression Transformation, Router Transformation, and Stored Procedure Transformation etc.) and Look ups.
- Analyzing session log files in session failures to resolve errors in mapping or session configuration.
- Verified test results (viewing the log files using VI editor, querying the database using the SQL) documenting the defects.
- Experience working on Lotus notes for the workspace environment.
- Working with Control-M team and Involving in Production Support in resolving issues and bugs.
- Experience in writing UNIX shell scripts to run and schedule batch jobs.
- Extracted the data from Oracle, DB2, XML and Flat Files into Data warehouse.
- Provided index tuning as required for performance improvement of the applications (SQL tuning).
- Proficient in Unit Testing and Integrated Testing.
Environment: SQL Server Reporting Services (SSRS), ETL, Informatica, DB2, Microsoft SQL Server 2012, Putty Unix Environment, Water Flow Methodology .