SUMMARY

• 10.5 years of extensive hands on experience in IT industry, with domain expertise on Big Data Technologies.
• Worked as a Subject Matter expert, Technology Analyst, Application Architect for Music Artist Royalties domain, Survey Text classification, Big Data Applications designing.
• Collaborated with Business Users & engineering team for requirements gathering & scoping and assisted them to solve problems and build applications.
• Vital role in different migration projects by using Data Structures as an ETL to migrate the data from Source to Target system in Big data as well as in Legacy systems using Cobol.
• Excellent Knowledge in understanding Big Data infrastructure, distributed file system, HDFS, parallel processing - MapReduce framework and complete Hadoop ecosystem - Hive, Pig, Mapreduce, Sqoop, Spark with Python and Scala, Oozie and Flume.
• Extensively used various types of ETL Activities using SPARK 1 &2.0 (DFs), FLUME, KAFKA, FLUME, HIVE, PIG, SQOOP, UNIX, EVENT ENGINE, MAGELLAN, ELASTIC SEARCH, KIBANA, TABLEAU, SCALA and PYTHON scripts.
• Working Knowledge in architecting Hadoop solutions including hardware recommendations, storage configurations, performance tuning, administration and support.
• Experience in developing ETL process using Map-Reduce Framework in java.
• Experience in working with different relational databases like MySQL, MS SQL and Oracle, Netezza, Data ware House.
• Proficient in Analyzing, Coding, Testing and implementation of COBOL Programming.
• Substantial development experience in creating stored procedure, PL/SQL Packages.
• Experience and Knowledge in design and deployment of Unix Shell scripting Programming.
• Strong Judgment, Analytical, Communication and Documentation skills in all phases of SDLC process.
• Strong conceptual and technical knowledge of entire SDLCE - Requirement Gathering & Analysis, Planning, Design, Development, Testing and Implementation.
• Have solid understanding in Agile methodology using Rally.
• Well-organized, efficient, quick learner, self-motivated, Good Team Player, ability to work independently and quick learner.
• Experience in troubleshooting, finding root causes, Debugging and automating solutions for operational issues in the production environment
• Handling Development, Testing and Support activities by involved in project estimation and project scheduling.
• Mentoring and training project members to enable them to perform their activities effectively.

TECHNICAL SKILLS

Operating System: OS/390, Windows 2003, Windows XP, Unix, RedHat/CentOS

Environment: Cloudera, MapR FS, HDFS, Windows, Unix, Oracle DB and Mainframe

Database: PL/SQL, Oracle 8i, DB2, HBase, NoSQL Databases.

Languages: PYTHON, SCALA, ACUCOBOL, COBOL, JCL, SQL, CORE JAVA, HTML-5, CSS3, UNIX SCRIPTING, JAVASCRIPT

Visualizations: Tableau, Kibana.

Big Data: HDFS, PySpark, Spark 2.0 Dataframes, Datasets, Kafka, Flume, Spark Streaming, Elastic Search, Kibana, MapReduce, Hive, HBase, Spark, Pig, Sqoop, Flume, Oozie, and Zookeeper

Hadoop Distributions: MapR, Apache, Hortonworks HDP, Cloudera

PROFESSIONAL EXPERIENCE

Confidential, PHOENIX, AZ

Senior Associate

Responsibilities:

• Collaborating with business users/product owners and contributing in analyzing the functional requirements of various use cases.
• Designing each and every applications and building entire product from design phase to deployment phase.
• Developing the applications using different tools & programming languages like Python, Scala, Spark with python and Scala, Spark with Hive Context, Sql-Context, Machine learning scripts using python to analyze the text and classify the text. Using organization specific tools like Magellan, Event engine for scheduling and ingestion.
• Developing applications for data ingestion of derived data or output of use cases into Data lake like cornerstone.
• Involved in Ingestion activities data mapping, data writing, event scheduling, and file format conversions, file compression activities etc.
• Developing numerous scripts in python, for data transformations, data transpose, web scrapping, Json to text conversion, different csv & text file processing etc.
• Developed different Hive UDF’s for business logics like date conversion, applying lookups, implemented complex logics on individual columns etc.
• Extracted data from different source systems using Python extraction scripts and loaded them into HDFS.
• Created numerous Internal and External tables, partitioning, bucketing concepts based on architectural design of applications.
• Implemented Hive-Hbase integrations for faster retrieval and huge data storages with in big data clusters.
• Used different file formats like Text, Parquet, ORC, Json etc. Most frequently used snappy compression techniques in the data lakes.
• For real-time scenarios, we integrated webserver logs with Flume agents and transformed the datasets into aggregated layers using spark streaming.
• Integrated AMEX credit card providers service logs and accessed the urls info using Flume Agent and transferred topics into Kafka Brokers which are setup as Sink, then using Spark stream accessed Kafka topics with KafkaUtils functions to aggregated the data and clean them as required.
• Used Sqoop to migrate the data from MySQL tables into HDFS for reporting the SPOT utility jobs performances and built dashboards which refreshes for every couple of hours. Implemented importing all tables into Hive DB, incremental scenarios, incremental appends and last modified updates etc.
• Automating the existing workflows for various use cases using shell scripts, oozie, python scripting etc.
• Scheduling is taken care using crontab, oozie and organization built in tool called Event Engine.
• Improving the existing performances of the applications by identifying the new tools and technologies in Big Data field. Like converting Hive and Pig scripts into Spark frame work, implementing different file formats and compression techniques to handle the storage and query performances.
• Used flume to extract the logging information from servers and apply transformations on top of it, for different POC’s.
• Performed data analytics in Hive and then exported this metrics back to Kognitio and Jethro using unix scripts for Tableau dashboard.
• Installation and release activities on products into Hadoop cluster. Debugging and troubleshooting the issues in development and Test environments.
• Conducting root cause analysis and resolve production problems and data issues.
• Proactively involved in ongoing maintenance, support and improvements in MapR cluster.

Confidential, DENVER, CO

System Analyst

Responsibilities:

• Collaborating with business users/product owners/developers to contribute to the analysis of functional requirements.
• Maintenance of the Dealer Web Site. Release activities, configuration activities.
• Work on Go Live activities as per the Implementation plan and manage any issues related to functionalities, user interface, performance, etc. that may arise.
• Develop code using knowledge of relevant technology as per design specifications and document artifacts such as unit test scripts, etc. independently and support peers in identifying code defects and ensuring that the output is as per the given specifications and SLAs.
• Implemented various enhancements using core java, ext-js, and webservices.
• New web pages are built or existing web pages are enhanced using Html and client side validations are done using ext-js framework.
• Database integrations are implemented using ejb framework.
• Various complex credit score logics are implemented using the responses from external data sources like Experian, Confidential etc. These third part data sources are communicated using soap web services and their responses are handled and processed, before providing offers to customers.
• All requests to the web pages are logged in databases for further processing and future lookups. As part of it, various DDL, DML scripts are created and implemented for multiple projects and enhancements.
• Conduct Impact Analysis, create Design Specifications as per the high-level design, and create Unit Test Plans IN ORDER TOdevelop / validate / maintain the application as per the requirements.
• Respond to the issues assigned, conduct analysis of the issues assigned, identify and evaluate different workarounds/ solution alternatives, implement the most optimal solution, support other team members on issue resolution in areas of expertise as required, manage stakeholder communication and close the issues assigned in order to ensure support availability as per agreed SLAs.
• Enhancements for the dealer website to in corporate all the new changes in Market like supporting HD receivers, clients etc.

Confidential, NEW YORK, NY

Hadoop Developer

Responsibilities:

• Cooperating with business users/product owners/developers to contribute to the analysis of functional requirements.
• Implemented extraction and transformation logics using big data tools like Pig, Hive, Map Reduce, Hive Java UDF’s, Python UDF’s etc.
• As part of the development, used concepts like Hive internal, external tables, partitioning and bucketing concepts.
• Developed Map Reduce programs to transform the data sets. As part of it used techniques like Distributed cache, Partitioner, Combiners, Secondary Sorting, integrating with Hive tables, Hbase tables, loading and extracting data from HCatalog etc.
• Used multiple ways to enhance the performance like changing the parameters like Number of mappers/reducers, using map-side joins, changing blocksize parameters, GetMerge utility, dist-copy utility for copying files from test/production clusters to development clusters.
• Data from SQL servers are extracted using SQOOP tool and different techniques are used to pull data from tables based on last updated value, create Hive tables, importing all tables into HDFS, Updating tables in HDFS etc.
• Transformed data and aggregated data are exported back to MySQL servers using sqoop.
• Automated the workflow for Artist Dashboard using shell scripts and Map-Reduce Programs.
• Design & Develop ETL workflow using oozie for business requirements that includes automating the extraction of data from MySQL database into HDFS using Sqoop scripts.
• Designed workflow by scheduling Hive processes for log files that are streamed into HDFS using Flume.
• Performed data analytics in Hive and then exported this metrics back to reporting databases using Sqoop.
• Installation and Administration of a Hadoop cluster. Debugging and troubleshooting the issues in development and Test environments.
• Conducting root cause analysis and resolve production problems and data issues.
• Proactively involved in ongoing maintenance, support and improvements in Hadoop cluster.

Confidential, NEW YORK, NY

Responsibilities:

• Preparation of Requirements Analysis, Impact Analysis and Test Results Documents.
• Preparation of Traceability Matrix.
• Review of Functional Specifications.
• Developing migrations logics using Unix scripts and validating the data.
• DE scoped existing applications, which are legacy technologies and removing the scheduling jobs, and making sure that the migrated data matches to the pre-migration data.
• DE scoped current tables and files, which are no longer used by the new technologies.
• Developing scripts and fixing code issues based on the impact of SAP migration.
• Preparation of test cases and execution (Unit Test Plan and Unit Test Results).
• Direct interaction with Business Manager and Business Team regarding planning and migration activities.
• Conducting user acceptance testing and communicating with users on the accuracy of the migrated data and getting their signoffs.
• Production implementation activities and change management activities.
• Attend peer reviews and internal inspections.

Confidential, NEW YORK, NY

Subject Matter Expert and Technology Analyst

Responsibilities:

• Involvement in system appreciating document preparation. Performing impact analysis on the requirements, creating design documents like High-level and Detail-level.
• Preparing and reviewing the Unit-Test case plan prepared by development team.
• Developing applications using Acu-Cobol technologies for calculating the Royalty payments, based on the business provided requirements.
• Developing reports for different life cycles like Domestic, Licensing, Monthly life cycles.
• Maintaining applications and supporting them during all these life cycles each year.
• Resolving different business users’ issues on data appearing in reports and implementing the fixes in production.
• Enhancing existing reports as suggested by consumers by adding new columns, new insights to reports.
• Modifying the scheduling frequency of the executions of reports from Semi-Annual to weekly/monthly.
• Modifying the Menu items and adding new features to menu items like Artist reports to Royalty reporting menu, publisher’s reports to Licensing reporting menu etc.
• Developing or changing the artist/publisher royalty calculations based on new regulations passed on by the government or different agencies.
• Performance enhancements on the existing process for faster completion and better archival/storage process.
• Providing support activities for the applications in production.
• Fixing of production incidents/adhoc tasks.
• Preparation of test cases and execution (Unit Test Plan and Unit Test Results) for implemented fixes or changes.
• Direct interaction with Business Manager and Business Team, for user acceptance signoffs.
• Preparation of Reconciliation for data processed/migrated during the Migration Projects.

Environment: ACUCOBOL, Unix Shell Scripting, PL/SQL, Oracle 8i, VB scripting, Mainframe, Unix file system.