PROFESSIONAL SUMMARY:

• Around 10.7 years of total IT experience as Ab Initio, Big data and spark with expertise in Financial & Banking.
• Experience in AGILE development methodologies.
• Experience in designing and developing applications in spark (Python).
• Strong experience in writing applications using python.
• Created re - usable templates using python for running creating configuration files for monthly run of feature set.
• Implemented spark using python and SparkSQL for faster testing and processing of data.
• Hands on experience in created RDD and data frames.
• Validation and analysis of output dataset using Apache Drill/Drill explorer.
• Created complex HQL for analysis of source parquet files and create join strategy.
• Good working knowldege of Ingestion of data into data lake.
• Good knowledge of Python libraries such as Pandas and Numpy.
• Good Knowledge of AWS: EC2, IAM, S3 etc.
• Exposure of AI and ML.
• Hands on experience in creating Continuous, Batch and generic graphs using Ab Initio.
• Have worked on XML processing with AbInitio, where XML files come to AbInitio through Java UI.
• Implemented Plans (Conduct>It), component folding and Micrographs.
• Worked on Ab initio advanced concepts such as Metaprogramming and PDL.
• Having working experience in creating different korn shell scripts and handling graphs and plans at backend.
• Experience in scheduling tools such as Op>Console/Control Center and Control-M.
• Extensive experience in writing and tuning complex SQL queries, Sub-queries.
• Hands on experience on version control system (EME).
• Exposure of data profiling and Metadata Hub.
• Well versed in EME administration, Code migration and environment check out.
• Working knowledge of QC and ALM tools for bug/defect tracking.
• Experience in reporting tools such as Business objects, Hyperion, Tableau.
• Team Leadership, Project Management, SOW, Fieldglass tool.

TECHNICAL SKILLS:

Hadoop ecosystem: HDFS, Hive, Apache Drill, Hue, PIG scripting, Python, spark (Pyspark, Scala)

Cloud: IAM, S3, EC2, Snowball, Lambda

ETL Tool: Ab>Initio, Informatica

Databases: Teradata, Oracle, MS SQL

Programming: C, C++, Korn Shell Scripting, Core Java, Python, Pandas, Numpy, SQL, T-SQL, PLSQL

Database Tools: Teradata SQL assistance, DB Visualizer, Squirrel, Hue, oracle SQL developer, SQLite

Reporting tool: MS Excel, Hyperion, Business Objects, Tableau

Scheduling tool: Control Center, Control-M, Maestro, Autosys, Dollar Universe, CA-7

Operating Systems: Windows, Unix, Linux

Other: GitHub, RLM, Fieldglass

PROFESSIONAL EXPERIENCE:

Confidential, Charlotte, NC

Environment: Python, Teradata, Squirrel, PySpark, Hive, Apache Drill, Jupiter Notebook, Hue, Autosys, GitHub, Parquet files, DCT Framework, DCI framework

Project Lead

Responsibilities:

• Involved in the Analysis, Design Phase and requirement gathering for data ingestion
• Working closely with Data science team for requirement gathering/ Feature model validation.
• Creating complex data frame, transformation and writing attributes as in parquet files.
• Writing configuration files which are input to the spark framework.
• Created reusable template for creating configuration files which is input for Spark framework.
• Writing complex hive SQL to analyze data and validate transformations.
• Written HQL’s by applying the transformation/Business rules.
• Used Apace Drill/ Drill explorer for querying HDFS data for analysis and validate output files.
• Identified areas of improvement in existing business by unearthing insights by analyzing vast amount of data using Hive and Pyspark.
• Analysis of various sources and consolidate all in data foundary.
• Building feature data model and by using Pyspark which is input for data science team to build chatbot etc.

Confidential, Tampa, FL

Environment: Python, Spark, Hive, Anaconda tools, HDFS, Hadoop, Parquet files, AbInitio, Teradata, Oracle, Autosys, MS SQL

Project Lead

Responsibilities:

• Working with business users for requirement gathering and analysis.
• Prepared Ab Initio ETL jobs inventory and document existing process, split it into small modules.
• Analyzed and estimated migration efforts to migrate ETL jobs in to Pyspark. the team with existing ETL design, data model and create development strategy.
• Created complex SparkSQL code and creating data frame.
• Design and development of spark jobs based on new data model build on top of data lake.
• Developed apache spark job to process and transform terabytes of data.
• Work with Business users to review pyspark code and spark output.
• Used apache spark to build data pipeline to ingest data into data lake.
• Validate output dataset by running statistics on output file and compare with existing system.
• Validate output attributes using Apache Drill.
• Involved with team to create Hive catalog on top of Source and target parquet files.
• Created complex HiveQL for validation and analysis of Source and target parquet files.
• Created automated validation process to validate data between existing output and Spark ETL output for 100+ tables.
• Schedule Pyspark jobs using Autosys scheduler.
• Code management and code control using GitHub tool.
• Preparing technical design documents, reviewing the codes of team members.

Confidential, Chicago, IL

Environment: Ab Initio, Unix, Oracle, Autosys

Senior Developer / Lead

Responsibilities:

• Direct interactions with Business analysts to collect the Business requirements.
• Write complex SQL query to analyze CRS application.
• Handling Production issues and working on Autosys scheduler tool.
• Developing wrapper scripts that can be used to execute the entire process and send mail notifications after the execution of every step.
• Enhancement in existing code for Crontab script and provide data security.
• Tuning of Ab Initio graphs for better performance and too ensure that any tuning in the existing code would not impact the process.
• Analyzing the changes for the percentage gain in the overall processing time.
• Involved in deployment of the changes in production and providing support for the applications as well.
• Experience on Agile Methodology Programming and Rally (Agile model) Tool.

Confidential, Atlanta, GA

Environment: Ab Initio, Unix, CA-7, Mainframe, MSSQL, Oracle

Senior Developer / Lead

Responsibilities:

• Analyze data flow in existing Dime system and create design doc to move complete system in Hadoop based Atlas data lake. Come up with High level and Low level Design document.
• Study the relation at the source systems.
• Provided end to end Architecture solution for the project.
• Analyzing CA-7 Jobs and create Schedule doc for Atlas data lake.
• Implement the ETL solution using Ab Initio (3. *) tools and involved in whole development life cycle of project like design, coding, testing, migration and production support.
• Creating Conduct>It generic Plans, Plan psets, graph psets, generic graphs and do rationalization.
• Identify the delta changes on these sources and ingest the data with updated code into Lake
• Worked on mainframe files, extracting them using Ingest process and Direct extract.
• Complex SQL using PB2EE and MYSQL server.

Confidential, Richmond, VA

Environment: Ab Initio, Unix, Control-M, EME, Version One

Team member

Responsibilities:

• Working closely with business line and risk managers to assess and manage firm wide risk management.
• Creating Ab Initio graphs for various matching process.
• Scheduling Job using Control-M.
• Development of UNIX wrapper shell scripts.
• Worked on Sed and awk commands.
• Interactions with BA & Dev teams to resolve the issues.
• Worked with using EME for Version control, Impact analysis, Dependence analysis for Common projects, higher environment migrations.
• Created Automation testing Unix script for various file validation.
• Involved in creation of mocked data for various scenarios of matching model for QA testing.
• Worked on Metadata management services (MDH).
• Check in, Check out and migration of project in different environments.
• Created Unix script for creating and sending status report of daily jobs and saving time for job monitoring.

Confidential, Richmond, VA

Environment: Ab Initio, UNIX, Control-M, Teradata, EME, Version One

Team member

Responsibilities:

• Analyzing the files in Production as well in EME to determine quality of the data.
• Having multiple rounds of discussion with the Source POC to clarify questions.
• Passing the requirements to the offshore team and reviewing the development work done by Team.
• Developing Ab Initio Generic Graphs, psets, xfr’s to SFTP files from various DDE server’s to IDQ server.
• Development of UNIX wrapper korn shell scripts.
• Doing research on dependency and create Sub-form spread Sheet to migrate jobs into Control-M
• Have done good analysis on control-M jobs and corrected schedules of existing jobs and finalizing new Jobs.
• Migrated the projects from Ab Initio 2.15 to 3.1.
• Getting involved in validation and testing of files and conduct Walkthroughs.
• Creating and maintaining various versions of documents generated during project.
• Performing the gap analysis for failed jobs to fix it and process file manually.

Confidential, Norwalk, CT

Environment: Informatica Power Center, Teradata, SQL Server, Hyperion, Business Objects, UNIX, ALM

Business Analyst

Responsibilities:

• Gathering business requirements, analyze the requirements and business. Evaluate information from multiple sources; decompose high level information into details and documenting (Business Requirement Documents) those.
• Explaining the business requirements, collaborate with design/architect team and developers for the overall design and low-level design for data model, ETL and Reporting and documenting (FSD, LLD, TDD) those.
• Analyzing different sub ledgers and creating complex SQL queries to Load Data into Stage to Core to Access Layer.
• Understanding the business Process definition, Risk Analysis and SDLC methodologies.
• Creating the proposal for Business and walk them through to get the agreement.
• Create high level, low level design documents for data integration.
• Involved in creating BO report on top of IRIS and Access Layer as per need of different business users.
• Worked on Hyperion EPM 11 Reports